Conference Proceeding Article
This paper aims to investigate efficient and scalable machine learning algorithms for resolving Non-negative Matrix Factorization (NMF), which is important for many real-world applications, particularly for collaborative filtering and recommender systems. Unlike traditional batch learning methods, a recently proposed online learning technique named "NN-PA" tackles NMF by applying the popular Passive-Aggressive (PA) online learning, and found promising results. Despite its simplicity and high efficiency, NN-PA falls short in at least two critical limitations: (i) it only exploits the first-order information and thus may converge slowly especially at the beginning of online learning tasks; (ii) it is sensitive to some key parameters which are often difficult to be tuned manually, particularly in a practical online learning system. In this work, we present a novel family of online Adaptive Passive-Aggressive (APA) learning algorithms for NMF, named "NN-APA", which overcomes two critical limitations of NN-PA by (i) exploiting second-order information to enhance PA in making more informative updates at each iteration; and (ii) achieving the parameter auto-selection by exploring the idea of online learning with expert advice in deciding the optimal combination of the key parameters in NMF. We theoretically analyze the regret bounds of the proposed method and show its advantage over the state-of-the-art NN-PA method, and further validate the efficacy and scalability of the proposed technique through an extensive set of experiments on a variety of large-scale real recommender systems datasets.
Non-Negative Matrix Factorization, Online Learning, Adaptive, Regularization, Learning with Expert Advice
Databases and Information Systems | Theory and Algorithms
Data Management and Analytics
CIKM 2016: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management: Indianapolis, October 24-28, 2016
City or Country
LIU, Chenghao; HOI, Steven C. H.; ZHAO, Peilin; SUN, Jianling; and LIM, Ee-Peng.
Online adaptive passive-aggressive methods for non-negative matrix factorization and its applications. (2016). CIKM 2016: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management: Indianapolis, October 24-28, 2016. 1161-1170. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/3450
Copyright Owner and License
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.