Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2016
Abstract
This paper aims to investigate efficient and scalable machine learning algorithms for resolving Non-negative Matrix Factorization (NMF), which is important for many real-world applications, particularly for collaborative filtering and recommender systems. Unlike traditional batch learning methods, a recently proposed online learning technique named "NN-PA" tackles NMF by applying the popular Passive-Aggressive (PA) online learning, and found promising results. Despite its simplicity and high efficiency, NN-PA falls short in at least two critical limitations: (i) it only exploits the first-order information and thus may converge slowly especially at the beginning of online learning tasks; (ii) it is sensitive to some key parameters which are often difficult to be tuned manually, particularly in a practical online learning system. In this work, we present a novel family of online Adaptive Passive-Aggressive (APA) learning algorithms for NMF, named "NN-APA", which overcomes two critical limitations of NN-PA by (i) exploiting second-order information to enhance PA in making more informative updates at each iteration; and (ii) achieving the parameter auto-selection by exploring the idea of online learning with expert advice in deciding the optimal combination of the key parameters in NMF. We theoretically analyze the regret bounds of the proposed method and show its advantage over the state-of-the-art NN-PA method, and further validate the efficacy and scalability of the proposed technique through an extensive set of experiments on a variety of large-scale real recommender systems datasets.
Keywords
Non-Negative Matrix Factorization, Online Learning, Adaptive, Regularization, Learning with Expert Advice
Discipline
Databases and Information Systems | Theory and Algorithms
Publication
CIKM 2016: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management: Indianapolis, October 24-28, 2016
First Page
1161
Last Page
1170
ISBN
9781450340731
Identifier
10.1145/2983323.2983786
Publisher
ACM
City or Country
New York
Citation
LIU, Chenghao; HOI, Steven C. H.; ZHAO, Peilin; SUN, Jianling; and LIM, Ee-Peng.
Online adaptive passive-aggressive methods for non-negative matrix factorization and its applications. (2016). CIKM 2016: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management: Indianapolis, October 24-28, 2016. 1161-1170.
Available at: https://ink.library.smu.edu.sg/sis_research/3450
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://doi.org./10.1145/2983323.2983786