Although both online learning and kernel learning have been studied extensively in machine learning, there is limited effort in addressing the intersecting research problems of these two important topics. As an attempt to fill the gap, we address a new research problem, termed Online Multiple Kernel Classification (OMKC), which learns a kernel-based prediction function by selecting a subset of predefined kernel functions in an online learning fashion. OMKC is in general more challenging than typical online learning because both the kernel classifiers and the subset of selected kernels are unknown, and more importantly the solutions to the kernel classifiers and their combination weights are correlated. The proposed algorithms are based on the fusion of two online learning algorithms, i.e., the Perceptron algorithm that learns a classifier for a given kernel, and the Hedge algorithm that combines classifiers by linear weights. We develop stochastic selection strategies that randomly select a subset of kernels for combination and model updating, thus improving the learning efficiency. Our empirical study with 15 data sets shows promising performance of the proposed algorithms for OMKC in both learning efficiency and prediction accuracy
Online learning, Kernel methods, Multiple kernels, Perceptron, Hedge, Classification
Computer Sciences | Databases and Information Systems
Data Management and Analytics
HOI, Steven C. H.; JIN, Rong; ZHAO, Peilin; and YANG, Tianbao.
Online Multiple Kernel Classification. (2013). Machine Learning. 90, (2), 289-316. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/2294
Copyright Owner and License
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.