Distance metric learning from uncertain side information with application to automated photo tagging
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2009
Abstract
Automated photo tagging is essential to make massive unlabeled photos searchable by text search engines. Conventional image annotation approaches, though working reasonably well on small testbeds, are either computationally expensive or inaccurate when dealing with large-scale photo tagging. Recently, with the popularity of social networking websites, we observe a massive number of user-tagged images, referred to as "social images", that are available on the web. Unlike traditional web images, social images often contain tags and other user-generated content, which offer a new opportunity to resolve some long-standing challenges in multimedia. In this work, we aim to address the challenge of large-scale automated photo tagging by exploring the social images. We present a retrieval based approach for automated photo tagging. To tag a test image, the proposed approach first retrieves k social images that share the largest visual similarity with the test image. The tags of the test image are then derived based on the tagging of the similar images. Due to the well-known semantic gap issue, a regular Euclidean distance-based retrieval method often fails to find semantically relevant images. To address the challenge of semantic gap, we propose a novel probabilistic distance metric learning scheme that (1) automatically derives constraints from the uncertain side information, and (2) efficiently learns a distance metric from the derived constraints. We apply the proposed technique to automated photo tagging tasks based on a social image testbed with over 200,000 images crawled from Flickr. Encouraging results show that the proposed technique is effective and promising for automated photo tagging.
Keywords
automated photo tagging, distance metric learning, uncertain side information
Discipline
Computer Sciences | Databases and Information Systems
Publication
MM'09: Proceedings of the 2009 ACM International Conference on Multimedia and co-located Workshops, October 19-24, 2009, Beijing, China
First Page
135
Last Page
144
ISBN
9781605586083
Identifier
10.1145/1631272.1631293
Publisher
ACM
City or Country
New York
Citation
WU, Lei; HOI, Steven C. H.; JIN, Rong; ZHU, Jianke; and YU, Nenghai.
Distance metric learning from uncertain side information with application to automated photo tagging. (2009). MM'09: Proceedings of the 2009 ACM International Conference on Multimedia and co-located Workshops, October 19-24, 2009, Beijing, China. 135-144.
Available at: https://ink.library.smu.edu.sg/sis_research/2369
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://dx.doi.org/10.1145/1631272.1631293