Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

6-2015

Abstract

Weak supervisory information of web images, such as captions, tags, and descriptions, make it possible to better understand images at the semantic level. In this paper, we propose a novel online multimodal co-indexing algorithm based on Adaptive Resonance Theory, named OMC-ART, for the automatic co-indexing and retrieval of images using their multimodal information. Compared with existing studies, OMC-ART has several distinct characteristics. First, OMCART is able to perform online learning of sequential data. Second, OMC-ART builds a two-layer indexing structure, in which the first layer co-indexes the images by the key visual and textual features based on the generalized distributions of clusters they belong to; while in the second layer, images are co-indexed by their own feature distributions. Third, OMC-ART enables flexible multimodal search by using either visual features, keywords, or a combination of both. Fourth, OMC-ART employs a ranking algorithm that does not need to go through the whole indexing system when only a limited number of images need to be retrieved. Experiments on two published data sets demonstrate the efficiency and effectiveness of our proposed approach.

Keywords

Hierarchical image co-indexing, multimodal search, online learning, clustering, weakly supervised learning

Discipline

Databases and Information Systems | Graphics and Human Computer Interfaces

Research Areas

Data Science and Engineering

Publication

Proceedings of the 5th ACM International Conference on Multimedia Retrieval, ICMR, Shanghai, China, June 23-26

First Page

219

Last Page

226

ISBN

9781450332743

Identifier

10.1145/2671188.2749362

Publisher

ACM

City or Country

Shanghai, China

Additional URL

https://doi.org/10.1145/2671188.2749362

Download

Share

COinS