Publication Type

Journal Article

Version

acceptedVersion

Publication Date

1-2010

Abstract

Based on the local keypoints extracted as salient image patches, an image can be described as a "bag-of-visual-words (BoW)" and this representation has appeared promising for object and scene classification. The performance of BoW features in semantic concept detection for large-scale multimedia databases is subject to various representation choices. In this paper, we conduct a comprehensive study on the representation choices of BoW, including vocabulary size, weighting scheme, stop word removal, feature selection, spatial information, and visual bi-gram. We offer practical insights in how to optimize the performance of BoW by choosing appropriate representation choices. For the weighting scheme, we elaborate a soft-weighting method to assess the significance of a visual word to an image. We experimentally show that the soft-weighting outperforms other popular weighting schemes such as TF-IDF with a large margin. Our extensive experiments on TRECVID data sets also indicate that BoW feature alone, with appropriate representation choices, already produces highly competitive concept detection performance. Based on our empirical findings, we further apply our method to detect a large set of 374 semantic concepts. The detectors, as well as the features and detection scores on several recent benchmark data sets, are released to the multimedia community.

Keywords

Bag-of-visual-words, representation choice, semantic concept detection

Discipline

Graphics and Human Computer Interfaces

Research Areas

Intelligent Systems and Optimization

Publication

IEEE Transactions on Multimedia

Volume

12

Issue

1

First Page

42

Last Page

53

ISSN

1520-9210

Identifier

10.1109/TMM.2009.2036235

Publisher

Institute of Electrical and Electronics Engineers

Download

Share

COinS