Efficient Benchmarking of Content-based Image Retrieval via Resampling

Publication Type

Conference Proceeding Article

Publication Date

10-2006

Abstract

While content-based image retrieval (CBIR) is an expanding field, and new approaches to ever more effective retrieval are frequently proposed, relatively little attention has so far been paid to the process of evaluating the effectiveness of CBIR methods. Most of the reported evaluations use standard IR evaluation methodologies, with little consideration of their statistical significance or appropriateness for CBIR, which makes it difficult to assess the precise impact of individual methods. In this paper, we present a new approach for evaluating CBIR systems which provides both efficient and statistically-sound performance evaluation. The approach is based on stratified sampling, and provides a significant improvement over existing evaluation approaches. Comprehensive experiments using our approach to evaluate a range of CBIR methods have shown that the approach reduces not only the estimation error, but also reduces the size of the test data set required to achieve specific estimation error levels.

Discipline

Databases and Information Systems | Numerical Analysis and Scientific Computing

Publication

MULTIMEDIA '06: Proceedings of the 14th ACM International Conference on Multimedia, October 23-27, Santa Barbara, CA

First Page

569

Last Page

578

ISBN

9781595934475

Identifier

10.1145/1180639.1180758

Publisher

ACM

City or Country

Santa Barbara, CA

Additional URL

http://dx.doi.org/10.1145/1180639.1180758

Link to Full Text

Share

COinS