Efficient Benchmarking of Content-based Image Retrieval via Resampling
Conference Proceeding Article
While content-based image retrieval (CBIR) is an expanding field, and new approaches to ever more effective retrieval are frequently proposed, relatively little attention has so far been paid to the process of evaluating the effectiveness of CBIR methods. Most of the reported evaluations use standard IR evaluation methodologies, with little consideration of their statistical significance or appropriateness for CBIR, which makes it difficult to assess the precise impact of individual methods. In this paper, we present a new approach for evaluating CBIR systems which provides both efficient and statistically-sound performance evaluation. The approach is based on stratified sampling, and provides a significant improvement over existing evaluation approaches. Comprehensive experiments using our approach to evaluate a range of CBIR methods have shown that the approach reduces not only the estimation error, but also reduces the size of the test data set required to achieve specific estimation error levels.
Databases and Information Systems | Numerical Analysis and Scientific Computing
Data Management and Analytics
MULTIMEDIA '06: Proceedings of the 14th ACM International Conference on Multimedia, October 23-27, Santa Barbara, CA
City or Country
Santa Barbara, CA
SHEN, Jialie and Shepherd, John.
Efficient Benchmarking of Content-based Image Retrieval via Resampling. (2006). MULTIMEDIA '06: Proceedings of the 14th ACM International Conference on Multimedia, October 23-27, Santa Barbara, CA. 569-578. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/1251