Cross-modal hashing integrates the advantages of traditional cross-modal retrieval and hashing, it can solve large-scale cross-modal retrieval effectively and efficiently. However, existing cross-modal hashing methods rely on either labeled training data, or lack semantic analysis. In this paper, we propose Cross-Modal Self-Taught Hashing (CMSTH) for large-scale cross-modal and unimodal image retrieval. CMSTH can effectively capture the semantic correlation from unlabeled training data. Its learning process contains three steps: first we propose Hierarchical Multi-Modal Topic Learning (HMMTL) to detect multi-modal topics with semantic information. Then we use Robust Matrix Factorization (RMF) to transfer the multi-modal topics to hash codes which are more suited to quantization, and these codes form a unified hash space. Finally we learn hash functions to project all modalities into the unified hash space. Experimental results on two web image datasets demonstrate the effectiveness of CMSTH compared to representative cross-modal and unimodal hashing methods.
Cross-modal hashing, Image retreival, Self-taught learning, Semantic correlation
Graphics and Human Computer Interfaces | Software Engineering
Information Systems and Management
XIE, Liang; ZHU, Lei; PAN, Peng; and LU, Yansheng.
Cross-Modal Self-Taught Hashing for large-scale image retrieval. (2016). Signal Processing. 124, 81-92. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/3587
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.