Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2022
Abstract
Known-item video search is effective with human-in-the-loop to interactively investigate the search result and refine the initial query. Nevertheless, when the first few pages of results are swamped with visually similar items, or the search target is hidden deep in the ranked list, finding the know-item target usually requires a long duration of browsing and result inspection. This paper tackles the problem by reinforcement learning, aiming to reach a search target within a few rounds of interaction by long-term learning from user feedbacks. Specifically, the system interactively plans for navigation path based on feedback and recommends a potential target that maximizes the long-term reward for user comment. We conduct experiments for the challenging task of video corpus moment retrieval (VCMR) to localize moments from a large video corpus. The experimental results on TVR and DiDeMo datasets verify that our proposed work is effective in retrieving the moments that are hidden deep inside the ranked lists of CONQUER and HERO, which are the state-of-the-art auto-search engines for VCMR.
Keywords
Interactive search, video corpus moment retrieval, reinforcement learning, user simulation
Discipline
Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces
Research Areas
Intelligent Systems and Optimization
Publication
Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal, 2022 October 10-14
First Page
296
Last Page
306
ISBN
9781450392037
Identifier
10.1145/3503161.3548277
Publisher
ACM
City or Country
Lisbon, Portugal
Citation
MA, Zhixin and NGO, Chong-wah.
Interactive video corpus moment retrieval using reinforcement learning. (2022). Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal, 2022 October 10-14. 296-306.
Available at: https://ink.library.smu.edu.sg/sis_research/7506
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://doi.org/10.1145/3503161.3548277
Included in
Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons