Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

6-2021

Abstract

Concept-free search, which embeds text and video signals in a joint space for retrieval, appears to be a new state-of-the-art. However, this new search paradigm suffers from two limitations. First, the search result is unpredictable and not interpretable. Second, the embedded features are in high-dimensional space hindering real-time indexing and search. In this paper, we present a new implementation of the Vireo video search system (Vireo-VSS), which employs a dual-task model to index each video segment with an embedding feature in a low dimension and a concept list for retrieval. The concept list serves as a reference to interpret its associated embedded feature. With these changes, a SQL-like querying interface is designed such that a user can specify the search content (subject, predicate, object) and constraint (logical condition) in a semi-structured way. The system will decompose the SQL-like query into multiple sub-queries depending on the constraint being specified. Each sub-query is translated into an embedding feature and a concept list for video retrieval. The search result is compiled by union or pruning of the search lists from multiple sub-queries. The SQL-like interface is also extended for temporal querying, by providing multiple SQL templates for users to specify the temporal evolution of a query.

Keywords

Concept-based search, Concept-free search, Interactive video search, SQL-like interpretable search, Video browser showdown

Discipline

Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

Multimedia Modeling, MMM 2021: 27th International Conference, Prague, June 22-24: Proceedings

First Page

391

Last Page

397

ISBN

9783030678340

Identifier

10.1007/978-3-030-67835-7_34

Publisher

Springer

City or Country

Cham

Copyright Owner and License

Publisher

Additional URL

https://doi.org/10.1007/978-3-030-67835-7_34

Download

Share

COinS