Publication Type
Journal Article
Version
publishedVersion
Publication Date
3-2022
Abstract
Conversational recommendation system (CRS) attracts increasing attention in various application domains such as retail and travel. It offers an effective way to capture users’ dynamic preferences with multi-turn conversations. However, most current studies center on the recommendation aspect while over-simplifying the conversation process. The negligence of complexity in data structure and conversation flow hinders their practicality and utility. In reality, there exist various relationships among slots and values, while users’ requirements may dynamically adjust or change. Moreover, the conversation often involves visual modality to facilitate the conversation. These actually call for a more advanced internal state representation of the dialogue and a proper reasoning scheme to guide the decision making process. In this paper, we explore multiple facets of multimodal conversational recommendation and try to address the above mentioned challenges. In particular, we represent the structured back-end database as a multimodal knowledge graph which captures the various relations and evidence in different modalities. The user preferences expressed via conversation utterances will then be gradually updated to the state graph with clear polarity. Based on these, we train an end-to-end State Graph-based Reasoning model (SGR) to perform reasoning over the whole state graph. The prediction of our proposed model benefits from the structure of the graph. It not only allows for zero-shot reasoning for items unseen in training conversations, but also provides a natural way to explain the policies. Extensive experiments show that our model achieves better performance compared with existing methods.
Keywords
Recommendation systems, conversation, knowledge graph
Discipline
Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces
Research Areas
Intelligent Systems and Optimization
Publication
IEEE Transactions on Multimedia
First Page
1
Last Page
12
ISSN
1520-9210
Identifier
10.1109/TMM.2022.3155900
Publisher
Institute of Electrical and Electronics Engineers
Citation
WU, Yuxia; LIAO, Lizi; ZHANG, Gangyi; LEI, Wenqiang; ZHAO, Guoshuai; QIAN, Xueming; and CHUA, Tat-Seng.
State graph reasoning for multimodal conversational recommendation. (2022). IEEE Transactions on Multimedia. 1-12.
Available at: https://ink.library.smu.edu.sg/sis_research/7581
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://doi.org/10.1109/TMM.2022.3155900
Included in
Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons