Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
7-2021
Abstract
Conversational recommender systems (CRS) enable the traditional recommender systems to explicitly acquire user preferences towards items and attributes through interactive conversations. Reinforcement learning (RL) is widely adopted to learn conversational recommendation policies to decide what attributes to ask, which items to recommend, and when to ask or recommend, at each conversation turn. However, existing methods mainly target at solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components, which restrict the scalability and generality of CRS and fall short of preserving a stable training procedure. In the light of these challenges, we propose to formulate these three decision-making problems in CRS as a unified policy learning task. In order to systematically integrate conversation and recommendation components, we develop a dynamic weighted graph based RL method to learn a policy to select the action at each conversation turn, either asking an attribute or recommending items. Further, to deal with the sample efficiency issue, we propose two action selection strategies for reducing the candidate action space according to the preference and entropy information. Experimental results on two benchmark CRS datasets and a real-world E-Commerce application show that the proposed method not only significantly outperforms state-of-the-art methods but also enhances the scalability and stability of CRS.
Keywords
Conversational Recommendation, Reinforcement Learning, Graph Representation Learning
Discipline
Databases and Information Systems | Graphics and Human Computer Interfaces
Research Areas
Data Science and Engineering
Areas of Excellence
Digital transformation
Publication
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Conference, July 11-15
First Page
1431
Last Page
1441
ISBN
9781450380379
Identifier
10.1145/3404835.3462913
Publisher
ACM
City or Country
New York
Citation
DENG, Yang; LI, Yaliang; SUN, Fei; DING, Bolin; and LAM, Wai.
Unified conversational recommendation policy learning via graph-based reinforcement learning. (2021). SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Conference, July 11-15. 1431-1441.
Available at: https://ink.library.smu.edu.sg/sis_research/9114
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3404835.3462913
Included in
Databases and Information Systems Commons, Graphics and Human Computer Interfaces Commons