Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
4-2022
Abstract
User Satisfaction Estimation (USE) is an important yet challenging task in goal-oriented conversational systems. Whether the user is satisfied with the system largely depends on the fulfillment of the user’s needs, which can be implicitly reflected by users’ dialogue acts. However, existing studies often neglect the sequential transitions of dialogue act or rely heavily on annotated dialogue act labels when utilizing dialogue acts to facilitate USE. In this paper, we propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction, by jointly learning User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific, we first employ a Hierarchical Transformer to encode the whole dialogue context, with two task-adaptive pre-training strategies to be a second-phase in-domain pre-training for enhancing the dialogue modeling ability. In terms of the availability of dialogue act labels, we further develop two variants of USDA to capture the dialogue act information in either supervised or unsupervised manners. Finally, USDA leverages the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets across different applications show that the proposed method substantially and consistently outperforms existing methods on USE, and validate the important role of dialogue act sequences in USE.
Keywords
User Satisfaction Estimation, Goal-oriented Conversational System, Dialogue Act Recognition
Discipline
Databases and Information Systems
Research Areas
Data Science and Engineering
Areas of Excellence
Digital transformation
Publication
WWW '22: Proceedings of the ACM Web Conference 2022, Virtual Conference, April 25-29
First Page
2998
Last Page
3008
Identifier
10.1145/3485447.3512020
Publisher
ACM
City or Country
New York
Citation
DENG, Yang; ZHANG, Wenxuan; LAM, Wai; CHENG, Hong; and MENG, Helen.
User satisfaction estimation with sequential dialogue act modeling in goal-oriented conversational systems. (2022). WWW '22: Proceedings of the ACM Web Conference 2022, Virtual Conference, April 25-29. 2998-3008.
Available at: https://ink.library.smu.edu.sg/sis_research/9141
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3485447.3512020