Publication Type

Journal Article

Version

acceptedVersion

Publication Date

4-2022

Abstract

Technical Q&A sites (e.g., Stack Overflow(SO)) are important resources for developers to search for knowledge about technical problems. Search engines provided in Q&A sites and information retrieval approaches have limited capabilities to retrieve relevant questions when queries are imprecisely specified, such as missing important technical details (e.g., the user's preferred programming languages). Although many automatic query expansion approaches have been proposed to improve the quality of queries by expanding queries with relevant terms, the information missed is not identified. Moreover, without user involvement, the existing query expansion approaches may introduce unexpected terms and lead to undesired results. In this paper, we propose an interactive query refinement approach for question retrieval, named Chatbot4QR, which assists users in recognizing and clarifying technical details missed in queries and thus retrieve more relevant questions for users. Chatbot4QR automatically detects missing technical details in a query and generates several clarification questions (CQs) to interact with the user to capture their overlooked technical details. To ensure the accuracy of CQs, we design a heuristic-based approach for CQ generation after building two kinds of technical knowledge bases: a manually categorized result of 1,841 technical tags in SO and the multiple version-frequency information of the tags. We collect 1.88 million SO questions as the repository for question retrieval. To evaluate Chatbot4QR, we conduct six user studies with 25 participants on 50 experimental queries. The results show that: (1) On average 60.8% of the CQs generated for a query are useful for helping the participants recognize missing technical details; (2) Chatbot4QR can rapidly respond to the participants after receiving a query within ~1.3 seconds; (3) The refined queries contribute to retrieving more relevant SO questions than nine baseline approaches. For more than 70% of the participants who have preferred techniques on the query tasks, Chatbot4QR significantly outperforms the state-of-the-art word embedding-based retrieval approach with an improvement of at least 54.6% in terms of Pre@k and NDCG@k; and (4)For 48%-88% of the assigned query tasks, the participants obtain more desired results after interacting with Chatbot4QR than directly searching from Web search engines (e.g., the SO search engine and Google) using the original queries.

Keywords

Stack Overflow, Chatbot, Interactive Query Refinement, Question Retrieval

Discipline

Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

IEEE Transactions on Software Engineering

Volume

48

Issue

4

First Page

1185

Last Page

1211

ISSN

0098-5589

Identifier

10.1109/TSE.2020.3016006

Publisher

IEEE

Embargo Period

5-11-2021

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1109/TSE.2020.3016006

Download

Share

COinS