Publication Type
Journal Article
Version
publishedVersion
Publication Date
10-2025
Abstract
Automated Program Repair (APR) aims to automatically generate patches for rectifying software bugs. Recentstrides in Large Language Models (LLM), such as ChatGPT, have yielded encouraging outcomes in APR,especially within the conversation-driven APR framework. Nevertheless, the efficacy of conversation-drivenAPR is contingent on the quality of the feedback information. In this article, we propose ContrastRepair, anovel conversation-based APR approach that augments conversation-driven APR by providing LLMs withcontrastive test pairs. A test pair consists of a failing test and a passing test, which offer contrastive feedback tothe LLM. Our key insight is to minimize the difference between the generated passing test and the given failingtest, which can better isolate the root causes of bugs. By providing such informative feedback, ContrastRepairenables the LLM to produce effective bug fixes. The implementation of ContrastRepair is based on the state-ofthe-art LLM, ChatGPT, and it iteratively interacts with ChatGPT until plausible patches are generated. Weevaluate ContrastRepair on multiple benchmark datasets, including Defects4J, QuixBugs, and HumanEval-Java.The results demonstrate that ContrastRepair significantly outperforms existing methods, achieving a newstate-of-the-art in program repair. For instance, among Defects4J 1.2 and 2.0, ContrastRepair correctly repairs143 out of all 337 bug cases, while the best-performing baseline fixes 124 bugs.
Keywords
Large language model, Program repair, Defects, Feedback, Human engineering, Iterative methods, Software testing
Discipline
Artificial Intelligence and Robotics | Databases and Information Systems | Programming Languages and Compilers
Research Areas
Data Science and Engineering; Intelligent Systems and Optimization
Publication
ACM Transactions on Software Engineering and Methodology
Volume
34
Issue
8
First Page
1
Last Page
31
ISSN
1049-331X
Identifier
10.1145/3719345
Publisher
Association for Computing Machinery (ACM)
Citation
KONG, Jiaolong; XIE, Xiaofei; CHENG, Mingfei; Liu, Shangqing; Du, Xiaoning; and Guo, Qi.
ContrastRepair: Enhancing conversation-based automated program repair via contrastive test case pairs. (2025). ACM Transactions on Software Engineering and Methodology. 34, (8), 1-31.
Available at: https://ink.library.smu.edu.sg/sis_research/11011
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Included in
Artificial Intelligence and Robotics Commons, Databases and Information Systems Commons, Programming Languages and Compilers Commons