Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
11-2020
Abstract
The 2019 edition of Stack Overflow developer survey highlights that, for the first time, Python outperformed Java in terms of popularity. The gap between Python and Java further widened in the 2020 edition of the survey. Unfortunately, despite the rapid increase in Python's popularity, there are not many testing and debugging tools that are designed for Python. This is in stark contrast with the abundance of testing and debugging tools for Java. Thus, there is a need to push research on tools that can help Python developers.One factor that contributed to the rapid growth of Java testing and debugging tools is the availability of benchmarks. A popular benchmark is the Defects4J benchmark; its initial version contained 357 real bugs from 5 real-world Java programs. Each bug comes with a test suite that can expose the bug. Defects4J has been used by hundreds of testing and debugging studies and has helped to push the frontier of research in these directions.In this project, inspired by Defects4J, we create another benchmark database and tool that contain 493 real bugs from 17 real-world Python programs. We hope our benchmark can help catalyze future work on testing and debugging tools that work on Python programs.
Keywords
Bugs database, Python, Testing and debugging
Discipline
Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering: 9-13 November, Virtual
First Page
1556
Last Page
1560
ISBN
9781450370431
Identifier
10.1145/3368089.3417943
Publisher
ACM
City or Country
New York
Citation
WIDYASARI, Ratnadira; SIM, Sheng Qin; LOK, Camellia; QI, Haodi; PHAN, Jack; TAY, Qijin; TAN, Constance; WEE, Fiona; TAN, Jodie Ethelda; YIEH, Yuheng; GOH, Brian; THUNG, Ferdian; KANG, Hong Jin; HOANG, Thong; David LO; and OUH, Eng Lieh.
BugsInPy: A database of existing bugs in Python programs to enable controlled testing and debugging studies. (2020). ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering: 9-13 November, Virtual. 1556-1560.
Available at: https://ink.library.smu.edu.sg/sis_research/5630
Copyright Owner and License
Publisher
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3368089.3417943