Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
7-2023
Abstract
In this study, we assess the efficacy of employing the ChatGPT language model to generate solutions for coding exercises within an undergraduate Java programming course. ChatGPT, a large-scale, deep learning-driven natural language processing model, is capable of producing programming code based on textual input. Our evaluation involves analyzing ChatGPT-generated solutions for 80 diverse programming exercises and comparing them to the correct solutions. Our findings indicate that ChatGPT accurately generates Java programming solutions, which are characterized by high readability and well-structured organization. Additionally, the model can produce alternative, memory-efficient solutions. However, as a natural language processing model, ChatGPT struggles with coding exercises containing non-textual descriptions or class files, leading to invalid solutions. In conclusion, ChatGPT holds potential as a valuable tool for students seeking to overcome programming challenges and explore alternative approaches to solving coding problems. By understanding its limitations, educators can design coding exercises that minimize the potential for misuse as a cheating aid while maintaining their validity as assessment tools.
Keywords
programming, Java, object-oriented, computer science education
Discipline
Programming Languages and Compilers | Software Engineering
Research Areas
Information Systems and Management
Publication
ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education, Turku, Finland, July 10-12
First Page
54
Last Page
60
ISBN
9798400701382
Identifier
10.1145/3587102.3588794
Publisher
ACM
City or Country
New York
Citation
OUH, Eng Lieh; GAN, Benjamin; SHIM, Kyong Jin; and WLODKOWSKI, Swavek.
ChatGPT, can you generate solutions for my coding exercises? An evaluation on its effectiveness in an undergraduate Java programming course. (2023). ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education, Turku, Finland, July 10-12. 54-60.
Available at: https://ink.library.smu.edu.sg/sis_research/8068
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3587102.3588794