Publication Type
Journal Article
Version
publishedVersion
Publication Date
9-2025
Abstract
Despite recent advances in code generation made possible by large language models (LLMs), programming is still an essential skill that computing students need to master now and in the foreseeable future. In learning programming, frequent practices with exercises set at an appropriate difficulty and knowledge level is of crucial importance for students. However, it’s not a trivial task for instructors to create many good quality exercises customized for each student. Programming problems found on Internet sources such as LeetCode are mostly too challenging for novice programmers with no prior coding knowledge. Recent work in AI-enabled education has been leveraging LLMs for adaptive feedback generation on code submitted by students. Not much work has been done in generating customized exercises for students to have more practice. In this work, we propose ExGen, an automatic exercise generation system which uses LLMs such as OpenAI’s GPT models to generate on-demand, customized, and ready-to-use programming exercises for individual students. ExGen is designed as a plugin to Visual Studio Code. It incorporates a set of prompting strategies for candidate exercise generation, and a novel chain of automatic filtering mechanisms to select ready-to-use exercises. ExGen is convenient to use as compared to chatbots such as ChatGPT. We have conducted an extensive performance evaluation using more than 1400 generated Python exercises. We considered several prompting strategies with various keyword and seed exercise types, filtering techniques, difficulty levels, and LLMs with different generative performance and cost. The results demonstrated the effectiveness of ExGen’s design and implementation.
Keywords
Natural language generation, OpenAI Codex, GPT-3, CS1, Programming exercises, Code explanations, Robosourcing, Exercise generation, Resource generation, Automated feedback, Large language models
Discipline
Programming Languages and Compilers | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Areas of Excellence
Digital transformation
Publication
Research and Practice in Technology Enhanced Learning
Volume
1
First Page
27
Last Page
43
ISSN
1793-7078
Identifier
10.58459/rptel.2026.21025
Publisher
SpringerOpen
Citation
TA, Nguyen Binh Duong; NGUYEN, Hua Gia Phuc; and GOTTIPATI Swapna.
Automatic generation of introductory programming exercises with large language models. (2025). Research and Practice in Technology Enhanced Learning. 1, 27-43.
Available at: https://ink.library.smu.edu.sg/sis_research/10392
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.58459/rptel.2026.21025