Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
9-2025
Abstract
Automatically generated unit tests-from searchbased tools like EvoSuite or LLMs-vary significantly in structure and readability. Yet most evaluations rely on metrics like Cyclomatic Complexity and Cognitive Complexity, designed for functional code rather than test code. Recent studies have shown that SonarSource's Cognitive Complexity metric assigns nearzero scores to LLM-generated tests, yet its behavior on EvoSuitegenerated tests and its applicability to test-specific code structures remain unexplored. We introduce CCTR, a Test-Aware Cognitive Complexity metric tailored for unit tests. CCTR integrates structural and semantic features like assertion density, annotation roles, and test composition patterns-dimensions ignored by traditional complexity models but critical for understanding test code. We evaluate 15,750 test suites generated by EvoSuite, GPT4o, and Mistral Large-1024 across 350 classes from Defects4J and SF110. Results show CCTR effectively discriminates between structured and fragmented test suites, producing interpretable scores that better reflect developer-perceived effort. By bridging structural analysis and test readability, CCTR provides a foundation for more reliable evaluation and improvement of generated tests. We publicly release all data, prompts, and evaluation scripts to support replication.
Keywords
Automatic Test Generation, Unit test, Cognitive complexity, Large Language Models, Metrics, Test Code Understandability, Software Quality, Empirical Software Engineering
Discipline
Artificial Intelligence and Robotics
Research Areas
Intelligent Systems and Optimization
Areas of Excellence
Digital transformation
Publication
Proceedings of the 41st IEEE International Conference on Software Maintenance and Evolution (ICSME 2025), Auckland, New Zealand, September 7-12
First Page
1
Last Page
6
Identifier
10.1109/ICSME64153.2025.00082
Publisher
IEEE
City or Country
Pistacataway
Citation
OUÉDRAOGO, Wendkûuni C.; LI, Yinghua; DANG, Xueqi; ZHOU, Xin; KOYUNCU, Anil; KLEIN, Jacques; LO, David; and BISSYANDÉ, Tegawendé F..
Rethinking cognitive complexity for unit tests: toward a readability-aware metric grounded in developer perception. (2025). Proceedings of the 41st IEEE International Conference on Software Maintenance and Evolution (ICSME 2025), Auckland, New Zealand, September 7-12. 1-6.
Available at: https://ink.library.smu.edu.sg/sis_research/10842
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/ICSME64153.2025.00082