Publication Type
Journal Article
Version
publishedVersion
Publication Date
7-2026
Abstract
Websites are critical in today’s digital world, with over 1.11 billion currently active and approximately 252,000 new sites launched daily. Converting website layout design into functional UI code is a time-consuming yet indispensable step of website development. Manual methods of converting visual designs into functional code present significant challenges, especially for non-experts. To explore automatic design-to-code solutions, we first conduct a motivating study on GPT-4o and identify three types of issues in generating UI code: element omission, element distortion, and element misarrangement. We further reveal that a focus on smaller visual segments can help multimodal large language models (MLLMs) mitigate these failures in the generation process.
In this paper, we propose DCGen, a divide-and-conquer-based approach to automate the translation of webpage design to UI code. DCGen starts by dividing screenshots into manageable segments, generating code for each segment, and then reassembling them into complete UI code for the entire screenshot. We conduct extensive testing with a dataset comprised of real-world websites and various MLLMs and demonstrate that DCGen achieves up to a 15% improvement in visual similarity and 8% in code similarity for large input images. Human evaluations show that DCGen can help developers implement webpages significantly faster and more similar to the UI designs. To the best of our knowledge, DCGen is the first segment-aware MLLM-based approach for generating UI code directly from screenshots.
Keywords
Multi-modal Large Language Model, Code Generation, User Interface, Web Development
Discipline
Databases and Information Systems | Software Engineering
Research Areas
Data Science and Engineering
Publication
Proceedings of the ACM on Software Engineering
Volume
2
Issue
FSE
First Page
2099
Last Page
2122
ISSN
2994-970X
Identifier
10.1145/3729364
Publisher
ACM
Embargo Period
10-19-2026
Citation
WAN, Yuxuan; WANG, Chaozheng; DONG, Yi; WANG, Wenxuan; LI, Shuqing; HUO, Yintong; and LYU, Michael R..
Divide-and-conquer: Generating UI code from screenshots. (2026). Proceedings of the ACM on Software Engineering. 2, (FSE), 2099-2122.
Available at: https://ink.library.smu.edu.sg/sis_research/11353
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Additional URL
https://doi.org/10.1145/3729364