Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

4-2026

Abstract

Software development remains predominantly text-centric, despite decades of evidence showing that developers think and communicate visually. While sketches and diagrams externalize developers’ mental models, they remain disconnected from source code and quickly become outdated. Recent advances in foundation models, capable of both code and visual reasoning, create an opportunity to unify these representations. In this vision paper, we introduce Visual Loop, a continuous visual development environment that keeps code and informal sketches in bidirectional synchronization. Our prototype connects a code editor with a tablet-based visualization workspace, allowing developers to explore, annotate, and modify systems through freehand sketches interpreted by multimodal LLMs. By grounding AI reasoning in static analysis and visual context, Visual Loop transforms sketching from passive documentation into an active interface for software evolution. We present illustrative scenarios, discuss cognitive and comprehension benefits, and outline directions for user studies and real-world integration.

Keywords

Code Visualization, Developer Tools, LLMs, Sketch Recognition

Discipline

Graphics and Human Computer Interfaces | Software Engineering

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

FORGE '26: Proceedings of the 2026 IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering, Rio de Janeiro, Brazil, April 12-13

First Page

195

Last Page

200

ISBN

9798400724770

Identifier

10.1145/3793655.3793729

Publisher

Association for Computing Machinery, Inc

City or Country

Rio de Janeiro

Additional URL

https://doi.org/10.1145/3793655.3793729

Download

Share

COinS