[Paper] Modular Layout Synthesis (MLS): Front-end Code via Structure Normalization and Constrained Generation

Source: arXiv - 2512.18996v1

Overview

The paper presents Modular Layout Synthesis (MLS), a new pipeline that turns UI screenshots into clean, framework‑agnostic front‑end code. By normalizing the visual hierarchy into a reusable component schema before generation, MLS produces modular, typed code that works out‑of‑the‑box with React, Vue, Angular, and similar ecosystems—addressing the “one‑size‑fits‑all” limitation of existing generative tools.

Key Contributions

• Hierarchical visual‑semantic encoder that converts a screen capture into a serialized tree representing the UI’s layout hierarchy.
• Structure normalization step that deduplicates repeated patterns and extracts reusable blocks, yielding a framework‑agnostic component schema.
• Constraint‑driven generation that steers a large language model (LLM) to emit production‑ready code with strict typing, explicit props, and proper imports for multiple front‑end frameworks.
• Comprehensive evaluation across React, Vue, and Angular showing significant gains in code modularity, reusability, and structural correctness compared with prior monolithic generators.

Methodology

1. Visual‑Semantic Encoding – A CNN‑backed encoder processes the UI screenshot and, together with a lightweight text encoder (for any embedded labels), produces a tree‑structured representation (nodes = UI elements, edges = containment).

2. Structure Normalization – The raw tree is passed through heuristic deduplication (identical sub‑trees) and pattern mining (common layout motifs). The result is a component schema that lists reusable blocks, their props, and their hierarchical relationships, independent of any specific framework.

3. Constrained Code Generation – The schema is fed to an LLM (e.g., GPT‑4) along with a set of generation constraints:

   • Target framework (React, Vue, Angular)
   • TypeScript/Flow typing rules
   • Prop‑validation rules
   • Naming conventions and import statements

   The LLM produces a set of component files that can be compiled directly.

Results & Findings

The authors report that MLS reduces the average lines of code per UI by ~35 % while preserving or improving functional fidelity, and that developers can edit the generated components with far fewer bugs.

Practical Implications

• Faster Prototyping – Designers can hand off screenshots and receive ready‑to‑use component libraries, cutting the UI hand‑off time from days to hours.
• Cross‑Framework Portability – Teams can target multiple front‑end stacks from a single source artifact, simplifying product line engineering.
• Maintainable Codebases – The modular output aligns with modern best practices (single‑responsibility components, typed props), making downstream refactoring and testing straightforward.
• Integration into CI/CD – Because MLS emits compilable code with strict typings, it can be slotted into automated pipelines that run linting, unit tests, and visual regression checks immediately after generation.

Limitations & Future Work

• Heuristic Deduplication may miss semantic equivalence in highly dynamic UIs, leading to over‑fragmented components.
• The current schema extraction relies on static visual cues; interactive states (hover, focus) are not captured.
• Evaluation is limited to three mainstream frameworks; extending to newer stacks (e.g., Svelte, Solid) and to mobile‑centric frameworks (React Native) remains open.
• Future research could explore learning‑based normalization (instead of heuristics) and incorporate behavioral specifications (event handling, animations) into the generation loop.

Authors

• Chong Liu
• Ming Zhang
• Fei Li
• Hao Zhou
• Xiaoshuang Chen
• Ye Yuan

Paper Information

• arXiv ID: 2512.18996v1
• Categories: cs.IR, cs.SE
• Published: December 22, 2025
• PDF: Download PDF