[Paper] AutoFigure: Generating and Refining Publication-Ready Scientific Illustrations

Source: arXiv - 2602.03828v1

Overview

Creating clear, publication‑ready figures is a hidden cost of every research project. The new AutoFigure system tackles this bottleneck by automatically turning long‑form scientific text (papers, surveys, textbooks, blogs) into polished illustrations. The authors also release FigureBench, the first large‑scale benchmark of 3.3 k text‑figure pairs, which makes it possible to evaluate and improve text‑to‑image models for scientific graphics.

Key Contributions

• FigureBench dataset – 3,300 high‑quality text‑figure pairs spanning multiple domains and figure types (diagrams, plots, schematics).
• AutoFigure framework – an “agentic” pipeline that (1) parses the input text, (2) reasons about the required visual components, (3) recombines them into a coherent layout, and (4) validates the result before rendering.
• State‑of‑the‑art performance – extensive experiments show AutoFigure outperforms existing text‑to‑image baselines on both objective metrics and human expert ratings.
• Open‑source release – code, data, and a Hugging Face demo are publicly available, enabling immediate experimentation and integration.

Methodology

1. Text Understanding – The system first runs a large language model (LLM) over the full scientific passage to extract visual concepts (e.g., “neural network architecture”, “phase diagram”) and structural constraints (e.g., “show three layers”, “include axis labels”).
2. Reasoning & Planning – An internal “thinking” module uses chain‑of‑thought prompting to decide how many sub‑figures are needed, their spatial arrangement, and which visual primitives (arrows, legends, color maps) are appropriate.
3. Component Generation – Each sub‑figure is generated by a specialized diffusion model conditioned on the extracted concept and the layout plan.
4. Validation & Refinement – A second LLM checks the rendered output against the original specification (e.g., “does the axis label match the described units?”). If mismatches are found, the pipeline iterates, tweaking prompts or layout until the figure passes the validation checklist.
5. Final Assembly – Validated sub‑figures are composited into a single, publication‑ready illustration with consistent styling and caption generation.

The whole pipeline runs end‑to‑end with minimal human intervention, yet retains a “human‑in‑the‑loop” fallback where developers can supply custom style guides or override decisions.

Results & Findings

• Quantitative gains: AutoFigure achieves a 23 % higher FID (Fréchet Inception Distance) and a 15 % boost in CLIP‑based similarity scores compared to the strongest baseline (a vanilla text‑to‑image diffusion model).
• Human evaluation: In a blind study with 30 domain experts, 78 % of AutoFigure’s outputs were rated “ready for submission” versus 42 % for the best baseline.
• Aesthetic consistency: The validation step reduces common errors (missing labels, mis‑aligned axes) by > 90 %, leading to cleaner, more trustworthy figures.
• Speed: Generating a multi‑panel figure (average 3 panels) takes ~45 seconds on a single A100 GPU, comparable to manual sketching time for a junior researcher.

Practical Implications

• Accelerated manuscript prep – Researchers can generate draft figures directly from their LaTeX or Markdown drafts, freeing up time for analysis and writing.
• Consistent corporate documentation – Tech companies producing internal white‑papers or API docs can enforce a unified visual style automatically.
• Educational content creation – Platforms that generate textbook or tutorial material can auto‑illustrate concepts at scale, reducing reliance on graphic designers.
• Rapid prototyping for ML pipelines – Data scientists can request visualizations of model architectures or data flows on the fly, integrating AutoFigure via its Python API or REST endpoint.

Limitations & Future Work

• Domain coverage – FigureBench, while diverse, still under‑represents highly specialized fields (e.g., quantum physics diagrams) where bespoke symbols are needed.
• Fine‑grained control – Current prompting allows high‑level layout decisions but lacks precise control over stroke width, font families, or exact color palettes without manual tweaks.
• Scalability of validation – The iterative validation loop can increase latency for very complex figures; future work will explore more efficient constraint solvers.
• User studies – Long‑term adoption impact (e.g., how researchers edit auto‑generated figures) remains to be measured.

The authors plan to expand FigureBench, integrate vector‑graphics back‑ends (SVG), and explore multimodal feedback (e.g., voice or sketch) to make AutoFigure an even more flexible assistant for scientific communication.

Authors

• Minjun Zhu
• Zhen Lin
• Yixuan Weng
• Panzhong Lu
• Qiujie Xie
• Yifan Wei
• Sifan Liu
• Qiyao Sun
• Yue Zhang

Paper Information

• arXiv ID: 2602.03828v1
• Categories: cs.AI, cs.CL, cs.CV, cs.DL
• Published: February 3, 2026
• PDF: Download PDF