[Paper] PPTArena: A Benchmark for Agentic PowerPoint Editing

Source: arXiv - 2512.03042v1

Overview

A new benchmark called PPTArena evaluates how well AI agents can edit PowerPoint decks directly, following natural‑language instructions. By focusing on real‑world slide modifications—text, charts, tables, animations, and master styles—PPTArena pushes beyond image‑to‑PDF or text‑to‑slide generation and measures both functional correctness and visual quality.

Key Contributions

• PPTArena benchmark: 100 diverse slide decks (2,125 slides) with >800 targeted edits covering a wide range of PowerPoint elements.
• Dual VLM‑as‑judge evaluation: Separate visual‑quality and instruction‑following scores using structural diffs and rendered slide images.
• PPTPilot agent: A structure‑aware editing system that (1) plans semantic edit sequences, (2) routes tasks to high‑level programmatic tools or low‑level XML operations, and (3) iteratively verifies results against task constraints.
• Comprehensive empirical study: PPTPilot beats leading proprietary agents and state‑of‑the‑art vision‑language models by >10 % on compound, layout‑sensitive, and cross‑slide edits.
• Insightful analysis of failure modes: Highlights persistent challenges for long‑horizon, document‑scale PPT editing.

Methodology

1. Dataset construction – Human annotators authored natural‑language edit instructions for real PowerPoint decks and produced ground‑truth “target” decks. Each edit targets a specific element (e.g., “increase font size of the title on slide 3” or “replace the bar chart on slide 7 with a stacked version”).
2. Evaluation pipeline – Two vision‑language models act as judges:
   • Instruction‑following score – compares the semantic intent of the edited deck to the target using structural diff (XML tree) analysis.
   • Visual‑quality score – renders before/after slides and measures pixel‑level similarity plus perceptual metrics.
3. PPTPilot architecture –
   • Planner parses the instruction, generates a sequence of high‑level edit actions (e.g., modify‑text, replace‑chart).
   • Router decides whether an action can be handled by a deterministic XML edit (precise control) or needs a higher‑level tool (e.g., chart regeneration via a VLM).
   • Executor applies the chosen operation, updates the PPTX file, and feeds the result back to the planner.
   • Verifier runs the dual‑judge pipeline after each step; if constraints are violated, PPTPilot revises the plan (plan‑edit‑check loop).

Results & Findings

• Compound edits (multiple changes on the same slide) see the biggest boost, confirming the benefit of the plan‑edit‑check loop.
• Cross‑slide consistency (e.g., unified color scheme) improves markedly when PPTPilot leverages master‑slide XML edits.
• Even the best agents still struggle with long‑horizon tasks that require >5 sequential edits across many slides, indicating room for more robust reasoning and memory mechanisms.

Practical Implications

• Enterprise automation – Companies can plug PPTPilot‑style agents into their workflow tools (e.g., Microsoft Teams bots) to auto‑update decks after meetings, saving hours of manual editing.
• Developer APIs – The benchmark and dual‑judge pipeline provide a ready‑to‑use evaluation harness for anyone building PowerPoint‑editing plugins or VLM‑backed assistants.
• Design consistency tools – By exposing master‑slide XML operations, developers can build “style‑enforcement” services that keep branding uniform across large slide collections.
• Rapid prototyping – Start‑ups can generate customized pitch decks on the fly: a natural‑language prompt (“Add a timeline chart for Q1‑Q4”) is reliably turned into a polished slide without hand‑crafting graphics.

Limitations & Future Work

• Scope of assets – PPTArena currently covers standard charts, tables, and animations but does not include embedded media (video/audio) or complex SmartArt objects.
• Judge reliability – While the dual VLM judges correlate well with human ratings, they can still misjudge subtle aesthetic nuances, suggesting a need for human‑in‑the‑loop validation for high‑stakes presentations.
• Scalability – The plan‑edit‑check loop incurs extra latency; optimizing the routing and verification steps is an open engineering challenge.
• Generalization – Extending the approach to other office formats (Word, Excel) and to multi‑modal inputs (voice + sketch) is a promising direction for future research.

Authors

• Michael Ofengenden
• Yunze Man
• Ziqi Pang
• Yu‑Xiong Wang

Paper Information

• arXiv ID: 2512.03042v1
• Categories: cs.CV, cs.AI
• Published: December 2, 2025
• PDF: Download PDF