[Paper] Towards a Neural Debugger for Python

Source: arXiv - 2603.09951v1

Overview

The paper “Towards a Neural Debugger for Python” explores how large language models (LLMs) can be turned into interactive debugging assistants. By fine‑tuning or pre‑training models on Python execution traces, the authors enable the model to mimic the behavior of a traditional debugger—supporting breakpoints, step‑into/over/out, and even reasoning backwards from a program state. This bridges the gap between static code‑generation models and the dynamic, step‑wise workflow developers use every day.

Key Contributions

• Neural Debugger Concept – Introduces the idea of treating an LLM as a debugger that can be queried with typical debugger commands.
• Bidirectional Execution Modeling – Demonstrates that the model can predict future program states (forward execution) and infer previous states or inputs (inverse execution) conditioned on debugger actions.
• Fine‑tuning & Scratch Pre‑training – Shows that both large pre‑trained LLMs (via fine‑tuning) and smaller models trained from scratch can learn the debugger behavior.
• Benchmark (CruxEval) – Provides a new evaluation suite for conditional execution tasks, measuring both output prediction and input reconstruction accuracy.
• Foundations for Agentic Coding – Positions neural debuggers as a world‑model component for autonomous coding agents that need to experiment with code in a simulated debugging environment.

Methodology

1. Data Collection – The authors generate massive Python execution traces by running a diverse corpus of scripts and recording the state after each line (variables, call stack, I/O).
2. Debugger Action Encoding – Each trace is annotated with a debugger command (e.g., step_over, breakpoint @ line 42). The command is tokenized and prepended to the model’s input, allowing the model to condition its prediction on the requested action.
3. Model Training
   • Fine‑tuning: Large LLMs (e.g., CodeGen‑2B/6B) are further trained on the annotated traces.
   • From‑scratch: Smaller transformer models (≈300 M parameters) are trained directly on the same data, testing whether the capability can be learned without a massive pre‑trained backbone.
4. Bidirectional Targets
   • Forward: Given the current state and a debugger command, predict the next program state and any output.
   • Inverse: Given a later state and a command, reconstruct the prior state or the missing input that led there.
5. Evaluation (CruxEval) – The benchmark measures exact match and token‑level accuracy for both forward and inverse tasks across a variety of Python constructs (loops, recursion, I/O, exceptions).

Results & Findings

• Robust Conditional Execution – Adding the debugger command token improves forward prediction by ~15 % over a vanilla neural interpreter.
• Inverse Reasoning – The models can recover missing inputs (e.g., user‑provided values) with high fidelity, a capability essential for “time‑travel” debugging.
• Generalization – Even the 300 M‑parameter model handles unseen libraries and custom functions, indicating that the debugger behavior is not tied to a specific codebase.

Practical Implications

• IDE Integration – A neural debugger could power “AI‑assisted breakpoints” that suggest likely variable values or highlight suspicious lines before a human even steps through the code.
• Automated Bug Localization – By stepping backwards from a crash state, the model can propose the minimal set of statements that likely introduced the bug, accelerating triage.
• Agentic Coding Assistants – Autonomous agents (e.g., GitHub Copilot‑style bots) can use the neural debugger as a sandbox to test hypotheses, iterate on patches, and verify fixes without spawning a real interpreter.
• Education & Onboarding – Novice programmers can ask the model “What would happen if I step into this function?” and receive a concise, line‑by‑line explanation, making learning interactive debugging more approachable.
• Performance‑Sensitive Scenarios – Because the model predicts execution outcomes without actually running the code, it can be used for quick “what‑if” analyses in CI pipelines where full execution would be too costly.

Limitations & Future Work

• Trace Fidelity – The model learns from deterministic traces; nondeterministic behavior (e.g., threading, random seeds) is not fully captured, limiting reliability in concurrent programs.
• Scalability to Large Codebases – Current experiments focus on relatively small scripts; scaling the approach to multi‑module projects will require smarter context‑window management.
• Real‑World Debugger Integration – The paper stops at a simulated debugger API; bridging to actual debuggers (e.g., pdb, VS Code) and handling side‑effects remains an open engineering challenge.
• Security & Trust – Since the model predicts execution rather than executing it, there is a risk of hallucinated states; future work should combine neural predictions with lightweight concrete execution for verification.

In short, this work lays the groundwork for turning LLMs into interactive, step‑wise debugging partners—an exciting step toward more autonomous, developer‑friendly AI tools.

Authors

• Maximilian Beck
• Jonas Gehring
• Jannik Kossen
• Gabriel Synnaeve

Paper Information

• arXiv ID: 2603.09951v1
• Categories: cs.LG, cs.AI, cs.SE
• Published: March 10, 2026
• PDF: Download PDF