[Paper] SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning

Source: arXiv - 2511.19422v1

Overview

The paper introduces SLMFix, a lightweight pipeline that uses a small language model (SLM) fine‑tuned with reinforcement learning (RL) to automatically repair syntax errors in code generated by large language models (LLMs). By focusing on low‑resource or domain‑specific languages, the authors show that you can dramatically boost the correctness of LLM‑generated programs without the massive compute budget normally required for full‑scale LLM fine‑tuning.

Key Contributions

• RL‑driven repair model: Applies reinforcement learning to a modest‑size model (≈ 1–2 B parameters) that learns to fix syntactic mistakes in LLM‑generated code.
• Dual‑reward signal: Combines a static syntax validator (pass/fail) with a static semantic similarity metric, encouraging both syntactic correctness and semantic fidelity to the original intent.
• Domain‑agnostic pipeline: Demonstrates the approach on several domain‑specific languages (DSLs) and low‑resource programming languages, achieving > 95 % pass rates on static validators.
• Cost‑effective alternative: Shows that SLMFix outperforms supervised fine‑tuning even for 7 B‑parameter LLMs, proving that a small model plus RL can replace expensive full‑model fine‑tuning.
• Open‑source potential: The methodology is compatible with any existing LLM code generator, making it easy to plug into current developer tooling.

Methodology

1. Generate candidate code with a pre‑trained LLM (e.g., GPT‑4, CodeLlama).
2. Pass the candidate to a small transformer‑based model (the SLM) that has been fine‑tuned to act as a “repair agent.”
3. Reinforcement learning loop:
   • State: The buggy program output by the LLM.
   • Action: Token‑level edits (insert, delete, replace) suggested by the SLM.
   • Reward:
     • Syntax validator → 1 if the edited program compiles/passes static checks, 0 otherwise.
     • Semantic similarity → a score (e.g., cosine similarity of AST embeddings) that penalizes over‑editing.
4. Policy update: The SLM’s parameters are updated via Proximal Policy Optimization (PPO) to maximize the combined reward.
5. Iterate until the repaired program consistently passes the validator across a validation set.

The pipeline is deliberately lightweight: the SLM can be trained on a single GPU in a few hours, and the RL reward is computed entirely offline (no runtime execution of the repaired code).

Results & Findings

• Generalizability: The same SLMFix pipeline, trained on one DSL, transferred to three unseen DSLs with less than 5 % performance loss.
• Efficiency: Training cost was ~0.3 × that of full supervised fine‑tuning for a 7 B model, and inference latency added < 30 ms per repair.
• Error types fixed: Missing brackets, mismatched indentation, undeclared identifiers, and simple type mismatches—all the most common syntax errors in the benchmark datasets.

Practical Implications

• Developer tooling: IDE plugins can embed SLMFix as a “quick‑fix” assistant that silently cleans up LLM‑generated snippets before insertion, reducing the manual debugging burden.
• CI/CD pipelines: Automated code review bots can run the SLMFix repair step on pull requests that contain AI‑generated code, guaranteeing syntactic compliance before compilation.
• Low‑resource language support: Companies building internal DSLs (e.g., hardware description languages, query languages) can now leverage cheap SLMs to improve code generation quality without needing massive data or compute.
• Cost‑effective AI adoption: Start‑ups and teams with limited GPU budgets can achieve near‑LLM‑level code quality by pairing a generic LLM with a small, RL‑trained repair model, sidestepping expensive fine‑tuning contracts.

Limitations & Future Work

• Static‑only validation: The reward relies on static checks; runtime bugs or logical errors remain undetected. Extending the reward to include unit‑test execution could close this gap.
• Semantic similarity metric: The current heuristic may penalize legitimate refactorings; more robust program‑level similarity measures (e.g., execution traces) are an open direction.
• Scalability to full‑scale languages: While DSLs and LRPLs are well‑served, applying SLMFix to massive ecosystems like Python or JavaScript may require richer context handling and larger SLMs.
• User‑controlled repair intensity: Future work could expose a “repair aggressiveness” knob, letting developers decide how much the model can deviate from the original LLM output.

TL;DR: SLMFix shows that a modest‑size model, trained with reinforcement learning, can act as an inexpensive “syntax‑doctor” for code generated by any LLM, delivering > 95 % syntactically correct output across multiple niche languages—making AI‑assisted coding more reliable and affordable for everyday developers.

Authors

• David Jiahao Fu
• Aryan Gupta
• Aaron Councilman
• David Grove
• Yu‑Xiong Wang
• Vikram Adve

Paper Information

• arXiv ID: 2511.19422v1
• Categories: cs.SE, cs.AI, cs.PL
• Published: November 24, 2025
• PDF: Download PDF