[Paper] SGCR: A Specification-Grounded Framework for Trustworthy LLM Code Review

Source: arXiv - 2512.17540v1

Overview

The paper introduces Specification‑Grounded Code Review (SGCR), a framework that steers large language models (LLMs) with human‑written specifications so they can give reliable, context‑aware feedback during code reviews. By coupling deterministic rule checking with a heuristic “creative” path, SGCR bridges the gap between the raw generative power of LLMs and the strict correctness expectations of production software teams.

Key Contributions

• Specification‑grounded prompting – a systematic way to inject formal or informal specifications (e.g., coding standards, API contracts) into the LLM’s context, turning vague suggestions into rule‑compliant advice.
• Dual‑pathway architecture
  • Explicit path: deterministic enforcement of extracted rules, guaranteeing that any feedback that cites a rule is provably correct.
  • Implicit path: a heuristic, generative stream that discovers issues not covered by the explicit rules (e.g., architectural smells, performance anti‑patterns).
• Live industrial deployment at HiThink Research, demonstrating end‑to‑end integration with existing CI/CD pipelines and developer tooling.
• Empirical validation – a 42 % adoption rate of SGCR’s suggestions (90.9 % relative lift over a vanilla LLM baseline).
• Open‑source reference implementation (code and specification extraction scripts) to accelerate reproducibility and community extensions.

Methodology

1. Specification Extraction – Developers author specifications in a lightweight DSL (e.g., “no global mutable state”, “all public functions must have unit tests”). A parser turns these into a rule set (pre‑conditions, post‑conditions, style constraints).
2. Prompt Construction – The rule set is injected into the LLM prompt as a “system message” that the model must treat as immutable knowledge.
3. Dual‑Pathway Inference
   • Explicit path: The LLM is asked to verify each rule against the submitted code snippet. The answer is forced into a yes/no plus a concise justification, guaranteeing deterministic compliance.
   • Implicit path: The same code is fed to the LLM with a more open‑ended prompt (“What potential issues do you see?”). The model can surface problems outside the rule set (e.g., hidden dead code, inefficient loops).
4. Result Fusion – Explicit findings are always presented first (high trust). Implicit findings are filtered through a lightweight static‑analysis sanity check before being shown to the developer.
5. Integration & Feedback Loop – SGCR hooks into the pull‑request workflow; developers can accept, reject, or comment on each suggestion, feeding the outcome back into a reinforcement‑learning‑style fine‑tuning loop for future improvements.

Results & Findings

• The explicit path eliminated all violations of the supplied specifications, confirming deterministic compliance.
• The implicit path uncovered 18 % of issues that were not captured by the rule set, showing the value of a heuristic “creative” stream.
• Developer surveys reported higher trust in SGCR’s feedback (4.3/5) versus the baseline (3.1/5).

Practical Implications

• Higher developer productivity – By surfacing rule‑compliant fixes instantly, teams spend less time hunting for style or contract violations.
• Reduced review bottlenecks – The 90 % relative lift in adoption means fewer back‑and‑forth comment cycles, accelerating merge times.
• Compliance‑as‑code – Organizations can codify security or regulatory policies as specifications, guaranteeing that every PR is automatically vetted against them.
• Plug‑and‑play integration – SGCR’s architecture works with any LLM that supports system‑message prompting (e.g., OpenAI GPT‑4, Anthropic Claude), making it adaptable to existing CI tools (GitHub Actions, GitLab CI).
• Scalable code‑review bots – The deterministic explicit path can be run on cheap inference hardware, while the implicit path can be throttled or run off‑peak, balancing cost and coverage.

Limitations & Future Work

• Specification quality dependency – SGCR’s deterministic guarantees are only as good as the authored specs; poorly written or incomplete specs can lead to missed defects.
• Implicit path still heuristic – Although filtered, the generative suggestions can produce false positives, especially on large, complex codebases.
• Model bias – The framework inherits any biases present in the underlying LLM; future work will explore bias‑mitigation techniques and domain‑specific fine‑tuning.
• Broader language support – Current experiments focus on Python and JavaScript; extending to statically typed languages (Java, Go) and low‑level code (C/C++) is planned.
• Continuous learning loop – The authors intend to close the loop by automatically updating the rule set from accepted implicit suggestions, moving toward a self‑evolving specification system.

Authors

• Kai Wang
• Bingcheng Mao
• Shuai Jia
• Yujie Ding
• Dongming Han
• Tianyi Ma
• Bin Cao

Paper Information

• arXiv ID: 2512.17540v1
• Categories: cs.SE
• Published: December 19, 2025
• PDF: Download PDF