[Paper] AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning

Source: arXiv - 2602.23258v1

Overview

AgentDropoutV2 tackles a long‑standing pain point in multi‑agent systems (MAS): a single faulty agent can drag the whole pipeline down by propagating wrong information. Instead of re‑architecting the whole system or fine‑tuning every participant, the authors introduce a test‑time “rectify‑or‑reject” pruning layer that sits between agents and the downstream consumer, catching and fixing errors on the fly.

Key Contributions

• Test‑time pruning framework that works without any additional training of the underlying agents.
• Retrieval‑augmented rectifier that iteratively corrects suspect outputs using a curated pool of failure‑driven indicators.
• Dynamic firewall logic that decides, per output, whether to rectify, pass through, or prune (reject) the result, preserving overall system stability.
• Context‑aware indicator pool that captures distilled failure patterns, enabling the system to recognize a wide variety of error signatures.
• Empirical validation on large math benchmark suites, showing an average +6.3 % accuracy lift and strong generalization to unseen difficulty levels.
• Open‑source release of code and datasets, facilitating reproducibility and community extensions.

Methodology

1. Failure‑Driven Indicator Pool – The authors first analyze a corpus of agent failures on a validation set, extracting concise “indicator” snippets (e.g., typical wrong reasoning steps, mis‑parsed symbols). These indicators are stored in a lightweight retrieval index.

2. Rectify‑or‑Reject Layer – At inference time, each agent’s output passes through this layer:

   • Detection: The output is compared against the indicator pool using similarity search (e.g., dense embeddings). A high similarity score flags a potential error.
   • Rectification: If flagged, a small retrieval‑augmented language model (the “rectifier”) fetches the most relevant indicator and attempts to rewrite the output, guided by the original context. This step can be repeated iteratively until the similarity drops below a threshold.
   • Rejection: When the rectifier cannot bring the similarity down (i.e., the error appears irreparable), the output is pruned. A fallback policy (e.g., using a default answer or deferring to another agent) ensures the pipeline does not break.

3. Dynamic Modulation – The system adapts the aggressiveness of pruning based on task difficulty estimated from input features, preventing over‑pruning on easy cases while being stricter on hard ones.

All of this runs post‑hoc, meaning the original agents remain untouched—no fine‑tuning, no extra parameters in the agents themselves.

Results & Findings

• Math Benchmarks: Across several standard arithmetic and algebra datasets (e.g., GSM8K, MATH), AgentDropoutV2 lifted the baseline MAS accuracy from ~71 % to ~77 %, a 6.3 pp gain.
• Error Pattern Coverage: The indicator pool captured >90 % of the most frequent error categories (mis‑aligned symbols, off‑by‑one reasoning steps).
• Generalization: When evaluated on out‑of‑distribution problems with higher difficulty, the framework still delivered a +4 % boost, showing that the rectifier can extrapolate from the distilled failure patterns.
• Efficiency: The additional latency per query stayed under 150 ms on a single GPU, making it viable for real‑time services.
• Ablation: Removing the retrieval component dropped the gain to ~2 pp, confirming that context‑aware indicators are the key driver.

Practical Implications

• Plug‑and‑Play Safety Net: Developers can wrap existing MAS deployments (e.g., autonomous planning bots, collaborative code assistants) with AgentDropoutV2 to gain immediate robustness without retraining each agent.
• Cost‑Effective Scaling: Since the framework operates at test time, organizations avoid expensive fine‑tuning cycles when adding new agents or updating models.
• Error Auditing: The indicator pool doubles as a diagnostic log, helping engineers understand systematic failure modes across the fleet.
• Adaptive Service Levels: By tuning the difficulty‑aware thresholds, SaaS providers can offer “high‑confidence” vs. “experimental” response modes, automatically degrading gracefully when confidence is low.
• Cross‑Domain Portability: Although evaluated on math reasoning, the same rectify‑or‑reject pipeline can be applied to other domains—e.g., multi‑agent dialogue systems, distributed recommendation pipelines, or sensor‑fusion in robotics.

Limitations & Future Work

• Indicator Dependence: The quality of the failure pool hinges on the representativeness of the validation set; rare or novel error patterns may slip through.
• Rectifier Capacity: The retrieval‑augmented model is relatively small to keep latency low; extremely complex errors might require a larger model or multi‑step reasoning.
• Pruning Side‑Effects: Aggressive rejection can reduce overall throughput if many agents are pruned, which may be undesirable in latency‑critical settings.
• Domain Transfer: While promising, the authors note that moving from math to natural‑language tasks will need domain‑specific indicator engineering.
• Future Directions: The paper suggests exploring online learning of indicators (continuously updating the pool from live failures) and hierarchical pruning where multiple layers of agents can be coordinated for even tighter error containment.

Authors

• Yutong Wang
• Siyuan Xiong
• Xuebo Liu
• Wenkang Zhou
• Liang Ding
• Miao Zhang
• Min Zhang

Paper Information

• arXiv ID: 2602.23258v1
• Categories: cs.AI, cs.CL
• Published: February 26, 2026
• PDF: Download PDF