[Paper] Understanding and Mitigating Dataset Corruption in LLM Steering

Source: arXiv - 2603.03206v1

Overview

This paper investigates how contrastive steering—a lightweight technique for nudging large language models (LLMs) toward or away from specific traits at inference time—holds up when the example data used to learn the steering direction is corrupted. The authors show that while modest noise is tolerable, targeted poisoning can cause harmful side‑effects, and they propose a simple robust‑statistics fix that dramatically improves safety.

Key Contributions

• Empirical robustness study of contrastive steering under various corruption scenarios (random noise, label flips, and adversarial poisoning).
• Geometric analysis of how corrupted examples distort the learned 1‑D steering subspace.
• Identification of a failure mode: when a non‑trivial fraction of the steering dataset is maliciously altered, the model can exhibit unintended behaviors.
• Robust mean estimator integration: swapping the standard high‑dimensional mean calculation with a recent robust estimator mitigates most malicious effects with negligible overhead.
• Practical safeguards and guidelines for safely deploying contrastive steering in production pipelines.

Methodology

1. Dataset Construction – The authors build steering datasets consisting of prompt‑response pairs labeled “with trait” vs. “without trait” (e.g., polite vs. blunt).
2. Corruption Types – They inject three kinds of noise:
   • Random: random swaps of labels or responses.
   • Systematic: consistent bias (e.g., all “with‑trait” examples are replaced by neutral text).
   • Adversarial: crafted examples designed to push the steering direction toward a harmful subspace.
3. Steering Direction Learning – Contrastive steering computes the mean activation vector for each class in a chosen intermediate layer and takes the difference as the steering direction (a 1‑D subspace).
4. Robust Mean Replacement – The standard mean is replaced by a robust high‑dimensional mean estimator (e.g., iterative filtering based on median‑of‑means) that tolerates outliers.
5. Evaluation – The authors measure:
   • Trait alignment (how well the model follows the intended direction).
   • Side‑effect leakage (unintended changes in unrelated attributes).
   • Sensitivity curves as a function of corruption fraction.

Results & Findings

• Baseline robustness: Up to ~15 % random corruption, the steering direction remains stable and trait alignment degrades only marginally.
• Adversarial vulnerability: With ~30 % targeted poisoning, the model starts to exhibit the malicious trait (e.g., generating disallowed content) while still appearing to follow the original steering cue.
• Geometric insight: Corrupted points shift the class means, rotating the steering subspace away from the true direction; the effect grows linearly with the fraction of poisoned data.
• Robust mean impact: Replacing the mean with the robust estimator cuts the malicious drift by >80 % even when 40 % of the dataset is poisoned, with less than a 2 % drop in intended trait performance.
• Computation cost: The robust estimator adds ~10 % runtime overhead, which is negligible compared to the overall inference cost of large models.

Practical Implications

• Safer model customization – Teams that use contrastive steering for safety filters, tone adjustments, or policy compliance can now guard against data‑poisoning attacks with a minimal code change.
• Low‑cost deployment – Because the robust estimator works on the same activation vectors used for steering, no extra model training or fine‑tuning is required.
• Auditability – The geometric analysis provides a diagnostic tool: monitoring the norm and direction of class means can flag when a steering dataset may have been tampered with.
• Broader applicability – Any workflow that relies on a small set of examples to compute a direction in activation space (e.g., prompt‑based alignment, LoRA‑style adapters) can benefit from the same robust‑mean safeguard.

Limitations & Future Work

• The study focuses on a single intermediate layer and a specific class of LLMs; robustness may differ for deeper or multi‑layer steering schemes.
• The robust estimator assumes a bounded fraction of outliers; extremely high‑scale poisoning (>50 %) still overwhelms the method.
• Real‑world adversaries might adapt to the robust estimator, prompting a need for adaptive defenses.
• Future research could explore online detection of corrupted examples, combine robust statistics with certified robustness guarantees, and evaluate the approach on multimodal models.

Authors

• Cullen Anderson
• Narmeen Oozeer
• Foad Namjoo
• Remy Ogasawara
• Amirali Abdullah
• Jeff M. Phillips

Paper Information

• arXiv ID: 2603.03206v1
• Categories: cs.LG, cs.AI, cs.CL
• Published: March 3, 2026
• PDF: Download PDF