[Paper] Training Introspective Behavior: Fine-Tuning Induces Reliable Internal State Detection in a 7B Model

Source: arXiv - 2511.21399v1

Overview

This paper shows that a 7‑billion‑parameter language model can be taught to reliably notice and report a “thought” that is briefly injected into its hidden state. By fine‑tuning on a simple single‑token injection task, the authors raise detection accuracy from virtually zero to ≈ 85 % while eliminating false positives, demonstrating that introspective behavior can be induced rather than left to chance emergence.

Key Contributions

• Fine‑tuning for introspection: Demonstrates that a modest amount of supervised training can turn a near‑useless model into one that consistently detects transient internal activations.
• High‑precision detection: Achieves 85 % accuracy on held‑out concepts with 0 % false‑positive rate (α = 40), meeting three of the four criteria proposed by Lindsey (2025) for introspective awareness.
• Generalization across concepts: Shows a 7.5 percentage‑point performance gap between seen and unseen concept vectors, indicating the model learns a transferable detection skill rather than memorizing specific patterns.
• Baseline for AI transparency: Provides the first concrete evidence that training can close the gap between models that spontaneously exhibit introspection and those that do not, opening a path toward built‑in self‑monitoring in LLMs.

Methodology

1. Injection task definition – The authors inject a single token representing a semantic concept (e.g., “apple”) into the model’s hidden state at a specific position during generation. The injection is fleeting: it disappears after the next forward pass.
2. Dataset creation – They generate thousands of short prompts, each paired with a random injection vector drawn from a pre‑computed concept embedding pool. For each prompt, the ground‑truth label is the injected concept.
3. Fine‑tuning regime – Using standard supervised learning, the model is trained to output a special “report” token that spells out the injected concept before it continues normal text generation. The loss combines a cross‑entropy term for the report and a regular language‑modeling term to preserve downstream generation quality.
4. Evaluation protocol – After training, the model is tested on two sets: (a) seen concepts that appeared during fine‑tuning, and (b) unseen concepts held out from training. Accuracy is measured as the proportion of correct reports, and false positives are counted when the model reports a concept despite no injection.

Results & Findings

• Reliability: The model reliably reports the injected thought before it continues generating text, satisfying the “internality” criterion.
• Transferability: The modest drop (≈ 7.5 pp) on unseen concepts suggests the model has learned a general detection mechanism rather than overfitting to specific vectors.
• No degradation: Standard language‑modeling perplexity remains unchanged, indicating that introspection training does not harm the model’s primary generation abilities.

Practical Implications

• Built‑in self‑monitoring: Developers can add a lightweight introspection head to existing LLMs, enabling the system to flag when an internal signal (e.g., a policy‑related token, a user‑provided cue) has been activated.
• Safety & compliance: In regulated domains, a model could be required to “self‑audit” whether a prohibited concept was ever internally considered, providing an audit trail for downstream decisions.
• Debugging & interpretability tools: Engineers can inject diagnostic tokens during inference to probe hidden‑state dynamics, using the trained detector to confirm that the model has actually “noticed” the probe.
• Modular transparency: Because the detection skill transfers across concepts, a single fine‑tuned model could serve multiple monitoring tasks (e.g., detecting bias‑related activations, privacy‑sensitive content, or policy violations) without retraining from scratch.

Limitations & Future Work

• Scope of introspection: The study only addresses detection of single‑token injections; richer, multi‑step internal reasoning remains untested.
• Metacognitive depth: While the model can report an injected concept, it does not demonstrate a deeper understanding or belief about that concept, so true metacognition is still out of reach.
• Scalability: Experiments are limited to a 7B model; it is unclear how the approach scales to larger models or multimodal architectures.
• Robustness to adversarial noise: Future work should examine whether the detector can be fooled by subtle perturbations or whether it generalizes to real‑world “thoughts” that arise naturally during generation.

Bottom line: By fine‑tuning a modest‑size language model, the authors prove that introspective detection is a learnable capability, paving the way for more transparent and self‑aware AI systems.

Authors

• Joshua Fonseca Rivera

Paper Information

• arXiv ID: 2511.21399v1
• Categories: cs.CL, cs.AI
• Published: November 26, 2025
• PDF: Download PDF