[Paper] Unambiguous Representations in Neural Networks: An Information-Theoretic Approach to Intentionality

Source: arXiv - 2512.11000v1

Overview

Francesco Lässig’s paper tackles a subtle but profound question: can artificial neural networks form unambiguous internal representations—states that can only be interpreted in one way, much like our conscious experience of a red square can’t simultaneously be read as a green square? By framing the problem with information theory, the work shows that the degree of ambiguity in a network’s “thoughts” can be measured, and that certain training regimes (e.g., dropout) dramatically reduce that ambiguity even when overall task performance stays the same.

Key Contributions

• Formal definition of representational ambiguity using conditional entropy (H(I|R)), where (I) = possible interpretations and (R) = a neural representation.
• Quantitative metric for ambiguity that can be computed from a trained model’s weights and activations.
• Empirical demonstration that dropout‑trained networks encode class identity with zero ambiguity (100 % decoding accuracy), while standard back‑propagation networks retain substantial ambiguity (≈38 % accuracy) despite identical classification scores on MNIST.
• Evidence that relational structure (the pattern of connections) carries semantic information independent of the learned decoder, enabling direct geometric matching to recover class identity.
• Showcase of spatial decoding: the physical layout of input neurons (their 2‑D positions) can be inferred from connectivity matrices with an (R^2) up to 0.844, indicating that low‑level geometry is also preserved in the network’s internal wiring.

Methodology

1. Network training – Two families of feed‑forward networks were trained on the MNIST digit classification task:
   • (a) standard stochastic gradient descent (SGD) with back‑propagation,
   • (b) the same architecture but with dropout applied to hidden units.
     Both families reached comparable test accuracies (~98 %).
2. Defining representations – For each input image, the activation vector of a chosen hidden layer is treated as the representation (R). The interpretation (I) is the digit class the network ultimately outputs.
3. Measuring ambiguity – Conditional entropy (H(I|R)) is estimated by building a decoder that maps (R) back to (I). Two decoders are used:
   • Learned decoder – a shallow classifier trained on a held‑out set of representations.
   • Geometric matcher – a nearest‑neighbor search in representation space, bypassing any learned parameters.
     Perfect decoding (zero conditional entropy) means the representation is unambiguous.
4. Connectivity analysis – The weight matrix between input and first hidden layer is examined as a graph. By treating each weight as an edge, the authors apply linear regression to predict the 2‑D coordinates of each input pixel from its outgoing weight pattern, yielding the reported (R^2).

Results & Findings

• Dropout eliminates ambiguity – For dropout‑trained nets, both the learned decoder and the geometric matcher recover the correct digit class 100 % of the time, implying (H(I|R)=0).
• Standard training leaves ambiguity – With vanilla back‑prop, the same decoders succeed only ~38 % of the time, despite the network still classifying digits correctly. This shows that high behavioral accuracy does not guarantee low‑ambiguity internal states.
• Relational structure matters – The success of the geometric matcher demonstrates that the pattern of connections alone (without any learned read‑out) can uniquely identify the represented class.
• Spatial information is retained – The regression from weight patterns to pixel coordinates reaches (R^2=0.844), indicating that the network’s wiring preserves a surprisingly faithful map of the input geometry.

Practical Implications

• Debugging & interpretability – Ambiguity metrics give developers a new lens to inspect whether a model’s hidden layers are “clean” or entangled. Low‑ambiguity representations could make feature visualizations and attribution methods more reliable.
• Robustness & safety – Models that encode information unambiguously are less likely to produce unexpected cross‑talk between classes, potentially reducing adversarial susceptibility and improving out‑of‑distribution detection.
• Model compression & pruning – If class identity is already encoded in the weight topology, aggressive pruning might preserve functionality while shedding redundant parameters, leading to leaner edge deployments.
• Neuro‑inspired architectures – The findings support incorporating stochastic regularizers (dropout, stochastic depth) not just for generalization but also for shaping clean internal representations, a design principle that could be baked into future AI frameworks.
• Meta‑learning of decoders – Since a simple geometric matcher can recover semantics, developers could build lightweight, task‑agnostic read‑outs for multi‑task systems, swapping decoders on the fly without retraining the whole network.

Limitations & Future Work

• Scope limited to simple feed‑forward nets and MNIST – It remains unclear how ambiguity behaves in convolutional, transformer, or recurrent architectures, especially on more complex datasets.
• Conditional entropy estimation relies on decoders – The metric is only as good as the decoder’s capacity; a more principled, decoder‑free estimator would strengthen the claim.
• Interpretation of “unambiguous” vs. “conscious” – While the paper draws parallels to philosophical accounts of consciousness, the operational link is speculative and would benefit from tighter neuroscientific validation.
• Future directions suggested by the author include extending the framework to multimodal models, exploring the trade‑off between ambiguity and robustness, and investigating whether training objectives that explicitly minimize (H(I|R)) improve downstream transfer learning.

Authors

• Francesco Lässig

Paper Information

• arXiv ID: 2512.11000v1
• Categories: q-bio.NC, cs.AI, cs.NE
• Published: December 10, 2025
• PDF: Download PDF