[Paper] Pseudo-Invertible Neural Networks

Source: arXiv - 2602.06042v1

Overview

The paper introduces Surjective Pseudo‑invertible Neural Networks (SPNN), a new class of neural architectures that extend the classic Moore‑Penrose pseudo‑inverse from linear algebra to the nonlinear world of deep learning. By providing a tractable, mathematically‑grounded “non‑linear pseudo‑inverse,” the authors enable zero‑shot inversion of complex, possibly semantic, degradations without having to retrain a generative model.

Key Contributions

• Formal definition of a non‑linear pseudo‑inverse that preserves essential geometric properties (e.g., null‑space projection).
• SPNN architecture: a design recipe guaranteeing that the pseudo‑inverse can be computed efficiently and exactly.
• Non‑Linear Back‑Projection (NLBP): a generalization of the classic linear back‑projection formula (x’ = x + A^{\dagger}(y-Ax)) to arbitrary nonlinear mappings (f(x)=y).
• Zero‑shot solution of nonlinear inverse problems: extending diffusion‑based back‑projection (previously limited to linear degradations) to tasks such as optical distortion correction, de‑blurring with learned kernels, and even semantic “undoing” of classification or style‑transfer operations.
• Demonstrations of precise semantic control over diffusion‑based generative models without any fine‑tuning.

Methodology

1. Surjectivity Requirement – SPNNs are built so that every possible output (y) has at least one pre‑image (x). This guarantees the existence of a pseudo‑inverse mapping.

2. Layer‑wise Construction – Each building block (e.g., affine transform, invertible activation, residual block) is paired with a closed‑form inverse or a tractable pseudo‑inverse that respects the Moore‑Penrose conditions.

3. Deriving the Non‑Linear PInv – By stacking these blocks, the authors derive an analytic expression for the overall pseudo‑inverse (f^{\dagger}(y)) that can be evaluated with a single forward pass through a mirror network.

4. Non‑Linear Back‑Projection (NLBP) – Given an initial estimate (\hat{x}) (e.g., from a diffusion prior), NLBP refines it via

   [ \hat{x}_{\text{new}} = \hat{x} + f^{\dagger}\bigl(y - f(\hat{x})\bigr), ]

   ensuring the updated (\hat{x}{\text{new}}) exactly satisfies the constraint (f(\hat{x}{\text{new}})=y).

5. Zero‑Shot Inversion Pipeline – The authors plug SPNN‑based NLBP into a pre‑trained diffusion model, using the diffusion prior only to propose a plausible (\hat{x}) and then projecting it onto the constraint manifold defined by the nonlinear degradation.

Results & Findings

Across all experiments, the SPNN pseudo‑inverse required ≤ 2 × the runtime of a standard forward pass, far cheaper than iterative optimization or training a dedicated inverse network.

Practical Implications

• Plug‑and‑play inverse modules: Developers can wrap any existing neural model (e.g., a super‑resolution or denoising network) with an SPNN wrapper to obtain an exact inverse, enabling on‑the‑fly correction of degradations.
• Zero‑shot restoration services: Cloud providers could expose a single diffusion‑based API that automatically adapts to user‑specified degradations (blur, lens distortion, compression artifacts) without per‑task fine‑tuning.
• Semantic editing tools: Graphic designers can now “undo” a classification or style‑transfer step, giving precise control over generated content while keeping the underlying diffusion prior untouched.
• Robustness & safety: In safety‑critical pipelines (e.g., medical imaging), NLBP guarantees that the reconstructed image exactly satisfies the physical forward model, reducing the risk of hallucinations.
• Research acceleration: Researchers can prototype new inverse problems (e.g., correcting neural‑network‑based compression) by defining the forward degradation and instantly obtaining a tractable pseudo‑inverse.

Limitations & Future Work

• Surjectivity constraint: Not all existing networks are surjective; adapting legacy models may require architectural changes or additional “expansion” layers.
• Memory overhead: Maintaining both the forward SPNN and its mirror pseudo‑inverse doubles the parameter count, which can be a bottleneck for very large models.
• Numerical stability: While the pseudo‑inverse is analytically defined, extreme non‑linearities (e.g., hard saturations) can amplify rounding errors; careful activation design is needed.
• Future directions suggested by the authors include: extending SPNNs to stochastic generative models (e.g., VAEs), exploring learned regularizers that work jointly with NLBP, and scaling the approach to video‑level inverse problems where temporal consistency must be preserved.

Authors

• Yamit Ehrlich
• Nimrod Berman
• Assaf Shocher

Paper Information

• arXiv ID: 2602.06042v1
• Categories: cs.LG, cs.CV
• Published: February 5, 2026
• PDF: Download PDF