[Paper] Realistic Face Reconstruction from Facial Embeddings via Diffusion Models

Source: arXiv - 2602.13168v1

Overview

The paper introduces FEM (Face Embedding Mapping), a novel attack pipeline that can turn the compact facial embeddings produced by modern face‑recognition (FR) and privacy‑preserving FR (PPFR) systems back into high‑resolution, photorealistic face images. By coupling a Kolmogorov‑Arnold Network (KAN) with a pre‑trained identity‑preserving diffusion model, the authors demonstrate that even “protected” embeddings leak enough information to reconstruct recognizable faces, raising fresh privacy concerns for deployed biometric services.

Key Contributions

• FEM framework: A generic, plug‑and‑play pipeline that maps any facial embedding to a realistic image using a KAN‑based mapper and a diffusion decoder.
• KAN for embedding‑to‑latent translation: Shows that the Kolmogorov‑Arnold Network can efficiently learn the highly non‑linear relationship between low‑dimensional embeddings and the latent space of a diffusion model.
• Robustness to partial and protected embeddings: The method works even when only a subset of the embedding dimensions is available or when embeddings have been deliberately obfuscated for privacy.
• Cross‑system leakage demonstration: Reconstructed faces successfully authenticate against several off‑the‑shelf FR services that were not used during training, confirming real‑world privacy leakage.
• Evaluation tool: Proposes FEM as a practical benchmark for measuring the privacy‑preserving strength of new FR/PPFR pipelines.

Methodology

1. Embedding Extraction – The target FR or PPFR system outputs a fixed‑size vector (e.g., 512‑D) that encodes identity information.
2. KAN Mapper – A lightweight Kolmogorov‑Arnold Network learns a deterministic mapping from this embedding to the latent space of a diffusion model. KANs are attractive because they approximate any continuous function with a shallow architecture, making training fast and stable.
3. Identity‑Preserving Diffusion Decoder – A pre‑trained diffusion model (e.g., Stable Diffusion fine‑tuned for faces) receives the latent code and iteratively denoises it into a high‑resolution image. The model is conditioned on an identity loss that forces the output to retain the original embedding’s identity cues.
4. Training Loop – The KAN and diffusion decoder are jointly optimized on public face datasets (e.g., CelebA‑HQ, FFHQ). The loss combines pixel‑level reconstruction, perceptual similarity, and an explicit embedding‑matching term.
5. Evaluation – Reconstructed images are fed to multiple commercial and open‑source FR APIs (e.g., ArcFace, FaceNet) to measure verification success rates. Experiments also test scenarios with missing embedding dimensions and with embeddings that have been transformed by common privacy mechanisms (e.g., quantization, random masking).

Results & Findings

• The reconstructed faces retain fine‑grained attributes (pose, lighting, expression) despite being generated from a compact vector.
• Cross‑system attacks succeed even when the target FR model differs in architecture and training data, indicating that the leakage is intrinsic to the embedding, not the specific classifier.
• Visual inspection shows that the diffusion decoder produces photorealistic results comparable to state‑of‑the‑art face generation, far surpassing earlier GAN‑based inversion attempts.

Practical Implications

• Privacy Audits – Companies deploying PPFR solutions can use FEM as a “red‑team” tool to quantify how much identity information leaks from their embeddings.
• Regulatory Compliance – Demonstrates that GDPR‑style “pseudonymisation” may be insufficient if embeddings can be inverted, prompting stricter data‑handling policies.
• Design of Safer Embeddings – Encourages research into embedding hardening techniques (e.g., differential privacy, adversarial perturbations) that survive attacks like FEM.
• Security‑Aware SDKs – Developers of authentication SDKs might integrate runtime checks that limit exposure of raw embeddings (e.g., on‑device verification, encrypted transmission).
• Synthetic Data Generation – The same pipeline could be repurposed to generate realistic avatars from anonymised embeddings, useful for VR/AR or gaming where identity preservation is optional.

Limitations & Future Work

• Dependence on a strong diffusion prior – The quality of reconstruction hinges on the availability of a well‑trained, identity‑preserving diffusion model; training such a model is computationally expensive.
• Dataset Bias – Experiments are limited to public, mostly Western‑centric face datasets; performance on under‑represented demographics remains unclear.
• Partial Embedding Scenarios – While the method tolerates some missing dimensions, extreme sparsity (e.g., < 10 % of features) still fails.
• Future Directions – The authors suggest exploring tighter integration of privacy mechanisms (e.g., homomorphic encryption) with FEM, extending the attack to video‑stream embeddings, and evaluating robustness against newer, transformer‑based FR backbones.

Authors

• Dong Han
• Yong Li
• Joachim Denzler

Paper Information

• arXiv ID: 2602.13168v1
• Categories: cs.CV, cs.LG
• Published: February 13, 2026
• PDF: Download PDF