[Paper] Smudged Fingerprints: A Systematic Evaluation of the Robustness of AI Image Fingerprints

Source: arXiv - 2512.11771v1

Overview

The paper “Smudged Fingerprints: A Systematic Evaluation of the Robustness of AI Image Fingerprints” investigates how well current model‑fingerprinting techniques hold up when an adversary tries to hide or forge the provenance of AI‑generated images. By treating fingerprint detection as a security problem, the authors expose a worrying gap between ideal (clean) performance and real‑world, adversarial scenarios—information that’s crucial for anyone building or defending AI‑generated content pipelines.

Key Contributions

• First security‑focused benchmark for AI image fingerprinting, covering both white‑box (full model knowledge) and black‑box (limited query) threat models.
• Two attack goals defined:
  1. Fingerprint removal – erase traces to evade attribution.
  2. Fingerprint forgery – inject false traces to misattribute an image to a target model.
• Implementation of five practical attack strategies (gradient‑based, optimization‑based, and query‑efficient methods).
• Comprehensive evaluation: 14 fingerprinting methods (RGB‑pixel, frequency‑domain, and learned‑feature approaches) tested on images from 12 state‑of‑the‑art generators (e.g., Stable Diffusion, DALL·E 2, Midjourney).
• Empirical discovery of a utility‑robustness trade‑off: the most accurate fingerprinting schemes tend to be the easiest to break.
• Guidelines for future research, highlighting which existing techniques are relatively more robust and where the biggest vulnerabilities lie.

Methodology

1. Threat Model Formalization – The authors spell out what an attacker can know (white‑box: full access to the fingerprint detector; black‑box: only query responses) and what they aim to achieve (removal vs. forgery).
2. Attack Suite – Five attacks are built on common adversarial‑image techniques:
   • Gradient‑based perturbations (FGSM, PGD) that directly minimize the fingerprint detector’s confidence.
   • Optimization‑based attacks that treat the fingerprint loss as an objective and iteratively refine the image.
   • Query‑efficient black‑box attacks (NES, bandit‑based) that estimate gradients from limited API calls.
3. Fingerprinting Baselines – The 14 methods span three families:
   • RGB‑domain (e.g., statistical moments of pixel values).
   • Frequency‑domain (e.g., DCT/FFT signatures).
   • Learned‑feature (deep‑network embeddings trained to discriminate models).
4. Evaluation Protocol – For each generator‑fingerprinter pair, the authors measure:
   • Attribution accuracy on clean images.
   • Success rate of removal (how often the detector’s confidence drops below a threshold).
   • Success rate of forgery (how often the image is misattributed to a chosen target).
     Results are aggregated across white‑box and black‑box settings.

Results & Findings

• Utility‑Robustness Trade‑off: Techniques that achieved > 95 % clean attribution (e.g., certain learned‑feature detectors) fell to < 20 % robustness under white‑box removal attacks.
• Domain Differences: Frequency‑domain fingerprints were slightly more resistant to black‑box removal than RGB‑based ones, but all fell short of providing both high accuracy and high robustness.
• Forgery Difficulty: While forging a fingerprint is tougher than removing one, targeted attacks against specific popular models (e.g., Stable Diffusion) succeeded over 50 % of the time in white‑box settings.
• No Universal Defender: No single method maintained > 80 % attribution accuracy and > 70 % robustness across all threat models.

Practical Implications

• Content‑Moderation Platforms: Relying solely on current fingerprint detectors to flag AI‑generated media could be bypassed by relatively simple adversarial edits, especially if attackers have white‑box knowledge (e.g., open‑source detectors).
• Intellectual‑Property Enforcement: Companies that use fingerprinting to prove ownership of a generative model’s output should treat the technology as a “soft” watermark—effective for casual detection but not for legal proof against determined adversaries.
• Tooling for Developers: The attack implementations are open‑source, meaning developers can now test the robustness of their own fingerprinting pipelines before deployment, similar to adversarial robustness testing for classifiers.
• Designing Safer Generators: Knowing that frequency‑domain signatures are marginally harder to erase suggests that future generators could embed robust, imperceptible signals at the synthesis stage (e.g., via loss‑function regularization).
• Policy & Regulation: Regulators aiming to mandate provenance tracking must consider that “technical compliance” (i.e., deploying a fingerprint detector) does not guarantee tamper‑proof traceability.

Limitations & Future Work

• Scope of Generators: Only 12 generators were examined; emerging diffusion models or multimodal generators may behave differently.
• Attack Realism: White‑box attacks assume full knowledge of the detector’s internals, which may not always be true in practice. Black‑box attacks were limited to a modest query budget; larger budgets could yield higher success rates.
• Metric Focus: The study centers on attribution accuracy and attack success; it does not deeply explore perceptual quality degradation caused by attacks, which matters for real‑world misuse.
• Future Directions:
  • Developing adaptive fingerprinting schemes that can detect when an image has been tampered with (meta‑robustness).
  • Exploring joint fingerprint‑and‑steganography approaches to combine statistical and cryptographic guarantees.
  • Extending the benchmark to video and audio generation pipelines, where temporal consistency adds new attack surfaces.

Bottom line: While AI image fingerprinting shows promise for provenance tracking, this systematic security evaluation reveals that current methods are far from battle‑ready. Developers and organizations should treat fingerprints as a complementary signal, not a silver bullet, and invest in robustness‑focused research before relying on them for high‑stakes applications.

Authors

• Kai Yao
• Marc Juarez

Paper Information

• arXiv ID: 2512.11771v1
• Categories: cs.CV, cs.AI
• Published: December 12, 2025
• PDF: Download PDF