[Paper] Memorization in 3D Shape Generation: An Empirical Study

Source: arXiv - 2512.23628v1

Overview

This paper tackles a surprisingly practical question: Do 3‑D shape generators simply memorize the training set, or can they truly create novel geometry? By introducing a systematic way to measure memorization in state‑of‑the‑art 3‑D generative models, the authors reveal when and why memorization happens and propose lightweight fixes that keep generation quality high while improving diversity.

Key Contributions

• A quantitative memorization benchmark for 3‑D shape generators that works across point clouds, meshes, and implicit fields.
• Empirical analysis of several popular 3‑D generative methods (e.g., PointFlow, ShapeGF, Diffusion‑based models) using the benchmark.
• Controlled experiments with a novel “Vecset” diffusion model, isolating the impact of data modality, conditioning granularity, and model hyper‑parameters on memorization.
• Practical mitigation strategies: moderate classifier‑free guidance, longer latent vector sets, and simple rotation augmentation—all shown to cut memorization without hurting visual fidelity.
• Open‑source implementation (https://github.com/zlab-princeton/3d_mem) for reproducibility and future research.

Methodology

1. Memorization Metric – The authors compute a nearest‑neighbor similarity between each generated shape and the entire training corpus, using Chamfer Distance (for point clouds) or IoU (for voxel/mesh representations). A high similarity score indicates the model is reproducing an existing example rather than inventing a new one.
2. Evaluation Framework –
   • Dataset splits: training, validation, and a held‑out “privacy” set that the model never sees.
   • Thresholding: shapes whose similarity exceeds a calibrated percentile are flagged as “memorized.”
   • Statistical reporting: memorization rate (percentage of generated samples flagged) plus distribution plots.
3. Vecset Diffusion Model – Instead of a single latent vector, the model samples a set of latent vectors (the “Vecset”) that are jointly denoised. This design lets the authors vary the set length and observe its effect on memorization.
4. Controlled Variables –
   • Data modality (point clouds vs. meshes vs. implicit fields).
   • Conditioning granularity (coarse class labels vs. fine part‑level cues).
   • Guidance scale (strength of classifier‑free guidance).
   • Data augmentations (random rotations).

All experiments run on standard 3‑D shape benchmarks (ShapeNet, ModelNet) with identical training pipelines to ensure fair comparisons.

Results & Findings

Across all tested baselines, memorization rates ranged from 5 % (point‑cloud diffusion) to 27 % (implicit‑field GAN). Applying the mitigation tricks reduced memorization by 15‑30 % while keeping FID‑like quality scores within 2 % of the original.

Practical Implications

• Data privacy compliance – Companies deploying 3‑D generative services (e.g., virtual asset marketplaces) can use the provided benchmark to audit whether proprietary CAD models are being unintentionally reproduced.
• Model selection – For applications where novelty matters (e.g., procedural content generation in games), point‑cloud diffusion models with moderate guidance and longer Vecsets are a safer default.
• Training pipelines – Adding cheap rotation augmentation and tuning the guidance scale are low‑cost steps that can be integrated into existing training scripts without extra compute.
• API design – Services can expose a “diversity knob” that internally adjusts Vecset length or guidance scale, giving developers control over the memorization‑quality trade‑off.
• Open‑source tooling – The released code makes it straightforward to plug the memorization metric into CI pipelines, automatically flagging regressions in model diversity.

Limitations & Future Work

• Metric sensitivity – The nearest‑neighbor similarity threshold is heuristic; edge cases (e.g., near‑duplicate but legally distinct designs) may be misclassified.
• Scope of modalities – The study focuses on static shapes; dynamic or textured 3‑D assets (e.g., animated characters) remain unexplored.
• Scalability – Computing pairwise distances for large training sets can be costly; approximate nearest‑neighbor methods may be needed for industrial‑scale datasets.
• Theoretical grounding – While the empirical trends are clear, a formal analysis of why moderate guidance maximizes memorization is left for future research.

The authors suggest extending the framework to text‑to‑3‑D pipelines, investigating cross‑modal memorization (e.g., from images to shapes), and exploring privacy‑preserving training (differential privacy, data sanitization) as promising next steps.

Authors

• Shu Pu
• Boya Zeng
• Kaichen Zhou
• Mengyu Wang
• Zhuang Liu

Paper Information

• arXiv ID: 2512.23628v1
• Categories: cs.CV, cs.LG
• Published: December 29, 2025
• PDF: Download PDF