[Paper] Geometry Gaussians: Decoupling Appearance and Geometry in Gaussian Splatting

Source: arXiv - 2606.05124v1

Overview

The paper Geometry Gaussians investigates why the popular 3‑D Gaussian Splatting (3DGS) pipeline struggles to capture both high‑fidelity appearance and accurate geometry at the same time. By training 3DGS with perfect texture + ground‑truth shape, the authors expose an inherent conflict in the original formulation and propose a lightweight fix: a per‑splat geometry‑opacity parameter (and an optional transparency‑aware optimization). The result is a system that delivers sharper renderings and cleaner surface reconstructions, even for challenging scenes with glass or water.

Key Contributions

• Diagnostic study showing that vanilla 3DGS cannot simultaneously encode detailed texture and precise geometry when supplied with full ground‑truth data.
• Geometry‑opacity extension: a single additional scalar per Gaussian that decouples surface visibility from appearance, enabling independent control of shape and color.
• Transparency‑curated training pipeline that leverages the new opacity to better handle semi‑transparent objects without sacrificing overall rendering quality.
• Extensive evaluation on synthetic ground‑truth datasets and on geometry extracted from vision foundation models, demonstrating consistent gains in both PSNR/SSIM (appearance) and Chamfer/IoU (geometry).
• Practical insight that the modification is a drop‑in replacement for existing 3DGS codebases, requiring virtually no extra compute or memory.

Methodology

1. Baseline 3DGS recap – 3DGS represents a scene as a cloud of anisotropic Gaussians, each storing position, covariance, color, and a shared opacity. Rendering is performed by splatting these Gaussians onto the image plane.
2. Problem identification – When the model is forced to fit both exact texture maps and exact surface depth, the shared opacity becomes a bottleneck: increasing opacity improves geometry but washes out fine texture, and vice‑versa.
3. Geometry‑opacity per splat – The authors augment each Gaussian with an independent geometry opacity ( \alpha_g ) that only influences the depth contribution during rasterization, while the original opacity ( \alpha_c ) continues to modulate color blending.
4. Training objective – Two loss terms are optimized jointly: (i) a photometric loss on rendered RGB images, and (ii) a geometric loss (e.g., signed‑distance or depth error) that is weighted by ( \alpha_g ). An optional “transparency curriculum” gradually relaxes the geometry loss for regions identified as semi‑transparent.
5. Implementation details – The extra scalar adds <0.1 % memory overhead. The authors reuse the existing splatting shader, inserting a simple multiplication by ( \alpha_g ) before depth accumulation, making the change compatible with real‑time pipelines.

Results & Findings

• Appearance improves modestly but consistently across all scenes, especially where the original method would over‑smooth textures to preserve geometry.
• Geometry sees a 30‑40 % reduction in surface error, with the biggest gains on objects that partially transmit light (glass, water).
• The transparency‑curated schedule further reduces artefacts like “ghost halos” around clear surfaces.

Practical Implications

• Real‑time AR/VR: Developers can now use 3DGS‑based assets that both look photorealistic and interact correctly with physics engines (collision meshes, occlusion culling).
• Digital twins & simulation: Accurate geometry extraction from existing 3DGS captures enables downstream tasks such as structural analysis or robotic navigation without a separate reconstruction pipeline.
• Content creation pipelines: Artists can keep a single Gaussian‑splat representation for rendering and export clean meshes for game engines, reducing the need for dual‑track workflows (texture‑only vs. mesh‑only).
• Handling transparency: The method offers a cheap alternative to full volumetric rendering for glass or water, making it feasible to integrate transparent objects into real‑time experiences without a performance hit.
• Plug‑and‑play upgrade: Since the change is a single extra scalar and a minor shader tweak, existing 3DGS codebases (e.g., Instant‑NGP‑style implementations) can adopt it with a few lines of code, preserving the original training speed.

Limitations & Future Work

• The geometry‑opacity parameter is still scalar, so it cannot capture complex intra‑splat opacity variations (e.g., thin‑film interference).
• The approach assumes reasonable initial geometry; extremely noisy depth priors may still lead to sub‑optimal decoupling.
• The transparency curriculum is hand‑crafted; learning an adaptive schedule could further improve results.
• Future research could explore multi‑layer splats or learned opacity fields to handle more intricate translucency and to push the method toward full‑scene reconstruction without any external geometry source.

Authors

• Hongyu Zhou
• Zorah Lähner

Paper Information

• arXiv ID: 2606.05124v1
• Categories: cs.GR, cs.CV, cs.LG
• Published: June 3, 2026
• PDF: Download PDF