[Paper] PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World

Source: arXiv - 2605.05163v1

Overview

PhysForge tackles a long‑standing pain point for game developers, VR creators, and robotics researchers: automatically generating 3D assets that are not only visually realistic but also behave correctly under physics simulation. By coupling a large‑scale, physics‑annotated dataset (PhysDB) with a two‑stage generation pipeline, the authors demonstrate that it’s possible to produce “simulation‑ready” objects that can be dropped, pushed, or articulated straight out of the generator.

Key Contributions

• PhysDB – a curated dataset of 150 k 3D assets annotated with a four‑tier hierarchy of physical properties (material, functional class, kinematic constraints, and simulation parameters).
• Hierarchical Physical Blueprint – a textual‑to‑visual plan generated by a vision‑language model (VLM) that encodes material, intended function, and motion limits before any geometry is created.
• Two‑stage generation pipeline
  1. Physical architect stage: the VLM produces the blueprint.
  2. Physics‑grounded diffusion stage: a diffusion model synthesizes geometry while injecting kinematic information via the novel KineVoxel Injection (KVI) technique.
• Simulation‑ready output – the pipeline directly yields meshes together with accurate collision shapes, mass, friction, and joint limits, eliminating the manual “physics rigging” step.
• Extensive evaluation – quantitative metrics (e.g., functional plausibility, physical parameter error) and user studies show PhysForge outperforms prior geometry‑only generators by a large margin.

Methodology

1. Data Preparation (PhysDB)

   • Collected 150 k assets from existing repositories.
   • Annotated each asset with:
     • Material type (wood, metal, fabric, etc.)
     • Functional class (door, chair, lever, etc.)
     • Kinematic constraints (hinge axis, sliding limits)
     • Simulation parameters (mass, friction, restitution).

2. Stage 1 – Blueprint Generation

   • A large vision‑language model (e.g., CLIP‑based) receives a textual prompt like “a wooden cabinet with two sliding doors”.
   • The model outputs a structured Hierarchical Physical Blueprint (HPB) that lists the material, functional intent, and a set of kinematic constraints in a machine‑readable format.

3. Stage 2 – Physics‑Grounded Diffusion

   • A 3‑D diffusion model (similar to DreamFusion) is conditioned on the HPB.
   • KineVoxel Injection (KVI): the blueprint’s kinematic constraints are rasterized into a voxel field that is injected into the diffusion latent space at each denoising step, steering the geometry toward shapes that can satisfy the specified motion.
   • The model simultaneously predicts a high‑resolution mesh and the associated physics parameters (mass, inertia tensor, joint limits).

4. Post‑Processing

   • Automatic generation of collision hulls and export to common formats (OBJ + URDF) ready for engines like Unity, Unreal, or ROS‑based simulators.

Results & Findings

• Qualitative: assets such as hinged doors, sliding drawers, and articulated robots exhibit correct joint axes and realistic material deformation when simulated.
• Ablation: removing KVI drops functional plausibility by ~1.2 points, confirming its role in enforcing kinematic feasibility.
• User Study: 30 developers rated PhysForge‑generated assets as “ready‑to‑use” 85 % of the time, versus 38 % for baseline generators.

Practical Implications

• Game & VR Development: Rapidly populate worlds with objects that already respect physics, cutting down weeks of manual rigging and testing.
• Robotics & Embodied AI: Generate training environments where agents interact with objects that behave realistically, improving sim‑to‑real transfer.
• Content Platforms: Asset marketplaces can offer “physics‑enabled” models, adding value for buyers who need simulation‑ready items.
• Design Automation: Engineers can prototype functional parts (e.g., brackets, levers) by simply describing the intended motion, receiving a CAD‑compatible mesh with correct joint specs.

Limitations & Future Work

• Complex Articulations: Current KVI handles single‑joint constraints well but struggles with multi‑link mechanisms (e.g., robotic arms).
• Material Diversity: While material types are annotated, nuanced properties like anisotropic friction or deformable behavior are not yet modeled.
• Scalability of Blueprint Language: The HPB schema is fixed; extending it to new functional categories requires manual updates.
• Future Directions: The authors plan to integrate reinforcement‑learning loops that validate generated assets in a physics engine during training, and to expand PhysDB with soft‑body and fluid‑interaction annotations.

Authors

• Yunhan Yang
• Chunshi Wang
• Junliang Ye
• Yang Li
• Zanxin Chen
• Zehuan Huang
• Yao Mu
• Zhuo Chen
• Chunchao Guo
• Xihui Liu

Paper Information

• arXiv ID: 2605.05163v1
• Categories: cs.CV
• Published: May 6, 2026
• PDF: Download PDF