[Paper] SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments

Source: arXiv - 2604.14144v1

Overview

SpatialEvo introduces a novel “self‑evolving” training loop for 3‑D spatial reasoning that eliminates the need for costly geometric annotations. By turning raw point‑cloud data and camera poses into a Deterministic Geometric Environment (DGE)—an error‑free oracle that can validate any spatial query—the authors let a single neural policy learn to both ask and answer questions about a scene, continuously improving itself without human labels.

Key Contributions

• Deterministic Geometric Environment (DGE): Formalizes 16 common 3‑D spatial reasoning tasks with exact geometric validation rules, turning any unlabelled scene into a zero‑noise interactive oracle.
• Unified Questioner‑Solver Policy: A single set of model parameters is trained to play both roles—generating physically plausible questions and producing precise answers—under the same DGE constraints.
• Task‑Adaptive Curriculum Scheduler: Automatically detects the model’s weakest reasoning categories and focuses training on them, removing the need for hand‑crafted curricula.
• Scalable Self‑Evolution: Demonstrates that the framework works at both 3 B and 7 B parameter scales, achieving state‑of‑the‑art scores on nine public 3‑D reasoning benchmarks while preserving performance on general vision‑language tasks.
• Annotation‑Free Learning: Shows that high‑quality spatial intelligence can be acquired without any human‑written geometric labels, dramatically reducing data collection costs.

Methodology

1. Building the DGE

   • Input: raw point clouds + known camera extrinsics.
   • The system computes exact geometric relationships (e.g., distances, occlusions, relative orientations) using deterministic algorithms (ray‑casting, convex hulls, etc.).
   • These computations serve as an oracle that can instantly verify whether a proposed spatial statement is true or false.

2. Dual‑Role Policy Architecture

   • A transformer‑based encoder‑decoder receives the current visual observation.
   • In questioner mode it outputs a natural‑language query that is guaranteed to be physically valid (the DGE rejects any illegal question).
   • In solver mode it consumes a query and produces an answer, which is then checked against the DGE ground truth.

3. Self‑Evolving Loop

   • The model generates a batch of question‑answer pairs on unlabelled scenes.
   • The DGE supplies the correct answer (zero‑noise) and a loss signal for the solver.
   • If the question is invalid, the DGE provides a corrective hint, guiding the questioner to improve.

4. Task‑Adaptive Scheduler

   • After each training epoch, the scheduler measures per‑category accuracy.
   • Categories with the lowest scores receive a higher sampling probability for the next epoch, forming a dynamic curriculum that automatically targets weaknesses.

Results & Findings

• Consistent improvements across all 16 task categories, with the largest gains on occlusion reasoning and relative orientation.
• Ablation studies confirm that removing the DGE or the adaptive scheduler drops performance by >4 pts, highlighting their importance.
• The model’s question‑generation quality improves over time, eventually producing human‑like spatial queries (e.g., “Is the red chair behind the blue table from the camera’s viewpoint?”).

Practical Implications

• Robotics & AR/VR: Developers can train embodied agents (drones, household robots, AR assistants) to understand spatial constraints without hand‑labelled 3‑D datasets, accelerating deployment in new environments.
• Simulation‑Free Data Augmentation: Existing point‑cloud repositories (e.g., ScanNet, Matterport3D) can be turned into infinite training sources for spatial reasoning, reducing reliance on expensive simulation pipelines.
• Zero‑Shot Spatial QA APIs: The unified policy can be exposed as a service that answers geometry‑related questions about any uploaded 3‑D scan, useful for architecture, construction, and e‑commerce (e.g., “Will this sofa fit through the doorway?”).
• Curriculum‑Free Model Scaling: The task‑adaptive scheduler removes the need for manual curriculum design when scaling models, simplifying the engineering effort for large‑scale training runs.

Limitations & Future Work

• Dependence on Accurate Pose Data: The DGE assumes precise camera extrinsics; noisy pose estimates can corrupt the oracle’s answers.
• Static Scenes Only: Current validation rules handle static geometry; extending to dynamic objects (e.g., moving humans) will require temporal reasoning extensions.
• Language Generalization: While the model retains general visual‑language abilities, its question‑generation style is biased toward the 16 predefined categories; broader open‑ended querying remains an open challenge.
• Future Directions: Incorporating probabilistic pose refinement, adding physics‑based simulation for dynamic interactions, and expanding the DGE to support multimodal queries (e.g., tactile or force feedback) are promising next steps.

Authors

• Dinging Li
• Yingxiu Zhao
• Xinrui Cheng
• Kangheng Lin
• Hongbo Peng
• Hongxing Li
• Zixuan Wang
• Yuhong Dai
• Haodong Li
• Jia Wang
• Yukang Shi
• Liang Zhao
• Jianjian Sun
• Zheng Ge
• Xiangyu Zhang
• Weiming Lu
• Jun Xiao
• Yueting Zhuang
• Yongliang Shen

Paper Information

• arXiv ID: 2604.14144v1
• Categories: cs.CV, cs.CL
• Published: April 15, 2026
• PDF: Download PDF