2.5-D Decomposition for LLM-Based Spatial Construction
Authors: Paul Whitten, Li-Jen Chen, Sharath Baddam
Summary
The authors solve the problem of LLMs making coordinate errors when building 3D structures by introducing "2.5-D decomposition": the LLM only plans in the horizontal (x-y) plane, and a deterministic executor computes all vertical (z) placement by tracking what's already stacked in each column. This eliminates an entire class of errors. On the Build What I Mean benchmark, GPT-4o-mini with this pipeline achieves 94.6% accuracy (within 3 points of the ceiling set by errors in understanding instructions), beating both GPT-4o (90.3%) and the best competing system (76.3%). The approach transfers directly to edge hardware (Nemotron on Jetson Thor) with no prompt changes.
Main takeaways:
- LLMs make systematic errors when generating 3D block placements; removing one dimension (vertical) from the LLM's output space and computing it deterministically fixes this
- GPT-4o-mini with 2.5-D decomposition achieves 94.6% structural accuracy, within 3 points of the ceiling and beating GPT-4o at 90.3%
- Controlled ablation shows 2.5-D decomposition accounts for 50.7 percentage points of accuracy improvement
- Approach transfers to edge hardware (Nemotron-3 120B on Jetson Thor) with no modifications, matching cloud performance at 94.5%
- Principle generalizes: remove deterministic dimensions from LLM output when physical constraints (gravity, assembly rules) fix degrees of freedom
Relevance
Not related to my persona or behavioral installation work — this is about spatial reasoning and neuro-symbolic architectures for construction tasks. Included because it's a frontier LLM paper, but no obvious connection to conditional behavior or midtraining.
Abstract
arXiv:2605.07066v1 Announce Type: new Abstract: Autonomous systems that build structures from natural-language instructions need reliable spatial reasoning, yet large language models (LLMs) make systematic coordinate errors when generating three-dimensional block placements. We present a neuro-symbolic pipeline based on \emph{2.5-D decomposition}: the LLM plans in the two-dimensional horizontal plane while a deterministic executor computes all vertical placement from column occupancy, eliminating an entire class of errors. On the Build What I Mean benchmark (160 rounds), GPT-4o-mini with this pipeline achieves 94.6% mean structural accuracy across 12 independent runs, within 3.0 percentage points of the 97.6% ceiling imposed by architect-agent errors that no builder-side improvement can address. This outperforms both GPT-4o at 90.3% and the best competing system at 76.3%. A controlled ablation confirms that 2.5-D decomposition is the dominant contributor, accounting for 50.7 percentage points of accuracy. The pipeline transfers directly to edge hardware: Nemotron-3 120B running locally on an NVIDIA Jetson Thor AGX matches the cloud result at 94.5% with no prompt modifications. The underlying principle, removing deterministic dimensions from the LLM's output space, applies to any autonomous construction or assembly task where gravity or other physical constraints fix one or more degrees of freedom. A transfer experiment on 500 IGLU collaborative building tasks confirm the effect generalizes beyond the primary benchmark.