SpaceCast-Bench

Headlines · 1

1. Oct 7
   SpaceCast-Bench: Evaluating predictive spatial reasoning in VLMs

   The paper introduces SpaceCast-Bench, which the authors describe as the first benchmark to directly and diagnostically evaluate predictive spatial reasoning, where a model must construct a scene from observations, anticipate how an intervention changes it, and reason about the unseen outcome. Built around an observe-transform-infer framework, it contains 3,862 questions from 182 real-world scenes across 16 task types at three levels: static perception, local prediction, and global prediction. Evaluating 21 models shows the strongest reaching only 58.0% versus 87.2% human performance, while spatially specialized models remain near random chance; fine-tuning on programmatically generated data lifts Qwen3-VL-4B from 34.0% to 65.7% with macro-average gains on six out-of-domain benchmarks.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my SpaceCast-Bench experienceAsk about SpaceCast-Bench