Split-and-Scale在推理阶段扩展3D潜变量容量
Apple Machine Learning Research收录了一篇NeurIPS论文,提出split-and-scale方法,在控制训练成本的同时于推理阶段扩展潜变量容量。该方法将3D场景表示为3D坐标的连续函数,使一个预训练分词器能够处理更小的场景子区域并保留更多细节。作者报告称,该方法在重建质量和生成建模方面取得了具有竞争力的结果。
据论文作者介绍,该方法把3D表示的细节扩展转移到推理阶段,以应对训练潜变量容量时高昂的计算与显存成本。
research area Computer Visionconference NeurIPS
content type paperpublished November 2026
Split-and-Scale Latent 3D Representations
AuthorsDorian Chan, Muhammed Kocabas, Xiaoming Zhao, Oncel Tuzel, Jen-Hao Rick Chang
Latent 3D representations have significantly expanded the capabilities of modern shape modeling, enabling compact encoding of 3D scenes and supporting powerful 3D generative models. Despite this progress, current representations struggle to capture the fine details of complex objects, fundamentally limited by the capacity of their latents. However, scaling latent capacity at training time is computationally expensive and often infeasible under memory limits, and naively increasing the number of latents at inference fails to make effective use of the added capacity. In this paper, we address both challenges with split-and-scale, a method that effectively expands latent capacity at inference while keeping training cost tractable. Our key observation is that by representing 3D scenes as continuous functions of 3D coordinates, a single pretrained tokenizer can be applied at inference to arbitrarily small sub-regions of a scene, allocating the same latent capacity to a smaller spatial extent and thus capturing far higher detail. We validate this insight by conducting extensive ablations to study the trade-offs between compute, memory, latent capacity, and reconstruction quality, demonstrating our inference-time scaling method achieves quality competitive with much larger models that exceed single-GPU memory limits. We further demonstrate that split-and-scale supports high-quality generative modeling, training an autoregressive 3D generator that performs competitively with state-of-the-art baselines.