Cola2提出连续语义压缩语言建模框架
AI 总结
Cola2论文作者提出一种分层语言建模框架,将连续语义压缩、重建与生成监督,以及连续和离散表示的协同适配结合起来。该系统动态划分文本语义片段,并通过无辅助损失的负反馈控制压缩率,再使用块因果掩码扩散解码器生成文本。作者报告了基于约2100亿训练词元、在五项下游任务上的实验结果,但这些性能数据尚未得到独立验证。
深度见解
Cola2为缩小连续语义表示与离散文本生成之间的差距提供了一种研究架构,并报告了大训练规模下的扩展结果。
Computer Science > Computation and Language
arXiv:2610.31305 (cs)
[Submitted on 22 Oct 2026]
Title:Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation
Authors:Hongcan Guo, Yian Zhao, Feng Wang, Shen Nie, Guoqiang Wei, Yan Zeng, Hengshuang Zhao
View a PDF of the paper titled Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation, by Hongcan Guo and 6 other authors