Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Cola2 proposes continuous semantic compression for language modeling

AI summary

The authors of the Cola2 paper propose a hierarchical language-modeling framework that combines continuous semantic compression, reconstruction and generation supervision, and co-adaptation between continuous and discrete representations. The system dynamically segments text into semantic spans and uses auxiliary-loss-free negative feedback to control compression, while a block-causal masked diffusion decoder handles generation. The authors report experiments on five downstream tasks using about 210 billion training tokens, but the performance claims have not been independently verified.

In-depth insight

Cola2 offers a research architecture for reducing the gap between continuous semantic representations and discrete text generation, with reported scaling results at large training-token volumes.

Computer Science > Computation and Language

arXiv:2610.31305 (cs)

[Submitted on 22 Oct 2026]

Title:Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation

Authors:Hongcan Guo, Yian Zhao, Feng Wang, Shen Nie, Guoqiang Wei, Yan Zeng, Hengshuang Zhao

View a PDF of the paper titled Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation, by Hongcan Guo and 6 other authors

#Cola2#Llama 3 8B

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.