Cola2 applies video-style diffusion modeling to language
ByteDance Seed researchers led by Yan Zeng published Cola2, which uses continuous latent representations and diffusion-inspired modeling to generate text more efficiently. The paper reports an average score of 60.10 across eight tasks using about 346 billion tokens, while the authors say the model had not converged and required more data and compute. TMTPost interprets the work as supporting a large-scale model direction at ByteDance, but the reported “10-trillion-parameter” roadmap and related personnel claims lack official confirmation.
Cola2 offers a non-autoregressive alternative for language modeling and illustrates ByteDance Seed’s reported interest in scaling diffusion-based approaches, although its claimed efficiency remains bounded by a paper that says training was not yet converged.
(本文作者为 字母AI,钛媒体经授权发布)
文 | 字母AI
95后的曾妍,正在快速晋升。
曾妍于1997年出生,2021年以校招的方式进入字节,后来成为了Seedance的预训练负责人。字母AI在文章《字节的“罗福莉”,撑起了Seedance的半边天》中曾对她进行过详细报道。
如今有传言称,曾妍的头衔已经上升为Seedance核心负责人。早在今年3月时,曾妍的职级还是3-2,而到了11月,她的职级达到了4-2,实现一年连跳两级。并且曾妍从原Seed核心负责人周畅手中,接过了越来越多的工作。
在人才密度极高的字节Seed团队,这样的速度并不多见。
#Cola2#ByteDance#ByteDance Seed#Seedance#Seedance 2.0#Seedream 5.0 Lite#Hugging Face#Llama-3-8B