Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Cola2 applies video-style diffusion modeling to language

AI summary

ByteDance Seed researchers led by Yan Zeng published Cola2, which uses continuous latent representations and diffusion-inspired modeling to generate text more efficiently. The paper reports an average score of 60.10 across eight tasks using about 346 billion tokens, while the authors say the model had not converged and required more data and compute. TMTPost interprets the work as supporting a large-scale model direction at ByteDance, but the reported “10-trillion-parameter” roadmap and related personnel claims lack official confirmation.

In-depth insight

Cola2 offers a non-autoregressive alternative for language modeling and illustrates ByteDance Seed’s reported interest in scaling diffusion-based approaches, although its claimed efficiency remains bounded by a paper that says training was not yet converged.

(本文作者为 字母AI,钛媒体经授权发布)

文 | 字母AI

95后的曾妍,正在快速晋升。

曾妍于1997年出生,2021年以校招的方式进入字节,后来成为了Seedance的预训练负责人。字母AI在文章《字节的“罗福莉”,撑起了Seedance的半边天》中曾对她进行过详细报道。

如今有传言称,曾妍的头衔已经上升为Seedance核心负责人。早在今年3月时,曾妍的职级还是3-2,而到了11月,她的职级达到了4-2,实现一年连跳两级。并且曾妍从原Seed核心负责人周畅手中,接过了越来越多的工作。

在人才密度极高的字节Seed团队,这样的速度并不多见。

#Cola2#ByteDance#ByteDance Seed#Seedance#Seedance 2.0#Seedream 5.0 Lite#Hugging Face#Llama-3-8B

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.