创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Transformers v5.24.0新增 Inkling 2 Mini

AI 总结

Transformers v5.24.0 新增 Thinking Machines Lab 的多模态模型 Inkling 2 Mini,可接收文本和图像输入并生成文本。据发布说明,其 24 层解码器会将第 6 至 17 层循环运行三次,并通过学习式消息传递机制混合前一遍的残差;每个 token 会路由至 128 个专家中的 8 个,隐藏层大小为 2304。此次版本还加入支持可变循环次数的 TML-d24-varloop-research 检查点,可用更多循环次数换取质量。

深度见解

此次更新让 Transformers 用户可直接使用 Inkling 2 Mini 及其可变循环研究配置,扩展了框架对循环式多模态 MoE 模型的支持。

Release v5.24.0

New Model additions

Inkling 2 Mini

image

Inkling 2 Mini is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs. It is a smaller sibling of Inkling 2 Small, added in v5.23.0. It has 24 decoder layers instead of 48, and compensates with a looping mechanism in which layers 6 to 17 run three times. A learned message-passing mechanism mixes the residuals of the previous pass into the attention and mixture-of-experts inputs of each loop layer. The decoder has a hidden size of 2304 and routes each token to 8 of 128 experts. The image encoder, tokenizer and processor are the same as Inkling 2 Small's.

A research checkpoint (TML-d24-varloop-research) has the same architecture but supports a variable number of loop passes. Fewer passes are faster, and more passes improve quality. Both checkpoints contain NVFP4 and MXFP8 quantized layers, which are dequantized at load time.

#Transformers v5.24.0#Thinking Machines Lab#Inkling 2 Mini#TML-d24-varloop-research#Inkling 2 Small#Hugging Face Transformers

原文约 16 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。