创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Transformers v5.23.0新增多种模型支持

AI 总结

据Transformers Releases发布说明,v5.23.0新增了对Inkling 2 Small、JEV、LFM3、Dots 3 Note Preview和LFM3-VL的支持。Inkling 2 Small被描述为支持文本和图像输入的多模态混合专家模型,JEV则提供用于单令牌标签评分的实验性决策头。此次版本还包含内核、注意力、生成和分布式训练等修复。

深度见解

该版本扩展了Transformers对多模态、语言模型和决策头的支持,并改进了推理与分布式训练相关功能。

Release v5.23.0

New Model additions

Inkling2

image

Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for use in English and other languages and across multiple coding languages. Its language backbone has 48 mixture-of-experts decoder layers, each routing tokens to 8 of 384 experts, for 594B total and 17B active parameters. It keeps the Inkling decoder (learned relative attention, short causal convolutions, mixture-of-experts layers) and adds block attention residuals, gated attention with a learned sink, and dynamic key/value convolutions. Images are encoded by a Perception Encoder-based vision transformer and projected into the language model embeddings, one embedding per 40x40 pixel patch.

Links: Documentation | Blog Post

部分来源内容未展示,可前往原文继续阅读。

#Transformers#Inkling 2 Small#Thinking Machines Lab#JEV#LFM3#Liquid AI#Dots 3 Note Preview#LFM3-VL

原文约 21 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。