创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

VocabX:按成本重构词表,推理成本降77%

AI 总结

Dharma AI 在 Hugging Face 文章中预告了 VocabX:它不再按词频、而是按每个 token 的成本重构语言模型词表,一方面重建面向特定领域的子词词表,另一方面把 JSON 外壳等重复片段合并为宏 token。作者称将该方法用于基于 Qwen3-4B 的生产级葡萄牙语作文评分系统,在基础模型、数据和训练配方均不变的情况下,实测推理成本降低 77%,输出层从约 15.2 万行缩减到约 6.5 万行。他们还称,在合同、医学论文、科学论文、新闻和 Python 代码五个领域与 Qwen3、Llama-3.1、Gemma-3 三个模型系列的 15 种组合上,VocabX 均优于 fast vocabulary transfer、multi-word tokenization 和 Liquid AI 的 in-place BPE。上述结果出自 Dharma AI 表示仍在评审、发表前可能变化的 VocabX 论文。

深度见解

文章认为,在对开源模型做专门化时通常被继承并原样保留的 tokenizer,其实是训练开始前就可设定的重要成本杠杆。

Models specialized for a domain or a task by fine-tuning general open-source models, such as Qwen and Gemma, inherit a vocabulary built for many domains and general applications.

In an essay grader we run in production, that vocabulary split the text the model writes most into many tokens. This article previews some results from the VocabX paper, which we will publish soon: how we specialized the vocabulary around the cost of each token rather than its frequency, and cut measured inference cost by 77% without changing the base model.

The tokenizer is usually treated as a fixed property of the base model, inherited and left alone. For a model specialized in a single domain, it can be one of the largest cost levers, and it is set before any training step.

One of the systems we run in production is an automated grader for Portuguese essays, built on Qwen3-4B. It receives the essay, the prompt and the supporting texts, and returns a JSON assessment with a grade and a comment for each competency.

The model was already small, at 4 billion parameters, and had already been trained for the task. Its tokenizer, however, was still the original Qwen3-4B one, built for many languages and many kinds of text. In the grader, it was used for a single language, Portuguese, and a single response format. That had two costs: the text the grader writes on every request took more tokens than it needed, and at every generated token the model scored a vocabulary of 151,669 entries, including many this task almost never uses.

#VocabX#Dharma AI#Qwen3-4B#Qwen3#Llama-3.1#Gemma-3#Liquid AI

原文约 10 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。