创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

BOS token改变6个模型回答

AI 总结

一项社区实验发现,在 Ollama 0.40.14 中使用 Bielik-Minitron 7B v3.0 Instruct 时,加入一个序列起始 token 会改变15个回答中的6个。全部135次试验中,不含 BOS 时通过11项任务,含 BOS 时通过12项,但作者明确表示,这一狭窄测试不能证明 BOS 通常能提升模型质量。Ollama 的该模型聊天模板是否遗漏此 token 并构成缺陷,目前仍不确定。

深度见解

这项 r/LocalLLaMA 报告表明,看起来相同的提示词在不同推理引擎中可能产生不同结果,基准测试需控制分词、聊天模板、缓存状态和计算细节。

I'm back with another experiment.

After the last heated discussion about KV cache, temperature 0, and reproducibility — where I got quite a beating for both my post and my writing style (partly deserved, to be fair ;) ) — I decided to investigate how models behave in Ollama and llama.cpp.

And, as usually happens with these experiments, I found something I wasn't even looking for.

The Bielik-Minitron 7B chat template stored in the GGUF includes the beginning-of-sequence token <s>, but the template Ollama uses for this model doesn't add it.

I thought to myself: come on, man, it's just one token.

But I decided to check whether its sudden appearance actually changes anything.

#Bielik-Minitron 7B v3.0 Instruct#Ollama#llama.cpp#GGUF

原文约 5 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。