Voyage相关测试称有标签时监督分类器优于Jev
一名就职于Voyage的作者报告测试了一套监督工作流：先使用开源或价格较低的嵌入模型处理输入，再用自己的标注数据训练sklearn分类器。作者称，该方案成本更低，较Jev降低了37%的错误率。作者同时说明，优势来自使用训练标签；Jev不使用训练标签，而监督方案在没有标签时表现较差。报告附有详细文章和代码。
这项比较展示了能够利用标注数据的监督模型与不使用训练标签的零样本决策模型之间的取舍。所称优势仅适用于有标签的场景，且报告者与Voyage有关联。
There’s a lot of hype right now around decision models like Jev. But in most real use cases, you have some labeled data, and can use it to personalize your model.
So we tested the old-school approach to classification and regression:
- Embed your inputs with an open-source or cheap embedding model
- Fit a simple sklearn model on top using your own labels
Turns out this approach was cheaper and almost always significantly more accurate (37% error reduction against Jev)
Write-up: https://blog.voyageai.com/2026/10/27/how-to-create-your-own-jev-using-voyage-models/ (Disclosure: I work at Voyage) Code: https://github.com/mishelsikder/mongo-predict
Curious whether others have seen the same thing, or found cases where zero-shot performs better
Edit: To be clear, the gains here come from having training labels, which only applies to some use cases. Jev doesn’t use training labels, so it's unsuprising that this approaches wins when you have them and loses when you don't.
submitted by /u/csinva [link] [comments]