Voyage-affiliated test favors supervised classifiers over Jev when labels are available
An author who works at Voyage reported testing a supervised workflow that embeds inputs with an open-source or inexpensive embedding model and fits an sklearn classifier using labeled data. The author said the approach was cheaper and delivered a 37% error reduction against Jev. They clarified that the gain came from using training labels: Jev does not use them, and the supervised approach performed worse when labels were unavailable. The report linked a write-up and code.
The comparison illustrates a tradeoff between supervised models that can use labeled data and zero-shot decision models that do not. Its reported advantage is limited to labeled use cases and comes from a Voyage-affiliated author.
There’s a lot of hype right now around decision models like Jev. But in most real use cases, you have some labeled data, and can use it to personalize your model.
So we tested the old-school approach to classification and regression:
- Embed your inputs with an open-source or cheap embedding model
- Fit a simple sklearn model on top using your own labels
Turns out this approach was cheaper and almost always significantly more accurate (37% error reduction against Jev)
Write-up: https://blog.voyageai.com/2026/10/27/how-to-create-your-own-jev-using-voyage-models/ (Disclosure: I work at Voyage) Code: https://github.com/mishelsikder/mongo-predict
Curious whether others have seen the same thing, or found cases where zero-shot performs better
Edit: To be clear, the gains here come from having training labels, which only applies to some use cases. Jev doesn’t use training labels, so it's unsuprising that this approaches wins when you have them and loses when you don't.
submitted by /u/csinva [link] [comments]