Unsloth Flint Q3 tested across three AI agent harnesses
AI summary
A r/LocalLLaMA user reported that the unsloth/flint-GGUF flint-UD-Q3_K_XL.gguf model completed all 12 tasks on OMP with a score of 71.2/100, solving seven, and on AtomicAgent with 39.2/100, solving three. The tester said Hermes stopped during task seven after broad filesystem searches and looping, while DeepSeek was not run because of previous harness failures and low scores.
In-depth insight
The report offers a concrete, if unverified, signal about how the same quantized model performs across different AI agent harnesses.
| Unsloth Flint Q3 Results Performed quite poorly on benchmark using 3 diff harnesses: OMP: Completed all 12 tasks; 71.2/100 (only 7 solved). AtomicAgent: Completed all 12 tasks; 39.2/100 (only 3 solved). Hermes: Stopped during Task 7 after it performed needless broad searches across the filesystem and looped. Deepseek: Due to previous harness failures and low scores, not run. Full leaderboard https://shortbench.vercel.app/ Current models with 100/100 Score submitted by /u/mesolimbic-bliss [link] [comments] |