Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Unsloth Flint Q3 tested across three AI agent harnesses

AI summary

A r/LocalLLaMA user reported that the unsloth/flint-GGUF flint-UD-Q3_K_XL.gguf model completed all 12 tasks on OMP with a score of 71.2/100, solving seven, and on AtomicAgent with 39.2/100, solving three. The tester said Hermes stopped during task seven after broad filesystem searches and looping, while DeepSeek was not run because of previous harness failures and low scores.

In-depth insight

The report offers a concrete, if unverified, signal about how the same quantized model performs across different AI agent harnesses.

Unsloth Flint Q3 Results Performed quite poorly on benchmark using 3 diff harnesses: OMP: Completed all 12 tasks; 71.2/100 (only 7 solved). AtomicAgent: Completed all 12 tasks; 39.2/100 (only 3 solved). Hermes: Stopped during Task 7 after it performed needless broad searches across the filesystem and looped. Deepseek: Due to previous harness failures and low scores, not run. Full leaderboard https://shortbench.vercel.app/ Current models with 100/100 Score submitted by /u/mesolimbic-bliss [link] [comments]

#Unsloth Flint Q3#OMP#AtomicAgent#Hermes#DeepSeek

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.