Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit User Reports Qwen Unsloth Q8 Outperforms Q3 in Financial PDF Test

AI summary

A Reddit user described a self-hosted test of four models processing a large set of bank, credit-card, mortgage, brokerage, and retirement PDF statements. The user said Qwen FN Unsloth Q8 extracted all data accurately and produced a polished, insightful frontend, but ran for about eight hours. Qwen FN Q3 produced a nice frontend while extracting PDF data incorrectly and providing basic analytics. Orca nano1 finished in about two hours with mostly correct data but some clear errors and a basic frontend. Maya-S24 reportedly stalled in PDF parsing and validation, and the user stopped waiting after more than 10 hours.

In-depth insight

The comparison offers a first-person account of a demanding local-model task involving financial documents that the user did not want sent to the cloud. The Q8-versus-Q3 difference may help users considering local processing, although the report provides only one user’s test results.

Just a PSA. I ran a realistic self host test on 3 models:

Inb4 : “hurr derr no duh” The point is people on here act like these Q3 versions have minimal loss. Do tests for yourself it’s not even close.

Qwen FN Q3 Qwen FN unsloth Q8 Orca nana-1 Maya GLM flash - Maya-S24

Prompt in comments. Harness : PI Project init layout:

/<my\_bank\_name> (83 monthly pdf statements) /credit_card/<card\_1> (84 pdf monthly pdf statements) /credit_card/<card\_2> (35 pdf monthly statements) /mortgage (24 pdf monthly statements) /brokage_account (105 pdf statements + ccs of all transactions) /retirement/<account> x4 (total 45 pdf statements 4 accounts 4)

Snapshot results (have details runtime/tps etc as well):

#Qwen#Unsloth#Strata#Orca nano-1#Maya-S24

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.