Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit user reports Strata underperformed llama.cpp on dual R9700

AI summary

A Reddit user testing the Strata inference engine with Qwen Next Flash on two Radeon R9700 cards said they reached only Q3 quantization and encountered hangs when experts were offloaded to RAM. The user reported about 60 tokens per second on paper, but HumanEval+ results of 93.9% for 3-bit Flash-Next versus 95.1% for Qwen3.8-27B Q8 under llama.cpp, with nearly twice as many tokens needed. They also questioned whether favorable reports about model-specific inference engines reflect hype or edge cases.

In-depth insight

The post provides a detailed community benchmark and operational account, but it remains a single user's setup and does not establish how Strata performs more generally.

My intention was to find a setup at least equal to qwen 3.8 27b Q8 MTP but with a larger context. Ideally 1M.

I was already running Next Flash in Q2 with llama. Good for conversational use but not for coding. Q4 would run but too slow. I did read many reports here on the good performance of strata with qwen next flash. I figured both are around for long enough to be stable so I gave it a shot.

The results however are very disappointing. All I was able to achieve was moving to Q3. Brining the question, are most of the posts on alternative engines just hype or edge cases? Certainly looks like that to me.

Some details on my try:

Two cards plus experts offloaded to RAM: hangs on every long tasks, whatever settings I tried.

Stable only when every expert fits in VRAM. On 64 GB that means a 3-bit quant.

Faster on paper: ~60 t/s against llama.cpp. But on HumanEval+ the 3-bit Flash-Next scored 93.9% against 95.1% for Qwen3.8-27B Q8 on plain llama.cpp, and it needed nearly twice the tokens to get there. No, advantage.

So for my use, no real-world gain over a mature stack. Instead of one stable engine, you get a single-model engine, its forks, special quants and a pile of environment-variable workarounds.

Is that just Strata being young, or is it systematic? Has anyone actually gotten a lasting real-world benefit from one of these model-specific engines over llama.cpp or vLLM?

submitted by /u/Don_Reuter [link] [comments]

#Strata#llama.cpp#vLLM#Qwen Next Flash#Qwen3.8-27B#AMD Radeon R9700

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.