Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit user reports 20–30% throughput gain from disabling MTP

AI summary

A Reddit user says disabling MTP improved token throughput by 20–30% in tests involving GyroStrata and Strata. The tester used a GeForce RTX 3060 with 32GB of RAM and reports that the issue may arise when a model is large relative to available memory; these results remain anecdotal.

In-depth insight

Local LLM users running large models on limited hardware may want to test whether disabling MTP improves throughput, but the reported gain still needs independent verification.

Little reminder, with all my testing I found out that mtp can decrease performance when using very big model, since many are starting to use strata or similar and I see many app activating it as default try and disable it, in my case, testing trough gyrostrata and strata with a 3060 and 32gb of ram both with gyro-m and swift 1.5 I got 20-30% improvement in tok/s just by disabling mtp.

#GyroStrata#Strata#gyro-m#Swift 1.5#GeForce RTX 3060

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.