Reddit 用户称关闭 MTP 可提升 20–30% 吞吐
AI 总结
一名 Reddit 用户称,在 GyroStrata 和 Strata 的测试中关闭 MTP 后,每秒 Token 数提升了 20%–30%。测试者使用 32GB 内存的 GeForce RTX 3060,并认为模型大小相对可用内存较高时,MTP 可能降低性能;该结果仍属个人经验。
深度见解
使用有限硬件运行大模型的本地 LLM 用户可测试关闭 MTP 是否能提高吞吐,但所称增幅仍需独立验证。
Little reminder, with all my testing I found out that mtp can decrease performance when using very big model, since many are starting to use strata or similar and I see many app activating it as default try and disable it, in my case, testing trough gyrostrata and strata with a 3060 and 32gb of ram both with gyro-m and swift 1.5 I got 20-30% improvement in tok/s just by disabling mtp.