Tesla P40
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Ornith 1.5 tuned for long-context inference on one Tesla P40
Developer sargetun123 reported running Ornith 1.5 35B-A3B on a single 24GB Tesla P40 using a custom llama.cpp fork, Q4_0 weights, q8_0 KV cache and MTP drafting. The reported setup supports a full 262k context and reached about 60.8 tokens/s decode without MTP at zero context and 29.1 tokens/s at 100k context, while a real server with MTP generated 35–39 tokens/s for a 101k-token prompt. The developer also reported that disabling ECC freed 1.5GB of VRAM and improved decoding by about 9%.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Tesla P40 experienceAsk about Tesla P40
Nobody has shared their experience with Tesla P40 yet.