cuDNN
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- LlamaIndex reports nearly 10x faster token generation for OCRia-1 on one B200
LlamaIndex reports that its optimizations raised OCRia-1 token generation from about 285 to 2,800 tokens per second on a single NVIDIA B200 GPU, a vendor-reported result not independently verified here. The stack combines workload-aware mixed-precision quantization, DFlash2 speculative decoding, ReplaySSM, custom kernels and 17 additional optimizations. LlamaIndex says OCRia-1 is available through LlamaParse across all tiers and plans to contribute generally applicable improvements to vLLM.
LlamaIndex Blog · 🔥 0
Experience and discussion from the community
Share my cuDNN experienceAsk about cuDNN
Nobody has shared their experience with cuDNN yet.