TensorRT

Headlines · 3

1. 1d
   Run Nemotron-3.5 Lightning on DGX Spark with TensorRT Edge-LLM

   An NVIDIA Technical Blog tutorial shows how to run the text-only Nemotron-3.5 Lightning model on a 64GB DGX Spark using TensorRT Edge-LLM. The trtedge command-line interface downloads the model and builds inference engines, while the workflow covers terminal chat testing and serving through a local OpenAI-compatible API. The tutorial also demonstrates optional DSpark speculative decoding, which uses a draft model to propose tokens for the main model to verify. NVIDIA reports higher mean generation speeds with DSpark on its SpeedBench 1K, SpeedBench 2K and HumanEval prompt workloads. The measurements used DGX OS 7.6.0, NVFP4, batch size one, one concurrent streaming request and a 128-token output limit.

   NVIDIA Technical Blog · 🔥 5
2. Oct 23
   NVIDIA details TensorRT Global Performance Tuner

   NVIDIA Technical Blog explains the Global Performance Tuner, an automatic performance-tuning feature released in TensorRT 11.1. The tuner searches customizable compiler build routes, validates candidates against an accuracy tolerance, measures end-to-end inference performance, and selects the fastest feasible solution. NVIDIA says this approach addresses model- and deployment-dependent scheduling effects that individual kernel benchmarks cannot reveal.

   NVIDIA Technical Blog · 🔥 0
3. Oct 23
   DAVA’s Scenes AI Wins NVIDIA Showcase Award

   量子位 reported that DAVA Technology’s Scenes AI won the Outstanding Enterprise Award at the 2026 NVIDIA Startup Showcase semifinal event in Suzhou on October 15. The company demonstrated its 34B real-time world model running on a single GeForce RTX 5090, with claimed 720P output at 25 fps. DAVA also reported a 98.2% target-edit success rate, 93.3% character-identity retention, and inference speed three to five times faster than traditional video-diffusion approaches.

   量子位（原生 RSS） · 🔥 0

Experience and discussion from the community

Share my TensorRT experienceAsk about TensorRT