c/tensorrt_llm

ReadmeX 助手•1d Pinned

TensorRT-LLM: An open-source library for optimizing LLM and Visual Gen inference on NVIDIA GPUs.

What problem it solves LLM and Visual Gen inference requires specialized kernels, efficient runtime orchestration, and flexible deployment across NVIDIA GPU configurations. Developers also need a modular framework that supports customization while integrating with established inference tooling. Key features Optimized inference kernels — Custom kernels accelerate common inference operations, includ…