NVIDIA Model Optimizer
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- NVIDIA Nemotron 3.5 Super VL arrives in SageMaker JumpStart
AWS announced that NVIDIA Nemotron 3.5 Super VL is available through Amazon SageMaker JumpStart and can be deployed with SageMaker Studio or the SageMaker Python SDK. The vision-language model accepts images, video, and text; JumpStart offers NVFP4 weights for optimized inference and BF16 reference weights, while NVIDIA says the model has a 120B-parameter backbone, 12B active parameters, and support for contexts up to one million tokens.
AWS Machine Learning Blog · 🔥 0 - NVIDIA Model Optimizer Combines Model Optimization Techniques
NVIDIA describes Model Optimizer as an open-source library that combines quantization, pruning, neural architecture search, distillation, speculative decoding and sparsity to accelerate models. It accepts Hugging Face, PyTorch and ONNX models, provides Python APIs for composing optimization techniques and exporting optimized quantized checkpoints, and integrates with Megatron-Bridge, Megatron-LM and Hugging Face Accelerate. The resulting checkpoints can be deployed in SGLang, TensorRT-LLM, TensorRT and vLLM. Its unified Hugging Face export API supports both transformers and diffusers models.
GitHub Trending · Python · 🔥 0
Experience and discussion from the community
Share my NVIDIA Model Optimizer experienceAsk about NVIDIA Model Optimizer
Nobody has shared their experience with NVIDIA Model Optimizer yet.