NVIDIA Model Optimizer

NVIDIA Model Optimizer is a unified library for state-of-the-art model optimization techniques including quantization, pruning, NAS, distillation, and speculative decoding. It supports Hugging Face, PyTorch, and ONNX models, and exports optimized checkpoints to TensorRT-LLM, TensorRT, vLLM, and SGLang. Recent results show up to 1.30x vLLM throughput and 3.1x smaller checkpoints for Qwen3.6-35B-A3B with NVFP4 quantization and QAD.

for who

AI engineers and ML practitioners who optimize and deploy large models for inference.

why now

New NVIDIA Model Optimizer tutorial shows 1.3x vLLM throughput with W4A4 NVFP4 quantization.

what changes

They can apply advanced optimization techniques with simple Python APIs and get deployment-ready quantized checkpoints, dramatically improving inference efficiency and reducing model size.

to do

Use the Model Optimizer library to quantize, prune, or distill models and export to TensorRT-LLM or vLLM for production inference.