Troubleshooting 10 Common vLLM Errors
An article documents 10 common vLLM startup and inference problems, with reproduced logs and suggested fixes. Cases include invalid model paths, context length above the model limit, insufficient KV-cache memory, repeated engine initialization in one process, unsupported dtypes, wrapped engine initialization errors, abnormal output or hangs associated with the FlashInfer sampler, occupied ports, multimodal image limits, and delays during CUDA graph capture. The author reports testing with vLLM 0.30.0, torch 2.13.0+cu130, an RTX 5070 Ti 16GB and Qwen2.5-VL-7B-Instruct-AWQ. Suggested steps include checking model paths, adjusting context and memory settings, using one engine configuration per process, disabling the FlashInfer sampler when testing abnormal output or hangs, changing occupied ports, adjusting multimodal limits, and using enforce_eager during tuning. The report cautions that error wording and default behavior may change across vLLM versions.
The article gives model-serving engineers concrete diagnostic checks and configuration changes for recurring vLLM failures, while tying its observations to a specific software, model and GPU test environment.
速查表
| 报错关键词 | 真实原因 | 解法 |
|---|---|---|
Repo id must be in the form 'repo_name' | 模型路径不存在/写错 | 用绝对路径,先 os.path.isdir() 检查 |
User-specified max_model_len (X) is greater than the derived max_model_len | 上下文超过模型上限 | 把 max_model_len 调到上限内 |
No available memory for the cache blocks | 显存不够给 KV cache | 提高 gpu_memory_utilization 或降上下文 |
| 同样报上一条,但你没有改配置 | 同进程开了两个引擎 | 一个配置一个进程 |
ValidationError ... dtype | dtype 写了不支持的值 | 只用 auto/half/float16/bfloat16/float32 |
Engine core initialization failed | 引擎子进程崩了 | 往上看第一条真实错误 |
| 推理结果异常/卡死 | flashinfer sampler 兼容问题 | VLLM_USE_FLASHINFER_SAMPLER=0 |
Address already in use | 端口被占 | 换端口或杀掉旧进程 |
| 图片数超过限制时报错 | limit_mm_per_prompt 设太小 | 调大或按需传图 |
| 启动卡在 CUDA graph capture | 编译优化耗时 | 调参阶段用 enforce_eager=True |
#vLLM#torch#RTX 5070 Ti#Qwen2.5-VL-7B-Instruct-AWQ#FlashInfer#CUDA