vLLM 首日支持 Recursive Flint 模型
vLLM 表示已从第一天起支持 Recursive 的 Flint，其 NVFP4 与 BF16 两个检查点均可单卡运行。Flint 是 40B 参数的混合专家模型，每个 token 约激活 2.5B 参数，49 层中多数采用滑动窗口注意力，上下文长度为 262K token。vLLM 称单请求在单块 GB200 上、上下文达 100K token 时解码速度约 500 token/s，并称针对 Flint 的专用内核将 NVFP4 解码从 4.81 毫秒降至 1.78 毫秒，单次前向的 kernel 数从 1,096 个减至 346 个。
该文记录了推理框架如何逐项内核优化来适配稀疏、以局部注意力为主的 MoE，包括面向不超过 8 个请求的低延迟解码路径。
vLLM supports Recursive's Flint from day 0. Both its NVFP4 and BF16 checkpoints run on a single GPU, and a single request decodes at about 500 tokens/s on one GB200, even with 100K tokens of context.
Flint is a 40B-parameter MoE that activates only about 2.5B parameters per token, with sliding-window attention in most layers and a 262K-token context. It ships as two separate checkpoints: NVFP4 (quantization-aware trained) and BF16.
Quick start
The day-0 Docker image vllm/vllm-openai:flint (x86_64 and arm64, CUDA 13.0) has Flint support built in:
# NVFP4 checkpoint (GGUF): config and tokenizer come from the BF16 repo
docker run --gpus all --ipc=host -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:flint \
RecursiveSI/flint-nvfp4-gguf \
--hf-config-path RecursiveSI/flint --tokenizer RecursiveSI/flint \
--enable-auto-tool-choice --tool-call-parser glm47 \
--reasoning-parser glm47 --exclude-tools-when-tool-choice-none
# BF16 checkpoint
docker run --gpus all --ipc=host -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:flint \
RecursiveSI/flint \
--enable-auto-tool-choice --tool-call-parser glm47 \
--reasoning-parser glm47 --exclude-tools-when-tool-choice-none