Whitepaper runs Qwen2.5-7B on an 8GB RTX 4060 with vLLM
A version 0.3 whitepaper documents an author-run deployment of Qwen2.5-7B-Instruct-AWQ and Qwen2.5-7B-Instruct-GPTQ-Int4 with vLLM 0.33.0 on a Windows 11 laptop containing an RTX 4060 Laptop GPU with 8GB of memory. Using 4-bit quantization, an 8,192-token context limit, 0.85 GPU-memory utilization and eager mode, the test used about 7.3GB of GPU memory and reported roughly 53-54 tokens per second for a single 256-token request. The paper also reproduces three vLLM issues and publishes logs, scripts and patches, although its results cover one Windows machine and cannot provide same-system comparisons for bf16 or GGUF.
The documented configuration and troubleshooting artifacts offer a concrete, reproducible reference for developers who want to serve a quantized 7B model on consumer hardware with only 8GB of VRAM.
完整白皮书、复现脚本与补丁已开源：github.com/Hayabusa-YL…
版本：v0.3（2026-11-05） 测试机：Windows 笔记本，RTX 4060 Laptop 8GB，24GB 内存 引擎：vLLM 0.33.0，模型 Qwen2.5-7B-Instruct-AWQ / GPTQ-Int4 目标读者：只有一张 8GB 显卡、想本地跑 7B 模型但被 OOM 和报错劝退的开发者 分工说明：选题、硬件环境与验收由作者本人负责；实测执行、排障、数据分析与初稿撰写由本地 AI 智能体（大肥鱼）完成，数据均来自本机真实运行日志。
第 1 章 为什么盯上 8GB 这个数字
RTX 4060 Laptop 8GB 是 2024 年以来最普及的消费级独显之一。几乎所有"本地跑大模型"的教程都以 24GB 卡（4090）为基准写参数，8GB 用户照抄就是一路 OOM。但 8GB 并非跑不了 7B：4-bit 量化后的 Qwen2.5-7B 权重约 5.2GB，剩下的 2.8GB 才决定你能不能跑、能跑多稳。
这本白皮书只回答三个问题：
#vLLM 0.33.0#Qwen2.5-7B-Instruct-AWQ#Qwen2.5-7B-Instruct-GPTQ-Int4#NVIDIA GeForce RTX 4060 Laptop GPU#NVIDIA#FlashInfer#PyTorch#Microsoft Windows 11#Windows Subsystem for Linux