Microsoft Windows 11
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Whitepaper runs Qwen2.5-7B on an 8GB RTX 4060 with vLLM
A version 0.3 whitepaper documents an author-run deployment of Qwen2.5-7B-Instruct-AWQ and Qwen2.5-7B-Instruct-GPTQ-Int4 with vLLM 0.33.0 on a Windows 11 laptop containing an RTX 4060 Laptop GPU with 8GB of memory. Using 4-bit quantization, an 8,192-token context limit, 0.85 GPU-memory utilization and eager mode, the test used about 7.3GB of GPU memory and reported roughly 53-54 tokens per second for a single 256-token request. The paper also reproduces three vLLM issues and publishes logs, scripts and patches, although its results cover one Windows machine and cannot provide same-system comparisons for bf16 or GGUF.
掘金 · 人工智能 · 🔥 0
Experience and discussion from the community
Share my Microsoft Windows 11 experienceAsk about Microsoft Windows 11
Nobody has shared their experience with Microsoft Windows 11 yet.