同一本地 Qwen 模型实测 10 个编程智能体
AI 总结
一名 Reddit 用户在配备 64GB 内存的 MacBook Pro M5 Max 上，通过 llama.cpp 使用同一个 Qwen 3.8 27B Q4 模型测试了 10 个编程智能体。在五个带陷阱的任务中，各智能体均通过四个或五个任务，但令牌消耗、完成时间、缓存复用和联网行为差异明显；Pi 的中位令牌消耗最低，Qwen Code 最高。Codex CLI、Cline、OpenCode 和 goose 每次运行都尝试联网，但请求被沙箱拦截。
深度见解
该测试显示，本地编程智能体即使任务通过率相近，也可能在效率与离线行为上存在显著差异，这是本地模型用户选择智能体时可直接比较的指标。
Part 2 of my agent test (part 1 was a 4B model)
Hardware I used: MacBook Pro M5 Max, 64 GB. Qwen 3.8 27B Q4 on llama.cpp, thinking off.
Setup
- same model and server for everyone
- 5 small coding tasks, each with a little trap
- each agent locked in a sandbox: it can only touch its own folder and only talk to the model
- auto-approve on, telemetry off, 5 min limit
Results
| # | Agent | Passed | Median time | Median tokens | Cache reuse | Tried to go online |
|---|---|---|---|---|---|---|
| 1 | Pi | 5/5 | 38 s | 9.2k | 72% | no |
| 2 | DeepSeek Harness | 5/5 | 60 s | 35.3k | 83% | no |
| 3 | Codex CLI | 5/5 | 94 s | 36.3k | 79% | every run |
| 4 | OpenClaw | 5/5 | 106 s | 39.0k | 72% | unclear |
| 5 | Cline | 5/5 | 152 s | 29.6k | 79% | every run |
| 6 | OpenCode | 4/5 | 50 s | 32.4k | 75% | every run |
| 7 | Hermes | 4/5 | 80 s | 64.9k | 93% | no |
| 8 | Atomic Agent | 5/5 | 56 s | 30.4k | 86% | no |
| 9 | goose | 4/5 | 98 s | 20.3k | 76% | every run |
| 10 | Qwen Code | 4/5 | 183 s | 107.9k | 74% | unclear |
#Qwen 3.8 27B#llama.cpp#Pi#DeepSeek Harness#Codex CLI#OpenClaw#Cline#OpenCode#Hermes#Atomic Agent#goose#Qwen Code