ModelAnalysis.si 推出模型成本与显存工具
开发者 Zack Ku 发布了免费面板 ModelAnalysis.si，用于比较大模型基准测试、API Token 价格、显存需求和自托管成本。该工具支持并排对比、雷达图、API 支出模拟、云 GPU 盈亏平衡计算，以及生成 Ollama 和 vLLM 部署命令。 Ku 称网站涵盖 250 多个模型，后一帖则称覆盖 280 多个模型；这些覆盖范围与准确性说法尚未得到独立验证。
ModelAnalysis.si 将模型质量、推理成本和硬件需求集中到同一工具中，但其基准、价格和模型覆盖数据仍主要来自开发者自述。
Hey everyone,
Every time a new model drops (like DeepSeek-R1 or the recent Kimi k3.1 Pro), finding reliable data usually means jumping between three or four places: checking leaderboard scores, digging up actual API pricing (inputs vs. outputs vs. caching), and guessing whether an open model will actually fit in VRAM with decent context.
I put together a clean dashboard to keep all of this in one spot: https://modelanalysis.si
Here is what it does:
- Benchmarks comparison: Instead of just standard MMLU, it pulls real-world benchmark data including SWE-bench Verified (actual GitHub issues resolved), LMSYS Arena-Hard, GPQA, and competition math. You can pick any models and overlay them on radar charts or compare them side-by-side.
- Price vs. Performance chart: A scatter plot showing which models deliver the best benchmark scores relative to their per-token API cost.
- VRAM calculator: Select model size, quantization (4-bit up to FP16), and context length to calculate the exact VRAM needed for model weights plus KV cache, along with ready-to-use Ollama and vLLM commands.
- Spend simulator: Drag sliders for monthly tokens and prompt cache percentage to project monthly API bills and see where self-hosting becomes cheaper.
It covers 280+ models, runs fast, requires no login, and has no ads or affiliate links.
Link: https://modelanalysis.si Benchmarks lab: https://modelanalysis.si/benchmarks/ Hardware/VRAM tool: https://modelanalysis.si/hardware/
Feedback or suggestions on other benchmarks you'd like to see are welcome.
submitted by /u/zackku [link] [comments]
#ModelAnalysis.si#OpenAI#Anthropic#DeepSeek#Qwen#Mistral#Meta#RunPod#Ollama#vLLM#SWE-bench Verified