Rivermark基准测试显示调优后的开放模型在设计任务上具竞争力
AI 总结
Rivermark开发者JannVanDam公布了初步测试结果：112个社交与营销设计请求共获得超过4700次盲测成对人工评判。他称，使用Rivermark的DeepSeek v4.1 Flash 1017和GLM 5.3 Flash可匹敌或超过普通Claude Code搭配Sonnet 5.5及/design，而Sonnet 5.5在Rivermark中表现最佳。该设计专用框架针对部分已在生产环境中使用的模型进行了调优，但未针对基准测试案例调优。作者仍在增加评委验证这些发现。
深度见解
报告表明，专门且针对生产使用调优的框架，可能让开放模型以更低成本取得与前沿模型配置相近的设计结果。作者同时表示，随着前沿模型能力提升，这类优势往往会减弱。
| Hey all! I've been working on a custom agentic harness called Rivermark for social, marketing & print design. Whoever has tried to use LLMs (even frontier ones) for this knows that they can be a bit hit or miss, compared to website and app design, where their skills are much better. The benchmark consisted of 112 requests for various social and marketing images that a business might ask an agent for. Some contain a proper brandkit, some have a website reference for the brand, and some start from nothing. Well, the first results, based on 4700+ pairwise blind human judgments, are in! My headline finding, and one I didn't quite expect, is that open models, such as DeepSeek v4.1 Flash 1017 and GLM 5.3 Flash, match or beat plain Claude Code with Sonnet 5.5 and /design. Sonnet 5.5 itself gets the largest lift with Rivermark - it jumps into an impressive lead over everything else. This was also surprising, as I'd expect advantages to be smaller for "smarter" models, as they have this know-how already encoded. I'm writing a blog post and validating the findings with more judges now, but figured I'd share these interesting preliminary results. In the first photo, you can see Bradley-Terry rankings for these model/harness combos. Second photo is the cost vs quality comparison, and in the third photo you can see some representative examples of what models output with Rivermark vs the normal harnesses. Of course, some very important caveats apply: Open to questions! submitted by /u/JannVanDam [link] [comments] |
#Rivermark#DeepSeek v4.1 Flash 1017#GLM 5.3 Flash#Claude Code#Sonnet 5.5