Rule-Based Router Sends 16 of 30 Requests to Cheaper Model
A report describes a rule-based router that scores requests by factors including length and technical keywords, then sends them to chat-cheap or chat-strong. In its 30-question evaluation, 16 requests were routed to chat-cheap and 14 to chat-strong, while seven routing unit tests passed. The report also identifies one misclassification and recommends testing whether chat-cheap can correctly handle each class of request before adjusting the rules. The routing distribution indicates potential cost savings, not realized savings, and the results come from a single report.
The report presents a concrete approach to balancing AI cost and quality: explainable routing rules determine which model receives a request, evaluations establish which requests the cheaper model can handle, route-decision monitoring supports recalibration, and failures can be retried with the stronger model.
系列《AI 应用生产化手册》第 16 篇(共 30 篇)|配套开源项目:github.com/ChenYingbo/… 本篇是模块四(成本与网关)第 3 篇:C01 把账算清、C02 把重复的钱省掉,C03 开始动最大的一块——该用便宜模型的别用贵的
一、先看问题:所有问题都花一样的钱
C01 算出了每请求 0.000461 元,C02 让重复问题不再花钱。但账上还剩一个大头没动:
- "1 加 1 等于几"和"设计一个多智能体架构"花一样的钱——所有请求都喂给最强最贵的模型。
- 想省钱又怕降质——知道该用便宜模型,但"哪些问题可以安全交给它"没依据。
- 路由基本靠拍脑袋——"简单问题走便宜模型"听起来对,可什么叫简单?规则怎么定?
解法:动态路由——按问题复杂度把请求分到不同模型,再用评测集验证"便宜模型能安全接管哪些问题"。
核心认知:路由的本质是"成本 × 质量"的取舍,而取舍的依据不是感觉,是评测。先测出便宜模型的"接管清单",再写路由规则——路由和评测是一套闭环。