StartLux open-sources Prism inference engine for decision models
StartLux has open-sourced StartLux-Prism, an inference engine designed for StartLux-Decision. According to StartLux’s tests on one NVIDIA H200, Prism ran 1.2 to 5.3 times faster than the original vLLM setup depending on the workload, while supporting combined tensor and data parallelism and deterministic output in supported configurations. The reported performance figures are vendor-published and vary with request pattern, shared-context reuse, input length, concurrency and hardware configuration.
Prism provides an open-source inference layer specialized for probability-based decision models. The reports identify classification, risk assessment and document review as relevant deployment scenarios, while noting that performance gains are workload-dependent.
最近，两家中国 AI 公司相继开源针对自家模型优化的推理系统。10 月 20 日，DeepSeek 开源面向自家模型的推理框架 DeepInfer；随后，StartLux 也宣布开源为 StartLux-Decision 设计的专用推理引擎 StartLux-Prism。两家公司的技术路线虽然不同，但都将优化重点从模型本身进一步延伸到推理系统。
根据 StartLux 公布的测试数据，在单张 NVIDIA H200 GPU 上，使用 StartLux-Decision v1.1 27B 模型，Prism 在测试场景中的推理速度达到原有 vLLM 方案的 1.2 至 5.3 倍。其中，针对同一份材料回答三个问题时，处理时间从 92.9 毫秒缩短至 17.4 毫秒，达到 5.3 倍加速；短问题场景下则实现了约 2.8 倍加速。
除了性能优化，Prism 还支持张量并行（TP）和数据并行（DP）组合部署。TP 主要用于缩短单个请求的处理时间，DP 则用于提高整体吞吐量，两者可以根据部署需求组合使用。
#StartLux#StartLux-Prism#StartLux-Decision v1.1#vLLM#NVIDIA H200