停止按钮不挂断连接,模型仍在后台生成
Sorami 在 r/OpenAI 发帖称,其在一个实验室环境中对自托管 vLLM 和 SGLang 引擎以及两个 AI 网关做了测试:只有应用真正关闭连接时,模型才立即停止生成,不会产生多余 token;如果应用只是不再显示输出但仍保持连接,模型会继续在后台生成,消耗看不见的 GPU 时间。帖子还称,应用超时后未关闭第一次连接就重试时,两个请求会同时运行,按时交付的回答几乎减半。作者建议停止按钮与重试逻辑应关闭连接而非仅隐藏输出,并说明本次未测试 OpenAI 自家的托管服务。
作者指出,仅隐藏输出的应用可能持续消耗无人看到的 GPU 时间,建议停止按钮与重试逻辑真正关闭连接。
We measured what happens when a user presses stop in an AI app.
The short version: the AI only stopped when the app actually hung up the connection. If the app just stopped showing the answer but stayed connected, the AI kept writing in the background, using GPU time nobody would see.
What we found on the setup we tested:
- When the app hung up, the AI stopped straight away. Nothing extra was generated.
- When the app stayed connected but stopped reading, the AI kept going.
- When the app timed out and retried without hanging up on the first attempt, both kept running. Answers delivered on time nearly halved.