Sorami
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Stop buttons only halt LLM output when the app actually closes the connection
In a post on r/OpenAI, Sorami reported a single-lab test of self-hosted vLLM and SGLang engines plus two AI gateways: generation stopped immediately with no extra tokens only when the app actually hung up the connection, while an app that merely stopped displaying output but stayed connected kept the model writing in the background and consuming unseen GPU time. The post also says that when an app timed out and retried without closing the first connection, both requests kept running and answers delivered on time nearly halved. The authors recommend that stop buttons and retry logic close the connection rather than just hide output, and note they did not test OpenAI's own hosted service.
r/OpenAI · 🔥 0
Experience and discussion from the community
Share my Sorami experienceAsk about Sorami
Nobody has shared their experience with Sorami yet.