Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Three Contract Mismatches in AI Agent SSE Streaming

AI summary

A Juejin technical analysis examines recurring SSE failures in AI Agent streaming stacks after connectivity is established. It identifies mismatched cancellation, asynchronous-generator shutdown, and client-termination semantics across application, framework, runtime, and client-library layers, then proposes fixes for disconnects, retries, resumption, graceful shutdown, and access control. The conclusions are based on Starlette 0.49.1+, uvicorn with ASGI spec_version 2.4, and @microsoft/fetch-event-source 2.0.1.

In-depth insight

The analysis shows why an AI Agent can continue producing output and incurring LLM usage after a client disconnects, while offering concrete ways to detect and correct the issue.

上一版文章把断连、续传、重试、优雅关闭的修复清单列了出来。这一版不增加新条目,只做一件事:把每条修复背后的“契约假设”说清楚。因为这些问题之所以反复出现,不是因为某一行代码写错了,而是因为应用层、框架层、运行时层、客户端库,对同一件事持有不一致的假设,而没有任何一层负责仲裁。

三处错配贯穿全文:

  • 服务端取消语义:Starlette 0.49+ 假设 send() 在客户端断开时抛 OSError,uvicorn 的 send() 在断开后直接 return。
  • 生成器关闭语义:应用层假设 aclose() 是同步的、可以随时调用,Python 异步生成器在“running”状态下并发调用 aclose() 会抛 RuntimeError,而这个 RuntimeError 会被 except Exception 捕获,掩盖关闭信号。
  • 客户端终止语义:SSE 规范用 204 表示“不要重连”,fetch-event-source 的默认策略在 204 和 clean close 两个点上都与标准错位。

下面逐个展开。

一、服务端取消:Starlette 的快速路径假设了 uvicorn 不履行的契约

Starlette 0.41.2 的 StreamingResponse.__call__ 无条件运行 stream_response(send) 和 listen_for_disconnect(receive) 的 anyio task group,谁先结束谁取消对方。客户端断开时,listen_for_disconnect 读到 http.disconnect,取消 body task,CancelledError 一路传播进生成器,finally 正常执行。

#Starlette#uvicorn#@microsoft/fetch-event-source#ASGI#SSE

Original · 9 min readRead the original
Group chat

No comments yet. Start the conversation.