创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

AWS降低大JSON智能体成本与延迟

AI 总结

AWS Machine Learning Blog介绍了 Flightscape 培训排班优化智能体采用的三种策略:将大型工件卸载至 Amazon S3、对工具结果分页,以及压缩上下文。该智能体可处理超过20 MB的 JSON 负载,并提供40多种工具,用于机组排班、资源分配和业务规则验证。AWS内部使用23 MB负载的基准测试显示,优化后令牌用量最多降低71%,平均延迟最多降低80%。

深度见解

该案例为需要反复处理大型结构化输入、同时避免完整负载不断进入模型上下文的生产级智能体提供了实用架构。

As enterprises build production agents with dozens of tools and large structured inputs, token consumption and latency become the primary blockers to a good user experience. When these agents process large JSON data, every additional tool invocation injects the full payload into the large language model (LLM) context window, causing token usage and response latency to spike. Over multi-turn conversations, this pressure compounds, driving up costs and degrading response quality. Effectively managing context-window pressure in agentic systems is key to reliability and responsiveness at scale. In this post, we share three composable strategies we used to build a production agent for Flightscape that handles 20 MB+ JSON payloads. You can apply them to reduce token costs, lower latency, and maintain full reasoning capability across extended sessions. Together, these optimizations reduced token consumption by up to 71% and cut average latency by up to 80% across our benchmark runs on a representative 23 MB payload.

Solution overview

Flightscape is an airline operations platform that helps airlines plan and manage their crews, flights, and operations. As part of Flightscape’s crew management capabilities, we built a Training Schedule Optimization agent that verifies crew training schedule assignments against business rules. With 40+ tools and JSON inputs in megabytes, this agent could answer complex queries across crew assignments, scheduling, and resource allocation. Although the agent delivered the required functionality, average request latencies exceeded 15 seconds across most calls. Every tool call injected its full payload into the model context, inflating token counts and costs. On larger datasets, the context window filled up before the agent could finish reasoning. The following diagram (Figure 1) details the agentic architecture and data flow:

#Flightscape Training Schedule Optimization agent#Flightscape#Amazon Bedrock AgentCore#Amazon S3#LangGraph#LangChain#Amazon Bedrock Guardrails

原文约 13 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。