创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

AWS发布可部署的Amazon Bedrock调用数据湖

AI 总结

AWS介绍了一套可部署的参考实现,可将关联账户中的Amazon Bedrock模型调用日志集中到一张可查询的数据表中。该架构使用Amazon CloudWatch、Amazon Data Firehose、Amazon S3、AWS Glue和Amazon Athena。AWS称,处理后的数据包括提示词、模型响应、输入与输出令牌数量、模型使用情况和停止原因,另有查询可计算单次调用成本。AWS提供两个CloudFormation模板、Lambda代码和SQL查询,以及架构说明、部署步骤和端到端测试脚本。

深度见解

这套实现为平台和机器学习团队提供了跨账户检查Bedrock调用、分析令牌用量和模型选择并将调用数据与成本信息关联的方法。AWS称,其基础设施即代码资源可以在组织内重复部署。

If you need an observability data lake for your Amazon Bedrock workloads, you’re not alone. When you’re running thousands of model invocations a day across multiple accounts, after a request is sent you might be wondering, what happened? You can see the aggregate bill, but you can’t see what happened inside each call.

You might start to ask questions: what prompt was sent, what the model returned, how many tokens it used, which model handled it, and whether the response was any good! If you’re a platform or machine learning (ML) engineer looking after a fleet of Amazon Bedrock workloads across teams, that’s a real problem. You can’t debug a spike, you can’t spot wasteful prompts, and you can’t tell one team’s usage from another’s.

We built something to help: a centralized observability data lake for your Amazon Bedrock invocations. In this post, I cover what native data you have and how we designed the lake. I also show how to use it to see your prompts and derive the per-invocation cost. The AWS CloudFormation templates, AWS Lambda code, and SQL queries are all in the AWS Sample Bedrock Invocation Data Lake GitHub repository. This post walks through the architecture and deployment steps. The repo has everything you need to deploy it.

#AWS#Amazon Bedrock#Amazon Bedrock Invocation Data Lake#Amazon CloudWatch#Amazon Data Firehose#Amazon S3#Amazon Athena#AWS CloudFormation#AWS Lambda#AWS Glue#AWS KMS#AWS IAM

原文约 10 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。