Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

AWS publishes deployable Bedrock invocation data lake

AI summary

AWS describes a deployable reference implementation that centralizes Amazon Bedrock model invocation logs from linked accounts into a queryable table. The architecture uses Amazon CloudWatch, Amazon Data Firehose, Amazon S3, AWS Glue and Amazon Athena. According to AWS, the processed data includes prompts, model responses, input and output token counts, model usage and stop reasons, while a separate query derives per-invocation cost. AWS provides two CloudFormation templates, Lambda code and SQL queries, along with architecture details, deployment steps and an end-to-end test script.

In-depth insight

The implementation gives platform and machine learning teams a way to inspect Bedrock invocations across accounts, analyze token use and model selection, and connect invocation data with cost information. AWS says its infrastructure-as-code resources can be deployed repeatedly across an organization.

If you need an observability data lake for your Amazon Bedrock workloads, you’re not alone. When you’re running thousands of model invocations a day across multiple accounts, after a request is sent you might be wondering, what happened? You can see the aggregate bill, but you can’t see what happened inside each call.

You might start to ask questions: what prompt was sent, what the model returned, how many tokens it used, which model handled it, and whether the response was any good! If you’re a platform or machine learning (ML) engineer looking after a fleet of Amazon Bedrock workloads across teams, that’s a real problem. You can’t debug a spike, you can’t spot wasteful prompts, and you can’t tell one team’s usage from another’s.

We built something to help: a centralized observability data lake for your Amazon Bedrock invocations. In this post, I cover what native data you have and how we designed the lake. I also show how to use it to see your prompts and derive the per-invocation cost. The AWS CloudFormation templates, AWS Lambda code, and SQL queries are all in the AWS Sample Bedrock Invocation Data Lake GitHub repository. This post walks through the architecture and deployment steps. The repo has everything you need to deploy it.

#AWS#Amazon Bedrock#Amazon Bedrock Invocation Data Lake#Amazon CloudWatch#Amazon Data Firehose#Amazon S3#Amazon Athena#AWS CloudFormation#AWS Lambda#AWS Glue#AWS KMS#AWS IAM

Original · 10 min readRead the original
Group chat

No comments yet. Start the conversation.