Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Amazon SageMaker AI adds metering for bidirectional streaming

AI summary

Amazon SageMaker AI introduced usage-based metering for bidirectional streaming inference, allowing model providers to report AWS Marketplace software charges as live sessions accumulate usage. A companion WebSocket carries incremental metering data, while Amazon SageMaker AI endpoint infrastructure charges remain separate. Deepgram is the launch partner, offering metered Speech-to-Text and Text-to-Speech billing, with AWS also providing integration guidance for model-server operators.

In-depth insight

The capability addresses a specific billing gap for long-running real-time AI inference by aligning AWS Marketplace software charges with reported session usage rather than endpoint availability time.

Usage-based metering for bidirectional streaming solves a software billing problem for real-time speech AI. Customers deploying streaming models on Amazon SageMaker AI bidirectional streaming pay for the endpoint infrastructure that hosts the model. Separately, model providers listing on AWS Marketplace need a way to charge for what customers consume during a live stream. Today, we’re announcing usage-based metering for Amazon SageMaker bidirectional (bidi) streaming inference, so model providers can report actual usage for the AWS Marketplace software charge. Amazon SageMaker AI endpoint infrastructure charges still apply.

Deepgram, the real-time speech AI company, is the launch partner for this capability, bringing usage-based Speech-to-Text (STT) and Text-to-Speech (TTS) billing to AWS Marketplace. This post builds on a series of innovations between Deepgram and AWS. First, we introduced bidirectional streaming for real-time inference on Amazon SageMaker AI, which supports long-lived bidirectional streams between clients and model servers. Then, Deepgram enhanced Amazon SageMaker AI support with AWS Identity and Access Management (AWS IAM) temporary delegation, simplifying secure authentication for enterprise deployments, and expanded observability with enhanced metrics. Now, with usage-based metering, we further strengthen the foundation for production real-time AI, aligning billing with how customers consume streaming and real-time AI workloads.

#Amazon SageMaker AI#AWS Marketplace#Deepgram#AWS Identity and Access Management

Original · 10 min readRead the original
Group chat

No comments yet. Start the conversation.