Amazon SageMaker AI adds metering for bidirectional streaming
Amazon SageMaker AI introduced usage-based metering for bidirectional streaming inference, allowing model providers to report AWS Marketplace software charges as live sessions accumulate usage. A companion WebSocket carries incremental metering data, while Amazon SageMaker AI endpoint infrastructure charges remain separate. Deepgram is the launch partner, offering metered Speech-to-Text and Text-to-Speech billing, with AWS also providing integration guidance for model-server operators.
The capability addresses a specific billing gap for long-running real-time AI inference by aligning AWS Marketplace software charges with reported session usage rather than endpoint availability time.
Usage-based metering for bidirectional streaming solves a software billing problem for real-time speech AI. Customers deploying streaming models on Amazon SageMaker AI bidirectional streaming pay for the endpoint infrastructure that hosts the model. Separately, model providers listing on AWS Marketplace need a way to charge for what customers consume during a live stream. Today, we’re announcing usage-based metering for Amazon SageMaker bidirectional (bidi) streaming inference, so model providers can report actual usage for the AWS Marketplace software charge. Amazon SageMaker AI endpoint infrastructure charges still apply.
Deepgram, the real-time speech AI company, is the launch partner for this capability, bringing usage-based Speech-to-Text (STT) and Text-to-Speech (TTS) billing to AWS Marketplace. This post builds on a series of innovations between Deepgram and AWS. First, we introduced bidirectional streaming for real-time inference on Amazon SageMaker AI, which supports long-lived bidirectional streams between clients and model servers. Then, Deepgram enhanced Amazon SageMaker AI support with AWS Identity and Access Management (AWS IAM) temporary delegation, simplifying secure authentication for enterprise deployments, and expanded observability with enhanced metrics. Now, with usage-based metering, we further strengthen the foundation for production real-time AI, aligning billing with how customers consume streaming and real-time AI workloads.
#Amazon SageMaker AI#AWS Marketplace#Deepgram#AWS Identity and Access Management