AssemblyAI cuts speech-to-text GPU inference costs with EC2 Spot
AWS Machine Learning Blog details AssemblyAI’s production speech-to-text inference architecture, combining Amazon ECS, EC2 Spot and On-Demand capacity across multiple NVIDIA GPU families, and Amazon SQS retries. The design supports regional data-residency requirements and absorbs Spot interruptions; AssemblyAI reports that it reduced inference-compute costs by 70% compared with On-Demand pricing.
The architecture offers a documented production pattern for running interruptible, independently retryable AI inference workloads while preserving service availability and latency commitments.
Speech-to-text inference is computationally intensive. Each audio file triggers a pipeline of deep learning models (acoustic models, language models, and post-processing stages) running on NVIDIA GPUs to meet latency requirements. At scale, this means thousands of GPU instances running continuously.
AssemblyAI needed a way to run GPU-intensive inference workloads cost-effectively at scale, without sacrificing the availability and latency guarantees their customers depend on. Running entirely on On-Demand Instances would be cost-prohibitive. Spot Instances offer significant savings, while Amazon Elastic Compute Cloud (Amazon EC2) can reclaim them with a 2-minute warning when capacity is needed elsewhere. AssemblyAI needed an architecture that absorbs Spot interruptions gracefully while maintaining service level agreement (SLA) commitments. Additionally, customers in regulated industries, including healthcare, finance, and legal, must keep audio data within specific geographic boundaries. They implement strict data residency configurations in European AWS Regions.
Available on AWS Marketplace, AssemblyAI provides an API for transcription, speaker detection, sentiment analysis, chapter detection, and personally identifiable information (PII) redaction. Their models transcribe and analyze speech for high-volume workloads across industries including technology, media, healthcare, and financial services.
#AssemblyAI#Amazon Web Services#AssemblyAI Speech-to-Text API#Amazon ECS#Amazon EC2 Spot Instances#Amazon SQS#Amazon EventBridge#AWS Lambda#Amazon CloudWatch#AWS Marketplace#NVIDIA#Amazon EC2 GPU instances