AssemblyAI用EC2 Spot降低语音推理成本
AWS Machine Learning Blog介绍了AssemblyAI的生产级语音转文字推理架构,该架构结合Amazon ECS、EC2 Spot与按需容量、多种NVIDIA GPU实例家族以及Amazon SQS重试机制。该设计支持区域数据驻留要求,并能吸收Spot实例中断;AssemblyAI称,相比按需定价,其推理计算成本降低了70%。
该架构为运行可中断且可独立重试的AI推理任务提供了一种生产实践,同时保持服务可用性和延迟承诺。
Speech-to-text inference is computationally intensive. Each audio file triggers a pipeline of deep learning models (acoustic models, language models, and post-processing stages) running on NVIDIA GPUs to meet latency requirements. At scale, this means thousands of GPU instances running continuously.
AssemblyAI needed a way to run GPU-intensive inference workloads cost-effectively at scale, without sacrificing the availability and latency guarantees their customers depend on. Running entirely on On-Demand Instances would be cost-prohibitive. Spot Instances offer significant savings, while Amazon Elastic Compute Cloud (Amazon EC2) can reclaim them with a 2-minute warning when capacity is needed elsewhere. AssemblyAI needed an architecture that absorbs Spot interruptions gracefully while maintaining service level agreement (SLA) commitments. Additionally, customers in regulated industries, including healthcare, finance, and legal, must keep audio data within specific geographic boundaries. They implement strict data residency configurations in European AWS Regions.
Available on AWS Marketplace, AssemblyAI provides an API for transcription, speaker detection, sentiment analysis, chapter detection, and personally identifiable information (PII) redaction. Their models transcribe and analyze speech for high-volume workloads across industries including technology, media, healthcare, and financial services.
#AssemblyAI#Amazon Web Services#AssemblyAI Speech-to-Text API#Amazon ECS#Amazon EC2 Spot Instances#Amazon SQS#Amazon EventBridge#AWS Lambda#Amazon CloudWatch#AWS Marketplace#NVIDIA#Amazon EC2 GPU instances