Amazon SageMaker AI
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 26
- AWS launches managed extensions for SageMaker HyperPod Slurm
AWS Machine Learning Blog says Amazon SageMaker HyperPod Slurm now offers managed cluster extensions for observability and user management. Administrators can configure running-node software and settings through APIs or the console while retaining SSH access to every node, including the controller. The extensions are available in all regions supporting SageMaker HyperPod, with no additional charge beyond the AWS resources they use.
AWS Machine Learning Blog · 🔥 6 - SageMaker AI adds automatic inference endpoint optimization monitoring
Amazon SageMaker AI now monitors generative AI inference endpoints using vLLM containers on GPU instances, detecting week-over-week latency drift, unused GPU capacity, and serving-configuration gaps. Findings are delivered through AWS Health and can be evaluated in Amazon SageMaker Studio using benchmarks that compare response time, throughput, and cost per million tokens without automatically changing production endpoints.
AWS Machine Learning Blog · 🔥 3 - AWS adds cross-Region inference routing to SageMaker AI
AWS announced that cross-Region inference routing is generally available in Amazon SageMaker AI for real-time and asynchronous inference in most commercial AWS Regions. Its inference router provides one invocation path across registered endpoints and inference components in different Regions and accounts, preferring healthy local capacity before shifting new requests and gradually restoring local traffic after recovery. AWS says the routing feature adds no fee beyond standard SageMaker AI and CloudWatch charges, while response streaming is unsupported and some quotas and regional exclusions apply.
AWS Machine Learning Blog · 🔥 3 - PolyAI Dialog-RSN-1 Launches on AWS Marketplace and SageMaker JumpStart
AWS Machine Learning Blog announced that PolyAI Dialog-RSN-1, a reasoning model for real-time complex conversations, is available through AWS Marketplace and Amazon SageMaker JumpStart. Customers can subscribe to the model and deploy it as a SageMaker endpoint within their AWS account or through SageMaker Studio. PolyAI claims the model directly processes caller audio, reasons about caller needs, and responds in real time, though these performance claims are vendor-sourced.
AWS Machine Learning Blog · 🔥 0 - SageMaker AI launches cross-Region inference routing
AWS Machine Learning Blog says Amazon SageMaker AI cross-Region inference routing is generally available as of November 18, 2026, in commercial Regions supporting real-time endpoint inference. Salesforce uses the feature to route requests across its existing endpoints in multiple Regions and accounts, adding serving capacity while preserving data residency commitments. The router does not create capacity or copy models, and operators can cap remote traffic to protect capacity reserved for local requests.
AWS Machine Learning Blog · 🔥 0 - AWS urges teams to question every generative AI model call
AWS Machine Learning Blog offers a decision framework for choosing between deterministic code, constrained decision calls, and appropriately sized Amazon Bedrock models at each workflow step. It says teams often send validation, formatting, and routing tasks to LLMs, adding costs, latency, and output variability at production scale. The guidance also recommends deterministic fast paths, labeled-data evaluation, configuration flags, and continuous integration for model changes.
AWS Machine Learning Blog · 🔥 0 - SageMaker AI adds KV cache-aware routing for real-time endpoints
AWS Machine Learning Blog reports that Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, a new RoutingStrategy option for LLM serving. The feature uses cache state reported by the model container to send requests to instances holding matching cached context while avoiding overly busy instances. Adoption requires the SM_VLLM_KV_EVENTS_CONFIG environment variable, the KV_AWARE routing strategy, and a supported container.
AWS Machine Learning Blog · 🔥 0 - AWS launches AI data preparation tools for Amazon SageMaker AI
AWS announced generally available AI Data Preparation capabilities for Amazon SageMaker AI, spanning auto-labeling, synthetic data generation, quality evaluation, transformation, and human review. Users can invoke the capabilities through SageMaker Python SDK APIs or the AWS AI-ML agent skill, with specifications and generated code available for inspection and modification. The tools are currently available in the US East (N. Virginia) and US West (Oregon) Regions.
AWS Machine Learning Blog · 🔥 0 - OpenFold3-preview now available on SageMaker JumpStart
AWS says the OpenFold3-preview, an open-source all-atom biomolecular structure prediction model developed by the OpenFold Consortium and Columbia University’s AlQuraishi Lab, can now be deployed through Amazon SageMaker JumpStart. It predicts structures for proteins, DNA, RNA, small-molecule ligands and their complexes. AWS describes asynchronous inference, parallel predictions on multi-GPU instances, network isolation, user-supplied multiple sequence alignments and scale-to-zero configurations. The developers describe the model as a research preview.
AWS Machine Learning Blog · 🔥 0 - Amazon SageMaker AI adds metering for bidirectional streaming
Amazon SageMaker AI introduced usage-based metering for bidirectional streaming inference, allowing model providers to report AWS Marketplace software charges as live sessions accumulate usage. A companion WebSocket carries incremental metering data, while Amazon SageMaker AI endpoint infrastructure charges remain separate. Deepgram is the launch partner, offering metered Speech-to-Text and Text-to-Speech billing, with AWS also providing integration guidance for model-server operators.
AWS Machine Learning Blog · 🔥 0 - Amazon details SageMaker AI pipeline for evaluating A+ Content
The AWS Machine Learning Blog describes an Amazon SageMaker AI pipeline that renders Marketplace A+ Content as a single snapshot, scores it with a vision-language model, and combines those scores with OCR readability features through CatBoost. On 5,478 product pages across 10 categories, Amazon reports 0.94 accuracy and 0.95 F1, while SHAP and an LLM turn quality signals into prioritized revision suggestions. The pipeline also uses a diffusion model to generate revised visual assets.
AWS Machine Learning Blog · 🔥 0 - AWS tutorial: distill a shopping agent with sequence-level knowledge distillation
The AWS Machine Learning Blog shows how to build a lower-cost shopping agent on Amazon SageMaker AI using sequence-level knowledge distillation (SeqKD), generating multi-step trajectories with a frontier teacher on Bedrock, keeping only those that solved the task with ShoppingBench's scorer, and fine-tuning a smaller student on serverless infrastructure. The post also covers encoding trajectories as chat-formatted messages, tracking training with MLflow, gating quality with SageMaker AI model evaluation and a custom tool-selection scorer, and serving the student behind an endpoint that runs the tool loop. AWS reports the payoff was a 4B student that recovers essentially all of its frontier teacher's task quality.
AWS Machine Learning Blog · 🔥 0 - AWS adds SOCI indexing for faster SageMaker AI jobs
AWS announced Seekable OCI (SOCI) indexing for Amazon SageMaker AI Training and processing jobs, allowing container images to be lazy-loaded instead of fully downloaded before execution. Users can select pre-indexed AWS Deep Learning Containers or index a custom image, though AWS says benefits vary with image size and startup behavior; its tests found GPU idle time during image pulls fell by up to 90%.
AWS Machine Learning Blog · 🔥 0 - Exowatt builds reusable SageMaker fine-tuning pipeline
AWS Machine Learning Blog reports that Exowatt used Amazon SageMaker AI serverless model customization to create a repeatable pipeline for task-specific fine-tuned models. A fine-tuned Qwen3-1.7B handled offline alarm triage, while a fine-tuned Qwen3-14B supported cloud energy-market analysis; AWS claims higher accuracy than frontier models at up to 49× lower cost, though those results were not independently verified here. The pipeline turns operational data into a certified training set, evaluated model and deployment artifact, reducing Exowatt's reported workflow from months to days.
AWS Machine Learning Blog · 🔥 0 - AWS Adds a Hands-On Multi-Turn RL Guide for SageMaker AI
AWS Machine Learning Blog published an end-to-end guide to multi-turn reinforcement learning (MTRL) on Amazon SageMaker AI, covering agent deployment, dataset preparation, training, MLflow monitoring and evaluation at serving settings. The workflow trains a policy using a reward for the outcome of an entire multi-step episode, without requiring the correct action trajectory, and supports agents hosted on Bedrock AgentCore or customer infrastructure. In AWS’s Wordle example, the company reports that gpt-oss-20b improved from 61% to 84% solve rate at default reasoning effort and from 15% to 69% at its trained low-effort setting, costing about $339 in training tokens.
AWS Machine Learning Blog · 🔥 0 - AWS CDK Construct Simplifies SageMaker AI Endpoint Autoscaling Updates
AWS Machine Learning Blog documents deployment failures when CloudFormation or AWS CDK updates a SageMaker AI real-time endpoint while Application Auto Scaling remains registered. It presents a reusable CDK construct that uses three custom resources to deregister and re-register autoscaling with explicit dependency ordering, allowing endpoint configuration and scaling parameters to be updated in one deployment.
AWS Machine Learning Blog · 🔥 0 - AWS makes SageMaker AI training sessions generally available
AWS announced the general availability of training sessions within Amazon SageMaker AI serverless model customization, letting developers write and coordinate their own post-training loop in Python — including custom loss functions or reward signals — while SageMaker AI handles GPU allocation, distributed execution, fault recovery and checkpoint storage. The launch supports gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B and Qwen3.8-27B, and AWS released a cookbook with ready-to-run SFT and RL recipes. AWS reports that a rank-32 LoRA adapter for GPT-OSS 20B trained for 50 steps on GSM8K rose from 0.46 to 0.60 training reward and from 40% to 50.5% held-out accuracy, figures the vendor supplied without independent replication.
AWS Machine Learning Blog · 🔥 0 - AWS shows sentence-by-sentence TTS streaming for voice agents with Kokoro-82M
An AWS Machine Learning Blog post walks through streaming each sentence an AI agent generates to a hosted Kokoro-82M text-to-speech model on Amazon SageMaker AI, so synthesis starts on sentence one instead of waiting for the finished response. The post frames the problem as a latency gap: a one-sentence response might generate in 600 milliseconds while a full response takes five seconds, during which the customer hears nothing. Kokoro-82M's 82 million parameters run in a custom container on an ml.g5.xlarge real-time endpoint with two routes, a POST streaming route and a bidirectional WebSocket route; AWS also compares returning audio to a browser over Server-Sent Events, where base64 encoding inflated payloads by close to the expected 33 percent and MP3 frames did not align with delivery chunks. AWS reports time to first audio fell by more than half in isolation and by 2.4 to 3.5 times in a deployed agent, and notes the handlers run a strictly sequential loop rather than concurrent synthesis.
AWS Machine Learning Blog · 🔥 0 - AWS shows how to set SageMaker TTS auto scaling targets from live measurements
An AWS Machine Learning Blog post describes deploying Kokoro-82M, an 82-million-parameter non-autoregressive text-to-speech model, as a custom container on an Amazon SageMaker AI real-time endpoint via AWS CDK stacks. Instead of guessing auto scaling target values, the authors measure two numbers against the live endpoint: Q_max, the highest concurrency one instance sustains before breaching a 3,000 ms latency SLO, and T_total, the end-to-end scale-out time. The post reports Q_max of 50 concurrent requests on ml.g5.xlarge (concurrency 60 breached the SLO) and a measured T_total of 260.2 seconds, planned against as 320.2 seconds with bounded policy detection lag, and derives scale-out and scale-in thresholds and cooldowns arithmetically from those values.
AWS Machine Learning Blog · 🔥 0 - AWS lets new SageMaker Studio users sign up with Google, GitHub or Apple accounts
AWS announced that new Amazon SageMaker AI customers can now sign up with an existing Google, GitHub, Apple or Amazon account, or an email address, in two to three steps, and then land in a fully configured SageMaker Studio environment created automatically in about 20 seconds. AWS says most customers no longer need the advanced AWS account sign-up, phone verification or a credit card, and that permissions for model customization, training, evaluation and deployment are pre-configured, with eligible accounts getting default access to G5, G6 and G4dn GPU instances. New customers get $100 in AWS Free Tier credits usable across SageMaker AI, including Studio notebooks, training jobs, model customization and endpoint deployments, with a chance to earn another $100; the Free Tier ends after six months or when credits run out, at which point usage stops automatically. The new sign-in experience applies only to new AWS accounts; existing accounts see no change.
AWS Machine Learning Blog · 🔥 0 - Deepgram ports Nova-3 speech model to AWS Trainium2, reports 11–70% more audio per dollar
Deepgram worked with AWS's Forward Deployed Engineering team to port its Nova-3 speech-to-text model from standard PyTorch code to AWS Trainium2, tuning it for offline batch workloads such as contact-center analytics and medical transcription. The final setup runs eight model replicas on a single Amazon EC2 trn2.3xlarge instance, using TorchNeuron, prebuilt NKILib kernels, hand-written NKI kernels and a Rust data-parallel runtime. According to the AWS blog, that configuration processed 11–70 percent more audio per dollar than the compute options Deepgram had been running on those offline throughput-oriented workloads, measured with instance pricing at evaluation time; no independent verification is cited.
AWS Machine Learning Blog · 🔥 0 - AWS details fused MoE kernel optimizations upstreamed to vLLM
An AWS Machine Learning Blog post describes several kernel optimizations AWS implemented for fused Mixture-of-Experts models running on GPUs. It says a swap-AB matrix transposition aligned with NVIDIA Hopper's WGMMA instruction layout produced a 2–4 percent end-to-end throughput improvement for large MoE models on H200, and was upstreamed to vLLM 0.24.0 or later. A second change added do_not_specialize annotations to eliminate repeated Triton JIT recompilation, which AWS says improved time to first token by 9 percent at 8-way tensor parallelism and is available in vLLM 0.22.0 or later. AWS says the optimizations are open sourced, applied in the LMI deep learning container on Amazon SageMaker AI, and together delivered up to 10 percent higher throughput and up to 9 percent faster TTFT; these figures are vendor-reported and not independently verified.
AWS Machine Learning Blog · 🔥 0 - AWS publishes guide to deploying decision models with llama.cpp on SageMaker AI
The AWS Machine Learning Blog published a tutorial on deploying decision models with the AWS Deep Learning Container for llama.cpp on Amazon SageMaker AI, serving quantized models on CPU or GPU infrastructure. The standard path exposes OpenAI-compatible chat completions through SageMaker AI InvokeEndpoint, while applications needing scores over a fixed set of choices can build a child image via AWS CodeBuild that swaps the default server for a SemIf (formerly OpenJev) adapter returning probabilities over declared options without decoding an answer token. The post includes request and response schemas, CPU-versus-GPU guidance, and advises applying application-level thresholds and fallbacks rather than treating a model decision as authorization.
AWS Machine Learning Blog · 🔥 0 - Qlik Answers scales grounded enterprise AI on Amazon Bedrock
In an AWS blog post, Qlik detailed the architecture behind Qlik Answers, its enterprise question-answering product: employees ask in natural language and get sourced answers drawn from knowledge bases, live analytics apps, glossary definitions or documents, across more than 40,000 customers and 11 Regions. The system is split into entry, routing, answer, specialist-agent, conversational-analytics, retrieval (Amazon OpenSearch Service) and model-access layers; model calls go through Qlik's own LLM gateway to Amazon Bedrock, with Bedrock Guardrails applying content filtering and contextual grounding validation to every request and response, and Amazon SageMaker AI hosting models not yet available in-Region. Qlik says that since February 2026 general availability its Discovery Agent has surfaced over 100,000 discoveries, that most Qlik Cloud accounts with agentic tools enabled keep using them, and that customer Lintech International saw 75% faster responses and up to 7 hours per week returned to managers — all figures reported by Qlik itself.
AWS Machine Learning Blog · 🔥 0 - Best practices for Amazon SageMaker HyperPod administration and governance
An AWS Machine Learning Blog post explains how to administer Amazon SageMaker HyperPod clusters through Amazon SageMaker Unified Studio while preserving underlying governance controls. It lays out four layers of control — organization, project, cluster, and workload — and covers designing identity, capacity, and observability policies across them, plus a "connection contract" record for each approved project-to-cluster connection.
AWS Machine Learning Blog · 🔥 0 - AWS adds SageMaker inference-optimization skill for coding agents
AWS introduced the aws-ai-ml skill through the Agent Toolkit for AWS, giving MCP-compatible coding agents such as Kiro, Claude Code and Codex SageMaker AI inference optimization and benchmarking expertise. The skill load-tests existing endpoints and reports measured throughput, latency percentiles and concurrency, ranks instance types for models stored in S3, in SageMaker JumpStart or on Hugging Face Hub, compares two benchmark runs, and generates runnable SageMaker Python SDK v3 code. It can be installed locally via an npx command or used in a preconfigured image inside a private Amazon SageMaker Studio JupyterLab space.
AWS Machine Learning Blog · 🔥 0
Experience and discussion from the community
Share my Amazon SageMaker AI experienceAsk about Amazon SageMaker AI
Nobody has shared their experience with Amazon SageMaker AI yet.