Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Amazon SageMaker Inference

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1

  1. Amazon SageMaker Inference adds NVIDIA MIG GPU partitioning for endpoints

    Amazon SageMaker Inference now supports GPU partitioning that uses NVIDIA Multi-Instance GPU (MIG) to split a physical GPU into independent partitions, each with a fixed share of compute units and dedicated memory isolated in hardware. According to the AWS Machine Learning Blog, this lets customers run more models than an instance has physical accelerators and choose a fraction of a newer accelerator instead of a whole GPU. The feature works on inference component endpoints and is available on P4d, P5, G7 and G7e instance families across seven AWS Regions, subject to instance and quota availability; single-model endpoints, serverless endpoints, heterogeneous instance pools and asynchronous invocation are not supported.

    AWS Machine Learning Blog · 🔥 0

Experience and discussion from the community

Share my Amazon SageMaker Inference experienceAsk about Amazon SageMaker Inference

Nobody has shared their experience with Amazon SageMaker Inference yet.