Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Together AI launches Storage Accelerator for GPU clusters

AI summary

Together AI launched Together Storage Accelerator (TSA), a distributed caching layer for Together GPU Clusters that is included with each cluster at no additional cost. TSA caches objects on cluster NVMe after the first read and serves later reads from within the cluster, while supporting existing S3-compatible and Hugging Face Hub workflows without new SDKs or rewritten data loaders. Together AI’s measurements found that warm reads of a 2 GiB Cloudflare R2 object took 0.64–0.72 seconds on an H100 node, compared with 12.7–15.0 seconds without TSA.

In-depth insight

TSA could reduce repeated object-storage traffic, latency and egress costs for AI training and inference clusters, although its published performance comes from the vendor’s own tests.

Your GPUs are only as fast as your data pipelines. Every training run that starts with a 30-minute model download, every node that re-pulls the same dataset from across the country, and every Hugging Face rate limit that stalls a job leads to compute resources sitting idle, waiting for your data, while you are paying for them.

Today we're launching Together Storage Accelerator (TSA), a distributed caching layer for Together GPU Clusters that alleviates the need for repeated data reads from object storage and Hugging Face Hub, dramatically accelerating your AI data pipelines while also cutting your data egress bills. TSA is now included free with every Together GPU Cluster.

The problem: your data lives far from your GPUs

Most AI teams keep their source of truth in object storage (Amazon S3, Cloudflare R2, Together AI's T4*) or pull models directly from Hugging Face Hub. That means every cold start pays a WAN tax: cross-cloud or cross-region reads are slow, expensive (egress from AWS runs up to $0.09/GB), and rate-limited. Multiplied across hundreds of nodes pulling the same checkpoint, the waste compounds and costs you time, money, and failed jobs when the origin throttles you.

#Together AI#Together Storage Accelerator#Together GPU Clusters#ASTRA#T4#Hugging Face Hub#Cloudflare R2#NVIDIA H100

Original · 4 min readRead the original
Group chat

No comments yet. Start the conversation.