Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

OpenAI details Habitat’s Azure Cosmos DB scale-up

AI summary

OpenAI says Habitat, the storage platform behind its products, was expanded from a client library and database into a globally distributed system serving more than 500 petabytes of data across 40 regions for over 1 billion users. The company describes co-managed Azure Cosmos DB containers, joint operations with Microsoft Azure after regional outages, a largest account handling more than 20 million QPS, and an in-development exabyte-scale system based on RocksDB Cloud and Raft replication.

In-depth insight

OpenAI’s engineering account provides concrete architectural and operational details about the infrastructure supporting ChatGPT and other products at very large scale.

All OpenAI products require fast, reliable access to data, provided by our storage platform, Habitat. It has to respond in milliseconds to over 1 billion users across 40 geographic regions. Our infrastructure faced an extraordinary challenge: supporting explosive growth and new use cases while continuing to improve reliability, performance, and product developer velocity. Solving that challenge required rethinking how our online storage platform was designed and operated. We constantly evaluated tradeoffs, following the pattern of: first become reliable enough for mission-critical product traffic, then fast enough for global users, and, finally, to deftly operate at massive scale.

This two-part series explains how we developed Habitat from a simple client library and database into a globally distributed storage platform serving over 500 petabytes of data. Part I focused on Habitat’s evolution from a Python client to a service. It also covered the techniques that allowed us to push a Python-based service far beyond what many would consider practical.

In this post, Part II, we walk through how we scaled the underlying storage component: Azure Cosmos DB. Through a close working partnership with the Azure team, we arrived at a viable multi-tenancy approach of co-managed containers, with excellent read performance at massive, global scale.

#OpenAI#Habitat#Azure Cosmos DB#ChatGPT#Microsoft Azure#RocksDB Cloud#Rockset#Raft

Original · 17 min readRead the original
Group chat

No comments yet. Start the conversation.