Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

ai-infra-book

ai-infra-book: An open-source book that derives LLM inference and training system design through quantitative analysis of hardware constraints and model architectures.

What problem it solves

Developers often need to understand why model applications are slow or costly, but system decisions require reasoning across model architecture, hardware limits, memory, computation, communication, and scheduling. Easy-to-miss factors such as KV cache, bandwidth limits, and inter-device communication can shift estimates by orders of magnitude, so system design should begin from explicit physical constraints.

Key features

  • Constraint-driven system analysis — The book derives system designs by first defining task and quality requirements, then enumerating computation, storage, communication, and dependencies against hardware capacity, bandwidth, and compute limits.

    Source: README

  • Broad AI infrastructure coverage — Twelve chapters connect model architecture and workloads with accelerator design, operators and runtimes, supernodes, data-center networking, inference and training systems, resource scheduling, and edge-cloud collaboration.

    Source: README, manuscripts, case-studies

  • Reproducible calculation tools — A Python CLI can reproduce calculations from the book and estimate resource requirements with alternative models and inputs. Static calculations use only the Python 3.10+ standard library and require neither a GPU nor model weights.

    Source: README, calculations

  • Experiments with documented conditions — Chapter-organized experiments include run instructions, input conditions, and result explanations. GPU-dependent work identifies its hardware and dependencies, while existing records can be reviewed without the required device.

    Source: README, experiments

  • Multi-format and multilingual editions — The manuscript is available through an online site, PDF, and EPUB, with community translations in English, Traditional Chinese, and Russian. The repository also includes scripts for local website and book builds.

    Source: README, website

Quick start

Copied verbatim from the project's README.

git clone https://github.com/bojieli/ai-infra-book.git
cd ai-infra-book

# 查看模型支持情况
python3 calculations/calc.py models

# 估算 Qwen3-8B 在 8192-token prefill 下的逐算子资源需求
python3 calculations/calc.py forward --model qwen3-8b --tokens 8192 --format md
python3 -m venv .venv-site
source .venv-site/bin/activate
python -m pip install -r website/requirements.txt
python scripts/build_site.py
python scripts/check_site.py
python scripts/build_site.py --serve

Who it may suit

This may suit developers who have called model APIs or run models locally and want to understand latency and cost, as well as systems, networking, and chip engineers, researchers, and students working on related problems.

Topics to discuss

  • Which hardware constraint most often invalidates quantitative estimates for LLM inference and training?

  • How should readers reproduce and document experiments when model versions, hardware, and inputs can affect results?

  • Which missing mechanism or tradeoff should receive deeper coverage in the manuscript?


Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.

Read the original README https://github.com/bojieli/ai-infra-book#readme

ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.

github.comhttps://github.com/bojieli/ai-infra-book

Comments

…