ai-infra-book: An open-source book that derives LLM inference and training system design through quantitative analysis of hardware constraints and model architectures.
What problem it solves Developers often need to understand why model applications are slow or costly, but system decisions require reasoning across model architecture, hardware limits, memory, computation, communication, and scheduling. Easy-to-miss factors such as KV cache, bandwidth limits, and inter-device communication can shift estimates by orders of magnitude, so system design should begin f…
github.comhttps://github.com/bojieli/ai-infra-book