ai-infra-book is a free open-source book, available in multi-language PDF, EPUB and web editions across 12 chapters, that teaches quantitative estimation of LLM inference and training system resource needs. It ships with companion Python calculation tools that recompute the book's numbers and work with different models and inputs.
Project overview
It derives LLM system design from first principles against hardware capacity, bandwidth and compute constraints, and its calculation tools run with only the Python 3.10+ standard library, requiring no GPU or model weights.
Project type
Infrastructure · Learning Resources
Use cases
Learning & Education
Deployment
Refer to project documentation
License
Apache-2.0
Best for
AI engineers, researchers and developers who want to learn quantitative estimation of LLM inference and training resource needs from hardware constraints and model architecture.
Readers who want numbers they can recompute themselves with a Python tool, including with their own models and inputs, without a GPU or model weights.
Key capabilities
A Python tool (calculations/calc.py) that recomputes the book's numbers and can estimate resource requirements with a different set of models and inputs; static calculation needs only the Python 3.10+ standard library, with no GPU or model weights.
Limitations and risks
The manuscript is still a first draft under continuous revision; expect content to change.
Building the full-book PDF yourself additionally requires Pandoc, XeLaTeX and fonts, beyond the Python-only requirements of the calculation tools.
Papers and large experiment records (~20 GB) are stored via Git LFS, so full cloning involves substantial data.
Getting started
Read online at bojieli.github.io/ai-infra-book or download the PDF/EPUB from Releases; reading requires no setup.
Alternatively, git clone the repository and run python3 calculations/calc.py models; static calculation needs only Python 3.10+ standard library, with no GPU or model weights.