datawhalechina/happy-llm

Project overview

A free, open-source tutorial that teaches large language model principles by covering architecture, pre-training, fine-tuning, and applications. It guides learners through building a LLaMA2 model from scratch using PyTorch.

Project type

Infrastructure

Use cases

Learning & Education

Deployment

Refer to project documentation

License

License pending

Best for

• Researchers, developers, and educators seeking a free, open-source tutorial to understand LLM architecture and training implementation.

Key capabilities

• Systematic tutorial covering NLP basics, Transformer architecture, pre-trained language models, and large language model concepts.
• Guides users to build a LLaMA2 model from scratch using PyTorch and implement pre-training and SFT pipelines.
• Covers training practices including pre-training, supervised fine-tuning, and LoRA/QLoRA efficient fine-tuning using industry frameworks.
• Introduces LLM applications including model evaluation, Retrieval-Augmented Generation (RAG), and Agent concepts.
• Collection of community-contributed extended reading materials on specific LLM topics like small model fine-tuning, Transformer design, and text data processing.

Limitations and risks

• The tutorial content and primary language badge indicate a Chinese-focused audience.
• Practical code reproduction may require specific hardware not fully documented in the main README.

Getting started

• Users need Python knowledge to reproduce code, should set up independent Python environments for different chapters, and follow the provided chapter instructions.

Evidence and sources

• GitHub project description: 📚 从零开始构建大模型
• README: 本项目是一个**系统性的 LLM 学习教程**
• README: 想要深入了解大语言模型的原理和训练过程
• README: 🏗️ **动手实现** 一个完整的 LLaMA2 模型
• README: 第五章 动手搭建大模型 | 实现 LLaMA2、训练 Tokenizer、预训练小型 LLM

Reviewed at: 2026-07-17T09:29:36Z