Verified project record

nndl/llm-beginner

llm-beginner is a progressive, hands-on tutorial that guides learners through six tasks—from implementing a mini-GPT and fine-tuning to building RAG systems and coding agents from scratch. It is designed for beginners with Python and deep learning backgrounds who want to understand and implement Large Language Models.

Project overview

It provides a structured, step-by-step path covering the full LLM stack, taking learners from writing self-attention blocks and training decoder-only models to building functional RAG systems and coding agents, which makes it a practical educational resource.

Project type: AI Agent · AI Coding · RAG
Use cases: Knowledge Q&A · Coding & Development · Learning & Education
Deployment: Refer to project documentation
License: MIT

Best for

Developers, researchers, and educators with Python and deep learning experience seeking a structured, hands-on approach to implementing LLMs and AI agents.
Learners who want to understand the transition from base models to chat models using SFT and DPO alignment.

Key capabilities

Hand-write self-attention and Transformer blocks, run them on small tasks, and visualize attention weights.
Build a decoder-only model from scratch using PyTorch, pre-train it, and integrate RoPE and KV cache.
Perform SFT and DPO alignment on a small pre-trained model to understand the transition from base to chat models.
Build an end-to-end Chinese Retrieval-Augmented Generation system and quantify improvements at each stage.
Implement a ReAct loop to let an LLM autonomously call tools for multi-step tasks.
Replicate a minimal version of an AI coding agent using MCP, Skills, and Subagents to modify code and run tests.

Limitations and risks

Tasks 5 and 6 recommend 16GB+ VRAM, or require Q4_K_M quantization to run on 8GB VRAM.
The tasks depend on downloading large external models and datasets which might require stable network or mirrors.

Getting started

To get started, install Python 3.10 or higher, navigate to the task-1-transformer directory, run pip install -r requirements.txt to install dependencies, and execute python data/download.py to fetch the necessary data and models.

Evidence and sources

README: 本系列是 NLP-Beginner 在大模型时代的重构版本，是一份**独立的入门教程**，面向有 Python 与深度学习基础的学习者。
README: 作为本教程的学习材料，同步撰写了**《大模型与智能体》**，围绕共用基础、大模型、智能体、边界与未来四条主线展开，共 17 章。
README: 手写 self-attention 与 Transformer block，在小任务上跑通并可视化注意力权重。
README: 用 PyTorch 从零搭一个 decoder-only 模型，先在中文小语料上预训练并自回归生成
README: 在小尺寸预训练模型上做 SFT + DPO 两阶段对齐，理解从 base model 到 chat model 的全过程。

Reviewed at: 2026-07-15T15:45:53Z