jingyaogong/minimind

Project overview

An open-source educational project that enables individuals to train and understand a 64M-parameter large language model from scratch in approximately 2 hours. It provides a full PyTorch-based training pipeline, model architecture code, open-source datasets, and OpenAI-compatible serving tools.

Project type

Model Development · Model Runtime

Use cases

Learning & Education

Deployment

Refer to project documentation

License

Apache-2.0

Best for

• Developers and educators seeking to understand and implement the complete LLM training lifecycle using native PyTorch.

Key capabilities

• Covers the complete training process including Pretrain, SFT, LoRA, RLHF-DPO, RLAIF, Tool Use, Agentic RL, and model distillation, implemented from scratch in PyTorch.
• Provides complete LLM structure code for Dense and MoE models, aligned with the Qwen3/Qwen3-MoE ecosystem, including RoPE and YaRN scaling.
• Provides a minimal server compatible with OpenAI API protocols, supporting reasoning_content, tool_calls, and open_thinking for integration with third-party UIs.
• Provides a minimal Streamlit-based web chat UI supporting thought display, tool selection, and multi-turn tool calls.
• Provides full-stage open-source data, covering collection, distillation, cleaning, and deduplication of high-quality datasets.

Limitations and risks

• The trained models have limited factual knowledge and generalization capabilities due to their small scale.

Getting started

• Setup is rated medium difficulty and requires GPU familiarity and command-line execution. To run inference for the first time, clone the repository, install dependencies, download desired pre-trained model weights such as minimind-3, and run CLI inference using eval_llm.py.

Evidence and sources

• GitHub project description: 🧠「大模型」2小时完全从0训练64M的小参数LLM！Train a 64M-parameter LLM from scratch in just 2h!
• README: 这不仅是一个大语言模型全阶段开源复现项目，也是一套面向 LLM 入门与实践的教程。
• README: 这更像是在教牛顿如何使用 21 世纪的智能手机——虽然有趣，却偏离了理解物理本质的初衷。
• README: 希望此项目能为更多人提供一个可复现、可理解、可扩展的起点，一起感受创造的乐趣，并推动更广泛 AI 社区的进步。
• README: 提供全阶段开源数据，覆盖收集、蒸馏、清洗与去重后的高质量数据集。

Reviewed at: 2026-07-17T06:35:22Z