PaddlePaddle/PaddleNLP

Project overview

PaddleNLP is a development kit for training, compressing, and serving Large and Small Language Models across multiple hardware backends. It provides 4D parallel training strategies, quantization, and an Auto API for model loading, requiring PaddlePaddle as a strict underlying dependency.

Project type

Model Development · Model Runtime · Infrastructure

Deployment

Refer to project documentation

License

Apache-2.0

Best for

• AI engineers and developers needing 4D parallel training, quantization, and multi-hardware adaptation for Large and Small Language Models.

Key capabilities

• Supports 4D parallel training strategies (data parallel, sharded data parallel, tensor parallel, pipeline parallel) and efficient fine-tuning algorithms with zero-padding data streams and FlashMask.
• Provides dynamic insertion and operator fusion for parallel inference acceleration, supporting INT8/FP8/4-bit quantization and speculative decoding.
• Provides an Auto API to load LLM models and tokenizers, supporting extensive model series including LLaMA, Qwen, DeepSeek, and ChatGLM.
• Supports multi-hardware training and inference across Nvidia GPU, Kunlun XPU, Ascend NPU, Enflame GCU, and Hygon DCU.
• Checkpoint storage tool supporting dynamic scaling of machine resources, saving up to 78.5% storage space and accelerating model storage by 95%.

Limitations and risks

• The project requires PaddlePaddle >= 3.0.0rc1 as a strict dependency to function.
• Setup requires Python coding knowledge, the installation of the underlying PaddlePaddle framework, and GPU configuration for model training.

Getting started

• Install paddlenlp and ensure the PaddlePaddle framework is present. The first success path includes running 'pip install --upgrade paddlenlp==3.0.0b4', verifying paddlepaddle >= 3.0.0rc1, and loading a model and tokenizer using the Auto API.

Evidence and sources

• GitHub project description: Easy-to-use and powerful LLM and SLM library with awesome model zoo.
• README: PaddleNLP 现已支持 Qwen3 系列模型
• README: 《飞桨产业级大语言模型开发利器 PaddleNLP 3.0 重磅发布》
• README: PaddleNLP是一款基于飞桨深度学习框架的大语言模型(LLM)开发套件，支持在多种硬件上进行高效的大模型训练、无损压缩以及高性能推理。
• README: 支持纯数据并行策略、分组参数切片的数据并行策略、张量模型并行策略和流水线模型并行策略的4D 高性能训练

Reviewed at: 2026-07-18T00:12:54Z