LlamaFactory is a unified framework for fine-tuning large language models and vision-language models through a zero-code CLI and Web UI. It supports a wide range of training algorithms, quantization methods, and inference backends.
Project overview
The framework addresses the complexity of managing diverse model architectures and hardware by providing zero-code fine-tuning for 100+ models.
Project type
AI Agent · Model Development
Deployment
Refer to project documentation
License
Apache-2.0
Best for
AI engineers, developers, and researchers who need a unified framework for zero-code model customization and deployment via CLI or Web UI.
Users who need to run parameter-efficient fine-tuning on limited hardware, such as 4-bit QLoRA for a 7B model on 6GB VRAM.
Key capabilities
Easily fine-tune 100+ large language models using zero-code CLI and Web UI.
Offers 16-bit full-tuning, freeze-tuning, LoRA, and 2/3/4/5/6/8-bit QLoRA via AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ.
Provides OpenAI-style API, Gradio UI, and CLI with vLLM worker or SGLang worker.
Fine-tune models through a web interface powered by Gradio.
Limitations and risks
Full fine-tuning of large models, such as a 70B model in pure_bf16, requires massive VRAM, specifically 600GB.
Requires manual installation of specific GPU versions of PyTorch and dependencies like bitsandbytes when running on Windows.
Getting started
Installation is rated as easy due to zero-code fine-tuning available through the Web UI and CLI, alongside pre-built Docker images. The first success path involves installing via pip or Docker, then running the llamafactory-cli webui command.
Alternatives and comparisons
Provides a unified framework for scaling Python and AI applications from a laptop to a cluster.
Provides examples, best practices, and reusable Python utilities for building, evaluating, and operationalizing recommendation algorithms.