ai-toolkit is a free, open-source training suite for fine-tuning diffusion models on image, video, and audio generation tasks. It is designed to run on consumer-grade hardware via a web interface or command line.
Project overview
The project addresses the need for an all-in-one, consumer-hardware-friendly fine-tuning suite for the latest diffusion models, offering both GUI and CLI workflows without requiring managed infrastructure.
Project type
Model Development · Image & Vision · Video · Audio & Speech
Use cases
Image Processing
Deployment
Refer to project documentation
License
MIT
Best for
Creators, developers, and AI engineers who need an all-in-one training suite for diffusion models and have an Nvidia GPU with sufficient RAM.
Key capabilities
Train LoRA models on diffusion models with configurable network parameters.
Train LoKr models by adjusting the network type in the config file.
Train or exclude specific layers using only_if_contains and ignore_if_contains network kwargs.
A web interface to easily start, stop, monitor jobs, and train models with a few clicks.
Automatically saves checkpoints and allows training to resume from the last checkpoint.
Automatically resizes images, handles varying aspect ratios without manual cropping, and supports trigger word replacement in captions.
Run training jobs on Modal cloud infrastructure.
Limitations and risks
WebP image format currently has issues in datasets.
MacOS support is experimental and untested on high RAM Macs.
Pressing ctrl+c while the toolkit is saving a checkpoint will likely corrupt it.
Getting started
Clone the repository, create and activate a Python virtual environment, install PyTorch (torch==2.9.1, torchvision==0.24.1, torchaudio==2.9.1), install requirements, then run a training job with a YAML config file.
Evidence and sources
README: AI Toolkit is an easy to use all in one training suite for diffusion models. I try to support all the latest models on consumer grade hardware. Image and video models. It can be r…