magic-animate generates temporally consistent human image animations from a reference image and a motion sequence using a diffusion model. It supports inference on both single and multiple GPUs and offers a local Gradio web interface for interactive use.
Project overview
The project addresses temporally consistent human image animation through a diffusion-based approach, with practical deployment paths including single-GPU and multi-GPU inference as well as an online demo.
Project type
Model Runtime · Image & Vision · Video
Use cases
Image Processing · Video Creation
Deployment
Refer to project documentation
License
BSD-3-Clause
Best for
Researchers and developers working on diffusion-based human image animation who need temporally consistent output and have access to CUDA-compatible GPU hardware.
Key capabilities
Generates temporally consistent human image animations using a diffusion model.
Runs inference to generate animations on a single GPU.
Distributes inference across multiple GPUs.
Launches a local web GUI to run the animation inference.
Provides a hosted online Gradio demo to try the model quickly.
Limitations and risks
The project requires manual downloading of specific pretrained models and checkpoints, including StableDiffusion V1.5, an MSE-finetuned VAE, and magic-animate checkpoints.
The project requires a specific CUDA version (>=11.3) and compatible GPUs to run efficiently.
Getting started
Setup is rated hard. The first success path involves downloading pretrained base models and checkpoints, installing prerequisites (python>=3.8, CUDA>=11.3, ffmpeg), creating a conda environment using environment.yaml, launching the local Gradio demo on a single GPU, and opening the demo in a local browser.
Evidence and sources
GitHub project description: MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
README: Release inference code and gradio demo. We are working to improve MagicAnimate, stay tuned!
README: Download the pretrained base models for StableDiffusion V1.5 and MSE-finetuned VAE. Download our MagicAnimate checkpoints.
README: prerequisites: `python>=3.8`, `CUDA>=11.3`, and `ffmpeg`.
README: Run inference on single GPU: ```bash bash scripts/animate.sh ``` Run inference with multiple GPUs: ```bash bash scripts/animate_dist.sh ```
AI Search
Find projects, verify facts, compare options, or turn a complex need into an actionable plan
Try a searchA click only fills the search box; you stay in control