first-order-model animates a still source image by transferring motion from a driving video, providing source code and pre-trained checkpoints for image animation and motion retargeting across various datasets.
Project overview
It offers a practical, self-hostable approach to motion transfer and image animation without requiring object-specific prior knowledge, with support for both absolute and relative keypoint-based animation.
Project type
Model Runtime · Image & Vision · Video
Use cases
Video Creation
Deployment
Refer to project documentation
License
MIT
Best for
Researchers and developers who need image animation and motion retargeting with access to source code and pre-trained checkpoints.
Creators who want to animate a still source image by transferring motion from a driving video.
Key capabilities
Animates a source image using the motion transferred from a driving video, with options for absolute or relative keypoint locations.
Allows training the motion model on specific datasets or custom user-provided video data.
Evaluates reconstruction performance and computes metrics using generated video outputs.
Generates semi-automatic crop suggestions for driving videos using ffmpeg and the face-alignment library.
Limitations and risks
Driving videos and source images should be cropped before being used in the method, though semi-automatic crop suggestions can be generated using ffmpeg and the face-alignment library.
The authors point out an improved framework for articulated objects in a new paper, which may be more suitable for certain use cases.
Getting started
Install Python dependencies via pip or set up Docker, download pre-trained checkpoints from Google Drive or Yandex Disk, prepare a cropped driving video and source image, and run the demo.py command.
Evidence and sources
README: !!! Check out our new paper and framework improved for articulated objects
README: This repository contains the source code for the paper First Order Motion Model for Image Animation
README: The bottom row contains the animated sequences with motion transferred from the driving video and object taken from the source image.
README: Checkpoints can be found under following link: google-drive or yandex-disk.
README: We support python3. To install the dependencies run: pip install -r requirements.txt