[AI Kuaishou LivePortrait] Leading the New Era of Efficient Portrait Animation

Kuaishou has launched LivePortrait, featuring efficient portrait animation with stitching and redirection control.

Quick Start

Download the code and prepare the environment

1
2
3
4
5
6
7
8


git clone https://github.com/KwaiVGI/LivePortrait
cd LivePortrait

# create env using conda
conda create -n LivePortrait python==3.9.18
conda activate LivePortrait
# install dependencies with pip
pip install -r requirements.txt

Download pre-trained weights

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


pretrained_weights
├── insightface
│   └── models
│       └── buffalo_l
│           ├── 2d106det.onnx
│           └── det_10g.onnx
└── liveportrait
    ├── base_models
    │   ├── appearance_feature_extractor.pth
    │   ├── motion_extractor.pth
    │   ├── spade_generator.pth
    │   └── warping_module.pth
    ├── landmark.onnx
    └── retargeting_models
        └── stitching_retargeting_module.pth

Inference

1
2
3
4


python inference.py

# specify source image and driving video
python inference.py -s assets/examples/source/s9.jpg -d assets/examples/driving/d0.mp4

Gradio Interface

1

python app.py

Inference Speed Evaluation

1

python speed.py

Examples

Tried it with an image

dutch-girl

The generated videos are as follows

The results look great! And the weights are quite small, around 600MB. Kuaishou is impressive.

FAQ

ImportError: libGL.so.1: cannot open shared object file: No such file or directory

apt-get update && apt-get install ffmpeg libsm6 libxext6 -y