LivePortrait

LivePortrait is the KwaiVGI (Kuaishou) team's open-source portrait animation project, first released in July 2024, licensed under Apache-2.0 (commercial-use OK), and written primarily in Python (PyTorch). Input a source portrait + a driving video (expression / head-motion reference); output a high-fidelity portrait animation. Significantly outperforms previous-generation methods like SadTalker and First Order Motion in facial expression transfer accuracy, natural head pose, and gaze direction detail. Mature ComfyUI node ecosystem.

Core capabilities center on high-fidelity expression / pose transfer + practical deployment. Architecture: three-stage — implicit keypoint extraction + MLP mixed deformation + appearance feature extraction — first extracts keypoints and appearance from source and driving frames, then MLP predicts motion fields for each keypoint, finally a decoder generates target frames. Input: source portrait (single JPG/PNG) + driving video (any person expression/motion reference) + optional editing templates (eye-close / mouth-open / head-pose); supports animal portraits (animal model) and stylization extensions. Output: MP4 video at 25-30 FPS; optional 512x512 / HD tiers. Ecosystem: ComfyUI-LivePortraitKJ node; Gradio Demo; Windows / Linux / macOS all supported; inference supports PyTorch + TensorRT (5-10x faster) + NCNN port; LivePortrait-Editor provides fine editing controls.

Use cases include: digital-human avatar animation (corporate / live / short video), film post-production character motion transfer (combined with DeepFaceLab etc.), e-commerce virtual streamer, real-time video meeting avatar/background replacement, and educational courseware personalization. Hardware floor: runs on CPU demo (slow), NVIDIA GPU recommended (6GB+ VRAM); TensorRT acceleration achieves 30 FPS real-time. Onboarding: git clone → download pretrained weights → python inference.py -s source.jpg -d driving.mp4 -o output.mp4; ComfyUI users install ComfyUI-LivePortraitKJ node.

Licensed under Apache-2.0, commercial use + patent grant + trademark disclaimer; the paper requires citing "LivePortrait: Efficient Portrait Animation with Implicit Structural Motion Priors (arXiv 2407.03168)". Positioning versus VisLane's already-listed HeyGen (commercial digital human) and the pending SadTalker (same-source open-source): SadTalker is "audio-driven" (audio → face animation); LivePortrait is "video-driven" (reference video → face animation). The two combine for "audio + pose collaborative" high-quality digital human (audio controls lip, video controls head pose); LivePortrait is currently the de-facto standard for open-source portrait animation; if only audio-driven is needed, choose SadTalker for lighter footprint.