CogVideoX

CogVideoX is an open-source text-to-video and image-to-video model family maintained by the THUDM lab at Tsinghua University (a co-founding party of Zhipu AI). Its predecessor, CogVideo, was open-sourced in May 2022 and accepted at ICLR 2023 as the first open-source large Transformer-based text-to-video generation model. The CogVideoX series started rolling out in 2024 with 2B / 5B / 1.5 tiers, paired with the CogKit fine-tuning and inference framework. It is one of the most important open-source video generation models in the Chinese-language ecosystem. The repository has since moved to zai-org/CogVideo (zai-org is the GitHub org associated with Zhipu AI), but THUDM/CogVideo remains accessible and continues to receive releases.

Core capabilities center on "Transformer + 3D Causal VAE + multi-task unification". The model family includes: CogVideoX-2B (the first open release, lowest VRAM floor), CogVideoX-5B (open-sourced August 2024 as a larger model, optimized for inference on RTX 3060-class desktop GPUs), CogVideoX-5B-I2V (open-sourced September 2024 for image-to-video, taking an image as background plus a text prompt), CogVideoX1.5-5B (open-sourced November 2024 as the 1.5 upgrade, supports 10-second videos at higher resolution), and CogVideoX1.5-5B-I2V (any-resolution image-to-video). Architectural components: a proprietary 3D Causal VAE (compresses videos into latent space with near-lossless reconstruction), an Expert Transformer backbone (with text, temporal, and spatial attention), and DDIM Inverse (added in February 2025 to 5B and 1.5-5B). Task coverage: text-to-video, video continuation, and image-to-video. Ecosystem tooling: CogKit (open-sourced March 2025) is the official fine-tuning and inference framework for the CogView4 and CogVideoX series; cogvideox-factory (community-maintained by a-r-r-o-w) is a cost-effective fine-tuning setup that runs on a single 4090 GPU. Prompting tooling: CogVLM2-Caption (open-sourced September 2024) is the caption model used during training to convert video data into text descriptions.

Use cases include: film / short-video teams doing AI video concept boards and storyboard drafts, e-commerce teams producing product rotation videos (I2V path), animation / motion-comic teams producing character-consistent shorts (with A2 ecosystem support), AI video researchers running controllable-generation experiments (multi-size, multi-task unified), and commercial teams doing brand-asset batch generation (Apache-2.0 permits commercial use). Hardware floor: CogVideoX-2B runs on older cards such as GTX 1080Ti / RTX 3060; CogVideoX-5B recommends RTX 3060+ / 4090 desktop GPUs; the latest SAT-version weights need multi-GPU; the diffusers integration path is friendlier to consumer cards. Deployment shapes: local clone + conda + pip install -r requirements.txt (SAT and diffusers paths coexist); online experience via HuggingFace Spaces / ModelScope Studio (5B model); higher-tier commercial versions through Zhipu's Qingying (chatglm.cn/video) and BigModel open platform (open.bigmodel.cn). Chinese support: a dedicated Chinese README is shipped (README_zh.md), and the model itself was trained with multilingual data including Chinese.

Licensed under Apache-2.0 (CogVideoX-2B moved to Apache-2.0 in August 2024; refer to official announcements for earlier models). Commercial use is allowed without special permission, making it one of the few commercially-friendly open-source video generation models. Important note: the GitHub API indicates the canonical repo has moved to zai-org/CogVideo, but THUDM/CogVideo remains reachable and continues to receive releases; this tool.json keeps github_url on the THUDM/CogVideo path to stay consistent with the README's self-references. Maintenance tail: the repository main branch has not received a new push since November 4, 2025; new model iterations (e.g. CogVideoX 2 series) are released via HuggingFace and Zhipu's Qingying / BigModel API.