Whisper

Whisper is OpenAI's open-source speech recognition (ASR) model, first released on September 21, 2022, with ★68k+ on GitHub, licensed under MIT, and written primarily in Python (PyTorch). Based on a Transformer encoder-decoder architecture, trained on 680,000 hours of multilingual supervised data, it supports 99 languages (Chinese, English, Japanese, French, German, Spanish, Arabic, etc.). Provides five size tiers (tiny / base / small / medium / large / large-v2 / large-v3) from 39M to 1550M parameters, with VRAM ranging from ~1GB to ~10GB; supports speech-to-text, speech translation (any language → English), language identification, and timestamp alignment.

Core capabilities center on "multilingual + multi-size + multi-task". Tasks: transcribe (original-language transcription), translate (any language → English translation), detect language, alignment (word-level timestamps). Models: tiny (39M, 1GB VRAM, English WER ~7.6%), base (74M, 1GB VRAM), small (244M, 2GB VRAM), medium (769M, 5GB VRAM), large-v3 (1550M, 10GB VRAM, English WER ~2.7%). Input: almost any audio format (WAV / MP3 / M4A / OGG / FLAC), supports long audio auto-chunking (default 30s segments). Output: SRT / VTT subtitles + TXT / JSON + word-level timestamps. Ecosystem: HuggingFace Transformers / faster-whisper (CTranslate2 4-5x acceleration) / whisper.cpp (C++ port for edge devices) / WhisperX (speaker diarization + forced alignment) / Insanely-fast-whisper (batch inference). Derivative models: Distil-Whisper (HuggingFace, 6x faster, 49% size), Whisper-Medusa (speculative decoding), Whisper-Streaming (streaming).

Use cases include: video auto-subtitle generation (alongside MoneyPrinterTurbo / ShortGPT / Auto-Editor), podcast-to-text, meeting minutes, voice search (with LLM), speech translation, teaching courseware transcription, and accessibility assistance. Hardware floor: tiny / base can run real-time on CPU; small / medium recommended on CPU/GPU; large needs NVIDIA GPU (10GB+ VRAM); Apple Silicon (MPS) supports medium; faster-whisper makes large model achieve 4-5x real-time speed on CPU. Onboarding: pip install -U openai-whisper → whisper audio.mp3 --language Chinese --model large-v3 --output_format srt; CLI whisper audio.mp3 defaults to base model; Python API whisper.load_model("base").transcribe("audio.mp3").

Licensed under MIT, allows commercial closed-source use, modification, and redistribution — this is OpenAI's unusually permissive disclosure of a core AI model. Positioning versus VisLane's already-listed descript (commercial transcription + editing): Whisper is "CLI-level open-source ASR core"; descript is "commercial SaaS" (transcription is just one sub-feature, main feature is text-driven editing); the two overlap heavily in transcription — Whisper's accuracy with large-v3 + faster-whisper is approaching human level, descript has stronger UI / editing UX. Whisper also complements VisLane's already-listed SadTalker / LivePortrait — Whisper transcribes audio to text → LLM processes → TTS re-synthesizes → SadTalker / LivePortrait generates digital-human video, forming a complete "AI digital-human content pipeline".