m-bain/whisperX

Project overview

WhisperX adds batched inference, word-level timestamps via wav2vec2 alignment, and pyannote-audio speaker diarization to Whisper large-v2, reaching up to 70x realtime transcription. It is designed for developers and researchers who need precise word timings and speaker labels but should be evaluated against its handling of overlapping speech and its reliance on language-specific alignment models.

Project type

Audio & Speech

Use cases

Meeting Notes · Audio & Speech

Deployment

Refer to project documentation

License

BSD-2-Clause

Best for

• Developers and researchers needing fast batched transcription, precise word-level alignments, and speaker labels.
• Environments equipped with under 8GB of GPU memory running large-v2 with beam_size=5.

Key capabilities

• Batched inference for 70x realtime transcription using whisper large-v2 with faster-whisper backend.
• Accurate word-level timestamps using wav2vec2 alignment.
• Multispeaker ASR using speaker diarization from pyannote-audio with speaker ID labels.
• Voice Activity Detection preprocessing reduces hallucination and enables batching with no WER degradation.
• Allows loading models, transcribing audio, aligning output, and assigning speaker labels via Python code.
• Command line tool to run transcription, alignment, and diarization on audio files.
• Option to run inference on CPU instead of GPU using compute_type int8.

Limitations and risks

• Transcript words lacking characters in the alignment model dictionary (e.g., "2014." or "£13.60") cannot be aligned and are not given a timing.
• Overlapping speech is not handled well by Whisper or WhisperX.
• Speaker diarization is not perfect.
• A language-specific wav2vec2 model is needed for alignment.
• The development version may contain experimental features and bugs; the stable PyPI release should be used for production environments.

Getting started

• Installation is evaluated as easy via pip or uvx, requiring ffmpeg and CUDA for GPU. The first success path involves installing CUDA toolkit 12.8 if using GPU, running pip install whisperx, and executing whisperx path/to/audio.wav.

Evidence and sources

• GitHub project description: WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)
• README: This repository provides fast automatic speech recognition (70x realtime with large-v2) with word-level timestamps and speaker diarization.
• README: Whilst it does produces highly accurate transcriptions, the corresponding timestamps are at the utterance-level, not per word, and can be inaccurate by several seconds. OpenAI's w…
• README: 1st place at [Ego4d transcription challenge](https://eval.ai/web/challenges/challenge-page/1637/leaderboard/3931/WER) 🏆
• Release: v3.8.6

Reviewed at: 2026-07-17T11:24:14Z