openai/whisper

Project overview

Whisper is a general-purpose speech recognition model that performs multilingual transcription, speech-to-English translation, and language identification. It replaces many stages of a traditional speech-processing pipeline using a single sequence-to-sequence model trained via large-scale weak supervision.

Project type

Audio & Speech · Infrastructure

Use cases

Meeting Notes · Translation & Subtitles · Audio & Speech

Deployment

Python environment

License

MIT

Best for

• Developers and AI engineers needing a self-hosted, local speech recognition, translation, and language identification model.

Key capabilities

• Transcribes speech from diverse audio into text using a multitasking model.
• Translates non-English speech into English text.
• Identifies the spoken language from an audio input.
• Performs voice activity detection as part of a joint sequence-to-sequence training format.

Limitations and risks

• The turbo model is not trained for translation tasks.
• English-only models tend to perform better only for English-only applications; the difference becomes less significant for small.en and medium.en models.
• May need rust installed if tiktoken does not provide a pre-built wheel for your platform.

Getting started

• Install openai-whisper via pip, install ffmpeg via a system package manager, and execute the whisper command against an audio file. Model weights are downloaded automatically.

Evidence and sources

• README: Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech reco…
• README: A Transformer sequence-to-sequence model is trained on various speech processing tasks, including multilingual speech recognition, speech translation, spoken language identificati…
• GitHub project description: Robust Speech Recognition via Large-Scale Weak Supervision
• README: pip install -U openai-whisper
• README: There are six model sizes, four with English-only versions, offering speed and accuracy tradeoffs.

Reviewed at: 2026-07-17T04:19:34Z