Verified project record

FunAudioLLM/SenseVoice

SenseVoice is a multilingual speech understanding model that performs speech recognition, language identification, emotion recognition, and audio event detection. It uses a non-autoregressive end-to-end framework to deliver high-accuracy, rich transcription with low-latency inference.

Project overview

By combining speech recognition, emotion recognition, and audio event detection in a single non-autoregressive model, SenseVoice runs significantly faster than comparable Whisper models while producing richly annotated transcriptions.

Project type: Model Runtime · Audio & Speech · Infrastructure
Use cases: Meeting Notes · Audio & Speech
Deployment: Refer to project documentation
License: MIT

Best for

Developers, AI engineers, and researchers who need high-accuracy multilingual speech recognition enriched with emotion and audio event detection.

Key capabilities

Performs high-accuracy speech recognition supporting Mandarin, Cantonese, English, Japanese, and Korean.
Recognizes emotions in speech, achieving and surpassing current best emotion recognition models on test data.
Detects common human-computer interaction events such as bgm, applause, laughter, crying, coughing, and sneezing.
Automatically identifies the spoken language in the audio input.
Uses a non-autoregressive end-to-end framework for low-latency inference, running significantly faster than comparable Whisper models.
Provides support for timestamps based on CTC alignment.
Runs on CPU/edge as a single self-contained binary with built-in VAD and no Python at runtime.
Supports exporting the model to ONNX and Libtorch formats for runtime deployment.

Limitations and risks

The released SenseVoiceSmall checkpoint supports only 5 languages and relies on the composed FunASR pipeline for diarization.
Event classification performance has some gaps compared to specialized AED models.

Getting started

Run pip install -r requirements.txt, prepare the SenseVoiceSmall model weights, and execute the inference script or webui.py.

Evidence and sources

GitHub project description: Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.
README: SenseVoice is a speech foundation model with multiple speech understanding capabilities, including automatic speech recognition (ASR), spoken language identification (LID), speech…
README: Released checkpoint scope: SenseVoiceSmall supports ASR and language ID for Mandarin, Cantonese, English, Japanese, and Korean, together with emotion and audio-event tags. Speaker…
README: SenseVoice focuses on high-accuracy multilingual speech recognition, speech emotion recognition, and audio event detection.
README: The released SenseVoiceSmall checkpoint linked above supports Mandarin, Cantonese, English, Japanese, and Korean; the benchmark comparisons below are task- and language-specific.

Reviewed at: 2026-07-18T04:15:49Z