Verified project record

livekit/agents

livekit/agents is a Python framework for building realtime, conversational, multi-modal voice agents that can see, hear, and understand. It runs programmable participants on servers and supports pluggable STT, LLM, TTS, and Realtime integrations via plugins.

Project overview

It addresses the lack of a unified framework for building realtime, server-side multimodal voice AI agents, combining core agent orchestration with native MCP tool support and a plugin ecosystem for mixing STT, LLM, TTS, and Realtime APIs from popular providers.

Project type: AI Agent · Audio & Speech
Use cases: Knowledge Q&A · Automation
Deployment: Refer to project documentation
License: Apache-2.0

Best for

Developers comfortable writing Python who want a unified framework for realtime, multi-modal voice AI agents.
Teams that want to mix and match STT, LLM, and TTS providers rather than commit to a single vendor's stack.

Key capabilities

Framework for building realtime, programmable participants that run on servers, creating conversational multi-modal voice agents that can see, hear, and understand.
Ecosystem to mix and match STT, LLM, TTS, and Realtime APIs via plugins for popular model providers.
Native support for MCP; integrate tools provided by MCP servers with one line of code.

Limitations and risks

Console mode is intended for testing; dev and production modes require a LiveKit server or LiveKit Cloud, configured with LIVEKIT_URL, an API key, and a secret.
Building an agent requires writing a Python agent script and installing provider plugins for STT, LLM, and TTS models.

Getting started

Install with pip (e.g. pip install "livekit-agents[openai,deepgram,cartesia]"), write an agent script using Agent/AgentSession/AgentServer, then run python myagent.py console for local terminal testing using a local microphone. Setup difficulty is medium because it requires writing a Python agent script and installing provider plugins.

Evidence and sources

README: The Agent Framework is designed for building realtime, programmable participants that run on servers.
README: Use it to create conversational, multi-modal voice agents that can see, hear, and understand.
README: **Flexible integrations**: A comprehensive ecosystem to mix and match the right STT, LLM, TTS, and Realtime API to suit your use case.
README: pip install "livekit-agents[openai,deepgram,cartesia]"
README: You'll need the following environment variables for this example: - LIVEKIT_URL - LIVEKIT_API_KEY - LIVEKIT_API_SECRET

Source updated at: 2026-10-25T18:08:18Z

Open original GitHub project