Verified project record

livekit/agents

livekit/agents is a framework for building realtime, programmable, multi-modal voice AI participants that run on servers. It provides integrated job scheduling and flexible integrations for STT, LLM, TTS, and Realtime APIs so agents can see, hear, and understand in conversation.

Project overview

The framework combines realtime multi-modal voice agent execution on servers with built-in job scheduling, dispatch APIs, telephony integration, and semantic turn detection using a transformer model to reduce interruptions.

Deployment: Refer to project documentation
License: Apache-2.0

Best for

Developers who need to build realtime, programmable, multi-modal voice AI participants that run on servers
AI engineers who want to mix and match STT, LLM, and TTS providers within a single agent framework

Key capabilities

Create conversational, multi-modal voice agents that can see, hear, and understand running as realtime programmable participants on servers.
An ecosystem to mix and match the right STT, LLM, TTS, and Realtime API to suit your use case.
Built-in task scheduling and distribution with dispatch APIs to connect end users to agents.
Allows your agent to make calls to or receive calls from phones via LiveKit's telephony stack.
Uses a transformer model to detect when a user is done with their turn, helps to reduce interruptions.
Native support for MCP. Integrate tools provided by MCP servers with one line of code.
Write tests and use judges to ensure your agent is performing as expected.
Support for multiple agents to handoff tasks between each other.

Limitations and risks

The framework is under active development in a rapidly evolving field. Expect ongoing changes as the project matures.
Because the framework is under active development in a rapidly evolving field, APIs may change frequently. Plan for potential breaking changes when upgrading.

Getting started

Run pip install livekit-agents[openai,deepgram,cartesia] to install the framework with common provider integrations.
Set LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET environment variables to connect the agent to a LiveKit Server or LiveKit Cloud instance. Model provider credentials (e.g., OpenAI, Deepgram, Cartesia) must also be configured.
Write agent code in Python using the framework, then run the agent with python myagent.py start. Coding is required to build an agent.

Evidence and sources

Open original GitHub project