livekit/agents is a framework for creating conversational, multi-modal voice agents that run as realtime programmable participants on servers. It provides integrated job scheduling and a selection of STT, LLM, TTS, and Realtime API integrations.
Project overview
It addresses the challenge of building realtime, programmable, multi-modal voice AI participants on servers, pairing integrated dispatch and job scheduling with a configurable ecosystem of model providers.
Project type
MCP · AI Agent · Audio & Speech
Use cases
Knowledge Q&A · Automation
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers and AI engineers building realtime, multi-modal conversational agents on servers who need configurable model integrations.
Key capabilities
Create conversational, multi-modal voice agents that can see, hear, and understand running as realtime programmable participants on servers.
An ecosystem to mix and match STT, LLM, TTS, and Realtime API providers to suit a given use case.
Built-in task scheduling and distribution with dispatch APIs to connect end users to agents.
Allows an agent to make calls to or receive calls from phones via the LiveKit telephony stack.
Uses a transformer model to detect when a user is done with their turn, helping to reduce interruptions.
Provides native support for MCP, allowing integration of tools provided by MCP servers with one line of code.
Write tests and use judges to ensure an agent is performing as expected.
Support for multiple agents to handoff tasks between each other.
Limitations and risks
The framework is under active development in a rapidly evolving field.
APIs may change frequently due to active development in a rapidly evolving field.
Getting started
Requires installing the Python package and configuring external credentials and environment variables. Run: pip install livekit-agents[openai,deepgram,cartesia]
Set the LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET environment variables to connect to a LiveKit Server or LiveKit Cloud.
Write agent code using the Python framework and execute it with the command: python myagent.py start
Evidence and sources
GitHub project description: A framework for building realtime voice AI agents 🤖🎙️📹
README: The Agent Framework is designed for building realtime, programmable participants that run on servers. Use it to create conversational, multi-modal voice agents that can see, hear,…
README: - **Extensive WebRTC clients**: Build client applications using LiveKit's open-source SDK ecosystem, supporting all major platforms.
README: - **Integrated job scheduling**: Built-in task scheduling and distribution with [dispatch APIs](https://docs.livekit.io/agents/build/dispatch/) to connect end users to agents.
README: - **Telephony integration**: Works seamlessly with LiveKit's [telephony stack](https://docs.livekit.io/sip/), allowing your agent to make calls to or receive calls from phones.
AI Search
Find projects, verify facts, compare options, or turn a complex need into an actionable plan
Try a searchA click only fills the search box; you stay in control