This project brings large language model voice interaction to ESP32 hardware devices. It pairs device-side peripheral control with cloud-side capability extension through a dual MCP architecture.
Project overview
This project brings large language model voice interaction to ESP32 hardware devices. It pairs device-side peripheral control with cloud-side capability extension through a dual MCP architecture.
Project type
MCP · AI Agent
Use cases
Automation
Deployment
Refer to project documentation
License
MIT
Best for
Developers and creators seeking to deploy voice-driven AI interactions on supported ESP32 chip platforms.
Key capabilities
Provides offline voice wake-up, ASR + LLM + TTS pipelines, and realtime end-to-end voice models.
Enables control of device peripherals (Speaker, LED, Servo, GPIO) via the MCP protocol.
Extends large model capabilities to smart home control, PC desktop operation, knowledge search, and email via cloud MCP.
Supports Wi-Fi, wired Ethernet, USB RNDIS, and 4G networking (ML307/EC801E or NT26 Cat.1).
Identifies the current speaker using 3D Speaker.
Supports OLED/LCD displays with emoji and camera vision input on supported boards.
Manages battery display and device power.
Allows customization of wake words, fonts, emojis, and chat backgrounds via a web-based editor.
Limitations and risks
ESP-IDF v5.5 is retained only for legacy hardware paths.
Getting started
Beginners can flash pre-compiled firmware without setting up a development environment. The first success path involves choosing a supported ESP32 board, following the beginner's firmware flashing guide, flashing firmware, connecting to Wi-Fi, and registering a personal account on xiaozhi.me to use for free.
Evidence and sources
GitHub project description: An MCP-based chatbot | 一个基于MCP的聊天机器人
README: As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.
README: This is an open-source ESP32 project, released under the MIT license
README: Wi-Fi, wired Ethernet, USB RNDIS, and ML307/EC801E or NT26 Cat.1 4G networking; supported boards can switch between Wi-Fi and 4G
README: For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.