bytedance/ui-tars-desktop is an open-source multimodal AI agent stack for automating human-like tasks via GUI, browser, and desktop control. It requires a compatible multimodal LLM and operates on Windows and MacOS.
Project overview
The project combines browser, desktop, and CLI agent workflows under a unified framework, using Vision-Language Models to interpret screenshots and execute mouse and keyboard actions.
Project type
MCP · AI Agent · Workflow
Use cases
Automation
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers, general users, and AI engineers needing a multimodal agent for browser and desktop automation tasks.
Environments equipped with a compatible multimodal LLM or model API key.
Key capabilities
Controls browsers using GUI Agent, DOM, or a hybrid strategy.
A desktop application providing native computer control based on the UI-TARS model.
CLI tool for executing agent workflows in headful or headless server modes.
Provides a web-based graphical user interface for interacting with the agent.
Powers natural language control using Vision-Language Models to recognize screen content.
Built on MCP and supports mounting MCP Servers to connect to real-world tools.
Protocol-driven Event Stream to drive Context Engineering and Agent UI.
Limitations and risks
The system requires external or local Vision-Language Models to function, such as Volcengine doubao, Anthropic Claude, or UI-TARS.
Using cloud-based proprietary models may incur API costs.
Getting started
Setup is rated as easy. Users can launch the tool directly without complex setup by running npx @agent-tars/cli@latest and providing a model API key.
Alternatives and comparisons
Automates the user's existing browser using AI applications while keeping logged-in sessions active and avoiding bot detection mechanisms.
An open-source Chromium fork and agentic browser suite that runs locally, preserving user logins and privacy, while allowing external AI tools or built-in agents to execute real web tasks.
A unified Android device control platform combining automation, remote operations, capture, reverse engineering, networking, AI agents, and MCP through one service and API.
GitHub project description: The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
README: Agent TARS</b> is a general multimodal AI Agent stack, it brings the power of GUI Agent and Vision into your terminal, computer, browser and product.
README: UI-TARS Desktop</b> is a desktop application that provides a native GUI Agent based on the <a href="https://github.com/bytedance/UI-TARS" target="_blank">UI-TARS</a> model.
README: Natural language control powered by Vision-Language Model - 🖥️ Screenshot and visual recognition support - 🎯 Precise mouse and keyboard control - 💻 Cross-platform support (Windows…