bytedance/UI-TARS-desktop

Project overview

An open-source multimodal AI agent stack for automating human-like tasks through GUI interaction, browser automation, and desktop control using natural language. It requires a compatible Vision-Language Model to function and supports multiple interaction modes including a desktop application, CLI, and Web UI.

Project type

MCP · AI Agent · Workflow

Use cases

Automation

Deployment

Refer to project documentation

License

Apache-2.0

Best for

• Developers, general users, or AI engineers looking to automate browser and desktop tasks via natural language using a multimodal AI agent.
• Users who have access to external or local Vision-Language Models and API keys for model providers.

Key capabilities

• Controls browsers using GUI Agent, DOM, or a hybrid strategy.
• A desktop application providing native computer control based on the UI-TARS model.
• CLI tool for executing agent workflows in headful or headless server modes.
• Provides a web-based graphical user interface for interacting with the agent.
• Powers natural language control using Vision-Language Models to recognize screen content.
• Built on MCP and supports mounting MCP Servers to connect to real-world tools.
• Protocol-driven Event Stream to drive Context Engineering and Agent UI.

Limitations and risks

• Requires external or local Vision-Language Models to function.
• Using cloud-based proprietary models may incur API costs.

Getting started

• The setup is straightforward and does not require coding. Users can launch the tool directly using npx via the CLI package and provide a model API key to begin.

Evidence and sources

• GitHub project description: The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
• README: Agent TARS</b> is a general multimodal AI Agent stack, it brings the power of GUI Agent and Vision into your terminal, computer, browser and product.
• README: UI-TARS Desktop</b> is a desktop application that provides a native GUI Agent based on the <a href="https://github.com/bytedance/UI-TARS" target="_blank">UI-TARS</a> model.
• README: Natural language control powered by Vision-Language Model - 🖥️ Screenshot and visual recognition support - 🎯 Precise mouse and keyboard control - 💻 Cross-platform support (Windows…
• Release: v0.3.0

Reviewed at: 2026-07-17T08:29:35Z