Verified project record

trycua/cua

An open-source framework enabling AI agents to autonomously operate computer interfaces and run in isolated sandboxes across operating systems. It provides background input drivers, a cross-OS sandbox API, Apple Silicon virtualization management, and agent benchmarks.

Project overview

Project type: AI Agent · Infrastructure
Use cases: Automation
Deployment: Refer to project documentation
License: MIT

Best for

Developers and AI engineers building autonomous agents that require direct interaction with cross-OS desktop environments.
Researchers evaluating computer-use agent performance and exporting training trajectories.

Key capabilities

Drive native desktop apps in the background. Agents click, type, and verify without stealing the cursor or focus.
One API to build agents that see screens, click buttons, and complete tasks autonomously across any VM or container image, cloud or local.
Evaluate computer-use agents on OSWorld, ScreenSpot, Windows Arena, and custom tasks. Export trajectories for training.
Create and manage macOS/Linux VMs with near-native performance on Apple Silicon using Apple's Virtualization.Framework.

Limitations and risks

The Sequoia macOS VM flow may still open the Accessibility step of Setup Assistant on its first display boot.
Linux supports X11 and compositor-specific Wayland routes with explicit limits for raw background input.
The optional cua-agent[omni] module includes ultralytics, which is licensed under AGPL-3.0.

Getting started

Install the package via pip, import and initialize the Sandbox or driver in your code, and execute commands or automate desktop applications. This requires Python 3.11+ or downloading Apple restore images for VMs.

Evidence and sources

GitHub project description: Scale computer-use 2.0 with open-source drivers, cross-OS fleets, and benchmarks for training, evaluation, and data generation.
README: Drive native desktop apps **in the background**. Agents click, type, and verify without stealing the cursor or focus.
README: Use the same CLI and MCP server on macOS, Windows, and Linux from Claude Code, Cursor, Codex, OpenClaw, and custom clients.
README: Linux supports X11 and compositor-specific Wayland routes with explicit limits for raw background input.
README: Build agents that see screens, click buttons, and complete tasks autonomously. One API for any VM or container image — cloud or local.

Reviewed at: 2026-07-17T22:21:06Z