open-autoglm is a multimodal AI phone agent framework that understands screen content and automates mobile device operations from natural language instructions. It supports Android, HarmonyOS, and iOS devices through debugging bridges and a Python API.
Project overview
The project addresses the difficulty of automating complex mobile interactions by combining vision-language screen understanding with natural language task planning, while incorporating a built-in sensitive operation confirmation mechanism for login and verification scenarios.
Project type
AI Agent · Model Runtime
Use cases
Automation
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers and researchers who need a programmable framework for automating mobile device tasks through natural language instructions and screen understanding.
Key capabilities
Understands phone screen content using a vision-language model for task automation.
Automatically parses intent, understands the current interface, plans the next action, and completes the entire workflow based on natural language descriptions.
Built-in sensitive operation confirmation mechanism with support for manual takeover during login or verification code scenarios.
Provides remote ADB debugging capability to connect devices via WiFi or network for flexible remote control.
Allows users to create a PhoneAgent, configure the model, and execute tasks programmatically via a Python API.
Configures WebDriverAgent and iPhone devices to use AutoGLM on iOS.
Limitations and risks
This project is intended for research and learning use only.
Strictly prohibited from being used for illegal acquisition of information, interfering with systems, or any illegal activities.
Getting started
Install Python dependencies via pip, configure ADB or HDC on a mobile device with developer mode and USB debugging enabled, and connect via USB. Run the agent with a command such as python main.py --base-url http://localhost:8000/v1 --model 'autoglm-phone-9b'. Optionally, deploy a large language model via Docker or pip.
Evidence and sources
GitHub project description: An Open Phone Agent Model & Framework. Unlocking the AI Phone for Everyone
README
README: | Model | Download Links | |-------------------------------|-----------------------------------------------------------------------------------------------------------------------…