MNN is a lightweight deep learning inference and training framework designed for resource-constrained mobile and embedded devices. It provides on-device inference, training, and runtimes for large language models and diffusion models.
Project overview
MNN has been integrated into more than 30 apps of Alibaba Inc covering more than 70 usage scenarios, demonstrating production-scale validation on mobile and embedded devices.
Project type
Infrastructure
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers and AI engineers who need to run or train deep learning models on resource-constrained mobile and embedded devices.
Teams deploying large language models or stable diffusion models locally on mobile phones, PCs, or IoT devices.
Key capabilities
Supports inference and training of deep learning models with on-device performance.
A large language model runtime solution to deploy LLM models locally on mobile phones, PCs, and IoT devices.
A stable diffusion model runtime solution to deploy stable diffusion models locally on various platforms.
Converts models from Tensorflow, Caffe, ONNX, and Torchscripts to MNN models for inference and performs graph optimization.
Compresses models to reduce size and increase performance and speed.
An OpenCV-like, lightweight image process library based on MNN.
Enables accelerated model inference on Qualcomm Hexagon DSPs.
Provides full multimodal LLM Android and iOS apps including text-to-text, image-to-text, audio-to-text, and text-to-image generation.
Limitations and risks
Using MNN_BUILD_MINI can reduce package size by about 25%, with a limit of fixed model input size.
Getting started
Begin by downloading prebuilt binaries or compiling the MNN C++ source code for the target platform.
Convert a Tensorflow, Caffe, ONNX, or Torchscripts model to an MNN model using MNN-Converter, which also performs graph optimization.
Integrate the MNN library into an existing application to load and run the converted model. Setup difficulty is medium because it requires compiling C++ source code or integrating into an existing app.
MNN provides visualized tools via MNN Workbench, and a desktop GUI is available as an interaction mode.