Verified project record

alibaba/MNN

alibaba/mnn is a lightweight deep learning framework for on-device model inference and training across mobile, IoT, and embedded devices. It supports converting models from TensorFlow, Caffe, ONNX, and TorchScripts, and includes runtimes for deploying large language models and diffusion models locally.

Project overview

The framework is optimized for minimal dependencies and has been adopted across more than 30 apps and 70 usage scenarios within Alibaba, demonstrating its capacity for efficient edge AI deployment.

Project type: Model Runtime · Infrastructure
Use cases: Chat Assistants
Deployment: Refer to project documentation
License: Apache-2.0

Best for

Developers and AI engineers needing a highly efficient, lightweight framework for on-device inference and training across mobile and IoT devices.
Enterprise teams looking to deploy large language models or diffusion models locally on edge hardware.

Key capabilities

A highly efficient and lightweight deep learning framework supporting comprehensive model inference and training on-device.
A large language model runtime solution to deploy LLM models locally on Mobile Phones, PCs, and IoT devices.
A stable diffusion model runtime solution to deploy diffusion models locally on various platforms.
Converts models from Tensorflow, Caffe, ONNX, and Torchscripts to MNN models while performing graph optimization to reduce computation.
Compresses models to reduce size and increase performance and speed.
An OpenCV-like library based on MNN that is much more lightweight (around 100k).
Supports models with control flow and allows using MNN's operations to do general-purpose computing.
Allows building and training models directly on PC or mobile devices.

Limitations and risks

Using MNN_BUILD_MINI to reduce package size imposes a limit of fixed model input size.
Specific architectures and precision combinations (e.g., CPU Native FP16, CPU BF16) have bugs, are unsupported, or are not recommended (rated C).

Getting started

Setup is of medium difficulty and requires building from source via CMake or using pre-built binaries and tools.

Evidence and sources

GitHub project description: MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
README: MNN is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models and has industry-leading performance for inference an…
README: [MNN-LLM](./transformers/README.md) is a large language model runtime solution developed based on the MNN engine. The mission of this project is to deploy LLM models locally on ev…
README: - MNN-Converter: Convert other models to MNN models for inference, such as Tensorflow(lite), Caffe, ONNX, Torchscripts. And do graph optimization to reduce computation.
README: - Optimized for devices, no dependencies, can be easily deployed to mobile devices and a variety of embedded devices. - iOS platform: static library size will full option for armv…

Reviewed at: 2026-07-17T14:02:28Z