MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
What is MNN?
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
Overview of MNN
What it is
MNN is a highly efficient and lightweight deep learning framework developed by Alibaba. It supports inference and training of deep learning models and has been integrated into more than 30 apps of Alibaba Inc, such as Taobao, Tmall, Youku, DingTalk, Xianyu, covering over 70 usage scenarios. MNN also includes MNN-LLM for large language model runtime and MNN-Diffusion for stable diffusion model runtime, both designed for local deployment on mobile phones, PCs, and IoT devices.
Key features
- Lightweight: No dependencies; iOS static library size about 12 MB (armv7+arm64), Android core so size about 800 KB (armv7a). Supports FP16/INT8 quantization reducing model size 50%-70%.
- Versatility: Supports TensorFlow, Caffe, ONNX, Torchscripts; covers CNN, RNN, GAN, Transformer. Converter supports 178 TensorFlow OPs, 52 Caffe OPs, 163 Torchscripts OPs, 158 ONNX OPs. Supports iOS 8.0+, Android 4.3+, and POSIX embedded devices.
- High performance: Optimized assembly for ARM/x64 CPU; GPU support via Metal, OpenCL, Vulkan, CUDA. Winograd convolution for symmetric kernels. ARM v8.2 FP16 half-precision gives 2x speedup; sdot on ARM v8.2 and VNNI gives 2.5x.
- Ease of use: Python API for inference, training, and image processing without C++. Lightweight image process module (~100 KB). Supports model building and training on PC/mobile.
Use cases
- On-device inference for live broadcast, short video capture, search recommendation, product searching by image, interactive marketing, equity distribution, security risk control.
- Embedded and IoT devices.
- Local deployment of large language models (e.g., Qianwen, Baichuan, LLAMA) and stable diffusion models on mobile phones, PCs, and IoT.
Getting started
See the repository README for installation and quick-start instructions.
License and pricing
Apache-2.0 license. Open source.
Based on the project's public README and repository metadata.