aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
What is aimet?
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
Overview of aimet
What it is
AI Model Efficiency Toolkit (AIMET) is a software library that provides advanced quantization and compression techniques for trained neural network models. It is designed to improve the runtime performance of deep learning models by reducing compute load and memory footprint. Models quantized with AIMET can be deployed on edge devices such as mobile phones or laptops. AIMET supports models from the ONNX and PyTorch frameworks. The toolkit employs post-training and fine-tuning techniques to minimize accuracy loss during quantization and compression.
Key features
- Post-Training Quantization (PTQ): Includes techniques such as Calibration, AdaRound, SeqMSE, BatchNorm Folding, Cross Layer Equalization, BatchNorm re-estimation, AdaScale, OmniQuant (PyTorch only), and SpinQuant.
- Quantization Aware Training (QAT): Supported via aimet-torch for training models with quantization in the loop.
- Model Compression: Provides Spatial SVD (tensor decomposition), Channel Pruning (removing redundant input channels), and per-layer compression-ratio selection.
- Visualization: Offers weight range inspection for Cross Layer Equalization and per-layer compression sensitivity feedback.
- Supports ONNX and PyTorch frameworks.
- User-friendly APIs that integrate directly into PyTorch pipelines.
Use cases
Quantizing and compressing trained ML models to reduce memory footprint and improve inference speed on edge devices. Suitable for deploying models on mobile phones, laptops, and Qualcomm Hexagon DSP. Can be used for computer vision models (e.g., MobileNet-v2, ResNet-50, DeepLab v3), ADAS object detection, semantic segmentation, and recurrent models (RNN, LSTM, GRU) like DeepSpeech2.
Getting started
aimet-onnx and aimet-torch packages are available on PyPI. See the repository's Quick Start guide for installation instructions. To build from source, refer to the Build, install and run AIMET from source guide in a Docker environment.
License and pricing
AIMET is licensed under the BSD 3-clause
Based on the project's public README and repository metadata.