ort
Fast ML inference & training for ONNX models in Rust
What is ort?
Fast ML inference & training for ONNX models in Rust
Overview of ort
What it is
ort is a Rust interface for performing hardware-accelerated inference and training on machine learning models in the Open Neural Network Exchange (ONNX) format. It is based on the now-inactive onnxruntime-rs crate and primarily wraps Microsoft's ONNX Runtime library, while also offering support for other pure-Rust runtimes.
Key features
- Hardware-accelerated inference and training for ONNX models.
- Wrapper for Microsoft's ONNX Runtime with support for other pure-Rust runtimes.
- Supports a wide range of hardware accelerators.
- Lightweight enough to run on user devices.
- Supports models from PyTorch, TensorFlow, Keras, scikit-learn, and PaddlePaddle.
- Includes documentation with a guide, API reference, examples, and migration guide from v1.x to v2.0.
Use cases
ort is designed for deploying machine learning models either on-device or in the datacenter. It is used in various open-source projects for tasks such as text embeddings, file type detection, OCR, transformer inference, text-to-speech, biodiversity conservation, animal detection in trail camera imagery, semantic search, and more.
Getting started
See the repository README for installation and quick-start instructions.
License and pricing
The project is licensed under Apache-2.0 and is open source.
Based on the project's public README and repository metadata.