onnxruntime
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
What is onnxruntime?
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
Overview of onnxruntime
What it is
ONNX Runtime is a cross-platform inference and training machine-learning accelerator. It supports models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries including scikit-learn, LightGBM, and XGBoost. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms.
Key features
- Supports inference and training acceleration for models from PyTorch, TensorFlow/Keras, scikit-learn, LightGBM, XGBoost, and others.
- Compatible with various hardware, drivers, and operating systems.
- Leverages hardware accelerators (e.g., NVIDIA GPUs) for optimal performance.
- Applies graph optimizations and transforms to improve efficiency.
- Training acceleration on multi-node NVIDIA GPUs for transformer models with a one-line addition to existing PyTorch training scripts.
Use cases
- Deploying machine learning models for inference in production environments where low latency and high throughput are required.
- Accelerating training of transformer models on multi-node GPU clusters.
- Running models across diverse hardware platforms (CPU, GPU, etc.) without code changes.
Getting started
See the repository README for installation instructions and usage documentation. The project provides general information at onnxruntime.ai and usage documentation at onnxruntime.ai/docs.
License and pricing
This project is licensed under the MIT License. It is open source and free to use.
Based on the project's public README and repository metadata.