oneDNN
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, Tens
What is oneDNN?
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
Overview of oneDNN
What it is
oneDNN is an open-source, cross-platform library of optimized deep learning operations for CPUs and GPUs. It provides highly optimized implementations of matrix multiplication, convolution, attention, normalization, and other operations. The library is optimized for Intel 64/AMD64 architecture based processors, Arm 64-bit Architecture (AArch64)-based processors, and Intel Graphics. It has experimental support for NVIDIA GPU, AMD GPU, OpenPOWER Power ISA (PPC64), IBMz (s390x), and RISC-V. oneDNN is the acceleration layer behind PyTorch, TensorFlow, OpenVINO, and many other AI frameworks.
Key features
- Single API, many devices: The same code targets Intel64/AMD64, AArch64, RISC-V CPUs, and Intel GPUs.
- Hardware specialization: Detects hardware features at runtime and JIT-generates kernels specialized for instruction set, number of cores, and cache sizes.
- Low-precision inference: Supports f32, bf16, f16, fp8, fp4, u8/s8, and u4/s4 data types with various quantization schemes including OCP microscaling formats.
- Operation fusion: Post-ops fuse activations, residual connections, and quantization into matmul and convolution. The Graph API fuses larger patterns such as scaled dot-product attention (SDPA) and grouped-query attention (GQA).
- Interoperability: Supports OpenMP, TBB, and custom threadpools on CPUs. SYCL and OpenCL on GPUs.
Use cases
oneDNN is intended for deep learning application and framework developers interested in improving application performance on CPUs and GPUs. It is used by frameworks such as Apache SINGA, DeepLearning4J, Flashlight, llama.cpp, ONNX Runtime, OpenNMT CTranslate2, OpenVINO toolkit, PaddlePaddle, PyTorch, and TensorFlow.
Getting started
Install pre-built oneDNN from conda-forge: sh conda install -c conda-forge onednn Build and run the matrix multiplication performance example: sh wget https://raw.githubusercontent.com/uxlfoundation/oneDNN/main/examples/matmul_perf.cpp https://raw.githubusercontent.com/uxlfoundation/oneDNN/main/examples/example_utils.hpp g++ matmul_perf.cpp -I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -ldnnl -o matmul_perf LD_LIBRARY_PATH=$CONDA_PREFIX/lib ./matmul_perf cpu
License and pricing
Licensed under Apache-2.0. Open source.
Based on the project's public README and repository metadata.
