zml

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

What is zml?

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

Overview of zml

What it is

ZML is a production inference stack built with Zig, MLIR, and Bazel. It is designed to decouple AI workloads from proprietary hardware, allowing the same codebase to run on multiple accelerators without rewriting.

Key features

  • Supports NVIDIA, AMD, Intel, Google TPU, and AWS Trainium/Inferentia 2 accelerators.
  • Compiled directly to each target hardware for peak performance.
  • Supports models including Llama 3.1/3.2, Qwen 3.5, and LFM 2.5.
  • Provides a unified LLM CLI for loading models from Hugging Face, local directories, or S3.
  • Includes examples for MNIST, sharding, I/O, and benchmarking.

Use cases

  • Running large language models (LLMs) on various hardware platforms.
  • Deploying AI inference workloads across different accelerators without code changes.
  • Experimenting with model inference on CPU, GPU, or TPU.
  • Benchmarking model loading and execution performance.

Getting started

Prerequisites: Install Bazel (recommended via Bazelisk). On macOS: bash brew install bazelisk On Linux: bash curl -L -o /usr/local/bin/bazel 'https://github.com/bazelbuild/bazelisk/releases/download/v1.28.0/bazelisk-linux-amd64' chmod +x /usr/local/bin/bazel Run the MNIST smoke test: bash bazel run //examples/mnist For LLM quickstart, authenticate with Hugging Face if needed: bash bazel run //tools/hf -- auth login Then run a prompt: bash bazel run //examples/llm -- --model=hf://meta-llama/Llama-3.2-1B-Instruct --prompt='What is the capital of France?' To run on GPU/TPU, append platform flags, e.g., for CUDA: bash bazel run //examples/llm --@zml//platforms:cuda=true -- --model=hf://meta-llama/Llama-3.2-1B-Instruct --prompt='Write a haiku about Zig'

License and pricing

ZML is licensed under the Apache 2.0 license. It is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/zmlzml.svg)](https://yunodb.com/tools/zmlzml?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/zmlzml?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/zmlzml.svg" alt="Featured on YunoDB" height="20"></a>