yzma
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows
What is yzma?
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
Overview of yzma
What it is
yzma is a Go library that directly integrates llama.cpp for fully local inference using hardware acceleration. It allows writing Go applications that run Vision Language Models (VLM) and Large/Small/Tiny Language Models (LLM) on Linux, macOS, or Windows. It also runs in a browser via TinyGo and WebAssembly, on CPU or GPU with WebGPU. yzma uses the purego and ffi packages so CGo is not needed. It works with the newest llama.cpp releases to use the latest features, performance improvements, and bugfixes.
Key features
- Supports hardware acceleration: CUDA, Metal, OpenVINO, Vulkan, HIP, ROCm, SYCL, OpenCL on various platforms.
- Runs in a browser with WebAssembly (CPU with SIMD, GPU with WebGPU).
- No CGo required; uses
puregoandffipackages. - Works with the latest
llama.cppreleases. - Supports multimodal models (image/audio) and text language models.
- Over 96% of
llama.cppfunctionality is supported (see ROADMAP.md).
Use cases
- Running large language models locally on desktop or server with hardware acceleration.
- Building Go applications that need local inference without external API calls.
- Deploying AI models in a browser with full privacy (model never leaves the user's machine).
- Interactive chat, vision-language tasks, and embedding generation.
Getting started
shellgo install github.com/hybridgroup/yzma@latestyzma install --lib /path/to/libexport YZMA_LIB=/path/to/libThen download a model:shellyzma model get -u https://huggingface.co/QuantFactory/SmolLM2-135M-GGUF/resolve/main/SmolLM2-135M.Q4_K_M.ggufRun an example:shellgo run ./examples/hello/
License and pricing
The license is not stated in the provided metadata (NOASSERTION). The project is listed as open source.
Based on the project's public README and repository metadata.