gufo
The open-source AI engine for AMD Strix Halo. Text, embeddings, speech, image and video models in one binary, with OpenAI-compatible APIs, caching for coding ag
What is gufo?
The open-source AI engine for AMD Strix Halo. Text, embeddings, speech, image and video models in one binary, with OpenAI-compatible APIs, caching for coding agents and model extensions. Linux and Windows.
Overview of gufo
What it is
Gufo is a vertical local inference engine specifically built and optimized for the AMD Strix Halo hardware: Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151), up to 128 GiB of unified memory.
Key features
- Supports multiple model families: Qwen3.8 27B, Qwen3.8 Flash-Next, DeepSeek V4 Flash, Qwen3-ASR 1.7B, Qwen3-TTS 1.7B, Qwen-Image-2.1, MiniMax H3.
- Covers text, audio, image, and video inference modes.
- Provides speculative decoding modes: DFlash2, MTP, DSpark.
- Treats concurrent requests, cancellation, and conversation caching as first-class workloads.
- Exposes an OpenAI-compatible API (Chat Completions and Responses) supporting text, images, function tools, structured output, and streaming.
- Delivered as a container image via Podman; also buildable from source with Nix or without Nix.
- Optimized specifically for Strix Halo 128 GiB hardware; smaller memory configurations still work.
Use cases
- Running large language models locally on AMD Strix Halo systems.
- Serving text, audio, image, and video generation models with low latency.
- Handling multiple concurrent users or requests with aggregated throughput.
- Integrating with OpenAI-compatible clients (e.g., Open WebUI, VS Code, OpenAI SDK).
Getting started
shhf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf --revision 4ca720788d1e01f1bff70c033e0d0028fd02e502 --repo-type model --local-dir models/Qwen3.8-27B-GGUFhf download z-lab/Qwen3.8-27B-DFlash2-GGUF Qwen3.8-27B-DFlash2-Q4_K_M.gguf --revision 2d9571f8ce46e151f61c6499c99dee6079e1d610 --repo-type model --local-dir models/Qwen3.8-27B-DFlash2-GGUFpodman pull ghcr.io/gufo-org/toolboxes/gufo-runtime:latestpodman run --rm --userns=keep-id:uid=1000,gid=1000 --device /dev/kfd --device /dev/dri --group-add keep-groups --ulimit memlock=-1 -p 8080:8080 -v ./models:/models:ro ghcr.io/gufo-org/toolboxes/gufo-runtime:latest gufo serve --host 0.0.0.0 --port 8080 llm --model /models/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q8_K_XL.gguf --speculative dflash2 --dflash-model /models/Qwen3.8-27B-DFlash2-GGUF/Qwen3.8-27B-DFlash2-Q4_K_M.gguf
License and pricing
MIT license, open source.
Based on the project's public README and repository metadata.