inferssd

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

What is inferssd?

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/wici-aiinferssd.svg)](https://yunodb.com/tools/wici-aiinferssd?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/wici-aiinferssd?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/wici-aiinferssd.svg" alt="Featured on YunoDB" height="20"></a>