olla

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local an

What is olla?

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/thushanolla.svg)](https://yunodb.com/tools/thushanolla?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/thushanolla?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/thushanolla.svg" alt="Featured on YunoDB" height="20"></a>