jitllm

An operating system for LLM inference that makes the most of your hardware: generate optimized compute, run models beyond available memory, and orchestrate and

What is jitllm?

An operating system for LLM inference that makes the most of your hardware: generate optimized compute, run models beyond available memory, and orchestrate and migrate layers across CPUs and GPUs at runtime to improve throughput without losing the conversation.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/jitllmjitllm.svg)](https://yunodb.com/tools/jitllmjitllm?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/jitllmjitllm?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/jitllmjitllm.svg" alt="Featured on YunoDB" height="20"></a>