PocketLLM

Inference engine for large language models on consumer GPUs, with deep optimization for older hardware

What is PocketLLM?

Inference engine for large language models on consumer GPUs, with deep optimization for older hardware