MACaQ

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and

What is MACaQ?

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/huawei-cslmacaq.svg)](https://yunodb.com/tools/huawei-cslmacaq?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/huawei-cslmacaq?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/huawei-cslmacaq.svg" alt="Featured on YunoDB" height="20"></a>