token-throttle

Multi-resource rate limiting for LLM APIs. Reserve tokens before you call, refund what you don't use, stay under the limit across workers.

What is token-throttle?

Multi-resource rate limiting for LLM APIs. Reserve tokens before you call, refund what you don't use, stay under the limit across workers.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/elijastoken-throttle.svg)](https://yunodb.com/tools/elijastoken-throttle?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/elijastoken-throttle?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/elijastoken-throttle.svg" alt="Featured on YunoDB" height="20"></a>