tiny-vllm
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
What is tiny-vllm?
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
Keep exploring
View all alternatives Related tools
100/100
meetily
Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local
100/100
langgraph
Build resilient agents.
100/100
unity-mcp
Unity MCP acts as a bridge between AI assistants and your Unity Editor. Give your LLM tools to manage assets, control scenes, edit scripts, and automate tasks w