Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

What is Mooncake?

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Overview of Mooncake

What it is

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. It is built on a KVCache-centric disaggregated architecture for LLM serving. Under real workloads, Mooncake’s architecture enables Kimi to handle 75% more requests while adhering to SLOs. The project is open source under the Apache-2.0 license and has been recognized with the Best Paper Award at FAST 2025. It has been integrated into the PyTorch Ecosystem and is used in production for models like Kimi K2.

Key features

  • KVCache-centric disaggregated architecture for LLM serving.
  • Mooncake Store: a distributed KVCache backend supporting content-addressed KV cache pooling and prefix reuse across serving instances.
  • Mooncake Transfer Engine: enables RDMA-based zero-copy transfer of KV cache and other tensors across devices and machines.
  • Supports PD (Prefill-Decode) and EPD (Encode-Prefill-Decode) disaggregation.
  • Integrated with multiple inference engines including vLLM, SGLang, TensorRT-LLM, LMDeploy, and others.
  • Provides a checkpoint-engine backend for peer-to-peer weight synchronization between training and rollout workers.

Use cases

  • Serving large language models (LLMs) with high throughput and low latency.
  • Disaggregated inference where prefill and decode stages are separated across different nodes.
  • Multi-node distributed reinforcement learning (RL) training with efficient weight and hidden state transfer.
  • Multimodal serving with encoder-decoder disaggregation.
  • Cloud-native deployment with elastic, scalable, and high-performance serving.

Getting started

See the repository README for installation and quick-start instructions.

License and pricing

Licensed under Apache-2.0. The project is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/kvcache-aimooncake.svg)](https://yunodb.com/tools/kvcache-aimooncake?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/kvcache-aimooncake?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/kvcache-aimooncake.svg" alt="Featured on YunoDB" height="20"></a>