curvine

AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sand

What is curvine?

AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sandbox project.

Overview of curvine

What it is

Curvine is an AI-native and cloud-native file system that provides a high-performance POSIX file semantic layer on top of cloud object storage. It integrates a multi-tier distributed cache and is designed for large-scale AI workloads and AI Agent platforms. It is a CNCF Sandbox Project.

Key features

  • AI-Native Positioning: First-class support for AI training acceleration and AI Agent cloud-native storage.
  • Multi-Cloud Object Storage Backend: Compatible with object storage services from multiple cloud providers as the durable underlying layer.
  • Cloud-Native Kubernetes Integration: Native CSI driver for dynamic PVC provisioning, volume expansion, and Helm-based deployment.
  • Multi-Tier Cache: Automatic tiering among Memory, SSD, and HDD with transparent hot data promotion.
  • Full POSIX Semantics via FUSE: Presents distributed cached data as a local file system, supporting operations like open, read, write, seek, rename, and list.
  • S3 & HDFS Protocol Compatibility: Read/write through both S3 and HDFS interfaces.
  • Massive Metadata Capacity: A single cluster supports 5 billion small files.

Use cases

  • AI Agent platform storage backing tens of thousands of stateful Pods on Kubernetes.
  • LLM training acceleration by caching datasets and checkpoints near GPU nodes.
  • LLM model distribution acceleration across regions.
  • Multimodal data lake access acceleration without copying data locally.
  • OLAP query acceleration with a hot data cache.
  • Multi-cloud data caching as a unified cache layer.

Getting started

See the repository README for installation and quick-start commands.

License and pricing

Curvine is licensed under the Apache License 2.0. The project is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/curvineiocurvine.svg)](https://yunodb.com/tools/curvineiocurvine?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/curvineiocurvine?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/curvineiocurvine.svg" alt="Featured on YunoDB" height="20"></a>