curvine
AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sand
What is curvine?
AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sandbox project.
Overview of curvine
What it is
Curvine is an AI-native and cloud-native file system that provides a high-performance POSIX file semantic layer on top of cloud object storage. It integrates a multi-tier distributed cache and is designed for large-scale AI workloads and AI Agent platforms. It is a CNCF Sandbox Project.
Key features
- AI-Native Positioning: First-class support for AI training acceleration and AI Agent cloud-native storage.
- Multi-Cloud Object Storage Backend: Compatible with object storage services from multiple cloud providers as the durable underlying layer.
- Cloud-Native Kubernetes Integration: Native CSI driver for dynamic PVC provisioning, volume expansion, and Helm-based deployment.
- Multi-Tier Cache: Automatic tiering among Memory, SSD, and HDD with transparent hot data promotion.
- Full POSIX Semantics via FUSE: Presents distributed cached data as a local file system, supporting operations like open, read, write, seek, rename, and list.
- S3 & HDFS Protocol Compatibility: Read/write through both S3 and HDFS interfaces.
- Massive Metadata Capacity: A single cluster supports 5 billion small files.
Use cases
- AI Agent platform storage backing tens of thousands of stateful Pods on Kubernetes.
- LLM training acceleration by caching datasets and checkpoints near GPU nodes.
- LLM model distribution acceleration across regions.
- Multimodal data lake access acceleration without copying data locally.
- OLAP query acceleration with a hot data cache.
- Multi-cloud data caching as a unified cache layer.
Getting started
See the repository README for installation and quick-start commands.
License and pricing
Curvine is licensed under the Apache License 2.0. The project is open source.
Based on the project's public README and repository metadata.