kuberay
A toolkit to run Ray applications on Kubernetes
What is kuberay?
A toolkit to run Ray applications on Kubernetes
Overview of kuberay
What it is
KubeRay is an open-source Kubernetes operator that simplifies the deployment and management of Ray applications on Kubernetes. It is a Cloud Native Computing Foundation sandbox project. The project provides several components: KubeRay core, which includes three custom resource definitions (RayCluster, RayJob, and RayService), and the KubeRay ecosystem, which includes optional components such as a kubectl plugin, an API server, and a dashboard.
Key features
- KubeRay fully manages the lifecycle of RayCluster, including cluster creation/deletion, autoscaling, and fault tolerance.
- RayJob automatically creates a RayCluster and submits a job when the cluster is ready; it can also be configured to automatically delete the RayCluster once the job finishes.
- RayService combines a RayCluster and a Ray Serve deployment graph, offering zero-downtime upgrades and high availability.
- The kubectl plugin (Beta, since v1.3.0) simplifies common workflows for deploying Ray on Kubernetes.
- The KubeRay APIServer (Alpha) provides a layer of simplified configuration for KubeRay resources.
- The KubeRay Dashboard (Experimental, since v1.4.0) enables users to view and manage KubeRay resources.
- KubeRay integrates with the Kubernetes ecosystem, including observability tools (e.g., Prometheus, Grafana, py-spy), queuing systems (e.g., Volcano, Apache YuniKorn, Kueue), and ingress controllers (e.g., Nginx).
Use cases
KubeRay is used for running a wide range of Ray workloads on Kubernetes, including training, LLM online inference, batch inference, and more. It is used by organizations such as Workday, Klaviyo, Niantic, Samsara, Google, DoorDash, Red Hat, Instacart, Spotify, Alibaba Cloud, Apple, Microsoft, AWS, Roblox, Airbnb, eBay, Reddit, IBM, Hinge, and Rubrik for scaling machine learning platforms, model serving, batch inference, and distributed training.
Getting started
See the repository README for quick start guides for RayCluster, RayJob, and RayService.
License and pricing
This project is licensed under the Apache-2.0 License. It is open source with no stated pricing.
Based on the project's public README and repository metadata.