trainer

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

What is trainer?

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

Overview of trainer

What it is

Kubeflow Trainer is a Kubernetes-native distributed AI platform for scalable large language model (LLM) fine-tuning, hyperparameter optimization, and training of AI models across a wide range of frameworks, including PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM, JAX, XGBoost, and more. It brings MPI to Kubernetes, orchestrating multi-node, multi-GPU distributed jobs efficiently across high-performance computing (HPC) clusters. The project is currently in alpha status.

Key features

  • Supports multiple AI frameworks: PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM, JAX, XGBoost, and more.
  • MPI integration for multi-node, multi-GPU distributed training on Kubernetes.
  • Integrates with Kueue for topology-aware scheduling and multi-cluster job dispatching, Slurm Bridge for hybrid Kubernetes/Slurm clusters, and KAI Scheduler for GPU-aware scheduling.
  • Reuses Kubernetes-native building blocks like JobSet and LeaderWorkerSet for workload orchestration.
  • Provides a distributed data cache for streaming large-scale data with zero-copy transfer to GPU nodes.
  • Offers a Python SDK with TrainJob and Runtime APIs for developing and fine-tuning LLMs.

Use cases

  • Scalable fine-tuning of large language models (LLMs) on Kubernetes.
  • Hyperparameter optimization for AI models.
  • Distributed training across multiple frameworks and hardware configurations.
  • High-performance computing (HPC) workloads requiring MPI communication between GPU nodes.

Getting started

To deploy the latest stable release, run the following Helm command: bash helm install kubeflow-trainer oci://ghcr.io/kubeflow/charts/kubeflow-trainer --namespace kubeflow-system --create-namespace --set runtimes.defaultEnabled=true For full installation and usage instructions, refer to the official Kubeflow Trainer documentation.

License and pricing

The project is licensed under Apache-2.0 and is open source (free to use).

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/kubeflowtrainer.svg)](https://yunodb.com/tools/kubeflowtrainer?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/kubeflowtrainer?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/kubeflowtrainer.svg" alt="Featured on YunoDB" height="20"></a>