higgsfield
Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters
What is higgsfield?
Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters
Overview of higgsfield
What it is
Higgsfield is an open-source, fault-tolerant GPU orchestration and machine learning framework. It is designed for training models with billions to trillions of parameters, such as Large Language Models (LLMs). The framework serves as a GPU workload manager, handling resource allocation, job queuing, and experiment monitoring.
Key features
- Allocates exclusive and non-exclusive access to compute resources (nodes) for training tasks.
- Supports ZeRO-3 DeepSpeed API and PyTorch’s fully sharded data parallel API for sharding trillion-parameter models.
- Provides a framework for initiating, executing, and monitoring the training of large neural networks on allocated nodes.
- Manages resource contention by maintaining a queue for running experiments.
- Facilitates continuous integration of machine learning development through integration with GitHub and GitHub Actions.
Use cases
Training large language models (LLMs) with billions to trillions of parameters in a distributed, multi-node setting. Managing GPU resource contention among multiple users or experiments. Automating deployment and experiment runs via GitHub workflows.
Getting started
bash$ pip install higgsfield==0.0.3See the repository README for detailed setup instructions on node configuration, environment setup, and running experiments.
License and pricing
Higgsfield is open-source under the Apache-2.0 license.
Based on the project's public README and repository metadata.
