paxml

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelizatio

What is paxml?

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry leading model flop utilization rates.

Overview of paxml

What it is

Paxml (also known as Pax) is a machine learning framework built on top of Jax, designed for training large-scale models. It enables advanced and fully configurable experimentation and parallelization.

Key features

  • Built on Jax for high-performance numerical computation.
  • Supports advanced and fully configurable experimentation.
  • Provides parallelization capabilities for large-scale model training.
  • Demonstrated industry leading model flop utilization rates.
  • Includes example configurations for models ranging from 1B to trillions of parameters.
  • Offers support for Cloud TPU VMs, including single-host and multi-slice setups.
  • Provides documentation and Jupyter Notebook tutorials.

Use cases

Paxml is used for training large-scale machine learning models, particularly language models. It is suitable for researchers and engineers working on advanced experimentation and parallelization with Jax, especially on Cloud TPU hardware. Example use cases include training 1B, 16B, and GPT3-XL models on the c4 dataset.

Getting started

To set up a Cloud TPU VM and install Paxml, follow the commands below. For detailed instructions, see the repository README.bashexport ZONE=us-central2-bexport VERSION=tpu-vm-v4-baseexport PROJECT=export ACCELERATOR=v4-8export TPU_NAME=paxmlgcloud compute tpus tpu-vm create $TPU_NAME --zone=$ZONE --version=$VERSION --project=$PROJECT --accelerator-type=$ACCELERATORAfter SSHing into the VM:bashpython3 -m pip install -U pippython3 -m pip install paxml jax[tpu] -f https://storage.googleapis.com/jax-releases/libtpu_releases.html

License and pricing

Paxml is licensed under Apache-2.0. It is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/googlepaxml.svg)](https://yunodb.com/tools/googlepaxml?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/googlepaxml?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/googlepaxml.svg" alt="Featured on YunoDB" height="20"></a>