optax
Optax is a gradient processing and optimization library for JAX.
What is optax?
Optax is a gradient processing and optimization library for JAX.
Overview of optax
What it is
Optax is a gradient processing and optimization library for JAX. It provides building blocks that can be recombined in custom ways to facilitate research. The library focuses on small, composable components that can be effectively combined into custom solutions. Implementations prioritize readability and structuring code to match standard equations over code reuse. Optax was originally developed as jax.experimental.optix and later moved out of experimental as a standalone open-source library renamed Optax.
Key features
- Provides simple, well-tested, efficient implementations of core optimization components.
- Enables combining low-level ingredients into custom optimizers or other gradient processing components.
- Includes implementations of many popular optimizers (e.g., Adam) and loss functions (e.g., mean squared error via
optax.l2_loss). - Supports easy recombination of building blocks for custom gradient processing.
- Allows anyone to contribute new ideas to accelerate adoption.
Use cases
Optax is used for gradient processing and optimization in JAX-based machine learning research and development. It is suitable for training neural networks, implementing custom optimizers, and experimenting with gradient transformations. The library is designed to improve research productivity by enabling researchers to easily combine low-level components into custom solutions.
Getting started
Install the latest released version from PyPI:shpip install optaxOr install the latest development version from GitHub:shpip install git+https://github.com/google-deepmind/optax.gitQuick usage example:pythonoptimizer = optax.adam(learning_rate)params = {'w': jnp.ones((num_weights,))}opt_state = optimizer.init(params)compute_loss = lambda params, x, y: optax.l2_loss(params['w'].dot(x), y)grads = jax.grad(compute_loss)(params, xs, ys)updates, opt_state = optimizer.update(grads, opt_state)params = optax.apply_updates(params, updates)
License and pricing
Optax is open source under the Apache-2.0 license. No pricing information is provided; it is free to use.
Based on the project's public README and repository metadata.