modin
Modin: Scale your Pandas workflows by changing a single line of code
What is modin?
Modin: Scale your Pandas workflows by changing a single line of code
Overview of modin
What it is
Modin is a drop-in replacement for pandas. While pandas is single-threaded, Modin lets you instantly speed up your workflows by scaling pandas so it uses all of your cores. Modin works especially well on larger datasets, where pandas becomes painfully slow or runs out of memory. Also, Modin comes with the additional APIs to improve user experience.
Key features
- Drop-in replacement for pandas: change one line of code (import modin.pandas as pd)
- Scales pandas workflows to use all CPU cores, providing speed-ups of up to 4x on a laptop with 4 physical cores
- Works especially well on larger datasets where pandas becomes slow or runs out of memory
- Supports multiple compute engines: Ray, Dask, and MPI through unidist
- Automatically detects which engine is installed and uses it for scheduling
Use cases
Scaling pandas workflows on larger datasets that are too slow or too large for pandas. Speeding up pandas operations on multi-core machines, including laptops. Handling datasets that exceed the available memory.
Getting started
pip install "modin[all]"
Then replace your pandas import with:
import modin.pandas as pd
For conda installation:
conda install -c conda-forge modin-all
License and pricing
Apache-2.0 license, open source (free).
Based on the project's public README and repository metadata.