kedro

Overview of kedro

What it is

Kedro is an open-source Python framework hosted by the LF AI & Data Foundation. It is a toolbox for production-ready data engineering and data science pipelines, designed to help create pipelines that are reproducible, maintainable, and modular.

Key features

• Project Template: A standard, modifiable project template based on Cookiecutter Data Science.
• Data Catalog: Lightweight data connectors for saving and loading data across various file formats and file systems, including local and network file systems, cloud object stores, and HDFS; includes data and model versioning for file-based systems.
• Pipeline Abstraction: Automatic dependency resolution between pure Python functions and pipeline visualisation using Kedro-Viz.
• Coding Standards: Test-driven development with pytest, well-documented code with Sphinx, linted code with support for ruff, and standard Python logging.
• Flexible Deployment: Deployment strategies for single or distributed machines, plus support for Argo, Prefect, Kubeflow, AWS Batch, and Databricks.

Use cases

Kedro addresses shortcomings of Jupyter notebooks, one-off scripts, and glue-code by focusing on creating maintainable data engineering and data science code. It enhances team collaboration among members with varying software engineering experience and increases efficiency through modularity and separation of concerns, enabling creation of reusable analytics code.

Getting started

To install Kedro from PyPI:uv pip install kedroAlternatively install with conda:conda install -c conda-forge kedroTo install from source (latest main branch):uv pip install git+https://github.com/kedro-org/kedro@main

License and pricing

Kedro is an open-source Python framework hosted by the LF AI & Data Foundation. The license is not explicitly stated in the provided metadata or README.

Based on the project's public README and repository metadata.