dagster

An orchestration platform for the development, production, and observation of data assets.

What is dagster?

An orchestration platform for the development, production, and observation of data assets.

Overview of dagster

What it is

Dagster is an orchestration platform for the development, production, and observation of data assets. It is designed for developing and maintaining data assets such as tables, data sets, machine learning models, and reports. Users declare data assets as Python functions, and Dagster helps run them at the right time to keep assets up-to-date. It is cloud-native and supports the whole development lifecycle from local development to production.

Key features

  • Declarative programming model: define data assets as Python functions.
  • Integrated lineage and observability: built-in metadata cataloging, diagnostics, and lineage tracking.
  • Best-in-class testability: supports unit tests, integration tests, and CI/CD best practices.
  • Robust multi-tenant, multi-tool orchestration engine that scales technically and organizationally.
  • Unified control plane: centralize metadata with observability, diagnostics, cataloging, and lineage.
  • Growing library of integrations with popular data tools.

Use cases

  • Developing and maintaining data assets like tables, datasets, ML models, and reports.
  • Running pipelines in production with a robust orchestration engine.
  • Managing data complexity with centralized metadata and observability.
  • Implementing CI/CD practices for data pipelines.

Getting started

Dagster is available on PyPI and supports Python 3.9 through 3.14. Install with:

uv add dagster dagster-webserver dagster-dg-cli

For a full tutorial, see the repository README.

License and pricing

Dagster is licensed under Apache 2.0 and is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/dagster-iodagster.svg)](https://yunodb.com/tools/dagster-iodagster?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/dagster-iodagster?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/dagster-iodagster.svg" alt="Featured on YunoDB" height="20"></a>