delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
What is delta?
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Overview of delta
What it is
Delta Lake is an open-source storage framework that enables building a Lakehouse architecture. It supports compute engines including Apache Spark, PrestoDB, Flink, Trino, and Hive, and provides APIs for Scala, Java, Rust, Ruby, and Python.
Key features
- Provides ACID transactions on data lakes with serializable isolation for concurrent reads and writes.
- Supports schema enforcement and evolution to handle changing data structures.
- Enables time travel and versioned data using the transaction log.
- Guarantees backward compatibility for all Delta Lake tables; newer versions can always read tables written by older versions.
- Integrates with multiple compute engines: Apache Spark, PrestoDB, Flink, Trino, Apache Hive, Delta Standalone, and Delta Rust API.
- Offers APIs in Scala, Java, Python, Rust, Ruby, and for DataFrameReader/Writer in Spark.
Use cases
Delta Lake is used to build Lakehouse architectures that combine data lake storage with warehouse-like reliability and performance. Typical use cases include reliable batch and streaming data ingestion, incremental data processing, and enabling ACID transactions on data lakes for analytics and machine learning workloads.
Getting started
For quick start guides with Scala, Java, and Python, see the repository README. To compile Delta Lake from source:build/sbt compileTo generate artifacts:build/sbt package
License and pricing
Delta Lake is open source under the Apache License 2.0. Pricing is not stated.
這是什麼
Delta Lake 是一個開放原始碼的儲存框架,可協助建置 Lakehouse 架構,支援 Apache Spark、PrestoDB、Flink、Trino 與 Hive 等運算引擎,並提供 Scala、Java、Rust、Ruby 與 Python 的 API。
主要功能
- 提供資料湖上的 ACID 交易,支援序列化隔離以處理並行讀寫。
- 支援 Schema 強制與演進,以因應資料結構的變化。
- 透過交易日誌實現時間旅行與版本化資料。
- 保證所有 Delta 資料表的回溯相容性;新版本永遠可以讀取舊版本寫入的資料表。
- 整合多種運算引擎:Apache Spark、PrestoDB、Flink、Trino、Apache Hive、Delta Standalone 及 Delta Rust API。
- 提供 Scala、Java、Python、Rust、Ruby 的 API,以及 Spark 中的 DataFrameReader/Writer 支援。
使用情境
Delta Lake 用於建置結合資料湖儲存與資料倉儲可靠性的 Lakehouse 架構。典型情境包含可靠的批次與串流資料導入、增量資料處理,以及在資料湖上對分析和機器學習工作負載啟用 ACID 交易。
快速開始
關於 Scala、Java 與 Python 的快速入門指南,請參閱儲存庫 README。若要從原始碼編譯 Delta Lake:build/sbt compile產生建置產物:build/sbt package
授權與價格
Delta Lake 採用 Apache License 2.0 開放原始碼授權。價格未說明。
Based on the project's public README and repository metadata.