olake

Overview of olake

What it is

OLake Go is an open-source data ingestion engine for replicating databases, S3, and Kafka into Apache Iceberg or plain Parquet. It supports sources including PostgreSQL, MySQL, MongoDB, Oracle, DB2, MSSQL, Kafka, and S3. Destinations include Iceberg (with Glue, Hive, JDBC, REST catalogs) and Parquet. It provides a web UI and CLI for managing pipelines.

Key features

• Smart sync: Full load and CDC replication with automatic schema discovery and evolution.
• High throughput: Benchmarks report 580,113 RPS for Postgres full load, 139,773 RPS for MySQL full load, and 55,555 RPS for Postgres CDC.
• Exactly once delivery and Arrow writes for accuracy and speed.
• Iceberg-native with support for Glue, Hive, JDBC, and REST catalogs.
• Self-serve UI deployable via Docker Compose.
• Infrastructure-light: no Spark, Flink, Kafka, or Debezium required.

Use cases

Replicating transactional databases (PostgreSQL, MySQL, MongoDB, Oracle, DB2, MSSQL) to Apache Iceberg for real-time analytics. Ingesting event streams from Apache Kafka into Iceberg. Copying data from Amazon S3 or S3-compatible storage to Iceberg or Parquet. Building open data lakehouses without vendor lock-in.

Getting started

Run the OLake UI stack using Docker Compose:shcurl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose.yml | docker compose -f - up -dAccess the UI at http://localhost:8000 and log in with default credentials admin / password. For CLI usage, see the repository README.

License and pricing

Licensed under Apache-2.0. Open source (free).

Based on the project's public README and repository metadata.