slothdb

An experimental embedded SQL engine in C++20. Query Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process. Early-stage.

What is slothdb?

An experimental embedded SQL engine in C++20. Query Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process. Early-stage.

Overview of slothdb

What it is

SlothDB is an experimental embedded SQL database engine written in C++20. It is in early development. It allows querying Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process, similar to DuckDB and SQLite. The project is a learning exercise and is early-stage; performance numbers should be treated as anecdotal until architecture work lands.

Key features

  • Query multiple file formats (Parquet, CSV, JSON, Arrow, Avro, SQLite, Excel) directly with SQL, in-process.
  • Natural language query support via a rules parser (sub-10 ms, no model) and an optional local Qwen2.5-Coder model (0.5B or 1.5B) for generating SQL from natural language.
  • Python, Node/JS, and shell interfaces; Python wheels available for Linux, macOS, Windows.
  • HTTP range requests for Parquet files, allowing partial downloads.
  • Predicate pushdown using row-group statistics to skip irrelevant row groups.
  • Top-N pushdown for ORDER BY ... LIMIT queries using a bounded heap.
  • Nested aggregate functions (e.g., ROUND(AVG(x))) and ORDER BY by aggregate alias.
  • Arithmetic type coercion and promotion.
  • GZIP and Zstd Parquet decoding support.
  • Glob pattern support for reading multiple files (e.g., data/*.parquet).
  • Single-threaded WebAssembly build for browser playground.
  • HTTPS Parquet fetching via a Cloudflare CORS proxy.
  • Discord community for bug reports and discussion.

Use cases

  • Ad-hoc querying of data files (Parquet, CSV, JSON, etc.) without loading into a database.
  • Embedding a lightweight SQL engine into applications for data analysis.
  • Natural language querying for non-technical users via the .ask sub-command.
  • Quick data exploration and prototyping in Python, Node.js, or the shell.

Getting started

Install the Python package and run the demo: bash pip install slothdb python -c 'import slothdb; slothdb.demo()' For Node.js: bash npm install @slothdb/wasm For the shell, download a prebuilt binary or build from source; see the repository for instructions.

License and pricing

Licensed under the MIT License. Open source and free.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/souravroy-etlslothdb.svg)](https://yunodb.com/tools/souravroy-etlslothdb?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/souravroy-etlslothdb?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/souravroy-etlslothdb.svg" alt="Featured on YunoDB" height="20"></a>