Daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

What is Daft?

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Overview of Daft

What it is

Daft is a high-performance data engine for AI and multimodal workloads. It processes images, audio, video, and structured data at any scale. The engine is Python-native with Rust under the hood, and can scale from local to distributed clusters on Ray or Kubernetes. It is licensed under Apache 2.0 and is open source.

Key features

  • Native multimodal processing: Process images, audio, video, and embeddings alongside structured data in a single framework - Built-in AI operations: Run LLM prompts, generate embeddings, and classify data at scale using OpenAI, Transformers, or custom models - Python-native, Rust-powered: Skip the JVM complexity with Python at its core and Rust under the hood for blazing performance - Seamless scaling: Start local, scale to distributed clusters on Ray, Kubernetes - Universal connectivity: Access data anywhere (S3, GCS, Iceberg, Delta Lake, Hugging Face, Unity Catalog)
  • Out-of-box reliability: Intelligent memory management and sensible defaults eliminate configuration headaches

Use cases

Daft is designed for AI and multimodal data pipelines. Use cases include processing large volumes of images, audio, video, and structured data for machine learning; running LLM prompts and generating embeddings at scale; building data preprocessing workflows for AI training; and performing classification or inference on multimodal datasets.

Getting started

Install Daft with pip: pip install daft Requires Python 3.10 or higher. For a quickstart with a real-world e-commerce dataset, see the repository README.

License and pricing

Licensed under Apache-2.0. Open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/eventual-incdaft.svg)](https://yunodb.com/tools/eventual-incdaft?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/eventual-incdaft?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/eventual-incdaft.svg" alt="Featured on YunoDB" height="20"></a>