cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

What is cocoindex?

Incremental engine for long horizon agents 🌟 Star if you like it!

Overview of cocoindex

What it is

CocoIndex is an incremental engine for long horizon agents. It turns codebases, meeting notes, inboxes, Slack, PDFs, and videos into live, continuously fresh context for AI agents and LLM apps to reason over effectively, with minimal incremental processing. It is declarative, Python-based, and parallel by default.

Key features

  • Incremental processing: only the delta (changed data) is recomputed, not the entire dataset.
  • Parallel by default: scales from a single repo to petabyte-scale stores.
  • Declarative Python API: define what should be in the target; CocoIndex keeps it in sync forever.
  • Supports multiple data sources: codebases, meeting notes, inboxes, Slack, PDFs, videos.
  • Built on a Rust engine for production-grade performance, parallel chunking, zero-copy transforms, and failure isolation.
  • Backfill and re-run: run once to backfill, re-run anytime to update only changed files.
  • Enterprise scale: designed for large corpora, incremental compute avoids re-embedding every cycle.

Use cases

  • Building AI agents that require continuously fresh context from diverse data sources.
  • Data transformation for AI workloads, with a smart incremental engine for always-fresh, explainable data.
  • Managing large corpora (e.g., petabyte-scale) where batch pipelines would be stale and inefficient.

Getting started

pip install -U cocoindex

Then declare what should be in your target using the Python API. See the repository README for a full example and more details.

License and pricing

Licensed under Apache 2.0. Open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/cocoindex-iococoindex.svg)](https://yunodb.com/tools/cocoindex-iococoindex?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/cocoindex-iococoindex?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/cocoindex-iococoindex.svg" alt="Featured on YunoDB" height="20"></a>