qsv

Blazing-fast, Neuro-symbolic Data-Wrangling toolkit

What is qsv?

Blazing-fast, Neuro-symbolic Data-Wrangling toolkit

Overview of qsv

What it is

qsv is a neuro-symbolic data-wrangling toolkit for querying, slicing, sorting, analyzing, filtering, enriching, transforming, validating, geocoding, formatting, converting, visualizing, FAIRifying, chatting, joining and documenting tabular data. It supports CSV, TSV, SSV, JSONL, parquet, spreadsheets and other formats. Commands are simple, composable and described as blazing fast.

Key features

  • Wide range of commands covering data manipulation, analysis, geocoding, validation, and AI-powered enrichment.
  • Supports multiple tabular formats including CSV, TSV, SSV, JSONL, parquet, and spreadsheets.
  • Includes neuro-symbolic capabilities with LLM integration via OpenAI API-compatible services (e.g., describegpt, apply with summarization).
  • Geocoding support with offline databases (Geonames, Maxmind GeoLite2) and online services (OpenCage, NYC Geoclient, US Census Geocoder).
  • Performance optimizations such as indexing, multithreading, external sorting, and memory-mapped hash tables for large files.
  • Commands are composable and can be piped together for complex workflows.

Use cases

  • Data cleaning and transformation (e.g., removing duplicates, filling missing values, reformatting).
  • Data analysis and exploration (e.g., frequency distributions, statistics, filtering).
  • Geocoding and spatial data enrichment (e.g., converting addresses to coordinates, adding census data).
  • Validation and quality assurance (e.g., detecting null sentinels, checking data integrity).
  • AI-assisted data documentation and querying (e.g., generating data dictionaries, asking questions about data).
  • Large-scale data processing with external sorting and deduplication.

Getting started

See the repository README for installation instructions.

License and pricing

The license is not stated (NOASSERTION). The project is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/dathereqsv.svg)](https://yunodb.com/tools/dathereqsv?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/dathereqsv?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/dathereqsv.svg" alt="Featured on YunoDB" height="20"></a>