qsv
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
What is qsv?
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
Overview of qsv
What it is
qsv is a neuro-symbolic data-wrangling toolkit for querying, slicing, sorting, analyzing, filtering, enriching, transforming, validating, geocoding, formatting, converting, visualizing, FAIRifying, chatting, joining and documenting tabular data. It supports CSV, TSV, SSV, JSONL, parquet, spreadsheets and other formats. Commands are simple, composable and described as blazing fast.
Key features
- Wide range of commands covering data manipulation, analysis, geocoding, validation, and AI-powered enrichment.
- Supports multiple tabular formats including CSV, TSV, SSV, JSONL, parquet, and spreadsheets.
- Includes neuro-symbolic capabilities with LLM integration via OpenAI API-compatible services (e.g.,
describegpt,applywith summarization). - Geocoding support with offline databases (Geonames, Maxmind GeoLite2) and online services (OpenCage, NYC Geoclient, US Census Geocoder).
- Performance optimizations such as indexing, multithreading, external sorting, and memory-mapped hash tables for large files.
- Commands are composable and can be piped together for complex workflows.
Use cases
- Data cleaning and transformation (e.g., removing duplicates, filling missing values, reformatting).
- Data analysis and exploration (e.g., frequency distributions, statistics, filtering).
- Geocoding and spatial data enrichment (e.g., converting addresses to coordinates, adding census data).
- Validation and quality assurance (e.g., detecting null sentinels, checking data integrity).
- AI-assisted data documentation and querying (e.g., generating data dictionaries, asking questions about data).
- Large-scale data processing with external sorting and deduplication.
Getting started
See the repository README for installation instructions.
License and pricing
The license is not stated (NOASSERTION). The project is open source.
Based on the project's public README and repository metadata.