fast_vector_similarity

High-performance vector similarity library in Rust with Python bindings: Spearman, Kendall, distance correlation, Jensen-Shannon, Hoeffding's D, and bootstrappe

What is fast_vector_similarity?

High-performance vector similarity library in Rust with Python bindings: Spearman, Kendall, distance correlation, Jensen-Shannon, Hoeffding's D, and bootstrapped confidence intervals

Overview of fast_vector_similarity

What it is

The Fast Vector Similarity Library is a high-performance Rust-based tool for computing similarity measures between vectors. It provides Python bindings for seamless integration with Python workflows, making it suitable for data analysis, machine learning, and statistical tasks.

Key features

  • Implements six similarity measures: Spearman's rank-order correlation, Kendall's tau rank correlation, approximate distance correlation, Jensen-Shannon dependency measure, Hoeffding's D measure, and normalized mutual information - Includes bootstrapping functionality to estimate the distribution of similarity measures and improve confidence in results - Performance optimizations: parallel processing using the rayon crate, efficient algorithms (e.g., merge sort for inversion counting), and vectorized operations via the ndarray crate - Benchmarking suite that verifies numerical correctness while measuring performance gains - Python bindings exposing two key functions: py_compute_vector_similarity_stats and py_compute_bootstrapped_similarity_stats, both returning JSON results

Use cases

  • Data analysis and statistical tasks requiring comparison of vectors - Machine learning workflows, such as analyzing text embeddings from large language models (e.g., Llama2)
  • Computing similarity measures between high-dimensional embeddings (e.g., 4096-dimensional vectors) and generating ranked lists of similar vectors

Getting started

bash pip install fast_vector_similarity For Rust projects, add the library to your Cargo.toml file. See the repository README for details.

License and pricing

Open source (license not explicitly stated).

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/dicklesworthstonefast-vector-similarity.svg)](https://yunodb.com/tools/dicklesworthstonefast-vector-similarity?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/dicklesworthstonefast-vector-similarity?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/dicklesworthstonefast-vector-similarity.svg" alt="Featured on YunoDB" height="20"></a>