upgini
Data search & enrichment library for Machine Learning → Easily find and add relevant features to your ML & AI pipeline from hundreds of public and premium exter
What is upgini?
Data search & enrichment library for Machine Learning → Easily find and add relevant features to your ML & AI pipeline from hundreds of public and premium external data sources, including open & commercial LLMs
Overview of upgini
What it is
Upgini is an intelligent data search and enrichment library for machine learning. It helps find and add relevant features to ML and AI pipelines from hundreds of public, community, and premium external data sources, including open and commercial large language models (LLMs). The library automatically optimizes connected data sources by generating features using LLMs, graph neural networks (GNNs), and recurrent neural networks (RNNs). It is designed to simplify the process of incorporating external data into supervised ML models, aiming to make external feature enrichment as standard as hyperparameter tuning.
Key features
- Automatically finds only relevant features that improve model accuracy, not just those correlated with the target variable.
- Automated feature generation from sources using LLM-based data augmentation, RNNs, and GraphNNs, with ensembling across multiple data sources.
- Automatic search key augmentation from all connected sources; if certain keys (e.g., postal code) are missing, Upgini attempts to add them based on provided keys.
- Calculates accuracy metrics and uplift after enriching an existing ML model with external features.
- Checks stability of accuracy gain on out-of-time intervals and verification datasets to mitigate risks of unstable external data dependencies.
- Scikit-learn-compatible interface for quick integration with existing ML pipelines.
- Supports binary classification, multiclass classification, regression, and time-series prediction.
Use cases
Upgini is suitable for any supervised ML task on tabular data where external features can boost model performance. Example use cases include salary prediction, retail sales forecasting, improving Kaggle competition solutions, low-code feature engineering for AutoML tools, multivariate time-series forecasting, and credit default prediction. It is also useful for speeding up feature engineering hypothesis tests by providing ready-to-use external features.
Getting started
Install from PyPI:python%pip install upginiFor a Docker setup, clone the repository and build the image as described in the repository README.
License and pricing
The library is licensed under BSD-3-Clause and is open source. Pricing for premium data sources is not stated in the provided metadata or README.
Based on the project's public README and repository metadata.