synthetic-data-generator
SDG is a specialized framework designed to generate high-quality structured tabular data.
What is synthetic-data-generator?
SDG is a specialized framework designed to generate high-quality structured tabular data.
Overview of synthetic-data-generator
What it is
The Synthetic Data Generator (SDG) is a specialized framework designed to generate high-quality structured tabular data. Synthetic data does not contain any sensitive information, yet it retains the essential characteristics of the original data, making it exempt from privacy regulations such as GDPR and ADPPA.
Key features
- Supports a wide range of statistical data synthesis algorithms, including an LLM-based synthetic data generation model.
- Optimized for big data, effectively reducing memory consumption.
- Supports differential privacy, anonymization and other methods to enhance the security of synthetic data.
- Supports expansion of models, data processing, data connectors, etc. in the form of plug-in packages.
- Includes a Data Processor module that converts data column formats (e.g., Datetime columns) before feeding into the model and reverses the conversion after generation.
- Supports automatic detection of data column relationships and allows for relationship specification.
Use cases
High-quality synthetic data can be safely utilized across various domains including data sharing, model training and debugging, system development and testing.
Getting started
bashdocker pull idsteam/sdgx:latest``````bashpip install sdgx``````bashgit clone [email protected]:hitsz-ids/synthetic-data-generator.gitpip install .# Or install from gitpip install git+https://github.com/hitsz-ids/synthetic-data-generator.git
License and pricing
License: Apache-2.0. Pricing: open source (no cost).
Based on the project's public README and repository metadata.