datasets
TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...
What is datasets?
TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...
Overview of datasets
What it is
TensorFlow Datasets (TFDS) is a collection of public datasets that are ready to use with TensorFlow, Jax, and other machine learning frameworks. It provides datasets as tf.data.Datasets objects, enabling easy integration into data pipelines. The library is designed to simplify dataset handling by offering standard interfaces for loading, preprocessing, and iterating over data. It is open source and licensed under Apache 2.0.
Key features
- Provides many public datasets as
tf.data.Datasetsobjects. - Supports standard use cases out of the box with simplicity.
- Follows best practices for performance, achieving state-of-the-art speed.
- Ensures determinism and reproducibility, giving all users the same examples in the same order.
- Allows advanced users fine-grained control for customization.
- Includes tutorials, guides, a list of all available datasets, and an API reference in its documentation.
Use cases
- Loading and preprocessing standard machine learning datasets (e.g., MNIST) for training models.
- Building input pipelines for TensorFlow, Jax, or other frameworks.
- Reproducible research requiring deterministic data loading.
- Rapid prototyping where simplicity and performance are needed.
Getting started
python# !pip install tensorflow-datasetsimport tensorflow_datasets as tfdsimport tensorflow as tf# Construct a tf.data.Datasetds = tfds.load('mnist', split='train', as_supervised=True, shuffle_files=True)# Build your input pipelineds = ds.shuffle(1000).batch(128).prefetch(10).take(5)for image, label in ds: pass
License and pricing
TensorFlow Datasets is open source and licensed under Apache 2.0. Pricing is not stated; it is an open source project.
Based on the project's public README and repository metadata.