DatasetsTensorFlow Datasets is a collection of datasets ready to use, with TensorFlow or other Python ML frameworks, such as Jax. All datasets are exposed as tf.data.Datasets, enabling easy-to-use and high-performance input pipelines. To get started see the guide and the list of datasets.This notebook shows how to load TensorFlow Datasets into a Document format that we can use downstream.Installation‚ÄãYou need to install tensorflow and tensorflow-datasets python packages.pip install tensorflowpip install tensorflow-datasetsExample‚ÄãAs an example, we use the mlqa/en dataset.MLQA (Multilingual Question Answering Dataset) is a benchmark dataset for evaluating multilingual question answering performance. The dataset consists of 7 languages: Arabic, German, Spanish, English, Hindi, Vietnamese, Chinese.Homepage: https://github.com/facebookresearch/MLQASource code: tfds.datasets.mlqa.BuilderDownload size: 72.21 MiB# Feature structure of `mlqa/en` dataset:FeaturesDict({    'answers': Sequence({        'answer_start': int32,        'text': Text(shape=(), dtype=string),    }),    'context': Text(shape=(), dtype=string),    'id': string,    'question': Text(shape=(), dtype=string),    'title': Text(shape=(), dtype=string),})import tensorflow as tfimport tensorflow_datasets as tfds# try directly access this dataset:ds = tfds.load('mlqa/en', split='test')ds = ds.take(1)  # Only take a single exampleds    <_TakeDataset element_spec={'answers': {'answer_start': TensorSpec(shape=(None,), dtype=tf.int32, name=None), 'text': TensorSpec(shape=(None,), dtype=tf.string, name=None)}, 'context': TensorSpec(shape=(), dtype=tf.string, name=None), 'id': TensorSpec(shape=(), dtype=tf.string, name=None), 'question': TensorSpec(shape=(), dtype=tf.string, name=None), 'title': TensorSpec(shape=(), dtype=tf.string, name=None)}>Now we have to create a custom function to convert dataset sample into a Document.This is a requirement. There is no standard format for the TF datasets that's why we
TensorFlow Datasets is a collection of datasets ready to use, with TensorFlow or other Python ML frameworks, such as Jax. All datasets are exposed as tf.data.Datasets, enabling easy-to-use and high-performance input pipelines. To get started see the guide and the list of datasets.
TensorFlow Datasets is a collection of datasets ready to use, with TensorFlow or other Python ML frameworks, such as Jax. All datasets are exposed as tf.data.Datasets, enabling easy-to-use and high-performance input pipelines. To get started see the guide and the list of datasets. ->: DatasetsTensorFlow Datasets is a collection of datasets ready to use, with TensorFlow or other Python ML frameworks, such as Jax. All datasets are exposed as tf.data.Datasets, enabling easy-to-use and high-performance input pipelines. To get started see the guide and the list of datasets.This notebook shows how to load TensorFlow Datasets into a Document format that we can use downstream.Installation‚ÄãYou need to install tensorflow and tensorflow-datasets python packages.pip install tensorflowpip install tensorflow-datasetsExample‚ÄãAs an example, we use the mlqa/en dataset.MLQA (Multilingual Question Answering Dataset) is a benchmark dataset for evaluating multilingual question answering performance. The dataset consists of 7 languages: Arabic, German, Spanish, English, Hindi, Vietnamese, Chinese.Homepage: https://github.com/facebookresearch/MLQASource code: tfds.datasets.mlqa.BuilderDownload size: 72.21 MiB# Feature structure of `mlqa/en` dataset:FeaturesDict({    'answers': Sequence({        'answer_start': int32,        'text': Text(shape=(), dtype=string),    }),    'context': Text(shape=(), dtype=string),    'id': string,    'question': Text(shape=(), dtype=string),    'title': Text(shape=(), dtype=string),})import tensorflow as tfimport tensorflow_datasets as tfds# try directly access this dataset:ds = tfds.load('mlqa/en', split='test')ds = ds.take(1)  # Only take a single exampleds    <_TakeDataset element_spec={'answers': {'answer_start': TensorSpec(shape=(None,), dtype=tf.int32, name=None), 'text': TensorSpec(shape=(None,), dtype=tf.string, name=None)}, 'context': TensorSpec(shape=(), dtype=tf.string, name=None), 'id': TensorSpec(shape=(), dtype=tf.string, name=None), 'question': TensorSpec(shape=(), dtype=tf.string, name=None), 'title': TensorSpec(shape=(), dtype=tf.string, name=None)}>Now we have to create a custom function to convert dataset sample into a Document.This is a requirement. There is no standard format for the TF datasets that's why we