Saving and reloading a dataset. In this video we'll take a look saving a dataset in various formats, and explore the ways to reload the saved data. When you download a dataset, the processing scripts and data are stored locally on your computer. The cache allows the Datasets library to avoid re-downloading or processing the entire dataset every time you use it. The data is stored in the form of Arrow tables whose location can be found by accessing the dataset's cache_files attribute. In this example, we've downloaded the allocine dataset from the Hugging Face Hub and you can see there are three Arrow files stored in the cache, one for each split. But in many cases, you'll want to save your dataset in a different location or format. As shown in the table, the Datasets library provides four main functions to achieve this. You're probably familiar with the CSV and JSON formats, both of which are great if you want to save small to medium-sized datasets. But if your dataset is huge, you'll want to save it in either the Arrow or Parquet formats. Arrow files are great if you plan to reload or process the data in the near future. Parquet files are designed for long-term disk storage and are very space efficient. Let's take a closer look at each format. To save a Dataset or a DatasetDict object in the Arrow format we use the save_to_disk function. As you can see in this example, we simply provide the path we wish to save the data to, and the Datasets library will automatically create a directory for each split to store the Arrow table and metadata. Since we're dealing with a DatasetDict object that has multiple splits, this information is also stored in the dataset_dict.json file. Now when we want to reload the Arrow datasets, we use the load_from_disk function. We simply pass the path of our dataset directory and voila the original dataset is recovered! If we want to save our datasets in the CSV format we use the to_csv function. In this case you'll need to loop over the splits of the DatasetDict object and save each dataset as an individual CSV file. Since the to_csv file is based on the one from Pandas, you can pass keyword arguments to configure the output. In this example, we've set the index argument to None to prevent the dataset's index column from being included in the CSV files. To reload our CSV files, we use the load_dataset function together with the csv loading script and data_files argument which specifies the filenames associated with each split. As you can see in this example, by providing all the splits and their filenames, we've recovered the original DatasetDict object. To save a dataset in the JSON or Parquet formats is very similar to the CSV case. We use either the to_json function for JSON files or the to_parquet function for Parquet ones. And just like the CSV case, we need to loop over the splits and save each one as an individual file. Once our datasets are saved as JSON or Parquet files, we can reload them again with the appropriate script in the load_dataset function, and a data_files argument as before. This example shows how we can reload our saved datasets in either format. And with that you now know how to save your datasets in various formats!