Hey there,
With datasets, all the pre-processed (binarized) data will be cached on the disk and will be loaded lazily by default. So you won’t run into any RAM issues at all.
So instead of first binarizing the data one could load it using datasets and then pre-process it using the .map method, this will cache the data which then you can load lazily.
You could refer to the training scripts in transformers examples 8.
Also, refer to the datasets dos 5
Hope this helps!