batch size = 4 is reasonable as well. I guess that it is set to this value in the paper for training. As you’re doing inference you can increase it if you want to.


Datasets are stored on disk and the library uses memory mapping to access them. This allows to load huge datasets at high speed without filling the RAM. The caching is also done on disk: when you call map, the new dataset is written on your disk and calling map again later will reuse the same file.


Hope this answers your questions