Hi Quentin! Thank you for your reply.
I have two more questions regarding the map method and caching.


On DPR paper they used batch size = 4 for larger datasets like NQ, Trivia and SQuAD. It’s probably better to use this batch size as well since I’m working with such a big dataset like MSMARCO, do you agree?


When I run the .map() method it appears a message in a logger saying: Caching processed dataset at....
Could you explain what is happening behind the scenes? Are you bringing data from disk to RAM and cache for faster read accesses?