I don’t know where the tokenizing is performed, but the __getitem__ method of your Dataset will be called to load and process each sample.
If the tokenizing (and other processing steps) are performed there, they will be executed in each iteration for all epochs, so you might want to preprocess the data beforehand.