Thanks, I’m having the save problem.
BTW, I have a question.
What is the best practice to train a model from scratch with large datasets like entire Wikipedia?
Tweak LineByLineTextDataset ?
or use other Dataset classes ?
or just use more powerful machine?