Hello,
I am trying to pretrain an XLM Roberta model, but I got some issue. I have trained the model for few thousand steps and get the checkpoint. Then I wanted to continue the pre-training from the checkpoint, but got a memory error “CUDA error memory”  after few steps. I wonder if there is a leak of something like that ? The memory used during the 1st pretraining used around 15.X GB / 16.2 GB so I quite don’t understand what’s going on.