I did, but I had to reduce the batch size to an unreasonable number like 11 per GPU, which is too small. I suspect that storing the result of the loss function is taking most of the memory. I’ll take a look as it further, for now I reduced the size of the model by a factor of 4.