Hi ptrblck,
How are you?
I am curretly running the model with:
import os
os.environ['CUDA_LAUNCH_BLOCKING'] = "1"

I haven’t tried to run it on the CPU. I will copy the stack trace whenn it fails at epoch 39.
this is releated to my other post about NaN when using torch.nn.BCEWithLogitsLoss at epoch 39…
cheers for now,
chaslie