I’ll try these changes.
But to let you know, I’m able to run training scripts without running into OOM when I do not have dropout layer. Shouldn’t train_loss += be a problem even without dropout then?