I realized that u are using restnet to train so no need to post ur code
In that case ur restnet model is too big for ur gpu to handle as it is 50 layers deep (as the name implies ‘restnet50’)
Measuring the  memory  use of  ResNet -50 training with a mini-batch of 32 on a typical high performance GPU shows that it needs over 7.5 GB of local DRAM. … This significantly slows down the training time and considerably increases power  consumption