You are bottlenecked by data loading time. The average time spent loading data in your log is 382 ms out of 465 ms. Make sure your data is on a local SSD and try increasing the number of data loading workers (-j in the ImageNet example code 3)