I’ve checked the code cudnn.benchmark is True. Though It is very unlikely that pytorch adds more than 10 gb for  8 more images.
I’m unable to understand the meaning of “some initial copies are needed if you are passing expanded tensors, which are needed in a contiguous format”. Can you please explain this? It might help me in locating the problem.