Note though that the Dataloader uses multiprocessing for loading when you use >0 workers. And autograd won’t track across these.
So in general, this might be tricky to get working if you use multiple workers for loading.