Okay, I have a doubt. Before following the tutorial, I was doing the data parallelism using the official Pytorch:DATA PARALLELISM -(https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html#optional-data-parallelism 1) tutorial.
It basically wraps up the model in nn.DataParallel(model).
So nn.DataParallel(model) doesn’t have any effect on loading the input data in parallel and data is still being loaded using a single core only?