Which PyTorch version are you using?
I assume you are using the default DataLoader from the tutorial, i.e. num_workers=0.
Could you execute the code line by line in a Python shell and see, if the creation of DataParallel hangs or if it’s the training loop using the DataLoader?