I am having sort of the same question.
My Geforce 1080 is hanging between those two print statements:
 print(f'test 3')
    for batch_idx, (inputs, targets) in enumerate(trainloader):
        print(f'test 2')

Test 3 is printed. Test 2 not.
After Ctrl+C test 2 is printed and the net is training for a short moment.
Torch version 1.5.0