@pritamdamania87
Thank you for your comment 
I am now still in debug, and experienced the same error on DataParallel setting (without hang; just training loop stops at the end of 54 epoch) 
Is this good evidence for the issue?