I’m using DistributedDataParallel to train my model. If one process met an exception and use the try...except block to catch the exception during forward then continue training with a new batch of data, all the process would hang (I guess that is because the fail of synchronization?). How can I handle exceptions in one process and continue training without hanging all the process? Thanks for the help!