we just want the training to continue normally for next iterations

Assuming you mean continue training with DDP, this exception basically indicates this is not possible - one rank has finished all its inputs while others have not. As a result, if you continue training DDP will hang (probably at SyncBN step) and the exception is designed to indicate this.
The main use case of the exception is to use it as a “signal” to finish the training process. i.e. all processes will raise this exception, and application code can catch it and terminate the main training loop, saving/evaluating the trained model appropriately. Usually inputs are only off by a few examples across ranks, so it should be fine to terminate training at this stage - if this isn’t true then you may want to look into how to better balance the dataset across ranks.