I checked using torch.sum(torch.isnan(val_pred)), and there are no nan or inf values in my validation data (predicted or ground truth masks).
This only seems to be a multi-gpu problem. When using a single GPU this problem doesn’t happen.