@jpeg729 When you run one model in parallel on 3 GPUs the error is duplicated I think. The reason for 3 values is due to an error telling me it was necessary ;-).
But you’re right about synchronizing the updates. I take the mean (torch.mean()) of the 3 outputs.