Hi,

It’s not easy to manage the gradients of all parameters. So if I need to sum the gradients across 2 GPUs, do you think that adding loss_total*=2 below 'loss_total=loss_b + loss_m + loss_c + loss_s` can achieve this effect?