When using DistributedDataParallel (DDP), loss is a local var. DDP will not communicate loss across processes. In order to make this work, you can do the following on each process

run forward on DDP model to calculate loss
create tensor to represent whether the loss is larger than a threshold.
use all_reduce or all_gather to collectively communicate this information to all processes.
After 3, all processes will have the same view on whether they should launch backward+step or not, and hence they can avoid run into desync problems now.