I think your example should works as expected whatever there is dist.barrier() or not. DDP are synced at loss.backward().
P.S.
Testing/inference with DDP is somewhat more tricky than training. If using DistributedSampler to scatter your data, You should ensure the number of testing data is divisible to the number of your GPUs, otherwise the results might be incorrect.
Say, there’re 100 batches in your testing set, while there are 8 GPUs (100 % 8 = 4). The DistributedSampler will repeat part of the data and expand it to 104 (104 % 8 = 0) such that the data could be evenly loaded into each GPU.