Hi,
As I understand it you’d like to broadcast the value of acc_max to all ranks from rank 0?
In that case, you can simply convert it to a tensor and call dist.broadcast:
acc_max = checkpoint['acc_max']
acc_tensor = torch.zeros(1) if args.local_rank != 0 else torch.tensor([acc_max])
torch.distributed.broadcast(acc_tensor)
acc_max_from_rank_0 = acc_tensor.item()


  
      pytorch.org
  
  
    

Distributed communication package - torch.distributed — PyTorch 1.7.0...