Hi, I’m learning to write distributed program by looking at the source code of torchvision
I was confused when I saw this part about evaluate.

  
      github.com
  
  
    pytorch/vision/blob/78159d61b966ec32ac330c28bea32e9e12fe6eb4/references/detection/engine.py#L103


    res = {target["image_id"].item(): output for target, output in zip(targets, outputs)}
    evaluator_time = time.time()
    coco_evaluator.update(res)
    evaluator_time = time.time() - evaluator_time
    metric_logger.update(model_time=model_time, evaluator_time=evaluator_time)

# gather the stats from all processes
metric_logger.synchronize_between_processes()
print("Averaged stats:", metric_logger)
coco_evaluator.synchronize_between_processes()

# accumulate predictions from all images
coco_evaluator.accumulate()
coco_evaluator.summarize()
torch.set_num_threads(n_threads)
return coco_evaluator



  
  
    
    
  
  


coco_evaluator is a warpper for  pycocotools.cocoeval.COCOeval
When all the processes have been evaluated, we need to gather all the results.
The core of synchronize_between_processes is torch.distributed.all_gather

  
      github.com
  
  
    pytorch/vision/blob/78159d61b966ec32ac330c28bea32e9e12fe6eb4/references/detection/utils.py#L75 1


    def __str__(self):
        return self.fmt.format(
            median=self.median,
            avg=self.avg,
            global_avg=self.global_avg,
            max=self.max,
            value=self.value)


def all_gather(data):
    """
    Run all_gather on arbitrary picklable data (not necessarily tensors)
    Args:
        data: any picklable object
    Returns:
        list[data]: list of data gathered from each rank
    """
    world_size = get_world_size()
    if world_size == 1:
        return [data]



  
  
    
    
  
  


That’s follow those steps:

coco.Eval computed on CPU in single process

synchronize_between_processes transfer coco evaluate result to GPU
doing all_gather over all process on GPU
all_gather result transfer to CPU for logger

I’s so trouble, Why not just use CPU  directly, here are some other ways，like, pipe or queue communicate between processes.
Is it because there is some reason here, to use GPU instead of  multiprocessing.pipe/multiprocessing.queue?