text: Title: Multi-gpu training
  Body: Hi, I am stuck on how multi-GPU training would work for loss functions with more than one negative, particularly [NTXentLoss](https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#ntxentloss). 
  
  In [SimCLR](https://arxiv.org/abs/2002.05709), the number of _negatives_ per some _positive_ pair is taken to be `2 * (N - 1)` (all examples in a minibatch of size `N` that don't belong to that positive pair), and they find ([as other works before them](https://github.com/KevinMusgrave/pytorch-metric-learning)) that the bigger the batch size, the larger the number of negatives, and the better the learned representations.
  
  `DataParallel` and `DistributedDataParallel` divide up a mini-batch, send each partition of examples to a GPU, compute gradients, and then average these gradients before backpropagating. But this means that each GPU is computing a loss with `N/n_gpus` examples and, therefore,  `2 * (N/n_gpus - 1)` negatives per positive pair.
  
  My question is: how might I use a loss function from this library, such as [NTXentLoss](https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#ntxentloss), with `DataParallel` or `DistributedDataParallel` that avoids this "issue"? I.e. that allows me to use multiple GPUs while maintaining `2 * (N - 1)` negatives per positive pair.