I get following warning when running pytorch 0.4.1 with python 3.6 on two K80 GPU
/python3.6/site-packages/torch/nn/parallel/_functions.py:58: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
I have a vector of loss returned. 2 GPU returns 2 losses. For now, I just use torch.mean().
Is this the right thing to do?