Solved by ptrblck in post #2 
              
              
                The problem arises, as the cosine similarity between zero vectors is undefined. 
In PyTorch nn.CosineEmbeddingLoss returns apparently a loss of 1 for zero tensors: 
loss = torch.nn.CosineEmbeddingLoss(reduction='none')
b = torch.tensor([[0.,0.,0.],[0.,1.,0.],[0.,0.,0.]])
print(loss(b, b, torch.ones_…