Hi,
I have an output vector and a ground truth vector. I want to minimize the cosine distance between them but i have a constraint - i want that the output vector will have a l2-norm of 1 so i have created the following custom loss:
class MultiCosineLoss(nn.Module):
    def __init__(self):
        super(MultiCosineLoss, self).__init__()
    def forward(self, outputs, targets):
        cosine_target = Variable(torch.ones(len(outputs)).cuda())
        loss_func = nn.CosineEmbeddingLoss().cuda()
        loss1 = loss_func(outputs, targets, cosine_target)

        loss_func2 = nn.L1Loss().cuda()
        output1_normed = output1.norm(p=2, dim=1)
        target = Variable(torch.ones(len(outputs), 1)).cuda()
        loss2 = loss_func2(outputs_normed, targets)

        return loss1 + loss2

Outputs is a batch of size 32 that each is a vector of 60 dims.
So i am trying to minimize the cosine distance + |1-norm(output)|
But when i add the L1Loss the network does not learn anything and without the L1Loss i get accuracy of 98%.
So i guess that i am not using the L1 loss appropriately.
Can you please advise how can i solve it?
Thanks!