Hi,
I am trying to implement triplet loss for maximising distance between anchor features and negative features while trying to minimise distance between anchor features and positive features.

class TripletLoss(nn.Module):
    """
    Triplet loss
    Takes embeddings of an anchor sample, a positive sample and a negative sample
    """

    def __init__(self, margin = 1.0):
        super(TripletLoss, self).__init__()
        self.margin = margin

    def forward(self, anchor, positive, negative, size_average=True):
        distance_positive = (anchor - positive).pow(2).sum(1)  # .pow(.5)
        distance_negative = (anchor - negative).pow(2).sum(1)  # .pow(.5)
        losses = F.relu(distance_positive - distance_negative + self.margin)
        return losses.mean() if size_average else losses.sum()


My input feature embeddings in this case are the 2048-dimensional outputs of the last fully connected layer of ResNet152 (from torchvision).
The triplet loss converges in an expected manner during the beginning of training and it’s value lies between 1-200 for a batch size of 10 images. But sometimes during the epoch the value suddenly goes very high like of the order of 10^32 while it was just of the order of 10^0 - 10^1 the batch before in the same epoch.
Could this be due to some overflow issues or is there a problem with my implementation. Any inputs are most welcome. Thanks.