Why does contrastive loss distinguish positive from negative samples? Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why the contrastive loss is not symmetric for positive and for negative examples.
The contrastive loss
$L(A, B) = y|f(A) - f(B)| + (1-y)max(0, m-|f(A) - F(B)|)$
uses a L2 norm for positive examples, but uses a hinge loss for negative examples.
The lecture doesn't explain why that is, and Googling also didn't get anything clear.
Would love a clarification on the deliberate distinction.

The contrastive loss has 2 components:

*

*The positives should be close together, so minimize $\| f(A) - f(B) \|$.

*The negative portion is less obvious, but the idea is that we want negatives to be farther apart.

We can't minimize $\| f(A) - f(B) \|$ for all $A,B$, because this would just map all inputs to the same point. Instead, we want to differentiate between the classes, so we want the dissimilar classes to be far apart.
One way to move negative pairs farther apart would be to just directly maximize the distance, equivalent to minimizing $- \| f(A) - f(B) \|$. But if we do this, then the loss can decrease arbitrarily by moving all points farther apart.
Instead, we just want to move $A, B$ far enough apart, which is controlled by the margin. The loss is positive, until $ \| f(A) - f(B) \| > m$, and then the loss is 0. That's why the loss term for comparing negatives is $ \max\left(0, m - \| f(A) - f(B) \|\right). $ This solves the problem that the loss can decrease indefinitely, because it places a lower-bound at 0.
