Answers.Body: <p>In the first you compare A and B. So if A is closer to B and y= 1 then it's better, that's it.</p>
  <p>In the second, you compare A to P and A to N simultaneously.</p>
  <p>So if A is closer to P it's good, but it's better if A is more distant to N. So with triplet you have a ranking.</p>
  <p>I hope this was clear.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-19T20:29:48.100Z
Answers.LastActivityDate: 2022-10-19T20:29:48.100Z
Answers.UserDisplayName: Skobo Do
Answers.Body: <p>I think the presenter is referring to ranking in terms of the sample triplet.  With the contrastive loss, we take a pair <span class="math-container">$(A, B)$</span> and simply measure how similar <span class="math-container">$A$</span> is to <span class="math-container">$B$</span>.  With triplet loss, we are measuring the similarity between an Anchor image <span class="math-container">$A$</span> and a positive sample <span class="math-container">$P$</span> and negative sample <span class="math-container">$N$</span>, so intuitively we are comparing the similarities <span class="math-container">$s(A, P)$</span> with <span class="math-container">$s(A, N)$</span>.  In the lecture, she presents this math.</p>
  <p><span class="math-container">$$||f(A)-f(P)||^2&lt;||f(A)-f(N)||^2$$</span></p>
  <p>which is how we derive the formula for triplet loss.  Triplet loss enforces this condition, i.e. that the distance between the embeddings for <span class="math-container">$A$</span> and <span class="math-container">$P$</span> are closer than the distance of the embeddings between <span class="math-container">$A$</span> and <span class="math-container">$N$</span>.  Think of rank as an ordering of the distances <span class="math-container">$s(A, P)$</span> and <span class="math-container">$s(A, N)$</span>.  We want to achieve the rank condition expressed in the above equation.  Therefore, the triplet loss measurement will be in terms of rank because it is dependent on <span class="math-container">$s(A, P)$</span> compared to <span class="math-container">$s(A, N)$</span>.  To summarize, contrastive loss is learning similarity by pushing apart dissimilar embeddings and pushing towards each other similar embeddings based purely on the distance between the embeddings.  Triplet loss is pushing together similar embeddings and pushing apart dissimilar embeddings concurrently by enforcing a ranking condition where <span class="math-container">$s(A, P) &lt; s(A, N)$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-19T21:27:24.597Z
Answers.Text: Please don't upload math equations and text as images, it isn't searchable. There are [tools available](https://or.meta.stackexchange.com/a/372/51) to convert PDFs to text and [MathJax](https://or.meta.stackexchange.com/q/5/51). Thanks.
Answers.UserDisplayName: Rob
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-19T15:44:17.777Z
Answers.LastActivityDate: 2022-10-20T08:33:24.227Z
Answers.LastEditDate: 2022-10-20T08:33:24.227Z
Answers.UserDisplayName: Jacob Regan
Body: <p>I am looking at <a href="https://youtu.be/6e65XfwmIWE?t=1175" rel="nofollow noreferrer">this lecture</a>, which states (link to exact time):</p>
  <blockquote>
  <p>What the triplet loss allows us in contrast to the contrastive loss is
  that we can learn a ranking. So it's not only about similarity, being
  closer together or being further apart, but now we want to learn how
  much closer am I compared to another image.</p>
  </blockquote>
  <p>The contrastive loss</p>
  <p><span class="math-container">$L(A, B) = y|f(A) - f(B)| + (1-y)max(0, m-|f(A) - F(B)|)$</span></p>
  <p>would push similar samples together, and dissimilar samples apart.</p>
  <p>The triplet loss</p>
  <p><span class="math-container">$L(A, P, N) = max(0, |f(A) - f(P)| - |f(A) - f(N)| + m) $</span></p>
  <p>would push the positive close to the anchor, and the negative away from the anchor.</p>
  <hr />
  <p>I fail to see why the quoted claim is or isn't true in either of these losses. To me, it looks like &quot;same&quot; samples are pushed together, and &quot;different&quot; samples are pushed apart by both.<br />
  Furthermore, with the contrastive loss, the distance in the embedding space would be, as I understand it, the ranking- which is claimed to only exist with the triplet loss.</p>
  <p>Is there a clearer reference for this, or just a simple answer?</p>
  
Tags: <deep-learning><reference-request><loss><triplet-loss-function><contrastive-learning>
Title: Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?
UserDisplayName: Gulzar
ThreadText: # Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?
  
  (Asked by: username_0 on 2022-05-03)
  
  I am looking at [this lecture](https://youtu.be/6e65XfwmIWE?t=1175), which states (link to exact time):
  
  > What the triplet loss allows us in contrast to the contrastive loss is
  > that we can learn a ranking. So it's not only about similarity, being
  > closer together or being further apart, but now we want to learn how
  > much closer am I compared to another image.
  
  The contrastive loss
  
  $L(A, B) = y|f(A) - f(B)| + (1-y)max(0, m-|f(A) - F(B)|)$
  
  would push similar samples together, and dissimilar samples apart.
  
  The triplet loss
  
  $L(A, P, N) = max(0, |f(A) - f(P)| - |f(A) - f(N)| + m) $
  
  would push the positive close to the anchor, and the negative away from the anchor.
  
  ---
  
  I fail to see why the quoted claim is or isn't true in either of these losses. To me, it looks like "same" samples are pushed together, and "different" samples are pushed apart by both.  
  Furthermore, with the contrastive loss, the distance in the embedding space would be, as I understand it, the ranking- which is claimed to only exist with the triplet loss.
  
  Is there a clearer reference for this, or just a simple answer?
  
  **Tags:** deep-learning, reference-request, loss, triplet-loss-function, contrastive-learning
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-19. Score: 1
  
  In the first you compare A and B. So if A is closer to B and y= 1 then it's better, that's it.
  
  In the second, you compare A to P and A to N simultaneously.
  
  So if A is closer to P it's good, but it's better if A is more distant to N. So with triplet you have a ranking.
  
  I hope this was clear.
  
  
  ---
  
  ### Answer by username_2 on 2022-10-19. Score: 0
  
  I think the presenter is referring to ranking in terms of the sample triplet. With the contrastive loss, we take a pair $(A, B)$ and simply measure how similar $A$ is to $B$. With triplet loss, we are measuring the similarity between an Anchor image $A$ and a positive sample $P$ and negative sample $N$, so intuitively we are comparing the similarities $s(A, P)$ with $s(A, N)$. In the lecture, she presents this math.
  
  $$||f(A)-f(P)||^2<||f(A)-f(N)||^2$$
  
  which is how we derive the formula for triplet loss. Triplet loss enforces this condition, i.e. that the distance between the embeddings for $A$ and $P$ are closer than the distance of the embeddings between $A$ and $N$. Think of rank as an ordering of the distances $s(A, P)$ and $s(A, N)$. We want to achieve the rank condition expressed in the above equation. Therefore, the triplet loss measurement will be in terms of rank because it is dependent on $s(A, P)$ compared to $s(A, N)$. To summarize, contrastive loss is learning similarity by pushing apart dissimilar embeddings and pushing towards each other similar embeddings based purely on the distance between the embeddings. Triplet loss is pushing together similar embeddings and pushing apart dissimilar embeddings concurrently by enforcing a ranking condition where $s(A, P) < s(A, N)$.
  
  > **Comments:**
  > **username_3**: Please don't upload math equations and text as images, it isn't searchable. There are [tools available](https://or.meta.stackexchange.com/a/372/51) to convert PDFs to text and [MathJax](https://or.meta.stackexchange.com/q/5/51). Thanks.  
  
  ---
  
  