N-pair loss vs InfoNCE? I was reading this awesome blog
And it introduces two losses:

*

*N-pair loss

$$
\begin{aligned}
\mathcal{L}_\text{N-pair}(\mathbf{x}, \mathbf{x}^+, \{\mathbf{x}^-_i\}^{N-1}_{i=1}) 
&= \log\big(1 + \sum_{i=1}^{N-1} \exp(f(\mathbf{x})^\top f(\mathbf{x}^-_i) - f(\mathbf{x})^\top f(\mathbf{x}^+))\big) \\
&= -\log\frac{\exp(f(\mathbf{x})^\top f(\mathbf{x}^+))}{\exp(f(\mathbf{x})^\top f(\mathbf{x}^+)) + \sum_{i=1}^{N-1} \exp(f(\mathbf{x})^\top f(\mathbf{x}^-_i))}
\end{aligned}
$$

*

*InfoNCE loss

$$
\mathcal{L}_\text{InfoNCE} = - \mathbb{E} \Big[\log \frac{f(\mathbf{x}, \mathbf{c})}{\sum_{\mathbf{x}' \in X} f(\mathbf{x}', \mathbf{c})} \Big]
$$
I think the InfoNCE loss can be better understood by papers like MOCO in which the loss is defined as
For query $q$ we have a set of keys $\{ k_1, ..., k_N\}$ where among those $N$ keys there is 1 positive key $k_+$, and InfoNCE is
$$
\mathcal{L}_q = - \log\dfrac{\exp(q k_+ / \tau)}{\sum_i \exp(q k_i / \tau)}
$$
Now my question is, other than the temperature, what are the differences between InfoNCE loss and N-pair loss?
Thanks!

N-pair loss and InfoNCE are actually the same thing! You could rewrite the InfoNCE loss like this:

This is actually the same as the n-pair loss. Replace $c$ with $x$, $x$ with the positive sample $x^+$, and all other $x'$ with negative samples $x_i^-$. And I changed $f$ to $g$ so that you can rewrite the score function $g(x,c) = \exp{f(x)^T f(c)}$ to match the one from the n-pair loss formulation.
By the way, the InfoNCE paper defines a similar score function with exponential in equation 3. You need the exponential to make the math work out, since it's a softmax.
If you want to learn more, you can check out my blog post about this: Demystifying Noise Contrastive Estimation
