Answers.Body: <p>The constraint is enforced with bespoke code. If FaceNet was implemented in NumPy, and the embedding layer vector (pre-constraint) was in the NumPy array <code>h</code>, then the code might look like:</p>
  
  <pre><code>e = h / np.linalg.norm(h)
  </code></pre>
  
  <p>The variable <code>e</code> would then contain the desired embedding with L2 distance of 1. In practice, even with NumPy, the code might be more complex due to handling mini-batches. More likely this will not be implemented in NumPy.</p>
  
  <p>In <a href="https://github.com/Skuldur/facenet-face-recognition" rel="nofollow noreferrer">this Keras/TensorFlow-based FaceNet implementation</a> you can see how it <a href="https://github.com/Skuldur/facenet-face-recognition/blob/11e44ec4ac803e69b46bb5407587ba32f06bec4f/inception_blocks_v2.py#L275-L276" rel="nofollow noreferrer">may be done in practice</a>:</p>
  
  <pre><code># L2 normalization
  X = Lambda(lambda  x: K.l2_normalize(x,axis=1))(X)
  </code></pre>
  
  <p>This scaling transformation is considered part of the neural network code (it is part of the Keras model building routine in the above snippet), so there needs to be corresponding support for back propagation through the embedding. When using an automatic differentiation framework such as Theano or TensorFlow, then no extra code is required.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-26T09:27:19.000Z
Answers.LastActivityDate: 2018-02-26T09:27:19.000Z
Answers.UserDisplayName: Neil Slater
Body: <p>In the <a href="https://arxiv.org/pdf/1503.03832.pdf" rel="nofollow noreferrer">FaceNet</a> paper, under section <strong>3.2</strong>, the authors mention that:</p>
  <blockquote>
  <p>The embedding is represented by <span class="math-container">$f(x) \in \mathbb{R}^{d}$</span>. It embeds an image <span class="math-container">$x$</span> into a <span class="math-container">$d$</span>-dimensional Euclidean space. Additionally, we constrain this embedding to live on the <span class="math-container">$d$</span> dimensional hypersphere, i.e. <span class="math-container">$\|f(x)\|_{2}=1$</span>.</p>
  </blockquote>
  <p>I don't quite understand how the above equation holds. As far as I understand, the <span class="math-container">$L_2$</span> norm is the same as the Euclidean distance, but I don't quite understand how this imposes
  <span class="math-container">$\|f(x)\|_{2}=1$</span> criteria.</p>
  
Tags: <papers><embeddings><facenet>
Title: How is the constraint $\|f(x)\|_{2}=1$ enforced for the embedding $f(x)$ in the FaceNet paper?
UserDisplayName: Abhijit Balaji
ThreadText: # How is the constraint $\|f(x)\|_{2}=1$ enforced for the embedding $f(x)$ in the FaceNet paper?
  
  (Asked by: username_0 on 2018-02-26)
  
  In the [FaceNet](https://arxiv.org/pdf/1503.03832.pdf) paper, under section **3.2**, the authors mention that:
  
  > The embedding is represented by $f(x) \in \mathbb{R}^{d}$. It embeds an image $x$ into a $d$-dimensional Euclidean space. Additionally, we constrain this embedding to live on the $d$ dimensional hypersphere, i.e. $\|f(x)\|\_{2}=1$.
  
  I don't quite understand how the above equation holds. As far as I understand, the $L\_2$ norm is the same as the Euclidean distance, but I don't quite understand how this imposes
  $\|f(x)\|\_{2}=1$ criteria.
  
  **Tags:** papers, embeddings, facenet
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-02-26. Score: 1 | Accepted answer
  
  The constraint is enforced with bespoke code. If FaceNet was implemented in NumPy, and the embedding layer vector (pre-constraint) was in the NumPy array `h`, then the code might look like:
  
  ```
  e = h / np.linalg.norm(h)
  
  ```
  
  The variable `e` would then contain the desired embedding with L2 distance of 1. In practice, even with NumPy, the code might be more complex due to handling mini-batches. More likely this will not be implemented in NumPy.
  
  In [this Keras/TensorFlow-based FaceNet implementation](https://github.com/Skuldur/facenet-face-recognition) you can see how it [may be done in practice](https://github.com/Skuldur/facenet-face-recognition/blob/11e44ec4ac803e69b46bb5407587ba32f06bec4f/inception_blocks_v2.py#L275-L276):
  
  ```
  # L2 normalization
  X = Lambda(lambda  x: K.l2_normalize(x,axis=1))(X)
  
  ```
  
  This scaling transformation is considered part of the neural network code (it is part of the Keras model building routine in the above snippet), so there needs to be corresponding support for back propagation through the embedding. When using an automatic differentiation framework such as Theano or TensorFlow, then no extra code is required.
  
  
  ---
  
  