Answers.Body: <p>The expression "latent space" explicitly indicates that the space is associated with the mathematical concept of an hidden (or latent) variable, which cannot be observed directly, but only indirectly.</p>
  
  <p>The expression "embedding space" refers to a vector space that represents an original space of inputs (e.g. images or words). For example, in the case of "word embeddings", which are vector representations of words. It can also refer to a latent space because a latent space can also be a space of vectors. However, an embedding space is not necessarily an hidden space. It is just another (vector) representation of another space.</p>
  
  <p>These two expressions can be used interchangeably, also because the expression "embedding space" is often not formally defined.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-24T12:22:09.307Z
Answers.LastActivityDate: 2019-05-24T12:22:09.307Z
Answers.UserDisplayName: nbro
Answers.Body: <p>To give a statistician's answer, the distinction is empirical (embedding) versus theoretical (latent positions). You define a statistical model which has latent positions that you could then try to estimate, given data. Or, given data, you might simply find a vector representation of each object of interest in a way that makes sense for the applications considered - and you would call that set of representations an embedding. There's a lot of work going on trying to re-interpret popular embeddings as estimating a compatible model's latent positions, but that's not always straightforward.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-18T14:25:33.897Z
Answers.LastActivityDate: 2021-06-18T14:25:33.897Z
Answers.UserDisplayName: PRD
Answers.Body: <p>The words latent space and embedding space are often used interchangeably. However, latent space can more specifically refer to the sample space of a stochastic representation, whereas embedding space more often refers to the space of a deterministic representation.</p>
  
  <p>This comes from latent referring to an unobserved random variable, for which we can infer a belief distribution over its plausible values, for example using an encoder network. You can then draw samples of the predicted distribution for further processing. To learn more, you can look into VAEs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-28T04:00:17.353Z
Answers.LastActivityDate: 2020-04-28T04:00:17.353Z
Answers.UserDisplayName: danijar
Answers.Body: <h2>Embedding vs Latent Space</h2>
  <p>Due to Machine Learning's recent and rapid renaissance, and the fact that it draws from many distinct areas of mathematics, statistics, and computer science, it often has a number of different terms for the same or similar concepts.</p>
  <p>&quot;Latent space&quot; and &quot;embedding&quot; both refer to an (often lower-dimensional) representation of high-dimensional data:</p>
  <ul>
  <li><em>Latent space</em> refers specifically to the space from which the low-dimensional representation is drawn.</li>
  <li><em>Embedding</em> refers to the way the low-dimensional data is mapped to (&quot;embedded in&quot;) the original higher dimensional space.</li>
  </ul>
  <p>For example, in this &quot;Swiss roll&quot; data, the 3d data on the left is sensibly modelled as a 2d manifold 'embedded' in 3d space. The function mapping the 'latent' 2d data to its 3d representation is the <em>embedding</em>, and the underlying 2d space itself is the <em>latent space</em> (or <em>embedded space</em>):</p>
  <p><a href="https://i.sstatic.net/ID6dP.png" rel="noreferrer"><img src="https://i.sstatic.net/ID6dP.png" alt="enter image description here" /></a></p>
  <h2>Synonyms</h2>
  <p>Depending on the specific impression you wish to give, &quot;embedding&quot; often goes by different terms:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Term</th>
  <th>Context</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>dimensionality reduction</td>
  <td>combating the &quot;curse of dimensionality&quot;</td>
  </tr>
  <tr>
  <td>feature extraction<br> feature projection<br> <a href="https://ai.stackexchange.com/questions/6556/what-is-feature-embedding-in-the-context-of-convolutional-neural-networks/20697">feature embedding</a><br> feature learning<br> representation learning</td>
  <td>extracting 'meaningful' features from raw data</td>
  </tr>
  <tr>
  <td>embedding<br> manifold learning<br> latent feature representation</td>
  <td>understanding the underlying topology of the data</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>However this is not a hard-and-fast rule, and they are often completely interchangeable.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-26T09:42:26.730Z
Answers.LastActivityDate: 2020-12-31T17:43:22.203Z
Answers.LastEditDate: 2020-12-31T17:43:22.203Z
Answers.UserDisplayName: brazofuerte
Body: <p>In general, the word &quot;latent&quot; means &quot;hidden&quot; and &quot;to embed&quot; means &quot;to incorporate&quot;. In machine learning, the expressions &quot;hidden (or latent) space&quot; and &quot;embedding space&quot; occur in several contexts. More specifically, an <a href="https://en.wikipedia.org/wiki/Word_embedding" rel="noreferrer">embedding</a> can refer to a vector representation of a word. An <a href="https://en.wikipedia.org/wiki/Embedding" rel="noreferrer">embedding space</a> can refer to a subspace of a bigger space, so we say that the subspace is <em>embedded</em> in the bigger space. The word &quot;latent&quot; comes up in contexts like <a href="https://en.wikipedia.org/wiki/Hidden_Markov_model" rel="noreferrer">hidden Markov models (HMMs)</a> or <a href="https://en.wikipedia.org/wiki/Autoencoder" rel="noreferrer">auto-encoders</a>.</p>
  <p>What is the difference between these spaces? In some contexts, do these two expressions refer to the same concept?</p>
  
Tags: <machine-learning><terminology><word-embedding><latent-variable><embeddings>
Title: What is the difference between latent and embedding spaces?
UserDisplayName: nbro
ThreadText: # What is the difference between latent and embedding spaces?
  
  (Asked by: username_0 on 2019-03-17)
  
  In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" and "embedding space" occur in several contexts. More specifically, an [embedding](https://en.wikipedia.org/wiki/Word_embedding) can refer to a vector representation of a word. An [embedding space](https://en.wikipedia.org/wiki/Embedding) can refer to a subspace of a bigger space, so we say that the subspace is *embedded* in the bigger space. The word "latent" comes up in contexts like [hidden Markov models (HMMs)](https://en.wikipedia.org/wiki/Hidden_Markov_model) or [auto-encoders](https://en.wikipedia.org/wiki/Autoencoder).
  
  What is the difference between these spaces? In some contexts, do these two expressions refer to the same concept?
  
  **Tags:** machine-learning, terminology, word-embedding, latent-variable, embeddings
  
  ---
  
  ## 4 answers
  
  ### Answer by username_3 on 2020-04-26. Score: 37 | Accepted answer
  
  ## Embedding vs Latent Space
  
  Due to Machine Learning's recent and rapid renaissance, and the fact that it draws from many distinct areas of mathematics, statistics, and computer science, it often has a number of different terms for the same or similar concepts.
  
  "Latent space" and "embedding" both refer to an (often lower-dimensional) representation of high-dimensional data:
  
  * *Latent space* refers specifically to the space from which the low-dimensional representation is drawn.
  * *Embedding* refers to the way the low-dimensional data is mapped to ("embedded in") the original higher dimensional space.
  
  For example, in this "Swiss roll" data, the 3d data on the left is sensibly modelled as a 2d manifold 'embedded' in 3d space. The function mapping the 'latent' 2d data to its 3d representation is the *embedding*, and the underlying 2d space itself is the *latent space* (or *embedded space*):
  
  [![enter image description here](https://i.sstatic.net/ID6dP.png)](https://i.sstatic.net/ID6dP.png)
  
  ## Synonyms
  
  Depending on the specific impression you wish to give, "embedding" often goes by different terms:
  
  | Term | Context |
  | --- | --- |
  | dimensionality reduction | combating the "curse of dimensionality" |
  | feature extraction  feature projection  [feature embedding](https://ai.stackexchange.com/questions/6556/what-is-feature-embedding-in-the-context-of-convolutional-neural-networks/20697)  feature learning  representation learning | extracting 'meaningful' features from raw data |
  | embedding  manifold learning  latent feature representation | understanding the underlying topology of the data |
  
  However this is not a hard-and-fast rule, and they are often completely interchangeable.
  
  
  ---
  
  ### Answer by username_0 on 2019-05-24. Score: 5
  
  The expression "latent space" explicitly indicates that the space is associated with the mathematical concept of an hidden (or latent) variable, which cannot be observed directly, but only indirectly.
  
  The expression "embedding space" refers to a vector space that represents an original space of inputs (e.g. images or words). For example, in the case of "word embeddings", which are vector representations of words. It can also refer to a latent space because a latent space can also be a space of vectors. However, an embedding space is not necessarily an hidden space. It is just another (vector) representation of another space.
  
  These two expressions can be used interchangeably, also because the expression "embedding space" is often not formally defined.
  
  
  ---
  
  ### Answer by username_1 on 2021-06-18. Score: 1
  
  To give a statistician's answer, the distinction is empirical (embedding) versus theoretical (latent positions). You define a statistical model which has latent positions that you could then try to estimate, given data. Or, given data, you might simply find a vector representation of each object of interest in a way that makes sense for the applications considered - and you would call that set of representations an embedding. There's a lot of work going on trying to re-interpret popular embeddings as estimating a compatible model's latent positions, but that's not always straightforward.
  
  
  ---
  
  ### Answer by username_2 on 2020-04-28. Score: 0
  
  The words latent space and embedding space are often used interchangeably. However, latent space can more specifically refer to the sample space of a stochastic representation, whereas embedding space more often refers to the space of a deterministic representation.
  
  This comes from latent referring to an unobserved random variable, for which we can infer a belief distribution over its plausible values, for example using an encoder network. You can then draw samples of the predicted distribution for further processing. To learn more, you can look into VAEs.
  
  
  ---
  
  