Answers.Body: <blockquote>
    <p>Do individual dimensions in vector space have meaning?</p>
  </blockquote>
  
  <p>IIRC, some dimensions are interpretable, but in general this is not the case. Also it is debatable as to wether it is actually learning the actual representation or just an approximation of it. But in any case its not very reliable outside from some edge cases.</p>
  
  <blockquote>
    <p>If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? </p>
  </blockquote>
  
  <p>Yes, but as to what that information entails in terms of "meaning" is lesser clear. You could say that if in a certain dimension the distance between two vectors is 0, then you have am estimate of the real distance that is better than guessing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-14T14:30:42.303Z
Answers.Text: IIRC means: if I recall correctly
Answers.UserDisplayName: gota
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-24T03:45:56.990Z
Answers.LastActivityDate: 2018-04-24T04:13:45.413Z
Answers.LastEditDate: 2018-04-24T04:13:45.413Z
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Body: <p>Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction). </p>
  
  <p>It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. For example "<em>king + woman - man = queen</em>".</p>
  
  <p>These terms are all N-dimensional vectors. Now, suppose, for simplicity, that <span class="math-container">$N=3$</span>, <span class="math-container">$\text{king} = [0, 1, 2], \text{woman} = [1, 1, 0], \text{man} = [2, 2, 2], \text{queen} = [-1, 0, 0]$</span>, then the expression above can be written as <span class="math-container">$[0, 1, 2] +  [1, 1, 0] - [2, 2, 2] = [-1, 0, 0]$</span>.</p>
  
  <p>In this (contrived) example, the last dimension (king/man=2, queen/woman=0) suggests a semantic concept of gender. Aside from semantics, a given dimension could "mean" a part of speech, first letter, or really any feature or set of features that the algorithm might have latched onto. However, any perceived "meaning" of a single dimension might well just be a simple coincidence.</p>
  
  <p>If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? Or is this purely a "random" artefact of the algorithm, with only the full N-dimensional vector distances mattering?</p>
  
Tags: <natural-language-processing><word2vec><word-embedding>
Title: Do individual dimensions in vector space have meaning?
UserDisplayName: G__
ThreadText: # Do individual dimensions in vector space have meaning?
  
  (Asked by: username_0 on 2018-04-23)
  
  Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction).
  
  It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. For example "*king + woman - man = queen*".
  
  These terms are all N-dimensional vectors. Now, suppose, for simplicity, that $N=3$, $\text{king} = [0, 1, 2], \text{woman} = [1, 1, 0], \text{man} = [2, 2, 2], \text{queen} = [-1, 0, 0]$, then the expression above can be written as $[0, 1, 2] + [1, 1, 0] - [2, 2, 2] = [-1, 0, 0]$.
  
  In this (contrived) example, the last dimension (king/man=2, queen/woman=0) suggests a semantic concept of gender. Aside from semantics, a given dimension could "mean" a part of speech, first letter, or really any feature or set of features that the algorithm might have latched onto. However, any perceived "meaning" of a single dimension might well just be a simple coincidence.
  
  If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? Or is this purely a "random" artefact of the algorithm, with only the full N-dimensional vector distances mattering?
  
  **Tags:** natural-language-processing, word2vec, word-embedding
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-04-24. Score: 2
  
  > Do individual dimensions in vector space have meaning?
  
  IIRC, some dimensions are interpretable, but in general this is not the case. Also it is debatable as to wether it is actually learning the actual representation or just an approximation of it. But in any case its not very reliable outside from some edge cases.
  
  > If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information?
  
  Yes, but as to what that information entails in terms of "meaning" is lesser clear. You could say that if in a certain dimension the distance between two vectors is 0, then you have am estimate of the real distance that is better than guessing.
  
  > **Comments:**
  > **username_2**: IIRC means: if I recall correctly  
  
  ---
  
  