Answers.Body: <p><span class="math-container">$USV^T$</span> refers to the result of the singular value decomposition (SVD).</p>
  <p>An <span class="math-container">$m \times n$</span> matrix <span class="math-container">$X$</span> can be written with the help of three matrices</p>
  <p><span class="math-container">$$X = USV^T,$$</span></p>
  <p>where <span class="math-container">$U$</span> is an <span class="math-container">$m \times m$</span> unitary matrix, <span class="math-container">$S$</span> is a diagonal <span class="math-container">$m \times n$</span> matrix with real entries called singular values, and <span class="math-container">$V$</span> is a unitary <span class="math-container">$n \times n$</span> matrix. The <span class="math-container">$^T$</span> is the Hermitian transpose. SVD has applications, e.g, in optimization problems, principal component analysis, etc. <a href="https://en.wikipedia.org/wiki/Singular_value_decomposition" rel="nofollow noreferrer">Wikipedia</a> has quite a long article.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-08-15T06:36:10.197Z
Answers.LastActivityDate: 2021-08-24T15:39:56.627Z
Answers.LastEditDate: 2021-08-24T15:39:56.627Z
Answers.UserDisplayName: Ari Hietanen
Body: <p>Here is an excerpt from the <a href="http://web.stanford.edu/class/cs224n/" rel="nofollow noreferrer">notes of the first lecture</a> of the course <a href="http://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes01-wordvecs1.pdf" rel="nofollow noreferrer">CS224n: Natural Language Processing with Deep Learning</a>.</p>
  <blockquote>
  <p>3 SVD Based Methods</p>
  <p>For this class of methods to find word embeddings (otherwise known
  as word vectors), we first loop over a massive data set and accumulate word co-occurrence counts in some form of a matrix <span class="math-container">$X$</span> and then perform Singular Value Decomposition on <span class="math-container">$X$</span> to get a <span class="math-container">$USV^T$</span> decomposition. We then use the rows of <span class="math-container">$U$</span> as the word embeddings for all words in our dictionary. Let us discuss a few choices of <span class="math-container">$X$</span>.</p>
  </blockquote>
  <p>What does <span class="math-container">$USV^T$</span> refer to in this context?</p>
  
Tags: <natural-language-processing><terminology><word-embedding><singular-value-decomposition>
Title: What is $USV^T$ in the context of word embeddings?
UserDisplayName: Daschin
ThreadText: # What is $USV^T$ in the context of word embeddings?
  
  (Asked by: username_0 on 2017-08-12)
  
  Here is an excerpt from the [notes of the first lecture](http://web.stanford.edu/class/cs224n/) of the course [CS224n: Natural Language Processing with Deep Learning](http://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes01-wordvecs1.pdf).
  
  > 3 SVD Based Methods
  >
  > For this class of methods to find word embeddings (otherwise known
  > as word vectors), we first loop over a massive data set and accumulate word co-occurrence counts in some form of a matrix $X$ and then perform Singular Value Decomposition on $X$ to get a $USV^T$ decomposition. We then use the rows of $U$ as the word embeddings for all words in our dictionary. Let us discuss a few choices of $X$.
  
  What does $USV^T$ refer to in this context?
  
  **Tags:** natural-language-processing, terminology, word-embedding, singular-value-decomposition
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-08-15. Score: 2
  
  $USV^T$ refers to the result of the singular value decomposition (SVD).
  
  An $m \times n$ matrix $X$ can be written with the help of three matrices
  
  $$X = USV^T,$$
  
  where $U$ is an $m \times m$ unitary matrix, $S$ is a diagonal $m \times n$ matrix with real entries called singular values, and $V$ is a unitary $n \times n$ matrix. The $^T$ is the Hermitian transpose. SVD has applications, e.g, in optimization problems, principal component analysis, etc. [Wikipedia](https://en.wikipedia.org/wiki/Singular_value_decomposition) has quite a long article.
  
  
  ---
  
  