Answers.Body: <p>The number of singular vectors we need to find during SVD is not unique. The possible values for k are from 1 to <span class="math-container">$r$</span>. Here, <span class="math-container">$r$</span> is the rank of matrix <span class="math-container">$A$</span>, on which we are performing decomposition.</p>
  <p>The same pdf says that</p>
  <blockquote>
  <p>First, in many applications, the data matrix <span class="math-container">$A$</span> is close to a matrix of
  low rank and it is useful to find a low rank matrix which is a good
  approximation to the data matrix .  We will show that from the singular
  value decomposition of <span class="math-container">$A$</span>, we can get the matrix <span class="math-container">$B$</span> of rank <span class="math-container">$k$</span> which best
  approximates <span class="math-container">$A$</span>; <strong>in fact we can do this for every <span class="math-container">$k$</span></strong>.  Also, singular
  value decomposition is defined for all matrices (rectangular or
  square)unlike the more commonly used spectral decomposition in Linear
  Algebra.</p>
  </blockquote>
  <p>So, the value of <span class="math-container">$k$</span> is up to the designer. If the designer selects the value of <span class="math-container">$k$</span> smaller than the rank <span class="math-container">$r$</span> of the matrix <span class="math-container">$A$</span>, then it is called as <a href="http://infolab.stanford.edu/pub/cstr/reports/na/m/86/36/NA-M-86-36.pdf" rel="nofollow noreferrer">truncated SVD</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-26T01:05:56.767Z
Answers.LastActivityDate: 2021-06-26T07:55:18.430Z
Answers.LastEditDate: 2021-06-26T07:55:18.430Z
Answers.UserDisplayName: hanugm
Body: <p>In the geometrical interpretation of SVD, the data points that we have need to be imagined as points in high dimensional space (say <span class="math-container">$d$</span>-dimensional space). But we need to find a hyperplane in <span class="math-container">$k-$</span>dimensional subspace that best fits the given data points</p>
  <blockquote>
  <p>To gain insight into the SVD, treat the rows of an <span class="math-container">$n \times d$</span> matrix <span class="math-container">$A$</span> as <span class="math-container">$n$</span> points in a <span class="math-container">$d$</span>-dimensional space and consider the problem of finding the best <strong><span class="math-container">$k$</span>-dimensional</strong> subspace with respect to the set of points.</p>
  </blockquote>
  <p>My doubt here is about the uniqueness of <span class="math-container">$k$</span>. Can we do decomposition for any <span class="math-container">$k \le d$</span> or for only certain values of <span class="math-container">$k$</span> or only for an unique <span class="math-container">$k$</span>?</p>
  <hr />
  <p>The paragraph is taken from the material on Singular Value Decomposition available <a href="https://www.cs.cmu.edu/%7Evenkatg/teaching/CStheory-infoage/book-chapter-4.pdf" rel="nofollow noreferrer">here</a>.</p>
  
Tags: <hyperparameter-optimization><hyper-parameters><feature-selection><singular-value-decomposition>
Title: How many singular vectors do we need to calculate for SVD?
UserDisplayName: hanugm
ThreadText: # How many singular vectors do we need to calculate for SVD?
  
  (Asked by: username_0 on 2021-06-24)
  
  In the geometrical interpretation of SVD, the data points that we have need to be imagined as points in high dimensional space (say $d$-dimensional space). But we need to find a hyperplane in $k-$dimensional subspace that best fits the given data points
  
  > To gain insight into the SVD, treat the rows of an $n \times d$ matrix $A$ as $n$ points in a $d$-dimensional space and consider the problem of finding the best **$k$-dimensional** subspace with respect to the set of points.
  
  My doubt here is about the uniqueness of $k$. Can we do decomposition for any $k \le d$ or for only certain values of $k$ or only for an unique $k$?
  
  ---
  
  The paragraph is taken from the material on Singular Value Decomposition available [here](https://www.cs.cmu.edu/%7Evenkatg/teaching/CStheory-infoage/book-chapter-4.pdf).
  
  **Tags:** hyperparameter-optimization, hyper-parameters, feature-selection, singular-value-decomposition
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-06-26. Score: 1
  
  The number of singular vectors we need to find during SVD is not unique. The possible values for k are from 1 to $r$. Here, $r$ is the rank of matrix $A$, on which we are performing decomposition.
  
  The same pdf says that
  
  > First, in many applications, the data matrix $A$ is close to a matrix of
  > low rank and it is useful to find a low rank matrix which is a good
  > approximation to the data matrix . We will show that from the singular
  > value decomposition of $A$, we can get the matrix $B$ of rank $k$ which best
  > approximates $A$; **in fact we can do this for every $k$**. Also, singular
  > value decomposition is defined for all matrices (rectangular or
  > square)unlike the more commonly used spectral decomposition in Linear
  > Algebra.
  
  So, the value of $k$ is up to the designer. If the designer selects the value of $k$ smaller than the rank $r$ of the matrix $A$, then it is called as [truncated SVD](http://infolab.stanford.edu/pub/cstr/reports/na/m/86/36/NA-M-86-36.pdf).
  
  
  ---
  
  