Answers.Body: <p>Good question! There's actually some ambiguity here: it's possible to consider the lower-dimensional projection with respect to the pixels within a single image <em>or</em> across a dataset of images.</p>
  <h3>A dataset of images</h3>
  <p>You're probably thinking of the case where you find principal components with respect to a dataset of images. A common example is &quot;<a href="https://machinelearningmastery.com/face-recognition-using-principal-component-analysis/" rel="nofollow noreferrer">eigenfaces</a>&quot;.</p>
  <p>Each sample corresponds to an image. <span class="math-container">$\mathbf{x}^i$</span> corresponds to (the grayscale pixel values) of a single image, and <span class="math-container">$\bar{\mathbf{x}}$</span> corresponds to the average (grayscale pixel values) of a single image. Here, <span class="math-container">$S$</span> corresponds to the variance between pixels across a dataset of images.</p>
  <h3>A single image</h3>
  <p>PCA greedily finds directions in feature space that have the greatest variance across samples. So if you have a single image, you need to split that image into equally shaped chunks. Each chunk in the image is a sample and each pixel within that chunk is a feature.</p>
  <p>For example, if you have a 128x128 image, you might choose each row to be a sample. You'd then have 128 samples of chunks, each with 128 features. <span class="math-container">$S$</span> then describes the covariance across the rows of the image. You can find an implementation of what I described in <a href="https://www.dataknowsall.com/imagepca.html" rel="nofollow noreferrer">this tutorial</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-16T23:20:47.853Z
Answers.Text: What do you mean by "$\tilde{\mathbf{x}}$ corresponds to the average (grayscale pixel values) of a single image." Do you mean the reconstructed image $\hat{\mathbf{x}}$? If so, why average?
Answers.UserDisplayName: DSPinfinity
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-16T23:51:17.340Z
Answers.Text: @DSPinfinity oh sorry I meant x^bar
Answers.UserDisplayName: Alexander Wan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-16T01:13:51.117Z
Answers.LastActivityDate: 2023-10-16T23:51:28.397Z
Answers.LastEditDate: 2023-10-16T23:51:28.397Z
Answers.UserDisplayName: Alexander Wan
Body: <p>If the feature vector is <span class="math-container">$\mathbf{x} \in \mathbb{R}^{d}$</span>,
  then to apply PCA we first need to construct the &quot;sample covariance matrix)
  <span class="math-container">\begin{align*}
  \underbrace{\frac{1}{N}\sum_{i=1}^N(\mathbf{x}^i-\bar{\mathbf{x}})(\mathbf{x}^i-\bar{\mathbf{x}})^T\bigg]}_{\text{$\mathbf{S}$ : sample covariance matrix}}
  \end{align*}</span>
  from <span class="math-container">$N$</span> data points.</p>
  <p>When I read a machine learning book, I encountered with the following:</p>
  <p><a href="https://i.sstatic.net/MFtFh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/MFtFh.png" alt="enter image description here" /></a></p>
  <p>What is <span class="math-container">$\mathbf{S}$</span> (sample covariance matrix) here in image compression based on PCA?</p>
  
Tags: <machine-learning><feature-extraction><dimensionality-reduction>
Title: What is $\mathbf{S}$ (sample covariance matrix) in image compression based on PCA?
UserDisplayName: DSPinfinity
ThreadText: # What is $\mathbf{S}$ (sample covariance matrix) in image compression based on PCA?
  
  (Asked by: username_0 on 2023-10-15)
  
  If the feature vector is $\mathbf{x} \in \mathbb{R}^{d}$,
  then to apply PCA we first need to construct the "sample covariance matrix)
  \begin{align\*}
  \underbrace{\frac{1}{N}\sum\_{i=1}^N(\mathbf{x}^i-\bar{\mathbf{x}})(\mathbf{x}^i-\bar{\mathbf{x}})^T\bigg]}\_{\text{$\mathbf{S}$ : sample covariance matrix}}
  \end{align\*}
  from $N$ data points.
  
  When I read a machine learning book, I encountered with the following:
  
  [![enter image description here](https://i.sstatic.net/MFtFh.png)](https://i.sstatic.net/MFtFh.png)
  
  What is $\mathbf{S}$ (sample covariance matrix) here in image compression based on PCA?
  
  **Tags:** machine-learning, feature-extraction, dimensionality-reduction
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-10-16. Score: 2 | Accepted answer
  
  Good question! There's actually some ambiguity here: it's possible to consider the lower-dimensional projection with respect to the pixels within a single image *or* across a dataset of images.
  
  ### A dataset of images
  
  You're probably thinking of the case where you find principal components with respect to a dataset of images. A common example is "[eigenfaces](https://machinelearningmastery.com/face-recognition-using-principal-component-analysis/)".
  
  Each sample corresponds to an image. $\mathbf{x}^i$ corresponds to (the grayscale pixel values) of a single image, and $\bar{\mathbf{x}}$ corresponds to the average (grayscale pixel values) of a single image. Here, $S$ corresponds to the variance between pixels across a dataset of images.
  
  ### A single image
  
  PCA greedily finds directions in feature space that have the greatest variance across samples. So if you have a single image, you need to split that image into equally shaped chunks. Each chunk in the image is a sample and each pixel within that chunk is a feature.
  
  For example, if you have a 128x128 image, you might choose each row to be a sample. You'd then have 128 samples of chunks, each with 128 features. $S$ then describes the covariance across the rows of the image. You can find an implementation of what I described in [this tutorial](https://www.dataknowsall.com/imagepca.html).
  
  > **Comments:**
  > **username_0**: What do you mean by "$\tilde{\mathbf{x}}$ corresponds to the average (grayscale pixel values) of a single image." Do you mean the reconstructed image $\hat{\mathbf{x}}$? If so, why average?  
  > **username_1**: @DSPinfinity oh sorry I meant x^bar  
  
  ---
  
  