Answers.Body: <p>This is a 'reversal' of the usual clustering approach. Normally you cluster objects, and you use their features to define similarity (as proximity in 'feature-space'). So you start off with a set of objects, and you end up with <em>k</em> groups of similar objects, for a specific value of <em>k</em>.</p>
  <p>In ML you might want to train a classifier based on features, but not all features are equally discriminant. So you turn the process on its head, and you <a href="https://en.wikipedia.org/wiki/Transpose" rel="nofollow noreferrer"><em>transpose</em></a> the object/feature matrix, so that you're now looking at features and which objects they occur in, rather than objects and the features they possess.</p>
  <p>You can now cluster this matrix, and you group features together by how similar they are. You don't really want features which are shared by too many objects, as they have less value in distinguishing between objects. You decide how many features you want (your value <em>k</em>), and then start clustering. You end up with <em>k</em> clusters of features which are similar to each other within the same cluster.</p>
  <p>You can now choose either the centroid, or an amalgamation of each of the cluster's elements, and you end up with <em>k</em> features that are of good quality. Essentially you have reduced the dimensionality of your feature space, which makes it easier to identify patterns using ML.</p>
  <p>This is similar to using <a href="https://en.wikipedia.org/wiki/Principal_component_analysis" rel="nofollow noreferrer">PCA</a> for reducing dimensionality, but a bit more flexible I guess, since you have more choice about what clustering algorithm to use.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-29T23:20:00.430Z
Answers.Text: can you give references on the first part? to me seems counterintuitive as $X^T$ will have a giant feature space, and very few samples (since we swap samples and features with the transposition), so all ML algorithm will fail due to curse of dimensionality
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-30T09:15:22.683Z
Answers.Text: @Alberto It all depends on how many objects and features you have. With a small-ish training set, the 'object space' will not be that big; if you try to do this with several million objects, then this is probably not the best approach.
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-29T10:06:36.287Z
Answers.LastActivityDate: 2023-11-29T10:06:36.287Z
Answers.UserDisplayName: Oliver Mason
Body: <p>In an ML book, it is said that:</p>
  <p>&quot;Similarly, clustering is a
  popular choice for feature selection in classification-clustering features based on
  their values across objects rather than clustering objects (<span class="math-container">$X^T$</span> rather tahn <span class="math-container">$X$</span>). In
  this instance, K should be chosen to give the best classification performance&quot;</p>
  <p>What does &quot;Clustering features based on
  their values across objects rather than clustering objects (<span class="math-container">$X^T$</span> rather than <span class="math-container">$X$</span>)&quot; mean? This part is not clear to me and I need a good explanation.</p>
  
Tags: <k-means>
Title: What does "Clustering features based on their values across objects rather than clustering objects ($X^T$ rather than$ X$)" mean?
UserDisplayName: DSPinfinity
ThreadText: # What does "Clustering features based on their values across objects rather than clustering objects ($X^T$ rather than$ X$)" mean?
  
  (Asked by: username_0 on 2023-11-28)
  
  In an ML book, it is said that:
  
  "Similarly, clustering is a
  popular choice for feature selection in classification-clustering features based on
  their values across objects rather than clustering objects ($X^T$ rather tahn $X$). In
  this instance, K should be chosen to give the best classification performance"
  
  What does "Clustering features based on
  their values across objects rather than clustering objects ($X^T$ rather than $X$)" mean? This part is not clear to me and I need a good explanation.
  
  **Tags:** k-means
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-11-29. Score: 1
  
  This is a 'reversal' of the usual clustering approach. Normally you cluster objects, and you use their features to define similarity (as proximity in 'feature-space'). So you start off with a set of objects, and you end up with *k* groups of similar objects, for a specific value of *k*.
  
  In ML you might want to train a classifier based on features, but not all features are equally discriminant. So you turn the process on its head, and you [*transpose*](https://en.wikipedia.org/wiki/Transpose) the object/feature matrix, so that you're now looking at features and which objects they occur in, rather than objects and the features they possess.
  
  You can now cluster this matrix, and you group features together by how similar they are. You don't really want features which are shared by too many objects, as they have less value in distinguishing between objects. You decide how many features you want (your value *k*), and then start clustering. You end up with *k* clusters of features which are similar to each other within the same cluster.
  
  You can now choose either the centroid, or an amalgamation of each of the cluster's elements, and you end up with *k* features that are of good quality. Essentially you have reduced the dimensionality of your feature space, which makes it easier to identify patterns using ML.
  
  This is similar to using [PCA](https://en.wikipedia.org/wiki/Principal_component_analysis) for reducing dimensionality, but a bit more flexible I guess, since you have more choice about what clustering algorithm to use.
  
  > **Comments:**
  > **username_2**: can you give references on the first part? to me seems counterintuitive as $X^T$ will have a giant feature space, and very few samples (since we swap samples and features with the transposition), so all ML algorithm will fail due to curse of dimensionality  
  > **username_1**: @Alberto It all depends on how many objects and features you have. With a small-ish training set, the 'object space' will not be that big; if you try to do this with several million objects, then this is probably not the best approach.  
  
  ---
  
  