Answers.Body: <p>Feature extraction (FE) is not the same as representation learning (RL), but they are similar and related.</p>
  <p>You describe accurately what <strong>feature extraction</strong> typically refers to, i.e. the process of extracting (new) features from existing ones or raw data (e.g. images). For example, let's say you have a dataset associated with a car. You have only two features in your dataset: distance and velocity. However, from these two, you can extract a third feature, e.g. the acceleration. So, feature extraction can be performed with a fixed algorithm (e.g. PCA for dimensionality reduction or <a href="https://www.cs.ubc.ca/%7Elowe/papers/ijcv04.pdf" rel="nofollow noreferrer">SIFT</a>) or manually.</p>
  <p><strong>Representation learning</strong> is the collection of all techniques that extract features <strong>automatically</strong> from the data (i.e. they <strong>learn</strong> the features or representations, hence the name <em>representation/feature learning</em>). So, for example, a convolutional neural network trained on ImageNet can (and/or needs to) learn general features in order to solve the corresponding classification task. (<a href="https://www.deeplearningbook.org/contents/convnets.html" rel="nofollow noreferrer">Chapter 9</a> of the book <em>Deep Learning</em> by Goodfellow et al. talks more about this topic.) These features are learned from the data (and that's why CNN's are <em>data-driven</em>), and they can later be exploited for transfer learning (TL), i.e. TL is based on the idea that neural networks learn general <em>representations</em> (which is thus a synonym for <em>features</em>) of data that can be exploited to solve other tasks (sometimes known as <em>downstream tasks</em>, especially in the context of <a href="https://ai.stackexchange.com/a/10624/2444">self-supervised learning</a>).</p>
  <p><a href="https://arxiv.org/pdf/1206.5538.pdf" rel="nofollow noreferrer">Yoshua Bengio et al.</a> defines representation learning as follows</p>
  <blockquote>
  <p>learning representations of the data that make it easier to extract useful information when building classifiers or other predictors</p>
  </blockquote>
  <p>So, RL is a subset of FE, given that RL also extracts features, but RL emphasizes the extraction of features automatically.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-28T10:15:43.263Z
Answers.LastActivityDate: 2022-01-14T23:28:50.967Z
Answers.LastEditDate: 2022-01-14T23:28:50.967Z
Answers.UserDisplayName: nbro
Body: <p><em>Feature selection</em> is a process of selecting a subset of features that contribute the most.</p>
  <p><em>Feature extraction</em> allows getting new features that are not actually present in the given set of features.</p>
  <p><em>Representation learning</em> is the process of learning a new representation that contributes the most.</p>
  <p>I can see no difference between feature extraction and representation learning.</p>
  <p>Is feature extraction the same as representation learning? If not, where do they differ? Do they differ at the application level only?</p>
  
Tags: <machine-learning><comparison><terminology><feature-extraction><representation-learning>
Title: Where do the feature extraction and representation learning differ?
UserDisplayName: hanugm
ThreadText: # Where do the feature extraction and representation learning differ?
  
  (Asked by: username_0 on 2021-05-28)
  
  *Feature selection* is a process of selecting a subset of features that contribute the most.
  
  *Feature extraction* allows getting new features that are not actually present in the given set of features.
  
  *Representation learning* is the process of learning a new representation that contributes the most.
  
  I can see no difference between feature extraction and representation learning.
  
  Is feature extraction the same as representation learning? If not, where do they differ? Do they differ at the application level only?
  
  **Tags:** machine-learning, comparison, terminology, feature-extraction, representation-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-05-28. Score: 5 | Accepted answer
  
  Feature extraction (FE) is not the same as representation learning (RL), but they are similar and related.
  
  You describe accurately what **feature extraction** typically refers to, i.e. the process of extracting (new) features from existing ones or raw data (e.g. images). For example, let's say you have a dataset associated with a car. You have only two features in your dataset: distance and velocity. However, from these two, you can extract a third feature, e.g. the acceleration. So, feature extraction can be performed with a fixed algorithm (e.g. PCA for dimensionality reduction or [SIFT](https://www.cs.ubc.ca/%7Elowe/papers/ijcv04.pdf)) or manually.
  
  **Representation learning** is the collection of all techniques that extract features **automatically** from the data (i.e. they **learn** the features or representations, hence the name *representation/feature learning*). So, for example, a convolutional neural network trained on ImageNet can (and/or needs to) learn general features in order to solve the corresponding classification task. ([Chapter 9](https://www.deeplearningbook.org/contents/convnets.html) of the book *Deep Learning* by Goodfellow et al. talks more about this topic.) These features are learned from the data (and that's why CNN's are *data-driven*), and they can later be exploited for transfer learning (TL), i.e. TL is based on the idea that neural networks learn general *representations* (which is thus a synonym for *features*) of data that can be exploited to solve other tasks (sometimes known as *downstream tasks*, especially in the context of [self-supervised learning](https://ai.stackexchange.com/a/10624/2444)).
  
  [Yoshua Bengio et al.](https://arxiv.org/pdf/1206.5538.pdf) defines representation learning as follows
  
  > learning representations of the data that make it easier to extract useful information when building classifiers or other predictors
  
  So, RL is a subset of FE, given that RL also extracts features, but RL emphasizes the extraction of features automatically.
  
  
  ---
  
  