Answers.Body: <p>After a lot of searches, I think self-taught learning is a Transfer learning category, I think when Self-taught learning paper published (2007), there isn't any good survey on transfer learning, and as seen by high citation of the paper of Pan, his paper (which published in 2009) describes transfer learning in a clear way that did not exist before it.</p>
  <p>Also, it is reasonable to consider Self-Taught learning a transfer learning category because he actually transfers the knowledge learned from unlabelled data to use it in the supervised task that we want to do (there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task).</p>
  <p>If someone could find something wrong with my answer or there is something missing, then please tell me.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-10T20:11:25.760Z
Answers.LastActivityDate: 2021-05-11T10:58:38.137Z
Answers.LastEditDate: 2021-05-11T10:58:38.137Z
Answers.UserDisplayName: Kais Hasan
Body: <p>I am new to transfer learning and I start by reading <a href="https://ieeexplore.ieee.org/document/5288526" rel="nofollow noreferrer">A Survey on Transfer Learning</a>, and it stated the following:</p>
  <blockquote>
  <p>according to different situations of labeled and unlabeled data in the source domain, we can
  further categorize the inductive transfer learning setting into two cases:</p>
  <p>case <span class="math-container">$(a)$</span> (It is irrelevant to my question).</p>
  <p>case <span class="math-container">$(b): $</span> No labeled data in the source domain are
  available. In this case, the inductive transfer
  learning setting is similar to the self-taught
  learning setting, which is first proposed by Raina
  et al. [22]. In the self-taught learning setting, the
  label spaces between the source and target
  domains may be different, which implies the
  side information of the source domain cannot be
  used directly. Thus, it’s similar to the inductive
  transfer learning setting where the labeled data
  in the source domain are unavailable.</p>
  </blockquote>
  <p>From that, I understand that self-taught learning is inductive transfer learning.</p>
  <p>But I opened the paper of self-taught learning that was mentioned (i.e <a href="https://www.researchgate.net/publication/200038912_Self-taught_learning_Transfer_learning_from_unlabeled_data" rel="nofollow noreferrer">paper by Raina
  et al. [22].</a>), and It stated the following in the introduction:</p>
  <blockquote>
  <p>Because
  self-taught learning places significantly fewer restrictions on
  the type of unlabeled data, in many practical applications
  (such as image, audio or text classification) it is much easier
  to apply than typical semi-supervised learning or transfer learning
  methods.</p>
  </blockquote>
  <p>And here it looks like transfer learning is different from self-taught learning.</p>
  <p>So what is the right relation between them?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-11T00:09:59.803Z
Comments.Text: I'm not familiar with the term _self-taught learning_, but I suspect it is related to _self-supervised learning_, which is turn is related to transfer learning, as SSL can be used for transfer learning. See [this post](https://ai.stackexchange.com/q/10623/2444) for more info about self-supervised learning.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-11T08:47:52.620Z
Comments.Text: @nbro It is good to know about self-supervised learning, Thank you.
  But what about the distribution of the data that used for it? and the actual labels of them?
  
  In Self_Taught learning, there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task.
Comments.UserDisplayName: Kais Hasan
Tags: <comparison><terminology><papers><unsupervised-learning><transfer-learning>
Title: What is the relation between self-taught learning and transfer learning?
UserDisplayName: Kais Hasan
ThreadText: # What is the relation between self-taught learning and transfer learning?
  
  (Asked by: username_0 on 2021-04-03)
  
  I am new to transfer learning and I start by reading [A Survey on Transfer Learning](https://ieeexplore.ieee.org/document/5288526), and it stated the following:
  
  > according to different situations of labeled and unlabeled data in the source domain, we can
  > further categorize the inductive transfer learning setting into two cases:
  >
  > case $(a)$ (It is irrelevant to my question).
  >
  > case $(b): $ No labeled data in the source domain are
  > available. In this case, the inductive transfer
  > learning setting is similar to the self-taught
  > learning setting, which is first proposed by Raina
  > et al. [22]. In the self-taught learning setting, the
  > label spaces between the source and target
  > domains may be different, which implies the
  > side information of the source domain cannot be
  > used directly. Thus, it’s similar to the inductive
  > transfer learning setting where the labeled data
  > in the source domain are unavailable.
  
  From that, I understand that self-taught learning is inductive transfer learning.
  
  But I opened the paper of self-taught learning that was mentioned (i.e [paper by Raina
  et al. [22].](https://www.researchgate.net/publication/200038912_Self-taught_learning_Transfer_learning_from_unlabeled_data)), and It stated the following in the introduction:
  
  > Because
  > self-taught learning places significantly fewer restrictions on
  > the type of unlabeled data, in many practical applications
  > (such as image, audio or text classification) it is much easier
  > to apply than typical semi-supervised learning or transfer learning
  > methods.
  
  And here it looks like transfer learning is different from self-taught learning.
  
  So what is the right relation between them?
  
  **Tags:** comparison, terminology, papers, unsupervised-learning, transfer-learning
  > **Comments:**
  > **username_1**: I'm not familiar with the term _self-taught learning_, but I suspect it is related to _self-supervised learning_, which is turn is related to transfer learning, as SSL can be used for transfer learning. See [this post](https://ai.stackexchange.com/q/10623/2444) for more info about self-supervised learning.  
  > **username_0**: @nbro It is good to know about self-supervised learning, Thank you. But what about the distribution of the data that used for it? and the actual labels of them?  In Self_Taught learning, there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-05-10. Score: 1
  
  After a lot of searches, I think self-taught learning is a Transfer learning category, I think when Self-taught learning paper published (2007), there isn't any good survey on transfer learning, and as seen by high citation of the paper of Pan, his paper (which published in 2009) describes transfer learning in a clear way that did not exist before it.
  
  Also, it is reasonable to consider Self-Taught learning a transfer learning category because he actually transfers the knowledge learned from unlabelled data to use it in the supervised task that we want to do (there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task).
  
  If someone could find something wrong with my answer or there is something missing, then please tell me.
  
  
  ---
  
  