Answers.Body: <p>No, the images do not need to be same. You can use different images for downstream task however you need to do some changes in model definition while loading model state_dict as CNN architecture used for pretext task 'relative patch location', assume Alexnet will expect 2 patches as input.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-08T11:19:46.977Z
Answers.LastActivityDate: 2021-05-08T11:19:46.977Z
Answers.UserDisplayName: Hemi
Body: <p>I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on <em><strong>downstream</strong></em> tasks when the network is trained on <em><strong>pretext</strong></em> tasks.</p>
  <p>Suppose that I want to do image classification on my own set of images. I have limited data on these images and maybe I can use self-supervised learning to achieve better accuracies on these limited data.</p>
  <p>Let's say that I try to train a neural network on a pretext task of <a href="https://arxiv.org/abs/1505.05192" rel="nofollow noreferrer">predicting the patch position relative to the center patch</a> on different images that are readily available in quantity, such as cats, dogs, etc.</p>
  <p>If I try to initialise the weights of my neural network, then do image classification on my own images, which are vastly different from that of the images used in the pretext task, would self-supervised learning work because the images for the pretext and downstream tasks are different?</p>
  <p><strong>TLDR</strong>: Must the images used in the pretext task and the downstream tasks be the same?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-04T12:35:18.750Z
Comments.Text: This question is related to https://ai.stackexchange.com/q/17231/2444, but probably not a duplicate, and no answer there actually answers the/your question.
Comments.UserDisplayName: nbro
Tags: <computer-vision><image-recognition><transfer-learning><self-supervised-learning><pretext-tasks>
Title: Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?
UserDisplayName: calveeen
ThreadText: # Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?
  
  (Asked by: username_0 on 2020-12-03)
  
  I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on ***downstream*** tasks when the network is trained on ***pretext*** tasks.
  
  Suppose that I want to do image classification on my own set of images. I have limited data on these images and maybe I can use self-supervised learning to achieve better accuracies on these limited data.
  
  Let's say that I try to train a neural network on a pretext task of [predicting the patch position relative to the center patch](https://arxiv.org/abs/1505.05192) on different images that are readily available in quantity, such as cats, dogs, etc.
  
  If I try to initialise the weights of my neural network, then do image classification on my own images, which are vastly different from that of the images used in the pretext task, would self-supervised learning work because the images for the pretext and downstream tasks are different?
  
  **TLDR**: Must the images used in the pretext task and the downstream tasks be the same?
  
  **Tags:** computer-vision, image-recognition, transfer-learning, self-supervised-learning, pretext-tasks
  > **Comments:**
  > **username_1**: This question is related to https://ai.stackexchange.com/q/17231/2444, but probably not a duplicate, and no answer there actually answers the/your question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-05-08. Score: 1
  
  No, the images do not need to be same. You can use different images for downstream task however you need to do some changes in model definition while loading model state\_dict as CNN architecture used for pretext task 'relative patch location', assume Alexnet will expect 2 patches as input.
  
  
  ---
  
  