Answers.Body: <p>what are the dimensions of the embeddings? The first thing that comes to mind is that you probably should be wary of using euclidean distance in high dimensions (Distribution of Distances between Elements in a Compact Set), as done when computing the Silhouette Coefficient.</p>
  <p>However, that's probably not an issue for typical dimensions of 10 to 100. It would be good to get more details on your training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-29T01:15:32.920Z
Answers.Text: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-28T14:34:31.533Z
Answers.LastActivityDate: 2022-06-28T14:34:31.533Z
Answers.UserDisplayName: Jonas
Body: <p>I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.</p>
  <p>The downstream task is a classification problem.</p>
  <p><strong>Transfer Learning</strong>
  Due to limited data, I tried to use Transfer learning model trained on &quot;Imagenet&quot;(ResNet50 V2 &quot;Deep Residual Learning for Image Recognition Kaiming He, et.al&quot;).</p>
  <p>I used the embedding from the pretrained model and trained Linear SVM and achieved a F1 score of 0.84.</p>
  <p><strong>Contrastive Learning</strong>
  I also trained a model for contrastive learning using Facenet technique(&quot;FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff, et.al&quot;) and further used the embedding for training a Linear SVM for classification problem. The achieved F1 score is 0.83.</p>
  <p><strong>Problem</strong>
  Though the scores of both the concept are closeby, I tried to evaluate the quality of both the embeddings using Silhouette Coefficient.</p>
  <p><em><strong>Overall Silhouette Coefficient:</strong></em></p>
  <ul>
  <li>Transfer Learning Embedding = 0.05</li>
  <li>Contrastive Learning Embedding = 0.49</li>
  </ul>
  <p>I do not understand this behaviour of the system that even with lower Silhouette Coefficient, the transfer learning model is able to perform well.</p>
  <p>Kindly provide me with your views on it</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-02T22:09:12.527Z
Comments.Text: Can you please be more **specific** and put your **specific question** in the title (too)? "Kindly provide me with your views on it" is not a question and it's also not specific and "A vs B" is also not a question.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-02T22:41:34.297Z
Comments.Text: Thanks a lot @nbro. I was bit unsure how to frame such a long question in the title.
  
  But I will try to explain the part which was unclear to you. Thanks once again.
Comments.UserDisplayName: Raj Rajeshwari Prasad
Tags: <machine-learning><deep-learning><transfer-learning><contrastive-learning>
Title: Embedding Quality of Transfer Learning model vs Contrastive learning model
UserDisplayName: Raj Rajeshwari Prasad
ThreadText: # Embedding Quality of Transfer Learning model vs Contrastive learning model
  
  (Asked by: username_0 on 2022-06-01)
  
  I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.
  
  The downstream task is a classification problem.
  
  **Transfer Learning**
  Due to limited data, I tried to use Transfer learning model trained on "Imagenet"(ResNet50 V2 "Deep Residual Learning for Image Recognition Kaiming He, et.al").
  
  I used the embedding from the pretrained model and trained Linear SVM and achieved a F1 score of 0.84.
  
  **Contrastive Learning**
  I also trained a model for contrastive learning using Facenet technique("FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff, et.al") and further used the embedding for training a Linear SVM for classification problem. The achieved F1 score is 0.83.
  
  **Problem**
  Though the scores of both the concept are closeby, I tried to evaluate the quality of both the embeddings using Silhouette Coefficient.
  
  ***Overall Silhouette Coefficient:***
  
  * Transfer Learning Embedding = 0.05
  * Contrastive Learning Embedding = 0.49
  
  I do not understand this behaviour of the system that even with lower Silhouette Coefficient, the transfer learning model is able to perform well.
  
  Kindly provide me with your views on it
  
  **Tags:** machine-learning, deep-learning, transfer-learning, contrastive-learning
  > **Comments:**
  > **username_1**: Can you please be more **specific** and put your **specific question** in the title (too)? "Kindly provide me with your views on it" is not a question and it's also not specific and "A vs B" is also not a question.  
  > **username_0**: Thanks a lot @nbro. I was bit unsure how to frame such a long question in the title.  But I will try to explain the part which was unclear to you. Thanks once again.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-06-28. Score: 0
  
  what are the dimensions of the embeddings? The first thing that comes to mind is that you probably should be wary of using euclidean distance in high dimensions (Distribution of Distances between Elements in a Compact Set), as done when computing the Silhouette Coefficient.
  
  However, that's probably not an issue for typical dimensions of 10 to 100. It would be good to get more details on your training.
  
  > **Comments:**
  > **username_3**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  