Answers.Body: <p>After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a &quot;<strong>Big Task agnostic CNN</strong>&quot; to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read <a href="https://arxiv.org/pdf/2006.10029.pdf" rel="nofollow noreferrer">Big Self-Supervised Models are Strong Semi-Supervised Learners</a> by <em>Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton</em>. The code for the following can be found <a href="https://github.com/google-research/simclr" rel="nofollow noreferrer">here</a>. But I feel that training such a model would take up a lot of computational resources.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-06T04:14:47.790Z
Answers.LastActivityDate: 2020-10-06T04:14:47.790Z
Answers.UserDisplayName: Saurav Maheshkar
Body: <p>I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.</p>
  <p>Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this &quot;larger&quot; CNN to multiple &quot;smaller&quot; more specialized ones that take in images from one particular sport?</p>
  <p>In other words, I want to know that if I have a &quot;larger&quot; CNN that gives me an output like &quot;football being kicked&quot;, is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a  system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?</p>
  <p>Kind of like having a smaller CNN as a &quot;ground-truth&quot; for the bigger one? In my head it kind of goes like this:</p>
  <pre><code>large_net_output = 'Football kick identified with 95.56% confidence' 
  
  for sport in large_net:
      if sport == 'football':
          access = small_net_for_football
          return small_net_for_football_output
  
      elif sport == 'swimming':
          access = small_net_for_swimming
          return small_net_for_swimming_output
  
      elif sport == 'baseball':
          access = small_net_for_baseball
          return small_net_for_baseball_output
  
  # and so on....
  &gt;&gt;&gt; small_net_for_football_output = 'Football kick identified with 97.32% confidence'
  
  robustness_check = large_net_output - small_net_for_football_output
  print(robustness_check)
  
  &gt;&gt;&gt; 'Your system is accurate within a good range of 1.76%'
       
  </code></pre>
  <p>I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-19T00:39:51.470Z
Comments.Text: Cross-posted: https://ai.stackexchange.com/q/23772/1794, https://stats.stackexchange.com/q/489205/2921.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068).
Comments.UserDisplayName: D.W.
Tags: <neural-networks><machine-learning><convolutional-neural-networks><classification>
Title: Comparing a large/general CNN to a smaller more specialized one?
UserDisplayName: User_13
ThreadText: # Comparing a large/general CNN to a smaller more specialized one?
  
  (Asked by: username_0 on 2020-09-26)
  
  I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.
  
  Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this "larger" CNN to multiple "smaller" more specialized ones that take in images from one particular sport?
  
  In other words, I want to know that if I have a "larger" CNN that gives me an output like "football being kicked", is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?
  
  Kind of like having a smaller CNN as a "ground-truth" for the bigger one? In my head it kind of goes like this:
  
  ```
  large_net_output = 'Football kick identified with 95.56% confidence' 
  
  for sport in large_net:
      if sport == 'football':
          access = small_net_for_football
          return small_net_for_football_output
  
      elif sport == 'swimming':
          access = small_net_for_swimming
          return small_net_for_swimming_output
  
      elif sport == 'baseball':
          access = small_net_for_baseball
          return small_net_for_baseball_output
  
  # and so on....
  >>> small_net_for_football_output = 'Football kick identified with 97.32% confidence'
  
  robustness_check = large_net_output - small_net_for_football_output
  print(robustness_check)
  
  >>> 'Your system is accurate within a good range of 1.76%'
       
  
  ```
  
  I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!
  
  **Tags:** neural-networks, machine-learning, convolutional-neural-networks, classification
  
  > **Comments:**
  > **username_1**: Cross-posted: https://ai.stackexchange.com/q/23772/1794, https://stats.stackexchange.com/q/489205/2921.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-10-06. Score: 1 | Accepted answer
  
  After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a "**Big Task agnostic CNN**" to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://arxiv.org/pdf/2006.10029.pdf) by *Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton*. The code for the following can be found [here](https://github.com/google-research/simclr). But I feel that training such a model would take up a lot of computational resources.
  
  
  ---
  
  