Answers.Body: <p>It takes at least 10 bits to represent any number between <span class="math-container">$1-1000$</span> because <span class="math-container">$2^{10} = 1024$</span>. This means that if one was trying to represent 1 of the 1000 classes, one would need at least 10 bits. However, having these 10 bits set correctly for each input is really hard and would require overfitting to ensure it. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-03-16T03:11:44.243Z
Answers.LastActivityDate: 2020-06-12T23:44:49.797Z
Answers.LastEditDate: 2020-06-12T23:44:49.797Z
Answers.UserDisplayName: Jaden Travnik
Answers.Body: <p>You need 10-bits (<span class="math-container">$2^{10} = 1024$</span>) to represent 1000 classes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-03-16T03:10:58.827Z
Answers.LastActivityDate: 2020-06-12T23:44:19.557Z
Answers.LastEditDate: 2020-06-12T23:44:19.557Z
Answers.UserDisplayName: Brian O'Donnell
Body: <p>I'm reading <a href="https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks" rel="nofollow noreferrer">the AlexNet paper</a>. In section 4, where the authors explain how they prevent overfitting, they mention </p>
  
  <blockquote>
    <p>Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label". </p>
  </blockquote>
  
  <p>What does this mean?</p>
  
Tags: <machine-learning><deep-learning><papers><alexnet>
Title: Why do we need 10 bits to represent the 1000 classes in AlexNet?
UserDisplayName: harupy
ThreadText: # Why do we need 10 bits to represent the 1000 classes in AlexNet?
  
  (Asked by: username_0 on 2018-03-15)
  
  I'm reading [the AlexNet paper](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks). In section 4, where the authors explain how they prevent overfitting, they mention
  
  > Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label".
  
  What does this mean?
  
  **Tags:** machine-learning, deep-learning, papers, alexnet
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-03-16. Score: 4 | Accepted answer
  
  You need 10-bits ($2^{10} = 1024$) to represent 1000 classes.
  
  
  ---
  
  ### Answer by username_1 on 2018-03-16. Score: 1
  
  It takes at least 10 bits to represent any number between $1-1000$ because $2^{10} = 1024$. This means that if one was trying to represent 1 of the 1000 classes, one would need at least 10 bits. However, having these 10 bits set correctly for each input is really hard and would require overfitting to ensure it.
  
  
  ---
  
  