Answers.Body: <p>Classification tasks with a large number of classes are usually handled with <a href="http://ruder.io/word-embeddings-softmax/index.html#hierarchicalsoftmax" rel="nofollow noreferrer">hierarchical softmax</a> to reduce the complexity of the final layer. This is useful, for example, in applications such as word embedding where you have hundreds of thousands of classes (words), like in your case.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-25T23:52:06.340Z
Answers.LastActivityDate: 2018-06-25T23:52:06.340Z
Answers.UserDisplayName: cantordust
Answers.Body: <p>A large one! </p>
  
  <p>In all seriousness, imagenet had roughly 1000 classes and did not require anything special from the top submissions. Depending on how deep(contextually) these classes are, you may want to do something like multi-label classification. Your biggest problems will likely be differentiating between classes, as well as class distribution.</p>
  
  <p>Good luck!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-26T20:52:08.487Z
Answers.Text: I agree with @hisairnessag3 also remember to apply one-hot encoding, multi-hot encoding or sparse categorical depending on your dataset structure.
Answers.UserDisplayName: Simbarashe Timothy Motsi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-25T13:35:18.567Z
Answers.LastActivityDate: 2018-06-25T13:35:18.567Z
Answers.UserDisplayName: hisairnessag3
Answers.Body: <p>As you can imagine and as it has already been said, a large one for your network to tune weights and biases. But I wanted to nuance this statement with two points</p>
  
  <p>First :  you can use an <a href="https://en.wikipedia.org/wiki/Autoencoder" rel="nofollow noreferrer">Autoencodeur</a> to pre-process your images. It can reduce dimensionality and so improve the learning capability and efficiency (in a generalization point of view). 
  This kind of NN takes your images as inputs, encode and then decode them to provie new representation of your initial images. Dealing with the decoded dataset can allow you to consider less hidden layer with less hidden nodes and then speed your work up.</p>
  
  <p>Second : architecture is sure a thing to deal with image recognition, but you can also play on the input representation (that is what the autoencoder aforesaid is about). You can look at <a href="https://en.wikipedia.org/wiki/Principal_component_analysis" rel="nofollow noreferrer">PCA</a> (Principal Component Analysis). It allows to reduce dimensionality to a certain number of components (that you specify). It is often used in face recognition where inputs and targets are various. </p>
  
  <p>All that to say that architecture is sure a thing when dealing with large datasets, but there also few tools to reshape the inputs so that then can be more easily learnt. </p>
  
  <p>And by doing so you can improve the capability of your network as much in term of time computation as in quality and accuracy of prediction</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-25T14:08:22.557Z
Answers.LastActivityDate: 2018-06-25T14:08:22.557Z
Answers.UserDisplayName: nsaura
Answers.Body: <p><a href="https://www.cs.toronto.edu/~fritz/absps/imagenet.pdf" rel="nofollow noreferrer">Alexnet</a> (2012), <a href="https://arxiv.org/abs/1312.6229" rel="nofollow noreferrer">Overfeat</a> (2013), <a href="https://arxiv.org/abs/1409.1556" rel="nofollow noreferrer">VGG</a> (2014) and <a href="http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html" rel="nofollow noreferrer">ResNet</a> (2016) are cited in many image recognition or segmentation applications. There is also <a href="https://arxiv.org/abs/1409.4842" rel="nofollow noreferrer">GoogleLeNet</a> (2015). 
  The lastest is the publication the denser is the network.</p>
  
  <p>The ResNet publication comments on how the network density affects accuracy depending on the image data set size. The article tends to give a motivated answer to the question </p>
  
  <blockquote>
    <p>Is learning better networks as easy as stacking more layers?</p>
  </blockquote>
  
  <p>You might consider the training time since you have your own image data set depending on the kind of hardware you can you use ( <a href="https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks/" rel="nofollow noreferrer">see this benchmark</a> for instance ). The denser the more time it will takes.</p>
  
  <p>You also have to consider the size of the traning data set w/r to the expected accuracy. If the set is too small the net will probably overfeat. In that case you migh consider a data augmentation strategy (one of the answers mentions <em>auto encoding</em>, I m not sure but this might help for this purpose).</p>
  
  <p>All these publications refer to the <a href="http://www.image-net.org/" rel="nofollow noreferrer">ImageNet</a> data base and the associated image classification/detection <a href="http://www.image-net.org/challenges/LSVRC/" rel="nofollow noreferrer">contest</a> which has 1000 classes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-03T09:14:33.997Z
Answers.LastActivityDate: 2020-02-03T09:14:33.997Z
Answers.UserDisplayName: pascal sautot
Body: <p>I have an image dataset where objects may belong to one of the hundred thousand classes.</p>
  
  <p>What kind of neural network architecture should I use in order to achieve this?</p>
  
Tags: <neural-networks><machine-learning><deep-learning><convolutional-neural-networks><classification>
Title: What kind of neural network architecture do I use to classify images into one hundred thousand classes?
UserDisplayName: Mohsin
ThreadText: # What kind of neural network architecture do I use to classify images into one hundred thousand classes?
  
  (Asked by: username_0 on 2018-06-25)
  
  I have an image dataset where objects may belong to one of the hundred thousand classes.
  
  What kind of neural network architecture should I use in order to achieve this?
  
  **Tags:** neural-networks, machine-learning, deep-learning, convolutional-neural-networks, classification
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2018-06-25. Score: 2
  
  Classification tasks with a large number of classes are usually handled with [hierarchical softmax](http://ruder.io/word-embeddings-softmax/index.html#hierarchicalsoftmax) to reduce the complexity of the final layer. This is useful, for example, in applications such as word embedding where you have hundreds of thousands of classes (words), like in your case.
  
  
  ---
  
  ### Answer by username_2 on 2018-06-25. Score: 1
  
  A large one!
  
  In all seriousness, imagenet had roughly 1000 classes and did not require anything special from the top submissions. Depending on how deep(contextually) these classes are, you may want to do something like multi-label classification. Your biggest problems will likely be differentiating between classes, as well as class distribution.
  
  Good luck!
  
  > **Comments:**
  > **username_3**: I agree with @hisairnessag3 also remember to apply one-hot encoding, multi-hot encoding or sparse categorical depending on your dataset structure.  
  
  ---
  
  ### Answer by username_4 on 2018-06-25. Score: 1
  
  As you can imagine and as it has already been said, a large one for your network to tune weights and biases. But I wanted to nuance this statement with two points
  
  First : you can use an [Autoencodeur](https://en.wikipedia.org/wiki/Autoencoder) to pre-process your images. It can reduce dimensionality and so improve the learning capability and efficiency (in a generalization point of view).
  This kind of NN takes your images as inputs, encode and then decode them to provie new representation of your initial images. Dealing with the decoded dataset can allow you to consider less hidden layer with less hidden nodes and then speed your work up.
  
  Second : architecture is sure a thing to deal with image recognition, but you can also play on the input representation (that is what the autoencoder aforesaid is about). You can look at [PCA](https://en.wikipedia.org/wiki/Principal_component_analysis) (Principal Component Analysis). It allows to reduce dimensionality to a certain number of components (that you specify). It is often used in face recognition where inputs and targets are various.
  
  All that to say that architecture is sure a thing when dealing with large datasets, but there also few tools to reshape the inputs so that then can be more easily learnt.
  
  And by doing so you can improve the capability of your network as much in term of time computation as in quality and accuracy of prediction
  
  
  ---
  
  ### Answer by username_5 on 2020-02-03. Score: 0
  
  [Alexnet](https://www.cs.toronto.edu/~fritz/absps/imagenet.pdf) (2012), [Overfeat](https://arxiv.org/abs/1312.6229) (2013), [VGG](https://arxiv.org/abs/1409.1556) (2014) and [ResNet](http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html) (2016) are cited in many image recognition or segmentation applications. There is also [GoogleLeNet](https://arxiv.org/abs/1409.4842) (2015).
  The lastest is the publication the denser is the network.
  
  The ResNet publication comments on how the network density affects accuracy depending on the image data set size. The article tends to give a motivated answer to the question
  
  > Is learning better networks as easy as stacking more layers?
  
  You might consider the training time since you have your own image data set depending on the kind of hardware you can you use ( [see this benchmark](https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks/) for instance ). The denser the more time it will takes.
  
  You also have to consider the size of the traning data set w/r to the expected accuracy. If the set is too small the net will probably overfeat. In that case you migh consider a data augmentation strategy (one of the answers mentions *auto encoding*, I m not sure but this might help for this purpose).
  
  All these publications refer to the [ImageNet](http://www.image-net.org/) data base and the associated image classification/detection [contest](http://www.image-net.org/challenges/LSVRC/) which has 1000 classes.
  
  
  ---
  
  