Answers.Body: <p>I see multiple reasons to take a different route:</p>
  
  <ol>
  <li><p>While in "classical" pattern recognition you might have done things like feature engineering outside of your model, one idea of deep learning was to "insource" it into the model and let the model take care of "everything". Following that, I have seen a general tendency in deep net architectures to let your deep net handle all the work in one single model. So your idea kind of contradicts the architectural mainstream.</p></li>
  <li><p>There are so many deep net architectures being published which are well engineered and tested for all kinds of tasks. Therefore, I would check if there is anything "ready off the shelf" for a task like yours in the very first place. If there is, then it will probably be better than your self-engineered model architecture. And if there is not, you might want to check related tasks and see how they solved this problem, e.g. medical applications where specific cells need to be detected with most examples being negative (and if there is a positive example the area in a picture might still be mostly negative as it is just a single positive cell among many others).  </p></li>
  <li><p>Your thought of feeding the "present/not present" binary classification to your task of locating an object is very closely related to what has been discussed as auxiliary tasks. I don't know if it has been applied to exactly your problem but there are similar applications. For example there is an application where "name recognition" is complemented by the auxiliary task of deciding whether any name is present in a given sentence or not. Which is somewhat similar to your case. The paper <a href="https://arxiv.org/pdf/1706.05098.pdf" rel="nofollow noreferrer">An Overview of Multi-Task Learning in Deep Neural Networks</a> provides an overview. A famous example using auxiliary tasks would be <a href="https://arxiv.org/pdf/1409.4842.pdf" rel="nofollow noreferrer">GoogleNet</a>. This also goes back to my first point of letting the deep net handle "everything" internally.  </p></li>
  </ol>
  
  <p>However, as a disclaimer, this is a theoretical perspective as I cannot speak from experience regarding your problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-05T20:38:09.160Z
Answers.LastActivityDate: 2019-12-05T20:38:09.160Z
Answers.UserDisplayName: Jonathan
Body: <p>I am building a Convolution neural network to predict certain categories based on images (the location of a pointer on a surface) . However in many cases there will be no pointer in the view or something that is not the pointer. Initially I was just going to train it with outputs of the different classifications including the null classification. However given that the null classification is far more common than the others (perhaps 1000 times more likely) would it be better to have a separate null classifier and then if this outputted non null then the second classifier would be used.</p>
  
  <p>Any suggestions?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-05T15:06:20.423Z
Comments.Text: Hi, can you share some more details on the area of application? What is it that you are trying to locate in the images? And what exactly does it mean when "nothing" is in view?
Comments.UserDisplayName: Jonathan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-05T15:16:57.527Z
Comments.Text: I've updated the question to reflect the problem is about locating a pointer on a surface in an image
Comments.UserDisplayName: Matthew Macfarlane
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-06T02:59:45.167Z
Comments.Text: You'll be fine so long as you use softmax and cross entropy loss. Take Alexnet. It has 1000 classes to classify, and since it only ever outputs 1, 999 times out of 1000, the desired output is 0. Despite this, it still operates fantastically due to its final activation and loss function.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-06T09:11:55.440Z
Comments.Text: I think there is a significant difference between an example where there are many classes each reasonably unlikely to an example where one state has around a 99.9% chance of occurring. In my example my main worry is making sure there are very few errors in this class.
Comments.UserDisplayName: Matthew Macfarlane
Tags: <convolutional-neural-networks><classification>
Title: structure of neural network for classification problems with large amounts of null classifications
UserDisplayName: Matthew Macfarlane
ThreadText: # structure of neural network for classification problems with large amounts of null classifications
  
  (Asked by: username_0 on 2019-12-05)
  
  I am building a Convolution neural network to predict certain categories based on images (the location of a pointer on a surface) . However in many cases there will be no pointer in the view or something that is not the pointer. Initially I was just going to train it with outputs of the different classifications including the null classification. However given that the null classification is far more common than the others (perhaps 1000 times more likely) would it be better to have a separate null classifier and then if this outputted non null then the second classifier would be used.
  
  Any suggestions?
  
  **Tags:** convolutional-neural-networks, classification
  
  > **Comments:**
  > **username_1**: Hi, can you share some more details on the area of application? What is it that you are trying to locate in the images? And what exactly does it mean when "nothing" is in view?  
  > **username_0**: I've updated the question to reflect the problem is about locating a pointer on a surface in an image  
  > **username_2**: You'll be fine so long as you use softmax and cross entropy loss. Take Alexnet. It has 1000 classes to classify, and since it only ever outputs 1, 999 times out of 1000, the desired output is 0. Despite this, it still operates fantastically due to its final activation and loss function.  
  > **username_0**: I think there is a significant difference between an example where there are many classes each reasonably unlikely to an example where one state has around a 99.9% chance of occurring. In my example my main worry is making sure there are very few errors in this class.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-12-05. Score: 1
  
  I see multiple reasons to take a different route:
  
  1. While in "classical" pattern recognition you might have done things like feature engineering outside of your model, one idea of deep learning was to "insource" it into the model and let the model take care of "everything". Following that, I have seen a general tendency in deep net architectures to let your deep net handle all the work in one single model. So your idea kind of contradicts the architectural mainstream.
  2. There are so many deep net architectures being published which are well engineered and tested for all kinds of tasks. Therefore, I would check if there is anything "ready off the shelf" for a task like yours in the very first place. If there is, then it will probably be better than your self-engineered model architecture. And if there is not, you might want to check related tasks and see how they solved this problem, e.g. medical applications where specific cells need to be detected with most examples being negative (and if there is a positive example the area in a picture might still be mostly negative as it is just a single positive cell among many others).
  3. Your thought of feeding the "present/not present" binary classification to your task of locating an object is very closely related to what has been discussed as auxiliary tasks. I don't know if it has been applied to exactly your problem but there are similar applications. For example there is an application where "name recognition" is complemented by the auxiliary task of deciding whether any name is present in a given sentence or not. Which is somewhat similar to your case. The paper [An Overview of Multi-Task Learning in Deep Neural Networks](https://arxiv.org/pdf/1706.05098.pdf) provides an overview. A famous example using auxiliary tasks would be [GoogleNet](https://arxiv.org/pdf/1409.4842.pdf). This also goes back to my first point of letting the deep net handle "everything" internally.
  
  However, as a disclaimer, this is a theoretical perspective as I cannot speak from experience regarding your problem.
  
  
  ---
  
  