Answers.Body: <blockquote>
    <p>The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. </p>
  </blockquote>
  
  <p>You need to train the network on noise well if you want it to be able to recognize it.</p>
  
  <blockquote>
    <p>or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?</p>
  </blockquote>
  
  <p>You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better</p>
  
  <p>See:</p>
  
  <blockquote>
    <p>Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-13T07:52:31.447Z
Answers.LastActivityDate: 2018-10-13T07:52:31.447Z
Answers.UserDisplayName: Martin Thoma
Body: <p>I have created a classifier for some simple gestures using an input layer, a hidden layer with tanh activation and an output softmax layer. I'm also using the Adam optimiser. The network classifies perfectly with validation data. However, I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. For example:</p>
  
  <p>One gesture input looks like this and is correctly classified as gesture 'A':
  <a href="https://i.sstatic.net/JlvjP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JlvjP.png" alt="enter image description here"></a></p>
  
  <p>However, when I pass this 'noise', which is clearly differentiable to the human eye, as input it still classifies it with 100% confidence that it is the same gesture 'A'.</p>
  
  <p><a href="https://i.sstatic.net/3DBKX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3DBKX.png" alt="enter image description here"></a></p>
  
  <p>I assume it's because the inputs are still very close to 0? My instinct is to scale up the inputs perhaps to increase the differentiation between the noise and the input. However, in real operation the noise will all be on a similar scale to the inputs and I won't know what is noise and what isn't so I will still have to apply the same scaling to that noise. Will I run into the same problem? </p>
  
  <p>On a more general note is there a teaching approach to prevent misclassifications, particularly if we know what they might look like? For example, in this case I thought I could perhaps generate some noise and use it at training time to create an extra noise class, or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold? For example, if the network only produces 50% classifcation confidence for an input then I can discard it as noise. Any suggestions much appreciated!  </p>
  
Tags: <convolutional-neural-networks><classification>
Title: Recognising Noise in Simple Classification
UserDisplayName: William Smith
ThreadText: # Recognising Noise in Simple Classification
  
  (Asked by: username_0 on 2018-10-12)
  
  I have created a classifier for some simple gestures using an input layer, a hidden layer with tanh activation and an output softmax layer. I'm also using the Adam optimiser. The network classifies perfectly with validation data. However, I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. For example:
  
  One gesture input looks like this and is correctly classified as gesture 'A':
  [![enter image description here](https://i.sstatic.net/JlvjP.png)](https://i.sstatic.net/JlvjP.png)
  
  However, when I pass this 'noise', which is clearly differentiable to the human eye, as input it still classifies it with 100% confidence that it is the same gesture 'A'.
  
  [![enter image description here](https://i.sstatic.net/3DBKX.png)](https://i.sstatic.net/3DBKX.png)
  
  I assume it's because the inputs are still very close to 0? My instinct is to scale up the inputs perhaps to increase the differentiation between the noise and the input. However, in real operation the noise will all be on a similar scale to the inputs and I won't know what is noise and what isn't so I will still have to apply the same scaling to that noise. Will I run into the same problem?
  
  On a more general note is there a teaching approach to prevent misclassifications, particularly if we know what they might look like? For example, in this case I thought I could perhaps generate some noise and use it at training time to create an extra noise class, or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold? For example, if the network only produces 50% classifcation confidence for an input then I can discard it as noise. Any suggestions much appreciated!
  
  **Tags:** convolutional-neural-networks, classification
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-10-13. Score: 0 | Accepted answer
  
  > The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently.
  
  You need to train the network on noise well if you want it to be able to recognize it.
  
  > or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?
  
  You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better
  
  See:
  
  > Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
  
  
  ---
  
  