Answers.Body: <p>This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and <em>identically</em> distributed, i.e., you do <em>not</em> expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit assumption that data comes from the same family of distributions (e.g. only Gaussians) and that all your training examples are independently drawn from the same distribution (specific mean and variance). Of course, this is a big limitation!</p>
  <p>A partial solution to your problem is to use a <a href="https://arxiv.org/pdf/1505.05424.pdf" rel="nofollow noreferrer">Bayesian neural network</a> (BNN). The idea of a BNN is to associate, rather than a single number, a distribution (usually a Gaussian distribution) with each unit (or neuron) of the neural network. Therefore, for each unit of the network, there are two learnable parameters: the mean and variance of a Gaussian distribution. Consequently, a BNN usually has the double of the number of parameters of a conventional (or non-Bayesian) neural network. However, by learning a distribution for each parameter, you also learn the uncertainty about the potential true value of each unit, based on the available training data.</p>
  <p>The forward passes of such a BNN are stochastic, i.e. you sample from each of these Gaussian distributions for every forward pass, so the output of the network is also stochastic (i.e. given the same input example, the output may be different each time).</p>
  <p>If your dataset is small, one expects the BNN to have wide Gaussian distributions, i.e. high uncertainty about the true value of the units. So, one expects a BNN to be able to deal with unexpected inputs more robustly. To be more precise, if you train a BNN with a small dataset, the hope is that Gaussian distributions will be wide and thus the outputs of the BNN will be highly variable (i.e. the model is highly uncertain). The more data is gathered, the less uncertain the BNN should be.</p>
  <p>This doesn't completely solve your problem, but it should at least mitigate it, i.e. if you provide an unseen example to the BNN, then ideally it should be uncertain about the actual label of that input.</p>
  <p>For simplicity, I didn't explain certain details of BNNs, but, at least, this answer gives you a potential solution. Of course, this doesn't exclude the possibility of having an <a href="https://ai.stackexchange.com/q/4889/2444">&quot;unknown&quot; class</a>. The approaches are not mutually exclusive. There may also be other solutions, but I am not aware of any.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-21T23:11:48.527Z
Answers.LastActivityDate: 2021-01-17T12:14:24.247Z
Answers.LastEditDate: 2021-01-17T12:14:24.247Z
Answers.UserDisplayName: nbro
Body: <p>I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. </p>
  
  <p>If I trained a data model to identify and classify different types of irises and I show it a picture of a cat, is there a way to add in an "unknown" or "not a" classification or would it necessarily have to guess what type of iris the cat most looks like?</p>
  
  <p>Further, I could easily just add another classification with the label "not an iris" and train it using pictures of cats, but then what if I show it a picture of a chair (the list of objects goes on).</p>
  
  <p>Another example would be in natural language processing. If I develop an application that takes the input language and spits out "I think this is Spanish", what if it encounters a language it doesn't recognize?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-21T22:50:33.023Z
Comments.Text: The answer will depend on your purpose for training the classifier, what kinds of "not a" target you expect it to have as inputs when used. There is not a general answer possible with current technology, but specific answers are possible depending on what you are trying to do. Could you clarify the use case you have in mind - including how you expect the network to be used in production?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-22T00:39:32.047Z
Comments.Text: I'm writing a paper on some research I did involving source code stylometry. At this point, I have no intent in operationalizing this, but essentially, given a source code, can we identify this as an author we have seen before.
Comments.UserDisplayName: Jeff
Tags: <neural-networks><deep-learning><classification><bayesian-deep-learning><bayesian-neural-networks>
Title: How should the neural network deal with unexpected inputs?
UserDisplayName: Jeff
ThreadText: # How should the neural network deal with unexpected inputs?
  
  (Asked by: username_0 on 2020-03-21)
  
  I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects.
  
  If I trained a data model to identify and classify different types of irises and I show it a picture of a cat, is there a way to add in an "unknown" or "not a" classification or would it necessarily have to guess what type of iris the cat most looks like?
  
  Further, I could easily just add another classification with the label "not an iris" and train it using pictures of cats, but then what if I show it a picture of a chair (the list of objects goes on).
  
  Another example would be in natural language processing. If I develop an application that takes the input language and spits out "I think this is Spanish", what if it encounters a language it doesn't recognize?
  
  **Tags:** neural-networks, deep-learning, classification, bayesian-deep-learning, bayesian-neural-networks
  
  > **Comments:**
  > **username_1**: The answer will depend on your purpose for training the classifier, what kinds of "not a" target you expect it to have as inputs when used. There is not a general answer possible with current technology, but specific answers are possible depending on what you are trying to do. Could you clarify the use case you have in mind - including how you expect the network to be used in production?  
  > **username_0**: I'm writing a paper on some research I did involving source code stylometry. At this point, I have no intent in operationalizing this, but essentially, given a source code, can we identify this as an author we have seen before.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-21. Score: 11 | Accepted answer
  
  This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and *identically* distributed, i.e., you do *not* expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit assumption that data comes from the same family of distributions (e.g. only Gaussians) and that all your training examples are independently drawn from the same distribution (specific mean and variance). Of course, this is a big limitation!
  
  A partial solution to your problem is to use a [Bayesian neural network](https://arxiv.org/pdf/1505.05424.pdf) (BNN). The idea of a BNN is to associate, rather than a single number, a distribution (usually a Gaussian distribution) with each unit (or neuron) of the neural network. Therefore, for each unit of the network, there are two learnable parameters: the mean and variance of a Gaussian distribution. Consequently, a BNN usually has the double of the number of parameters of a conventional (or non-Bayesian) neural network. However, by learning a distribution for each parameter, you also learn the uncertainty about the potential true value of each unit, based on the available training data.
  
  The forward passes of such a BNN are stochastic, i.e. you sample from each of these Gaussian distributions for every forward pass, so the output of the network is also stochastic (i.e. given the same input example, the output may be different each time).
  
  If your dataset is small, one expects the BNN to have wide Gaussian distributions, i.e. high uncertainty about the true value of the units. So, one expects a BNN to be able to deal with unexpected inputs more robustly. To be more precise, if you train a BNN with a small dataset, the hope is that Gaussian distributions will be wide and thus the outputs of the BNN will be highly variable (i.e. the model is highly uncertain). The more data is gathered, the less uncertain the BNN should be.
  
  This doesn't completely solve your problem, but it should at least mitigate it, i.e. if you provide an unseen example to the BNN, then ideally it should be uncertain about the actual label of that input.
  
  For simplicity, I didn't explain certain details of BNNs, but, at least, this answer gives you a potential solution. Of course, this doesn't exclude the possibility of having an ["unknown" class](https://ai.stackexchange.com/q/4889/2444). The approaches are not mutually exclusive. There may also be other solutions, but I am not aware of any.
  
  
  ---
  
  