Answers.Body: <p>I understand your questions as follows:</p>
  
  <p><strong>Is it valid to have an all-zeros output for a certain input?</strong> </p>
  
  <p>Yes its possible in some cases like:</p>
  
  <ol>
  <li><p>In the tutorial "jigsaw-toxic-comment-classification-challenge", the data is taken from wikipedia comments, so generally there is no toxic behavior in people's comment, it is very rare that a person posts "bad" comments in an informative source like wikipedia, this might lead to all the labels being zero for a particular example.</p></li>
  <li><p>In a single label classification like predicting whether a person has a rare disease,the dataset would contain labels that are mostly "zero"-(no disease)-skewed towards zero</p></li>
  </ol>
  
  <p>This happens in datasets where the positive output to be predicted is a very rare case.</p>
  
  <p><strong>I guess your second question is -whether "accuracy" is a good way of evaluating a model for this type of a problem</strong></p>
  
  <p>You are right, in such a case <strong>even</strong> a simple program/model that outputs zeros for all the inputs would achieve an accuracy of more than 90%, <strong>so</strong> here <strong>accuracy</strong> is <strong>not</strong> a good metric to evaluate the model on.</p>
  
  <p>You should go through the metrics <strong>f1_score</strong>,<strong>recall</strong>, and <strong>precision</strong> which are ideal for this type of problems.</p>
  
  <p>Basically here we are interested in "out of those which are predicted positive, how many are really positive -precision" and "out of those which should have been predicted positive, how many are really predicted positive"</p>
  
  <p>If my definitions seem confusing to understand, please got through the link below:</p>
  
  <p><a href="https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9" rel="nofollow noreferrer">f1_score/recall/precision</a></p>
  
  <p>Hope I was helpful</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-04T10:56:25.700Z
Answers.LastActivityDate: 2019-09-04T10:56:25.700Z
Answers.UserDisplayName: SUMAN
Body: <p>I tried <a href="https://androidkt.com/multi-label-text-classification-in-tensorflow-keras/" rel="nofollow noreferrer">this</a> example for a multi class classifier, but when looking at the data I realized two things:</p>
  
  <ol>
  <li>There are many examples of "all zeros" vectors, that is, messages that don't belong in any classification.</li>
  <li>These all-zeros are actually the majority, by far.</li>
  </ol>
  
  <p>Is it valid to have an all-zeros output for a certain input? I would guess a Sigmoid activation would have no problems with this, by simply not trying to force a one out of all the "near zero" outputs.</p>
  
  <p>But I also think an "accuracy" metric will be skewed too optimistically: if all outputs are zero 90% of the time, the network will quickly overfit to always output 0 all the time, and get 90% score.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-05T12:13:11.947Z
Comments.Text: are you sure that this is not an effect of processing mistake to have all zeros?
Comments.UserDisplayName: quester
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-05T12:14:46.457Z
Comments.Text: also if you have all zeros in training data so maybe add additional column that will be = 1 when this situation happens else 0
Comments.UserDisplayName: quester
Tags: <supervised-learning>
Title: Class imbalance and "all zeros" one-hot encoding?
UserDisplayName: hjf
ThreadText: # Class imbalance and "all zeros" one-hot encoding?
  
  (Asked by: username_0 on 2019-09-03)
  
  I tried [this](https://androidkt.com/multi-label-text-classification-in-tensorflow-keras/) example for a multi class classifier, but when looking at the data I realized two things:
  
  1. There are many examples of "all zeros" vectors, that is, messages that don't belong in any classification.
  2. These all-zeros are actually the majority, by far.
  
  Is it valid to have an all-zeros output for a certain input? I would guess a Sigmoid activation would have no problems with this, by simply not trying to force a one out of all the "near zero" outputs.
  
  But I also think an "accuracy" metric will be skewed too optimistically: if all outputs are zero 90% of the time, the network will quickly overfit to always output 0 all the time, and get 90% score.
  
  **Tags:** supervised-learning
  
  > **Comments:**
  > **username_1**: are you sure that this is not an effect of processing mistake to have all zeros?  
  > **username_1**: also if you have all zeros in training data so maybe add additional column that will be = 1 when this situation happens else 0  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-09-04. Score: 0 | Accepted answer
  
  I understand your questions as follows:
  
  **Is it valid to have an all-zeros output for a certain input?**
  
  Yes its possible in some cases like:
  
  1. In the tutorial "jigsaw-toxic-comment-classification-challenge", the data is taken from wikipedia comments, so generally there is no toxic behavior in people's comment, it is very rare that a person posts "bad" comments in an informative source like wikipedia, this might lead to all the labels being zero for a particular example.
  2. In a single label classification like predicting whether a person has a rare disease,the dataset would contain labels that are mostly "zero"-(no disease)-skewed towards zero
  
  This happens in datasets where the positive output to be predicted is a very rare case.
  
  **I guess your second question is -whether "accuracy" is a good way of evaluating a model for this type of a problem**
  
  You are right, in such a case **even** a simple program/model that outputs zeros for all the inputs would achieve an accuracy of more than 90%, **so** here **accuracy** is **not** a good metric to evaluate the model on.
  
  You should go through the metrics **f1\_score**,**recall**, and **precision** which are ideal for this type of problems.
  
  Basically here we are interested in "out of those which are predicted positive, how many are really positive -precision" and "out of those which should have been predicted positive, how many are really predicted positive"
  
  If my definitions seem confusing to understand, please got through the link below:
  
  [f1\_score/recall/precision](https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9)
  
  Hope I was helpful
  
  
  ---
  
  