Answers.Body: <p>You could say that the class of the data (e.g. spam vs not spam) is a hidden quality that can be inferred through the observable features (e.g. message subject contains &quot;bitcoin&quot;),</p>
  <p><span class="math-container">$$P(C \; | \; F)$$</span></p>
  <p>which says that the probability of the class <span class="math-container">$C$</span> is conditioned on the visibility of the feature <span class="math-container">$F$</span>.</p>
  <p>Using Bayes' theorem we can write</p>
  <p><span class="math-container">$$P(C \; | \; F) = \frac{P(F \; | \; C) \, P(C)}{P(F)}$$</span></p>
  <p>Your problem is that you are not taking into account the <strong>evidence</strong>, <span class="math-container">$P(F)$</span>, denominator in this expression.</p>
  <p><a href="https://i.sstatic.net/3ujvJ.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3ujvJ.jpg" alt="Bayes' theorem label factors" /></a></p>
  <p>You can have multiple features in your <strong>likelihood</strong> function</p>
  <p><span class="math-container">$$P(F \; | \; C) = P(F_1,F_2,F_3,\ldots \; | \; C)$$</span></p>
  <p>and these can be independent</p>
  <p><span class="math-container">$$\prod_k P(F_k \; | \; C)$$</span></p>
  <p>and then you have your <strong>prior</strong></p>
  <p><span class="math-container">$$P(C) \prod_k P(F_k \; | \; C)$$</span></p>
  <p>but what you are missing is the <strong>evidence</strong> factor in the denominator</p>
  <p><span class="math-container">$$P(C \; | \; F) = \frac{P(C) \prod_k P(F_k \; | \; C)}{\sum_C \scriptsize{P(C) \prod_k P(F_k \; | \; C)}}$$</span></p>
  <p>This is the general view of the problem. Specifically, to fix your issue, evaluate the probabilities for all your classes, add them together, then divide each probability by that sum.</p>
  <p>For instance, if you have two classes and the probabilities are <span class="math-container">$p_0$</span> and <span class="math-container">$p_1$</span>, then write</p>
  <p><span class="math-container">$$P(C=0) = \frac{p_0}{p_0 + p_1}$$</span></p>
  <p>and</p>
  <p><span class="math-container">$$P(C=1) = \frac{p_1}{p_0 + p_1}$$</span></p>
  <p>which guarantees that</p>
  <p><span class="math-container">$$P(C=0) + P(C=1) = 1$$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-19T10:46:27.660Z
Answers.LastActivityDate: 2020-07-19T10:46:27.660Z
Answers.UserDisplayName: someone_new
Body: <p>When applying multinomial Naive Bayes text classification, I get very small probabilities (around <span class="math-container">$10e^{-48}$</span>), so there's no way for me to know which classes are valid predictions and which ones are not. I'd the probabilities to be in the interval <span class="math-container">$[0,1]$</span>, so I can exclude classes in the prediction with say a score of 0.5 or less. How do I go about doing this?</p>
  <p>This is what I've implemented:</p>
  <p><span class="math-container">$$c_{\operatorname{map}}=\underset{\operatorname{c \in C}}{\arg \max }(P(c \mid d))=\underset{\operatorname{c \in C}}{\arg \max }\left(P(c) \prod_{1 \leq k \leq n_d} P\left(t_{k} \mid c\right)\right)$$</span></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-17T18:30:50.667Z
Comments.Text: I think your probabilities are split up among all possible outputs. If you add them all together, do they come out to 1? Order them by value and you have your predictions
Comments.UserDisplayName: Zakk Diaz
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-04-16T00:33:15.620Z
Comments.Text: Use log probabilities instead to avoid underflow when dealing with naive bayes probability. instead of a product of probabilities, compute the argmax summation of log probabilities
Comments.UserDisplayName: calveeen
Tags: <machine-learning><classification><probability><naive-bayes>
Title: Why do I get small probabilities when implementing a multinomial naive Bayes text classification model?
UserDisplayName: Yacine al
ThreadText: # Why do I get small probabilities when implementing a multinomial naive Bayes text classification model?
  
  (Asked by: username_0 on 2018-07-17)
  
  When applying multinomial Naive Bayes text classification, I get very small probabilities (around $10e^{-48}$), so there's no way for me to know which classes are valid predictions and which ones are not. I'd the probabilities to be in the interval $[0,1]$, so I can exclude classes in the prediction with say a score of 0.5 or less. How do I go about doing this?
  
  This is what I've implemented:
  
  $$c\_{\operatorname{map}}=\underset{\operatorname{c \in C}}{\arg \max }(P(c \mid d))=\underset{\operatorname{c \in C}}{\arg \max }\left(P(c) \prod\_{1 \leq k \leq n\_d} P\left(t\_{k} \mid c\right)\right)$$
  
  **Tags:** machine-learning, classification, probability, naive-bayes
  
  > **Comments:**
  > **username_1**: I think your probabilities are split up among all possible outputs. If you add them all together, do they come out to 1? Order them by value and you have your predictions  
  > **username_2**: Use log probabilities instead to avoid underflow when dealing with naive bayes probability. instead of a product of probabilities, compute the argmax summation of log probabilities  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-07-19. Score: 0
  
  You could say that the class of the data (e.g. spam vs not spam) is a hidden quality that can be inferred through the observable features (e.g. message subject contains "bitcoin"),
  
  $$P(C \; | \; F)$$
  
  which says that the probability of the class $C$ is conditioned on the visibility of the feature $F$.
  
  Using Bayes' theorem we can write
  
  $$P(C \; | \; F) = \frac{P(F \; | \; C) \, P(C)}{P(F)}$$
  
  Your problem is that you are not taking into account the **evidence**, $P(F)$, denominator in this expression.
  
  [![Bayes' theorem label factors](https://i.sstatic.net/3ujvJ.jpg)](https://i.sstatic.net/3ujvJ.jpg)
  
  You can have multiple features in your **likelihood** function
  
  $$P(F \; | \; C) = P(F\_1,F\_2,F\_3,\ldots \; | \; C)$$
  
  and these can be independent
  
  $$\prod\_k P(F\_k \; | \; C)$$
  
  and then you have your **prior**
  
  $$P(C) \prod\_k P(F\_k \; | \; C)$$
  
  but what you are missing is the **evidence** factor in the denominator
  
  $$P(C \; | \; F) = \frac{P(C) \prod\_k P(F\_k \; | \; C)}{\sum\_C \scriptsize{P(C) \prod\_k P(F\_k \; | \; C)}}$$
  
  This is the general view of the problem. Specifically, to fix your issue, evaluate the probabilities for all your classes, add them together, then divide each probability by that sum.
  
  For instance, if you have two classes and the probabilities are $p\_0$ and $p\_1$, then write
  
  $$P(C=0) = \frac{p\_0}{p\_0 + p\_1}$$
  
  and
  
  $$P(C=1) = \frac{p\_1}{p\_0 + p\_1}$$
  
  which guarantees that
  
  $$P(C=0) + P(C=1) = 1$$
  
  
  ---
  
  