Answers.Body: <p>Usually you use cross entropy loss for classification, and you can use class probabilities as the label instead of hard labels.</p>
  <p><a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html" rel="nofollow noreferrer">https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-16T15:20:22.040Z
Answers.Text: The OP seems to have the categorical outcomes. Could you please clarify the advantages of disregarding them over using the approach described in my answer?
Answers.UserDisplayName: Dave
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T08:34:26.243Z
Answers.LastActivityDate: 2022-07-22T08:34:26.243Z
Answers.UserDisplayName: DKDK
Answers.Body: <p>use categorical cross-entropy, the model will learn the distribution by itself (so given the same input, say there are 50% samples that are 1 and the rest 0, the model will predict 50%)</p>
  <p>it’s literally the definition of crossentropy</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-16T16:56:08.967Z
Answers.LastActivityDate: 2024-10-16T16:56:08.967Z
Answers.UserDisplayName: Alberto
Answers.Body: <p>In machine learning, it is called soft-label. The output of the algorithms for classification problem is a probability too. You can use cross entropy loss with soft-labels.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-16T15:20:25.093Z
Answers.Text: The OP seems to have the categorical outcomes. Could you please clarify the advantages of disregarding them over using the approach described in my answer?
Answers.UserDisplayName: Dave
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-12T18:48:57.490Z
Answers.LastActivityDate: 2024-09-12T18:48:57.490Z
Answers.UserDisplayName: morteza
Answers.Body: <p><strong>TOUGH</strong></p>
  <p>Think about what you've written: given the same input, you have limited ability to distinguish between the two outcomes and determine which is going to occur. The best, then, you can do is be honest about the probability of each outcome, which you know is <span class="math-container">$60\%$</span> dog, <span class="math-container">$40\%$</span> cat.</p>
  <p>Remember that most models explicitly predict values on the interval <span class="math-container">$[0,1]$</span> rather than make hard classifications, and <a href="https://stats.stackexchange.com/a/638777/247274">such predictions can be interpreted as probabilities (despite some issues)</a>.</p>
  <p>If you want to have a tighter fit and more reliably predict if the outcome will be a dog or a cat, you need some additional features to distinguish the dogs from the cats. Those additional features might be easy to acquire, but they might not be, and then you are stuck with the best prediction being that a dog has a probability of <span class="math-container">$0.6$</span> while a cat has a probability of <span class="math-container">$0.4$</span>.</p>
  <p>Frank Harrell has written about working with probability predictions instead of categorical predictions (&quot;classifications&quot;), noting that in his field of biostatistics, it is totally reasonable to have two patients present the same way (same features) yet experience different outcomes. His classic blog posts are below.</p>
  <p><a href="https://www.fharrell.com/post/class-damage/" rel="nofollow noreferrer">Damage Caused by Classification Accuracy and Other Discontinuous Improper Accuracy Scoring Rules</a></p>
  <p><a href="https://www.fharrell.com/post/classification/" rel="nofollow noreferrer">Classification vs. Prediction</a></p>
  <p>The links in my profile about class imbalance and classification accuracy that get into proper scoring rules may be helpful, too.</p>
  <p>This is not a matter of using soft labels, as you have the categorical outcomes and can simply feed the features and categorical outcomes into your machine learning model as you usually would. Using soft labels wrecks your ability to use the sample sizes. One set of features giving 60:40 is not the same as another set of features giving 60000:40000, even though both are the same probability of being a dog.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-16T15:15:20.540Z
Answers.LastActivityDate: 2024-10-16T16:00:40.000Z
Answers.LastEditDate: 2024-10-16T16:00:40.000Z
Answers.UserDisplayName: Dave
Body: <p>In the usual classification problems, the label for the <strong>same input</strong> is usually the same. For example, if I have an image of a dog, then the true label for that exact input is dog every time.</p>
  <p>However, for my dataset, the label for the same (or very similar) input is probabilistic, i.e. if I have the <strong>same input</strong> (or very similar inputs), the label follow a statistical distribution, e.g. 60% of the time it is a dog, 40% of the time it is a cat</p>
  <p>How should I solve such a machine learning task?</p>
  <p>Example: given a sequence of stock prices <code>p[t-T:t]</code>, if you buy at time <code>t</code>, the chance that you will make 2% profit in 1 hour is 60%, and 40% otherwise.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-18T16:38:18.287Z
Comments.Text: Why do you have the distribution as 60/40? Is it because you observed those conditions 100 times and observed 60 instances of one category and 40 of the other?
Comments.UserDisplayName: Dave
Tags: <machine-learning><classification>
Title: How can I solve a classification problem where the label for the same input is probabilistic?
UserDisplayName: em1971
ThreadText: # How can I solve a classification problem where the label for the same input is probabilistic?
  
  (Asked by: username_0 on 2022-07-21)
  
  In the usual classification problems, the label for the **same input** is usually the same. For example, if I have an image of a dog, then the true label for that exact input is dog every time.
  
  However, for my dataset, the label for the same (or very similar) input is probabilistic, i.e. if I have the **same input** (or very similar inputs), the label follow a statistical distribution, e.g. 60% of the time it is a dog, 40% of the time it is a cat
  
  How should I solve such a machine learning task?
  
  Example: given a sequence of stock prices `p[t-T:t]`, if you buy at time `t`, the chance that you will make 2% profit in 1 hour is 60%, and 40% otherwise.
  
  **Tags:** machine-learning, classification
  
  > **Comments:**
  > **username_1**: Why do you have the distribution as 60/40? Is it because you observed those conditions 100 times and observed 60 instances of one category and 40 of the other?  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_2 on 2022-07-22. Score: 0
  
  Usually you use cross entropy loss for classification, and you can use class probabilities as the label instead of hard labels.
  
  <https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html>
  
  > **Comments:**
  > **username_1**: The OP seems to have the categorical outcomes. Could you please clarify the advantages of disregarding them over using the approach described in my answer?  
  
  ---
  
  ### Answer by username_3 on 2024-10-16. Score: 0
  
  use categorical cross-entropy, the model will learn the distribution by itself (so given the same input, say there are 50% samples that are 1 and the rest 0, the model will predict 50%)
  
  it’s literally the definition of crossentropy
  
  
  ---
  
  ### Answer by username_4 on 2024-09-12. Score: 0
  
  In machine learning, it is called soft-label. The output of the algorithms for classification problem is a probability too. You can use cross entropy loss with soft-labels.
  
  > **Comments:**
  > **username_1**: The OP seems to have the categorical outcomes. Could you please clarify the advantages of disregarding them over using the approach described in my answer?  
  
  ---
  
  ### Answer by username_1 on 2024-10-16. Score: 0
  
  **TOUGH**
  
  Think about what you've written: given the same input, you have limited ability to distinguish between the two outcomes and determine which is going to occur. The best, then, you can do is be honest about the probability of each outcome, which you know is $60\%$ dog, $40\%$ cat.
  
  Remember that most models explicitly predict values on the interval $[0,1]$ rather than make hard classifications, and [such predictions can be interpreted as probabilities (despite some issues)](https://stats.stackexchange.com/a/638777/247274).
  
  If you want to have a tighter fit and more reliably predict if the outcome will be a dog or a cat, you need some additional features to distinguish the dogs from the cats. Those additional features might be easy to acquire, but they might not be, and then you are stuck with the best prediction being that a dog has a probability of $0.6$ while a cat has a probability of $0.4$.
  
  Frank Harrell has written about working with probability predictions instead of categorical predictions ("classifications"), noting that in his field of biostatistics, it is totally reasonable to have two patients present the same way (same features) yet experience different outcomes. His classic blog posts are below.
  
  [Damage Caused by Classification Accuracy and Other Discontinuous Improper Accuracy Scoring Rules](https://www.fharrell.com/post/class-damage/)
  
  [Classification vs. Prediction](https://www.fharrell.com/post/classification/)
  
  The links in my profile about class imbalance and classification accuracy that get into proper scoring rules may be helpful, too.
  
  This is not a matter of using soft labels, as you have the categorical outcomes and can simply feed the features and categorical outcomes into your machine learning model as you usually would. Using soft labels wrecks your ability to use the sample sizes. One set of features giving 60:40 is not the same as another set of features giving 60000:40000, even though both are the same probability of being a dog.
  
  
  ---
  
  