Answers.Body: <p>Bucketing is usually used in industry because we are not sure what we are going to get. Some amount of observation on the distribution will give you a good idea of bucketing. But, there is caveat, if there isn't a lot of variance then bucketing won't help much.</p>
  <p>Also, in terms of losses, since different buckets will have different number of examples, class imbalance will arise. So, focal loss is the better loss here.</p>
  <p>I hope you get results you can write home about.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-02T14:48:03.813Z
Answers.Text: Hi, thanks for your input. I have thought of another approach, and would like to perhaps get your opinion. I am still running a classification based on my buckets, but I am using a KL divergence loss function. Additionally, for the labels I am "adding" noise to them -- essentially, centring a truncated normal (so that the pdf remains in my domain) around the observed label. In this sense, I am training a network to be a "soft" classifier. Does this make sense to you, and do you know if there is any existing approaches like this? The results are rather good for me.
Answers.UserDisplayName: reimama
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-03T00:58:03.213Z
Answers.Text: The approach seems a bit convoluted. You are first assuming a discrete distribution over continuous values then you are using losses suitable for distributions (it is working on the label distribution). Also, the noise part is easily there in cross entropy function (see label smoothing). This can be a experiment and may even perform. But, traditionally, the pipeline would be (continuous -> discrete -> NN with cross entropy (with label smoothing for regularization). Do tell how that worked out for you.
Answers.UserDisplayName: Abhishek Verma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-27T19:06:29.810Z
Answers.LastActivityDate: 2021-05-27T19:06:29.810Z
Answers.UserDisplayName: Abhishek Verma
Body: <p>I am considering a rather typical regression problem, but, for practice, I am trying to implement this as a classification problem.</p>
  <p>The setup is as follows. I have <span class="math-container">$\mathbb{R}$</span>-valued labels <span class="math-container">$y_i \in [-1,1]$</span>, which I then discretize to <span class="math-container">$N$</span> buckets -- my classification problem is to then predict the labels to the nearest bucket.</p>
  <p>This is rather straightforward and easy to implement with a cross-entropy loss function. However, I do not believe that this is the best option, as I would ideally like my predictions to be close to their correct bucket, even if I do not predict them correctly (which will be more difficult as if I take <span class="math-container">$N$</span> larger).</p>
  <p>My current approach involves using a mean-squared error loss function. My network outputs logits for each bucket, I apply a softargmax (so the network remains differentiable) and then convert the output of the network into the <span class="math-container">$\mathbb{R}$</span>-valued prediction.</p>
  <p>My (very premature) results are nothing to write home about. So, I ask, is there a more natural loss function that I could consider for this exercise?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-27T09:47:16.103Z
Comments.Text: What code are you using ? share what you have tried
Comments.UserDisplayName: FabioSpaghetti
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-17T18:57:03.840Z
Comments.Text: This is called ordinal regression, which could be a term to search, but [binning tends to be discouraged.](https://stats.stackexchange.com/a/68839/247274)
Comments.UserDisplayName: Dave
Tags: <machine-learning><classification><objective-functions><regression>
Title: Which loss function could I use to solve a regression problem as a classification problem (where we discretize the labels into buckets)?
UserDisplayName: reimama
ThreadText: # Which loss function could I use to solve a regression problem as a classification problem (where we discretize the labels into buckets)?
  
  (Asked by: username_0 on 2021-05-26)
  
  I am considering a rather typical regression problem, but, for practice, I am trying to implement this as a classification problem.
  
  The setup is as follows. I have $\mathbb{R}$-valued labels $y\_i \in [-1,1]$, which I then discretize to $N$ buckets -- my classification problem is to then predict the labels to the nearest bucket.
  
  This is rather straightforward and easy to implement with a cross-entropy loss function. However, I do not believe that this is the best option, as I would ideally like my predictions to be close to their correct bucket, even if I do not predict them correctly (which will be more difficult as if I take $N$ larger).
  
  My current approach involves using a mean-squared error loss function. My network outputs logits for each bucket, I apply a softargmax (so the network remains differentiable) and then convert the output of the network into the $\mathbb{R}$-valued prediction.
  
  My (very premature) results are nothing to write home about. So, I ask, is there a more natural loss function that I could consider for this exercise?
  
  **Tags:** machine-learning, classification, objective-functions, regression
  > **Comments:**
  > **username_1**: What code are you using ? share what you have tried  
  > **username_2**: This is called ordinal regression, which could be a term to search, but [binning tends to be discouraged.](https://stats.stackexchange.com/a/68839/247274)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2021-05-27. Score: 0 | Accepted answer
  
  Bucketing is usually used in industry because we are not sure what we are going to get. Some amount of observation on the distribution will give you a good idea of bucketing. But, there is caveat, if there isn't a lot of variance then bucketing won't help much.
  
  Also, in terms of losses, since different buckets will have different number of examples, class imbalance will arise. So, focal loss is the better loss here.
  
  I hope you get results you can write home about.
  
  > **Comments:**
  > **username_0**: Hi, thanks for your input. I have thought of another approach, and would like to perhaps get your opinion. I am still running a classification based on my buckets, but I am using a KL divergence loss function. Additionally, for the labels I am "adding" noise to them -- essentially, centring a truncated normal (so that the pdf remains in my domain) around the observed label. In this sense, I am training a network to be a "soft" classifier. Does this make sense to you, and do you know if there is any existing approaches like this? The results are rather good for me.  
  > **username_3**: The approach seems a bit convoluted. You are first assuming a discrete distribution over continuous values then you are using losses suitable for distributions (it is working on the label distribution). Also, the noise part is easily there in cross entropy function (see label smoothing). This can be a experiment and may even perform. But, traditionally, the pipeline would be (continuous -> discrete -> NN with cross entropy (with label smoothing for regularization). Do tell how that worked out for you.  
  
  ---
  
  