Answers.Body: <blockquote>
  <p>but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes</p>
  </blockquote>
  <p>I don't agree with this statement. I already trained many CNNs for regressions tasks where a continous output is trained and they generally perform very well.</p>
  <p>I think the general &quot;advantage&quot; for a classification approach over a regression approach is that there is usually some margin where the output of the NN can still be clipped to a specific class, which might not be the case in regression tasks. Furthermore a lot of the &quot;better classification performance&quot; is due to the fact that the cross-entropy loss of classification tasks always tries to increase probability of one class while reducing all others. But this advantage might not be really important in your case where a specific range of days (e.g 3-5 days) can be probable instead of just one single day (e.g. exactly day 4).</p>
  <p>Thats why I think you should defenitly try the regression approach. For this just normalize/clip your output layer to the min/max range of days. You should also defenitley decrease the learning rate at the end of training, I found this much further improves regression predictions to be more accurate.</p>
  <p>And lastly if you want to achieve higher performance and more robustness, in your case I highly recommend a bayesian approach where you average over an ensemble of NNs (this ensemble can be your single NN with different parts truned off via Dropout layers during inferencing). With this average you can get an estimate of the uncertainty (the variance over different predictions) which gives you a much better idea of the probability distribution over the different days for a specifiy input.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-21T17:49:30.640Z
Answers.Text: Thanks, this is very interesting. I'll wait a bit to see if there are other answers and if not I'll award you the bounty. I'll try to read on the bayesian approach, I don't know much about it (if you have any reference I'd be happy to see it).
Answers.UserDisplayName: Antoine Labelle
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-21T11:28:29.083Z
Answers.LastActivityDate: 2021-01-21T11:28:29.083Z
Answers.UserDisplayName: Marcel_marcel1991
Body: <p>I want to train some IA algorithm to be able to evaluate the maturity of a fruit (say, measured in numbers of days before rotten) based on an image of the fruit.
  My first instinct is to go with convolutional neural network (CNN), since those have proven very efficient for recognizing images. However, I am not sure what the output layer should look like in this case.</p>
  <p>I could separate the data into a bunch of classes (1 day left, 2 days left, 3 days left, etc.) and use one output node for each of these classes, as in an usual classification task, but in doing so I completely lose the continuous nature of the output, which makes me think it might not be the optimal way to proceed.</p>
  <p>Another option would be to just have a unique output node, whose activation would correspond to the continuous value to predict, here the number of days left (normalized appropriately to lie between 0 and 1). This would have the advantage of taking the continuity into account, but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes.</p>
  <p>What do you think would be the best way to proceed? Is there another way to nudge a neural network so that its output is continuous? Or maybe CNN just aren't suited for this task? If you have any suggestions of other algorithms that would be efficient for this kind of task, I would be happy to know them.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-21T01:19:40.270Z
Comments.Text: Just to make sure. Do you actually have continuous-valued labels corresponding to the input training data? Also, I am not sure what would per-se be bad about generating a probability distribution over a discrete set of classes represented by the output nodes. As long as you set granularity of the discrete classes to an appropriate level and train the classifier in that way, you would achieve your goal as well. Or is there a reason that you really need continuous outputs (except for subjectively preferring it)?
Comments.UserDisplayName: Daniel B.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-21T02:10:47.480Z
Comments.Text: My labels are integers corresponding to the number of days. I just feel like we're losing some information by representing the maturity as a discrete unordered set of classes, I feel like somehow the NN should know that if the answer is 2 days then predicting 3 days is a much better prediction than 10 days. But maybe I am wrong and the NN will just learn this by itself (I don't have any specific reason for preferring continuous output beside that).
Comments.UserDisplayName: Antoine Labelle
Tags: <convolutional-neural-networks><image-recognition><continuous-tasks>
Title: Predicting continous value with CNN (prediction of fruit maturity)
UserDisplayName: Antoine Labelle
ThreadText: # Predicting continous value with CNN (prediction of fruit maturity)
  
  (Asked by: username_0 on 2021-01-18)
  
  I want to train some IA algorithm to be able to evaluate the maturity of a fruit (say, measured in numbers of days before rotten) based on an image of the fruit.
  My first instinct is to go with convolutional neural network (CNN), since those have proven very efficient for recognizing images. However, I am not sure what the output layer should look like in this case.
  
  I could separate the data into a bunch of classes (1 day left, 2 days left, 3 days left, etc.) and use one output node for each of these classes, as in an usual classification task, but in doing so I completely lose the continuous nature of the output, which makes me think it might not be the optimal way to proceed.
  
  Another option would be to just have a unique output node, whose activation would correspond to the continuous value to predict, here the number of days left (normalized appropriately to lie between 0 and 1). This would have the advantage of taking the continuity into account, but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes.
  
  What do you think would be the best way to proceed? Is there another way to nudge a neural network so that its output is continuous? Or maybe CNN just aren't suited for this task? If you have any suggestions of other algorithms that would be efficient for this kind of task, I would be happy to know them.
  
  **Tags:** convolutional-neural-networks, image-recognition, continuous-tasks
  
  > **Comments:**
  > **username_1**: Just to make sure. Do you actually have continuous-valued labels corresponding to the input training data? Also, I am not sure what would per-se be bad about generating a probability distribution over a discrete set of classes represented by the output nodes. As long as you set granularity of the discrete classes to an appropriate level and train the classifier in that way, you would achieve your goal as well. Or is there a reason that you really need continuous outputs (except for subjectively preferring it)?  
  > **username_0**: My labels are integers corresponding to the number of days. I just feel like we're losing some information by representing the maturity as a discrete unordered set of classes, I feel like somehow the NN should know that if the answer is 2 days then predicting 3 days is a much better prediction than 10 days. But maybe I am wrong and the NN will just learn this by itself (I don't have any specific reason for preferring continuous output beside that).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-01-21. Score: 1
  
  > but I have been told that neural networks aren't made to predict values in that way, they really are best suited for classification into discrete classes
  
  I don't agree with this statement. I already trained many CNNs for regressions tasks where a continous output is trained and they generally perform very well.
  
  I think the general "advantage" for a classification approach over a regression approach is that there is usually some margin where the output of the NN can still be clipped to a specific class, which might not be the case in regression tasks. Furthermore a lot of the "better classification performance" is due to the fact that the cross-entropy loss of classification tasks always tries to increase probability of one class while reducing all others. But this advantage might not be really important in your case where a specific range of days (e.g 3-5 days) can be probable instead of just one single day (e.g. exactly day 4).
  
  Thats why I think you should defenitly try the regression approach. For this just normalize/clip your output layer to the min/max range of days. You should also defenitley decrease the learning rate at the end of training, I found this much further improves regression predictions to be more accurate.
  
  And lastly if you want to achieve higher performance and more robustness, in your case I highly recommend a bayesian approach where you average over an ensemble of NNs (this ensemble can be your single NN with different parts truned off via Dropout layers during inferencing). With this average you can get an estimate of the uncertainty (the variance over different predictions) which gives you a much better idea of the probability distribution over the different days for a specifiy input.
  
  > **Comments:**
  > **username_0**: Thanks, this is very interesting. I'll wait a bit to see if there are other answers and if not I'll award you the bounty. I'll try to read on the bayesian approach, I don't know much about it (if you have any reference I'd be happy to see it).  
  
  ---
  
  