Answers.Body: <p>I've gotten no answer on this, but after some reflection <strong>I've come to accept R^2 as regression's analogue to accuracy.</strong> I have no idea why it is not used by more people in deep learning, but I recently started using it and it's exactly what I was hoping for! It scales from 0 to 1 and tells you in no uncertain terms how useful the model is and when it becomes unrealistic to improve the model further (e.g. at R^2=0.99). Additionally, R^2=0 when the model starts guessing only the mean of the data, which means it immediately reveals an important and recurring issue in regression which can otherwise go undetected for a while.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-03T20:05:44.930Z
Answers.LastActivityDate: 2021-11-03T20:05:44.930Z
Answers.UserDisplayName: profPlum
Body: <p>I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand.</p>
  <p>Whereas regression models seem to have a very large number of possible evaluation strategies and to me at least it is not clear which (if any) of them is as reliable/interpretable as accuracy is in classification problems.</p>
  <p>Possible Candidates:</p>
  <ul>
  <li>Regular loss (e.g MAE): MAE is potentially quite interpretable, but again interpretation depends upon distribution statistics which vary across regression problems.</li>
  <li><a href="https://en.wikipedia.org/wiki/Mean_absolute_percentage_error" rel="nofollow noreferrer">MAPE/Relative Loss</a>: This is interesting and is potentially decently similar to accuracy. Yet it has obvious draw backs, like the true value being extremely small causing explosion of loss values &amp; there being no incorporation of overall distribution statistics for the output values.</li>
  <li><a href="https://en.wikipedia.org/wiki/Chi-squared_test" rel="nofollow noreferrer">Chi-squared test</a>: I like the idea of this but I have not seen it used at all for NN regression for some reason. I'm not sure why and I'm curious if people think it would be a good idea to use it for that.</li>
  <li><a href="https://en.wikipedia.org/wiki/Coefficient_of_determination" rel="nofollow noreferrer">(adjusted) R^2 coefficient</a>: Another statistic that seems great in theory, but again I see almost never being used for NNs and I'm not sure why. This has the great advantage of being a 'bounded'/'normalized' metric like accuracy and in theory is should be just as interpretable. Why is it not used for NNs?</li>
  </ul>
  
Tags: <neural-networks><regression>
Title: Best way to measure regression accuracy?
UserDisplayName: profPlum
ThreadText: # Best way to measure regression accuracy?
  
  (Asked by: username_0 on 2021-10-10)
  
  I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand.
  
  Whereas regression models seem to have a very large number of possible evaluation strategies and to me at least it is not clear which (if any) of them is as reliable/interpretable as accuracy is in classification problems.
  
  Possible Candidates:
  
  * Regular loss (e.g MAE): MAE is potentially quite interpretable, but again interpretation depends upon distribution statistics which vary across regression problems.
  * [MAPE/Relative Loss](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error): This is interesting and is potentially decently similar to accuracy. Yet it has obvious draw backs, like the true value being extremely small causing explosion of loss values & there being no incorporation of overall distribution statistics for the output values.
  * [Chi-squared test](https://en.wikipedia.org/wiki/Chi-squared_test): I like the idea of this but I have not seen it used at all for NN regression for some reason. I'm not sure why and I'm curious if people think it would be a good idea to use it for that.
  * [(adjusted) R^2 coefficient](https://en.wikipedia.org/wiki/Coefficient_of_determination): Another statistic that seems great in theory, but again I see almost never being used for NNs and I'm not sure why. This has the great advantage of being a 'bounded'/'normalized' metric like accuracy and in theory is should be just as interpretable. Why is it not used for NNs?
  
  **Tags:** neural-networks, regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-11-03. Score: 0 | Accepted answer
  
  I've gotten no answer on this, but after some reflection **I've come to accept R^2 as regression's analogue to accuracy.** I have no idea why it is not used by more people in deep learning, but I recently started using it and it's exactly what I was hoping for! It scales from 0 to 1 and tells you in no uncertain terms how useful the model is and when it becomes unrealistic to improve the model further (e.g. at R^2=0.99). Additionally, R^2=0 when the model starts guessing only the mean of the data, which means it immediately reveals an important and recurring issue in regression which can otherwise go undetected for a while.
  
  
  ---
  
  