Answers.Body: <p>I think you are confused, here are some facts about overfitting:</p>
  <ul>
  <li>Overfitting is only evaluated on the loss used to train the model.</li>
  <li>Overfitting depends on the difference between training loss and validation/test loss, not on a specific threshold, you should evaluate a plot of train/val loss across epochs during training.</li>
  <li>Overfitting is not detectable on metrics. Only on the loss.</li>
  </ul>
  <p>So you cannot use any of the metrics you mentioned to detect overfitting, unless they are actually the loss used to train the model. Also a threshold for RMSE does not make sense to detect overfitting, it simply does not work that way.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T22:21:29.827Z
Answers.Text: I use loss function, and the both validation and training loss have almost the same values: ```Epoch 1/100
  1640/1640 - 24s - loss: 0.0150 - val_loss: 7.6152e-04 - 24s/epoch - 2ms/step
  Epoch 2/100
  1640/1640 - 22s - loss: 2.6332e-04 - val_loss: 6.1577e-04 - 22s/epoch - 2ms/step``` So, because it confused me I am trying different methods of identifying the overfitting...
Answers.UserDisplayName: jim
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T23:52:55.733Z
Answers.Text: @just_learning These results do not show overfitting, so nothing to worry.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T22:13:09.973Z
Answers.LastActivityDate: 2023-01-11T22:13:09.973Z
Answers.UserDisplayName: Dr. Snoopy
Body: <p>How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely see some thresholds in RMSE, for example, that above x value it means that I have overfitting...</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-11T20:13:16.973Z
Comments.Text: What do you want to train with your LSTM-RNN?
Comments.UserDisplayName: Cloud Cho
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-11T20:14:04.893Z
Comments.Text: Time-series forecasting on temperature...
Comments.UserDisplayName: jim
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-11T20:28:50.490Z
Comments.Text: I see. Do you use other data other than temperature for the training such as location, date, time, and etc?
Comments.UserDisplayName: Cloud Cho
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-11T20:29:18.937Z
Comments.Text: No, only temperature...
Comments.UserDisplayName: jim
Tags: <long-short-term-memory>
Title: How to identify overfitting in LSTM-RNN using metrics?
UserDisplayName: jim
ThreadText: # How to identify overfitting in LSTM-RNN using metrics?
  
  (Asked by: username_0 on 2023-01-11)
  
  How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely see some thresholds in RMSE, for example, that above x value it means that I have overfitting...
  
  **Tags:** long-short-term-memory
  
  > **Comments:**
  > **username_1**: What do you want to train with your LSTM-RNN?  
  > **username_0**: Time-series forecasting on temperature...  
  > **username_1**: I see. Do you use other data other than temperature for the training such as location, date, time, and etc?  
  > **username_0**: No, only temperature...  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-11. Score: 2 | Accepted answer
  
  I think you are confused, here are some facts about overfitting:
  
  * Overfitting is only evaluated on the loss used to train the model.
  * Overfitting depends on the difference between training loss and validation/test loss, not on a specific threshold, you should evaluate a plot of train/val loss across epochs during training.
  * Overfitting is not detectable on metrics. Only on the loss.
  
  So you cannot use any of the metrics you mentioned to detect overfitting, unless they are actually the loss used to train the model. Also a threshold for RMSE does not make sense to detect overfitting, it simply does not work that way.
  
  > **Comments:**
  > **username_0**: I use loss function, and the both validation and training loss have almost the same values: ```Epoch 1/100 1640/1640 - 24s - loss: 0.0150 - val_loss: 7.6152e-04 - 24s/epoch - 2ms/step Epoch 2/100 1640/1640 - 22s - loss: 2.6332e-04 - val_loss: 6.1577e-04 - 22s/epoch - 2ms/step``` So, because it confused me I am trying different methods of identifying the overfitting...  
  > **username_2**: @just_learning These results do not show overfitting, so nothing to worry.  
  
  ---
  
  