Answers.Body: <p>This really depends on your data. MSE and its variant, the RMSE, are good for regression problems. In other words, when you want to produce a real number as an output, for example, in a time series forecasting situation. The MAPE is good and can be interpreted as a percentage, but it does not work well if you have zeros in your dataset. As the other answer mentions, BLEU is good too, but only if your model is doing machine translation, or working with some other type of categorical data; however, you would not use BLEU if you are trying to predict house prices, for example.</p>
  <p>To find the best evaluation metric you have to consider your data and the problem you are trying to solve. Its good to go over a list of potential loss functions and see what fits your problem best.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-03T20:16:23.327Z
Answers.LastActivityDate: 2021-05-03T20:16:23.327Z
Answers.UserDisplayName: Marcus
Answers.Body: <p>I would recommend taking a look at Bilingual Evaluation Understudy(BELU) score which is commonly used in evaluating machine translation results by sequence to sequence model. Here is the reference <a href="https://en.wikipedia.org/wiki/BLEU" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/BLEU</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-04T03:59:49.720Z
Answers.LastActivityDate: 2020-12-04T03:59:49.720Z
Answers.UserDisplayName: Srivatsan Ramesh
Answers.Body: <p>Also consider the soft-dtw  loss <a href="https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.soft_dtw.html#r26a17581b9fd-1" rel="nofollow noreferrer">https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.soft_dtw.html#r26a17581b9fd-1</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-19T09:29:32.260Z
Answers.LastActivityDate: 2024-01-19T09:29:32.260Z
Answers.UserDisplayName: Ggjj11
Body: <p>I am solving many sequence-to-sequence prediction problems using RNN/LSTM. </p>
  
  <p>What type of evaluation metrics can be used for sequence prediction problems?</p>
  
  <p>One metric is the mean squared error (MSE) that we can give as a parameter during the training model. Currently, the accuracy of my sequence-to-sequence problems is very low. </p>
  
  <p>What are other ways through which we can compare the performance of our models?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-14T07:37:14.300Z
Comments.Text: It depends on what type of data you're having and what kind of results you're interested in. For example you could use root mean square error (RMSE) which will greatly penalize large errors. If you want your results to be easily interpreted you could use mean absolute percentage error.
Comments.UserDisplayName: razvanc92
Tags: <recurrent-neural-networks><long-short-term-memory><sequence-modeling><metric>
Title: What evaluation metric are used for sequence-to-sequence prediction problems?
UserDisplayName: Asif Khan
ThreadText: # What evaluation metric are used for sequence-to-sequence prediction problems?
  
  (Asked by: username_0 on 2019-11-14)
  
  I am solving many sequence-to-sequence prediction problems using RNN/LSTM.
  
  What type of evaluation metrics can be used for sequence prediction problems?
  
  One metric is the mean squared error (MSE) that we can give as a parameter during the training model. Currently, the accuracy of my sequence-to-sequence problems is very low.
  
  What are other ways through which we can compare the performance of our models?
  
  **Tags:** recurrent-neural-networks, long-short-term-memory, sequence-modeling, metric
  > **Comments:**
  > **username_1**: It depends on what type of data you're having and what kind of results you're interested in. For example you could use root mean square error (RMSE) which will greatly penalize large errors. If you want your results to be easily interpreted you could use mean absolute percentage error.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2021-05-03. Score: 1
  
  This really depends on your data. MSE and its variant, the RMSE, are good for regression problems. In other words, when you want to produce a real number as an output, for example, in a time series forecasting situation. The MAPE is good and can be interpreted as a percentage, but it does not work well if you have zeros in your dataset. As the other answer mentions, BLEU is good too, but only if your model is doing machine translation, or working with some other type of categorical data; however, you would not use BLEU if you are trying to predict house prices, for example.
  
  To find the best evaluation metric you have to consider your data and the problem you are trying to solve. Its good to go over a list of potential loss functions and see what fits your problem best.
  
  
  ---
  
  ### Answer by username_3 on 2020-12-04. Score: 0
  
  I would recommend taking a look at Bilingual Evaluation Understudy(BELU) score which is commonly used in evaluating machine translation results by sequence to sequence model. Here is the reference <https://en.wikipedia.org/wiki/BLEU>
  
  
  ---
  
  ### Answer by username_4 on 2024-01-19. Score: 0
  
  Also consider the soft-dtw loss <https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.soft_dtw.html#r26a17581b9fd-1>
  
  
  ---
  
  