Answers.Body: <p>Simply said, predicting pseudo random number is just not possible for now. Pseudo random numbers generated now have a high enough "randomness" so that it cannot be predicted. Pseudo random numbers is the basis of modern cryptography which is widely used in the world wide web and more. It may be possible in the future through faster computers and stronger AI, but for now it is not possible. If you train a model to fit on pseudo random numbers, the model will just overfit and thus creating a scenario as shown in the question. The training loss will be very low while test loss will be extremely high. The model will just "remember" the training data instead of generalising to all pseudo random numbers, thus the high test loss.</p>
  
  <p>Also, as a side note, loss is not represented by %, instead it is just a raw numeric value.</p>
  
  <p>See <a href="https://ai.stackexchange.com/q/3850/23713">this stack exchange answer</a> for details.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-06T04:46:36.040Z
Answers.Text: >"Also, as a side note, loss is not represented by %, instead it is just a raw numeric value."
  
  Thanks, but that came as a default with the software :-)
Answers.UserDisplayName: Damir Olejar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-06T02:36:21.470Z
Answers.LastActivityDate: 2020-01-06T19:26:54.270Z
Answers.LastEditDate: 2020-01-06T19:26:54.270Z
Answers.UserDisplayName: Clement
Body: <p>I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model overfits and returns very bad results. The correlation coefficient and error metrics are both not performing well.</p>
  
  <p>What would be some of the ways to solve this issue?</p>
  
  <p>For example, if I train it with 5000 rows of data and test it with 1000, I get:</p>
  
  <pre><code>Correlation coefficient                  0.0742
  Mean absolute error                      0.742 
  Root mean squared error                  0.9407
  Relative absolute error                146.2462 %
  Root relative squared error            160.1116 %
  Total Number of Instances             1000     
  </code></pre>
  
  <p>As mentioned, I can train it with as many training samples as I want and still have the model overfits. If anyone is interested, I can provide/generate some data and post it online.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:24:24.573Z
Comments.Text: Did you mention your data is pseudo-random? If that is the case the network must overfits as currently we cannot predict pseudo random numbers using neural network, or this will break all current cryptography in our society
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:26:23.280Z
Comments.Text: Also what is the output you are going to predict?
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:32:58.007Z
Comments.Text: @ClementHui excellent point and you guessed the topic in question too, the output is just a single number from -1 to 1. Regardless, I would like to learn if there are any solutions that might even slightly suggest improved results.
Comments.UserDisplayName: Damir Olejar
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:34:07.947Z
Comments.Text: So you are trying to predict pseudo random numbers
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:35:26.363Z
Comments.Text: @ClementHui lets say yes, however, I can filter the noise out... in that case any suggestion on which filter to apply (or any other method) ?
Comments.UserDisplayName: Damir Olejar
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:37:31.163Z
Comments.Text: Please refer to this https://ai.stackexchange.com/q/3850/23713
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-06T02:36:41.997Z
Comments.Text: Added an answer. Hope I can help you @Damir Olejar
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-06T02:41:04.747Z
Comments.Text: Btw is this a possible duplicate of https://ai.stackexchange.com/q/3850/23713 @nbro
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-06T04:51:37.000Z
Comments.Text: @ClementHui Not a duplicate, but for the simplicity's sake I said that it is (pseudo) random given the noisy data.
Comments.UserDisplayName: Damir Olejar
Tags: <neural-networks><training><overfitting><multilayer-perceptrons>
Title: Why does my model overfit on pseudo-random numbers training data?
UserDisplayName: Damir Olejar
ThreadText: # Why does my model overfit on pseudo-random numbers training data?
  
  (Asked by: username_0 on 2020-01-05)
  
  I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model overfits and returns very bad results. The correlation coefficient and error metrics are both not performing well.
  
  What would be some of the ways to solve this issue?
  
  For example, if I train it with 5000 rows of data and test it with 1000, I get:
  
  ```
  Correlation coefficient                  0.0742
  Mean absolute error                      0.742 
  Root mean squared error                  0.9407
  Relative absolute error                146.2462 %
  Root relative squared error            160.1116 %
  Total Number of Instances             1000     
  
  ```
  
  As mentioned, I can train it with as many training samples as I want and still have the model overfits. If anyone is interested, I can provide/generate some data and post it online.
  
  **Tags:** neural-networks, training, overfitting, multilayer-perceptrons
  
  > **Comments:**
  > **username_1**: Did you mention your data is pseudo-random? If that is the case the network must overfits as currently we cannot predict pseudo random numbers using neural network, or this will break all current cryptography in our society  
  > **username_1**: Also what is the output you are going to predict?  
  > **username_0**: @ClementHui excellent point and you guessed the topic in question too, the output is just a single number from -1 to 1. Regardless, I would like to learn if there are any solutions that might even slightly suggest improved results.  
  > **username_1**: So you are trying to predict pseudo random numbers  
  > **username_0**: @ClementHui lets say yes, however, I can filter the noise out... in that case any suggestion on which filter to apply (or any other method) ?  
  > **username_1**: Please refer to this https://ai.stackexchange.com/q/3850/23713  
  > **username_1**: Added an answer. Hope I can help you @username_0  
  > **username_1**: Btw is this a possible duplicate of https://ai.stackexchange.com/q/3850/23713 @nbro  
  > **username_0**: @ClementHui Not a duplicate, but for the simplicity's sake I said that it is (pseudo) random given the noisy data.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-06. Score: 1 | Accepted answer
  
  Simply said, predicting pseudo random number is just not possible for now. Pseudo random numbers generated now have a high enough "randomness" so that it cannot be predicted. Pseudo random numbers is the basis of modern cryptography which is widely used in the world wide web and more. It may be possible in the future through faster computers and stronger AI, but for now it is not possible. If you train a model to fit on pseudo random numbers, the model will just overfit and thus creating a scenario as shown in the question. The training loss will be very low while test loss will be extremely high. The model will just "remember" the training data instead of generalising to all pseudo random numbers, thus the high test loss.
  
  Also, as a side note, loss is not represented by %, instead it is just a raw numeric value.
  
  See [this stack exchange answer](https://ai.stackexchange.com/q/3850/23713) for details.
  
  > **Comments:**
  > **username_0**: >"Also, as a side note, loss is not represented by %, instead it is just a raw numeric value."  Thanks, but that came as a default with the software :-)  
  
  ---
  
  