Answers.Body: <p>There are a few potential explanations for this. One possibility is that your model is overfitting to the training data. This means that it is memorizing the training data instead of learning to generalize from it. Another possibility is that your error function is not well-suited to the task at hand. This could be due to the fact that it is not a convex function, or it could be because it is not differentiable. Finally, it is also possible that your learning rate is too high, causing the model to diverge instead of converge.</p>
  <p>Yes, the error function you are using is known as the Frobenius Norm. It is a measure of the distance between two matrices. In this case, it is measuring the distance between the original features and the reconstructed features. The fact that the error is staying huge at the last stages could be due to overfitting or a poor learning rate, as mentioned before.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-16T22:23:05.833Z
Answers.LastActivityDate: 2022-10-16T22:23:05.833Z
Answers.UserDisplayName: Faizy
Body: <p>I am training an auto-encoder over <span class="math-container">$10^4$</span> epochs. I get a converging learning curve. However the error at the last stages stays huge <span class="math-container">$\sim10^{15}$</span>. What does this mean? does it mean that my auto-encoder isn't trained properly? Is it a question of the error function?</p>
  <p>The error function I am using is: <span class="math-container">$\mathcal{L} = \|X-\hat X\|_F^2$</span> where <span class="math-container">$X\in\mathbb{R}^{n\times d}$</span> is the original features (with <span class="math-container">$n$</span> the number of rows and <span class="math-container">$d$</span> the number of features), <span class="math-container">$\hat X\in\mathbb{R}^{n\times d}$</span> is the reconstructed features (by the auto-encoder) and <span class="math-container">$\|.\|_F^2$</span> refers to the Frobenius Norm.</p>
  <p><a href="https://i.sstatic.net/aBDzD.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/aBDzD.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-13T02:26:37.873Z
Comments.Text: It could mean anything. Start tweaking parts of the network and see the effect that has on the learning curve.
Comments.UserDisplayName: Recessive
Tags: <machine-learning><deep-learning><objective-functions><unsupervised-learning>
Title: Learning curve converges with huge errors
UserDisplayName: devCharaf
ThreadText: # Learning curve converges with huge errors
  
  (Asked by: username_0 on 2022-10-12)
  
  I am training an auto-encoder over $10^4$ epochs. I get a converging learning curve. However the error at the last stages stays huge $\sim10^{15}$. What does this mean? does it mean that my auto-encoder isn't trained properly? Is it a question of the error function?
  
  The error function I am using is: $\mathcal{L} = \|X-\hat X\|\_F^2$ where $X\in\mathbb{R}^{n\times d}$ is the original features (with $n$ the number of rows and $d$ the number of features), $\hat X\in\mathbb{R}^{n\times d}$ is the reconstructed features (by the auto-encoder) and $\|.\|\_F^2$ refers to the Frobenius Norm.
  
  [![enter image description here](https://i.sstatic.net/aBDzD.png)](https://i.sstatic.net/aBDzD.png)
  
  **Tags:** machine-learning, deep-learning, objective-functions, unsupervised-learning
  > **Comments:**
  > **username_1**: It could mean anything. Start tweaking parts of the network and see the effect that has on the learning curve.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-10-16. Score: 1
  
  There are a few potential explanations for this. One possibility is that your model is overfitting to the training data. This means that it is memorizing the training data instead of learning to generalize from it. Another possibility is that your error function is not well-suited to the task at hand. This could be due to the fact that it is not a convex function, or it could be because it is not differentiable. Finally, it is also possible that your learning rate is too high, causing the model to diverge instead of converge.
  
  Yes, the error function you are using is known as the Frobenius Norm. It is a measure of the distance between two matrices. In this case, it is measuring the distance between the original features and the reconstructed features. The fact that the error is staying huge at the last stages could be due to overfitting or a poor learning rate, as mentioned before.
  
  
  ---
  
  