Answers.Body: <p>You're loss is converging fast simply cause you're totally overfitting on you're toy input samples, which is good, at least it proves that your training script works.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-04T10:47:46.137Z
Answers.Text: Thank you. I was using the toy samples for that purposes, indeed.
Answers.UserDisplayName: tail
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-04T07:31:00.900Z
Answers.LastActivityDate: 2022-10-04T07:45:29.073Z
Answers.LastEditDate: 2022-10-04T07:45:29.073Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>I am implementing an ANN whose training loss is in Figure:</p>
  <p><a href="https://i.sstatic.net/VIgHL.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/VIgHL.png" alt="training loss" /></a></p>
  <p>As you can see training loss decreases very fast and it is approximately 3.2 at epochs 2, 3, ..., 8, ... 10, and so on. (batch learning)</p>
  <p>The diagram is similar even though I increases MAX_EPOCHS to 2000.</p>
  <p>Is it correct?</p>
  <p>Note that my training set is a toy:</p>
  <pre><code>X = np.asmatrix([
          [2, 5, 3.2, 7.2],
          [1, 0.8, 1, 8.5],
          [2, 7, 3, 1.4]
      ])
  
  targets = np.asarray([0,2,1])
  targets = np.asmatrix(np.eye(np.max(targets) + 1)[targets])  # 1-hot
  </code></pre>
  <p>And I am using Gradient Descent with momentum</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-03T21:18:38.517Z
Comments.Text: What is the architecture  of your ANN? The dataset seems extremely small (with only 3 samples) so it is difficult to explain the behaviour of the ANN. It is possible that with the given architecture and this dataset, the loss cannot be reduced beyond 3.2.
Comments.UserDisplayName: Ganesh Tata
Tags: <neural-networks><machine-learning><training><loss>
Title: Training loss decreases very fast after few epochs
UserDisplayName: tail
ThreadText: # Training loss decreases very fast after few epochs
  
  (Asked by: username_0 on 2022-10-03)
  
  I am implementing an ANN whose training loss is in Figure:
  
  As you can see training loss decreases very fast and it is approximately 3.2 at epochs 2, 3, ..., 8, ... 10, and so on. (batch learning)
  The diagram is similar even though I increases MAX_EPOCHS to 2000.
  Is it correct?
  Note that my training set is a toy:
  X = np.asmatrix([
          [2, 5, 3.2, 7.2],
          [1, 0.8, 1, 8.5],
          [2, 7, 3, 1.4]
      ])
  
  targets = np.asarray([0,2,1])
  targets = np.asmatrix(np.eye(np.max(targets) + 1)[targets])  # 1-hot
  
  And I am using Gradient Descent with momentum
  
  **Tags:** neural-networks, machine-learning, training, loss
  > **Comments:**
  > **username_1**: What is the architecture  of your ANN? The dataset seems extremely small (with only 3 samples) so it is difficult to explain the behaviour of the ANN. It is possible that with the given architecture and this dataset, the loss cannot be reduced beyond 3.2.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-10-04. Score: 1 | Accepted answer
  
  You're loss is converging fast simply cause you're totally overfitting on you're toy input samples, which is good, at least it proves that your training script works.
  
  > **Comments:**
  > **username_0**: Thank you. I was using the toy samples for that purposes, indeed.  
  
  ---
  
  