Answers.Body: <p>Having <strong>NaN</strong> might be because of the <em><strong>exploding gradient</strong></em> problem. This happens when your model learns very slowly and perhaps the training stagnates at a very early stage just after a few iterations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-30T11:03:30.213Z
Answers.Text: I don't think it is because of slow learning because when I made the learning rate 0.001, it stopped giving NaN values. I think it is the opposite.
Answers.UserDisplayName: Ege
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-30T09:27:29.357Z
Answers.LastActivityDate: 2023-08-30T09:27:29.357Z
Answers.UserDisplayName: Kanan Suleyman
Body: <p>I made my own Neural Network from scratch in unity with C# and I am using it as DQN. <em><strong>I set up my network which has 4 layers: 9 input values, 20 nodes in the second layer, 15 nodes in the third layer, and 8 outputs</strong></em>. I set all the layers' <strong>activation functions as Identity</strong> (which has no impact on the value), and the loss function as <strong>MSE</strong> I scaled my inputs and rewards, so my input values are between 0-1, and targets are between -1 and 1.</p>
  <p>But after the first training, the weights become <strong>NaN</strong>. On another project I have a <strong>much simpler network which is 5-4-3-2 nodes</strong> in layers. And there it works perfectly. But when I increase the nodes for layers it immediately makes the weights NaN. If I add a <strong>coefficient factor</strong> to Identity something like <strong>0.1</strong> it doesn't give NaN but this time the 8 output values become very <em><strong>similar</strong></em> to each other.</p>
  <p>I debugged the delta values and after 6 or 7 epochs later the weights go to very large numbers like 2345221. Than after that they become NaN, I think due to large values.</p>
  <p>With other <em><strong>activation function(TanH, sigmoid)</strong></em> they don't give that NaN values but the outputs are become very <strong>close to 1</strong>(I understand that because the network has very nodes and the outputs are larger than 3 or 4 and this functions converge to 1 with larger values than 2). But this NaN thing happens with the ReLU functions too.</p>
  <p><strong>My question is:</strong> Is it normal to have this numbers or something is wrong with my code? If it is normal what can I do because the other functions also give bad results, I am trying to predict the cumulative reward here.</p>
  
Tags: <neural-networks><activation-functions>
Title: Is It Reasonable to Get NaN Outputs With Identity Activation Functions
UserDisplayName: Ege
ThreadText: # Is It Reasonable to Get NaN Outputs With Identity Activation Functions
  
  (Asked by: username_0 on 2023-08-29)
  
  I made my own Neural Network from scratch in unity with C# and I am using it as DQN. ***I set up my network which has 4 layers: 9 input values, 20 nodes in the second layer, 15 nodes in the third layer, and 8 outputs***. I set all the layers' **activation functions as Identity** (which has no impact on the value), and the loss function as **MSE** I scaled my inputs and rewards, so my input values are between 0-1, and targets are between -1 and 1.
  
  But after the first training, the weights become **NaN**. On another project I have a **much simpler network which is 5-4-3-2 nodes** in layers. And there it works perfectly. But when I increase the nodes for layers it immediately makes the weights NaN. If I add a **coefficient factor** to Identity something like **0.1** it doesn't give NaN but this time the 8 output values become very ***similar*** to each other.
  
  I debugged the delta values and after 6 or 7 epochs later the weights go to very large numbers like 2345221. Than after that they become NaN, I think due to large values.
  
  With other ***activation function(TanH, sigmoid)*** they don't give that NaN values but the outputs are become very **close to 1**(I understand that because the network has very nodes and the outputs are larger than 3 or 4 and this functions converge to 1 with larger values than 2). But this NaN thing happens with the ReLU functions too.
  
  **My question is:** Is it normal to have this numbers or something is wrong with my code? If it is normal what can I do because the other functions also give bad results, I am trying to predict the cumulative reward here.
  
  **Tags:** neural-networks, activation-functions
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-30. Score: 0
  
  Having **NaN** might be because of the ***exploding gradient*** problem. This happens when your model learns very slowly and perhaps the training stagnates at a very early stage just after a few iterations.
  
  > **Comments:**
  > **username_0**: I don't think it is because of slow learning because when I made the learning rate 0.001, it stopped giving NaN values. I think it is the opposite.  
  
  ---
  
  