Body: <p><strong>tldr</strong>; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which produces two outcomes (<code>p</code> and <code>v</code>) always predicts exactly <code>0</code> for <code>v</code> and I would like to change that.</p>
  
  <p>In the following post I will provide all code necessary to reproduce the problem.<br>
  I am training a neural network on the same input. The wanted outcome for a value "<code>v</code>" is <code>1</code>. If I create the network and train it, sometimes the predicted outcome will be <code>1</code>, sometimes it will be <code>-</code>1.<br>
   Also, the <code>loss</code> seems to flip between <code>0</code> and <code>4</code> during training epochs.<br>
   Additionally, the loss blows up immensly, even though both losses for the outcome layers are close to zero.<br>
   I do not understand where this behaviour comes from. I used <code>Leaky-ReLU</code> to make sure it can handle negative input, I used a high <code>learning rate</code> to make sure the data in this example is sufficient on the training, and the input is the same all the time.</p>
  
  <p>My Neural network looks like this:</p>
  
  <pre><code>input_layer = keras.Input(shape=(6,7),)    
  formatted_input_layer = keras.layers.Reshape((6,7, 1))(input_layer)       
  conv_layer1 = self.create_conv_layer(formatted_input_layer)
  res_layer1 = self.create_res_layer(conv_layer1)
  res_layer2 = self.create_res_layer(res_layer1)
  res_layer3 = self.create_res_layer(res_layer2)
  res_layer4 = self.create_res_layer(res_layer3)
  policy_head = self.create_policy_head(res_layer4)
  value_head = self.create_value_head(res_layer4)
  model = keras.Model(inputs=input_layer, outputs=[policy_head, value_head])
  optimizer = keras.optimizers.SGD(lr=args['lr'],momentum=args['momentum'])
  model.compile(loss = {'policy_head' : 'categorical_crossentropy', 'value_head' : 'mean_squared_error'}, optimizer=optimizer, loss_weights={'policy_head':0.5, 'value_head':0.5})
  </code></pre>
  
  <p>Methods for the different layers:<br>
  conv_layer:</p>
  
  <pre><code>def create_conv_layer(self, input_layer):
      conv_layer = keras.layers.Conv2D(filters=256,
                                       kernel_size=3,
                                       strides=1,
                                       padding='same',
                                       use_bias=False,
                                       data_format="channels_last",
                                       activation = "linear",
                                       kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
      conv_layer= keras.layers.BatchNormalization(axis=-1)(conv_layer)
      conv_layer = keras.layers.LeakyReLU()(conv_layer)
      return conv_layer
  </code></pre>
  
  <p>res_layer:  </p>
  
  <pre><code>def create_res_layer(self, input_layer):
          conv_layer = self.create_conv_layer(input_layer)
          res_layer = keras.layers.Conv2D(filters=256,
                                           kernel_size=3,
                                           strides=1,
                                           padding='same',
                                           use_bias=False,
                                           data_format="channels_last",
                                           activation = "linear",
                                           kernel_regularizer = keras.regularizers.l2(0.0001))(conv_layer)
          res_layer= keras.layers.BatchNormalization(axis=-1)(res_layer)
          res_layer = keras.layers.add([input_layer, res_layer])
          res_layer = keras.layers.LeakyReLU()(res_layer)
          return res_layer
  </code></pre>
  
  <p>policy head:  </p>
  
  <pre><code>def create_policy_head(self, input_layer):
          policy_head = keras.layers.Conv2D(filters=2,
                                            kernel_size=1,
                                            strides=1,
                                            padding='same',
                                            use_bias = False,
                                            data_format='channels_last',
                                            activation='linear',
                                            kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
          policy_head = keras.layers.BatchNormalization(axis=-1)(policy_head)
          policy_head = keras.layers.LeakyReLU()(policy_head)
          policy_head = keras.layers.Flatten()(policy_head)
          policy_head = keras.layers.Dense(units = 7,
                                           use_bias = False,
                                           activation = 'softmax',
                                           kernel_regularizer = keras.regularizers.l2(0.0001),
                                           name = "policy_head"
                                           )(policy_head)
          return policy_head
  </code></pre>
  
  <p>value head:  </p>
  
  <pre><code>def create_value_head(self, input_layer):
          value_head = keras.layers.Conv2D(filters=1,
                                            kernel_size=1,
                                            strides=1,
                                            padding='same',
                                            use_bias = False,
                                            data_format='channels_last',
                                            activation='linear',
                                            kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
          value_head = keras.layers.BatchNormalization(axis=-1)(value_head)
          value_head = keras.layers.LeakyReLU()(value_head)  
          value_head = keras.layers.Flatten()(value_head)        
          value_head = keras.layers.Dense(units = 21,
                                           use_bias = False,
                                           activation = 'linear',
                                           kernel_regularizer = keras.regularizers.l2(0.0001)
                                           )(value_head)
          value_head = keras.layers.LeakyReLU()(value_head)      
          value_head = keras.layers.Dense(units = 1,
                                           use_bias = False,
                                           activation = 'tanh',
                                           kernel_regularizer = keras.regularizers.l2(0.0001),
                                           name = "value_head"                                     
                                           )(value_head)
          return value_head
  
                                  )(value_head)
  </code></pre>
  
  <p>The way I am testing my NN:</p>
  
  <pre><code>    canonicalBoard = np.zeros(shape = (6,7), dtype=int) 
  
  
      Pi = [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0] 
  
      trainExamples = [[canonicalBoard, Pi, 1]]*50        
  
      nnetwrapper.train(trainExamples)
  
      board = canonicalBoard[np.newaxis, :, :]
  
      p, v = nnetwrapper.nnet.model.predict(board)
  </code></pre>
  
  <p>which results in the training looking like this:</p>
  
  <pre><code>Epoch 1/10
  50/50 [==============================] - 4s 71ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000
  Epoch 2/10
  50/50 [==============================] - 1s 15ms/step - loss: 2.3218 - policy_head_loss: 3.8470 - value_head_loss: 0.3768
  Epoch 3/10
  50/50 [==============================] - 1s 13ms/step - loss: 4456112.5000 - policy_head_loss: 0.9027 - value_head_loss: 0.8510
  Epoch 4/10
  50/50 [==============================] - 1s 14ms/step - loss: 16085884.0000 - policy_head_loss: 0.0945 - value_head_loss: 3.9925
  Epoch 5/10
  50/50 [==============================] - 1s 14ms/step - loss: 32722448.0000 - policy_head_loss: 2.6572 - value_head_loss: 4.0000
  Epoch 6/10
  50/50 [==============================] - 1s 14ms/step - loss: 52690084.0000 - policy_head_loss: 9.6345 - value_head_loss: 3.1810e-12
  Epoch 7/10
  50/50 [==============================] - 1s 14ms/step - loss: 74703120.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  Epoch 8/10
  50/50 [==============================] - 1s 14ms/step - loss: 97784832.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  Epoch 9/10
  50/50 [==============================] - 1s 14ms/step - loss: 121202520.0000 - policy_head_loss: 2.0802e-05 - value_head_loss: 4.0000
  Epoch 10/10
  50/50 [==============================] - 1s 14ms/step - loss: 144415040.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  </code></pre>
  
  <p>and my prediction looking like this:</p>
  
  <pre><code>p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[-1.]]
  </code></pre>
  
  <p>another outcome could be:</p>
  
  <pre><code>Epoch 1/10
  50/50 [==============================] - 4s 82ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000
  Epoch 2/10
  50/50 [==============================] - 1s 17ms/step - loss: 2.2826 - policy_head_loss: 2.0001 - value_head_loss: 2.1454
  Epoch 3/10
  50/50 [==============================] - 1s 16ms/step - loss: 1718694.1250 - policy_head_loss: 0.5434 - value_head_loss: 3.9772
  Epoch 4/10
  50/50 [==============================] - 1s 14ms/step - loss: 6204218.0000 - policy_head_loss: 9.4180e-05 - value_head_loss: 0.0000e+00
  Epoch 5/10
  50/50 [==============================] - 1s 14ms/step - loss: 12620835.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 6/10
  50/50 [==============================] - 1s 14ms/step - loss: 20322232.0000 - policy_head_loss: 7.7489 - value_head_loss: 0.0000e+00
  Epoch 7/10
  50/50 [==============================] - 1s 14ms/step - loss: 28812526.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 1.5966
  Epoch 8/10
  50/50 [==============================] - 1s 14ms/step - loss: 37715012.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 9/10
  50/50 [==============================] - 1s 15ms/step - loss: 46747064.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 10/10
  50/50 [==============================] - 1s 14ms/step - loss: 55699992.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  </code></pre>
  
  <p>with the predictions:</p>
  
  <pre><code>p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[1.]]
  </code></pre>
  
  <p>which are the correct ones as I would have expected.</p>
  
  <p>How come on some trainings, my NN doesnt fit the data at all?   I wanted to start the training process for a whole week now, but before I do so I want to make sure there are no errors in the way I layed out my NN.   And this looks like I am missing something here.</p>
  
  <p>And here at the <code>predict</code> / <code>train</code> methods of my neuralnet, (the NN is part of an alpha-zero replica and the involved game is connect4, I omitted these in the example to make it easier to actually replicate the problem. This is why you see some transform operations in <code>predict</code> and <code>train</code> methods)</p>
  
  <pre><code>def train(self, examples):
          input_boards, target_pis, target_vs = list(zip(*examples))       
          input_boards = np.asarray(input_boards)
          target_pis = np.asarray(target_pis)
          target_vs = np.asarray(target_vs)        
          logger.debug("Passing to nn: x: {}, y: {}, batch_size: {}, epochs: {}".format(input_boards, [target_pis, target_vs], self.args["batch_size"], self.args["epochs"]))
          self.nnet.model.fit(x = input_boards, y = [target_pis, target_vs], batch_size = self.args["batch_size"], epochs = self.args["epochs"])
  
  
  def predict(self, board):
          board = board.nn_board_2d
          # preparing input
          board = board[np.newaxis, :, :] # this has to be done for the conv2d to work
  
          # run
          pi, v = self.nnet.model.predict(board)
          return pi[0], v[0]
  </code></pre>
  
  <p>The parameters I used for this example:</p>
  
  <pre><code>'lr': 0.2
  'dropout': 0.1
  'epochs': 10
  'num_channels': 512,
  'filters': 256
  'momentum':0.9
  </code></pre>
  
  <p>EDIT: As soon as I use a lower learning rate and more iterations, my <code>p</code> changes, but my <code>v</code> stays exactly at <code>0</code>. This is what was bothering me in the first place:</p>
  
  <pre><code>Epoch 1/10
  118/118 [==============================] - 5s 44ms/step - loss: 1.8037 - policy_head_loss: 2.4305 - value_head_loss: 0.7578
  Epoch 2/10
  118/118 [==============================] - 2s 14ms/step - loss: 1.1666 - policy_head_loss: 1.7723 - value_head_loss: 0.1416
  Epoch 3/10
  118/118 [==============================] - 2s 13ms/step - loss: 1.0987 - policy_head_loss: 1.6832 - value_head_loss: 0.0948
  Epoch 4/10
  118/118 [==============================] - 2s 13ms/step - loss: 1.0430 - policy_head_loss: 1.5787 - value_head_loss: 0.0876
  Epoch 5/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9943 - policy_head_loss: 1.4859 - value_head_loss: 0.0826
  Epoch 6/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9469 - policy_head_loss: 1.3959 - value_head_loss: 0.0777
  Epoch 7/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9046 - policy_head_loss: 1.3180 - value_head_loss: 0.0707
  Epoch 8/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.8629 - policy_head_loss: 1.2403 - value_head_loss: 0.0647
  Epoch 9/10
  118/118 [==============================] - 2s 14ms/step - loss: 0.8068 - policy_head_loss: 1.1344 - value_head_loss: 0.0583
  Epoch 10/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.7335 - policy_head_loss: 0.9922 - value_head_loss: 0.0538
  I0916 14:24:38.595780  7116 trainingonly1NN.py:232] Values for empty board: new network: P : [0.20057818 0.11068489 0.15129891 0.20042823 0.13117987 0.04705378
   0.15877616]  v: 0
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-16T12:04:52.563Z
Comments.Text: You misunderstood about the learning rate. Your network is diverging. Use a **low** learning rate (and lots of iterations) instead.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-16T12:08:55.880Z
Comments.Text: But both losses I am tracking (policy and value, these are output units) reach 0, how can the overall loss go up?  
  I wonder if there is a weight initiliazation issue or something
Comments.UserDisplayName: Leviathan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-16T14:55:11.173Z
Comments.Text: Sorry, I missed the combined nature of the losses. I'm not sure what is going on. However your premise "I used a high learning rate to make sure the data in this example is sufficient on the training" is wrong in general. High learning rates will often lead to diverging loss values. What happens when you try a lower learning rate and more iterations?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-16T15:51:46.610Z
Comments.Text: Then P does get updated correctly,  but v stays at 0. This is what is so odd to me.
  I mean it stays exactly 0, not 0.0001 or -0.0001 or something, it always stays at 0. and that is pretty bad
Comments.UserDisplayName: Leviathan
Tags: <neural-networks><deep-learning><architecture><overfitting>
Title: Neural Network training on one example to try overfitting leads to strange predictions
UserDisplayName: Leviathan
ThreadText: # Neural Network training on one example to try overfitting leads to strange predictions
  
  (Asked by: username_0 on 2019-09-16)
  
  **tldr**; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which produces two outcomes (`p` and `v`) always predicts exactly `0` for `v` and I would like to change that.
  
  In the following post I will provide all code necessary to reproduce the problem.  
  I am training a neural network on the same input. The wanted outcome for a value "`v`" is `1`. If I create the network and train it, sometimes the predicted outcome will be `1`, sometimes it will be `-`1.  
  Also, the `loss` seems to flip between `0` and `4` during training epochs.  
  Additionally, the loss blows up immensly, even though both losses for the outcome layers are close to zero.  
  I do not understand where this behaviour comes from. I used `Leaky-ReLU` to make sure it can handle negative input, I used a high `learning rate` to make sure the data in this example is sufficient on the training, and the input is the same all the time.
  
  My Neural network looks like this:
  
  ```
  input_layer = keras.Input(shape=(6,7),)    
  formatted_input_layer = keras.layers.Reshape((6,7, 1))(input_layer)       
  conv_layer1 = self.create_conv_layer(formatted_input_layer)
  res_layer1 = self.create_res_layer(conv_layer1)
  res_layer2 = self.create_res_layer(res_layer1)
  res_layer3 = self.create_res_layer(res_layer2)
  res_layer4 = self.create_res_layer(res_layer3)
  policy_head = self.create_policy_head(res_layer4)
  value_head = self.create_value_head(res_layer4)
  model = keras.Model(inputs=input_layer, outputs=[policy_head, value_head])
  optimizer = keras.optimizers.SGD(lr=args['lr'],momentum=args['momentum'])
  model.compile(loss = {'policy_head' : 'categorical_crossentropy', 'value_head' : 'mean_squared_error'}, optimizer=optimizer, loss_weights={'policy_head':0.5, 'value_head':0.5})
  
  ```
  
  Methods for the different layers:  
  conv\_layer:
  
  ```
  def create_conv_layer(self, input_layer):
      conv_layer = keras.layers.Conv2D(filters=256,
                                       kernel_size=3,
                                       strides=1,
                                       padding='same',
                                       use_bias=False,
                                       data_format="channels_last",
                                       activation = "linear",
                                       kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
      conv_layer= keras.layers.BatchNormalization(axis=-1)(conv_layer)
      conv_layer = keras.layers.LeakyReLU()(conv_layer)
      return conv_layer
  
  ```
  
  res\_layer:
  
  ```
  def create_res_layer(self, input_layer):
          conv_layer = self.create_conv_layer(input_layer)
          res_layer = keras.layers.Conv2D(filters=256,
                                           kernel_size=3,
                                           strides=1,
                                           padding='same',
                                           use_bias=False,
                                           data_format="channels_last",
                                           activation = "linear",
                                           kernel_regularizer = keras.regularizers.l2(0.0001))(conv_layer)
          res_layer= keras.layers.BatchNormalization(axis=-1)(res_layer)
          res_layer = keras.layers.add([input_layer, res_layer])
          res_layer = keras.layers.LeakyReLU()(res_layer)
          return res_layer
  
  ```
  
  policy head:
  
  ```
  def create_policy_head(self, input_layer):
          policy_head = keras.layers.Conv2D(filters=2,
                                            kernel_size=1,
                                            strides=1,
                                            padding='same',
                                            use_bias = False,
                                            data_format='channels_last',
                                            activation='linear',
                                            kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
          policy_head = keras.layers.BatchNormalization(axis=-1)(policy_head)
          policy_head = keras.layers.LeakyReLU()(policy_head)
          policy_head = keras.layers.Flatten()(policy_head)
          policy_head = keras.layers.Dense(units = 7,
                                           use_bias = False,
                                           activation = 'softmax',
                                           kernel_regularizer = keras.regularizers.l2(0.0001),
                                           name = "policy_head"
                                           )(policy_head)
          return policy_head
  
  ```
  
  value head:
  
  ```
  def create_value_head(self, input_layer):
          value_head = keras.layers.Conv2D(filters=1,
                                            kernel_size=1,
                                            strides=1,
                                            padding='same',
                                            use_bias = False,
                                            data_format='channels_last',
                                            activation='linear',
                                            kernel_regularizer = keras.regularizers.l2(0.0001))(input_layer)
          value_head = keras.layers.BatchNormalization(axis=-1)(value_head)
          value_head = keras.layers.LeakyReLU()(value_head)  
          value_head = keras.layers.Flatten()(value_head)        
          value_head = keras.layers.Dense(units = 21,
                                           use_bias = False,
                                           activation = 'linear',
                                           kernel_regularizer = keras.regularizers.l2(0.0001)
                                           )(value_head)
          value_head = keras.layers.LeakyReLU()(value_head)      
          value_head = keras.layers.Dense(units = 1,
                                           use_bias = False,
                                           activation = 'tanh',
                                           kernel_regularizer = keras.regularizers.l2(0.0001),
                                           name = "value_head"                                     
                                           )(value_head)
          return value_head
  
                                  )(value_head)
  
  ```
  
  The way I am testing my NN:
  
  ```
      canonicalBoard = np.zeros(shape = (6,7), dtype=int) 
  
      Pi = [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0] 
  
      trainExamples = [[canonicalBoard, Pi, 1]]*50        
  
      nnetwrapper.train(trainExamples)
  
      board = canonicalBoard[np.newaxis, :, :]
  
      p, v = nnetwrapper.nnet.model.predict(board)
  
  ```
  
  which results in the training looking like this:
  
  ```
  Epoch 1/10
  50/50 [==============================] - 4s 71ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000
  Epoch 2/10
  50/50 [==============================] - 1s 15ms/step - loss: 2.3218 - policy_head_loss: 3.8470 - value_head_loss: 0.3768
  Epoch 3/10
  50/50 [==============================] - 1s 13ms/step - loss: 4456112.5000 - policy_head_loss: 0.9027 - value_head_loss: 0.8510
  Epoch 4/10
  50/50 [==============================] - 1s 14ms/step - loss: 16085884.0000 - policy_head_loss: 0.0945 - value_head_loss: 3.9925
  Epoch 5/10
  50/50 [==============================] - 1s 14ms/step - loss: 32722448.0000 - policy_head_loss: 2.6572 - value_head_loss: 4.0000
  Epoch 6/10
  50/50 [==============================] - 1s 14ms/step - loss: 52690084.0000 - policy_head_loss: 9.6345 - value_head_loss: 3.1810e-12
  Epoch 7/10
  50/50 [==============================] - 1s 14ms/step - loss: 74703120.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  Epoch 8/10
  50/50 [==============================] - 1s 14ms/step - loss: 97784832.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  Epoch 9/10
  50/50 [==============================] - 1s 14ms/step - loss: 121202520.0000 - policy_head_loss: 2.0802e-05 - value_head_loss: 4.0000
  Epoch 10/10
  50/50 [==============================] - 1s 14ms/step - loss: 144415040.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 4.0000
  
  ```
  
  and my prediction looking like this:
  
  ```
  p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[-1.]]
  
  ```
  
  another outcome could be:
  
  ```
  Epoch 1/10
  50/50 [==============================] - 4s 82ms/step - loss: 1.6829 - policy_head_loss: 1.9459 - value_head_loss: 1.0000
  Epoch 2/10
  50/50 [==============================] - 1s 17ms/step - loss: 2.2826 - policy_head_loss: 2.0001 - value_head_loss: 2.1454
  Epoch 3/10
  50/50 [==============================] - 1s 16ms/step - loss: 1718694.1250 - policy_head_loss: 0.5434 - value_head_loss: 3.9772
  Epoch 4/10
  50/50 [==============================] - 1s 14ms/step - loss: 6204218.0000 - policy_head_loss: 9.4180e-05 - value_head_loss: 0.0000e+00
  Epoch 5/10
  50/50 [==============================] - 1s 14ms/step - loss: 12620835.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 6/10
  50/50 [==============================] - 1s 14ms/step - loss: 20322232.0000 - policy_head_loss: 7.7489 - value_head_loss: 0.0000e+00
  Epoch 7/10
  50/50 [==============================] - 1s 14ms/step - loss: 28812526.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 1.5966
  Epoch 8/10
  50/50 [==============================] - 1s 14ms/step - loss: 37715012.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 9/10
  50/50 [==============================] - 1s 15ms/step - loss: 46747064.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  Epoch 10/10
  50/50 [==============================] - 1s 14ms/step - loss: 55699992.0000 - policy_head_loss: 1.1921e-07 - value_head_loss: 0.0000e+00
  
  ```
  
  with the predictions:
  
  ```
  p: [[0. 0. 0. 0. 0. 1. 0.]]  v: [[1.]]
  
  ```
  
  which are the correct ones as I would have expected.
  
  How come on some trainings, my NN doesnt fit the data at all? I wanted to start the training process for a whole week now, but before I do so I want to make sure there are no errors in the way I layed out my NN. And this looks like I am missing something here.
  
  And here at the `predict` / `train` methods of my neuralnet, (the NN is part of an alpha-zero replica and the involved game is connect4, I omitted these in the example to make it easier to actually replicate the problem. This is why you see some transform operations in `predict` and `train` methods)
  
  ```
  def train(self, examples):
          input_boards, target_pis, target_vs = list(zip(*examples))       
          input_boards = np.asarray(input_boards)
          target_pis = np.asarray(target_pis)
          target_vs = np.asarray(target_vs)        
          logger.debug("Passing to nn: x: {}, y: {}, batch_size: {}, epochs: {}".format(input_boards, [target_pis, target_vs], self.args["batch_size"], self.args["epochs"]))
          self.nnet.model.fit(x = input_boards, y = [target_pis, target_vs], batch_size = self.args["batch_size"], epochs = self.args["epochs"])
  
  def predict(self, board):
          board = board.nn_board_2d
          # preparing input
          board = board[np.newaxis, :, :] # this has to be done for the conv2d to work
  
          # run
          pi, v = self.nnet.model.predict(board)
          return pi[0], v[0]
  
  ```
  
  The parameters I used for this example:
  
  ```
  'lr': 0.2
  'dropout': 0.1
  'epochs': 10
  'num_channels': 512,
  'filters': 256
  'momentum':0.9
  
  ```
  
  EDIT: As soon as I use a lower learning rate and more iterations, my `p` changes, but my `v` stays exactly at `0`. This is what was bothering me in the first place:
  
  ```
  Epoch 1/10
  118/118 [==============================] - 5s 44ms/step - loss: 1.8037 - policy_head_loss: 2.4305 - value_head_loss: 0.7578
  Epoch 2/10
  118/118 [==============================] - 2s 14ms/step - loss: 1.1666 - policy_head_loss: 1.7723 - value_head_loss: 0.1416
  Epoch 3/10
  118/118 [==============================] - 2s 13ms/step - loss: 1.0987 - policy_head_loss: 1.6832 - value_head_loss: 0.0948
  Epoch 4/10
  118/118 [==============================] - 2s 13ms/step - loss: 1.0430 - policy_head_loss: 1.5787 - value_head_loss: 0.0876
  Epoch 5/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9943 - policy_head_loss: 1.4859 - value_head_loss: 0.0826
  Epoch 6/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9469 - policy_head_loss: 1.3959 - value_head_loss: 0.0777
  Epoch 7/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.9046 - policy_head_loss: 1.3180 - value_head_loss: 0.0707
  Epoch 8/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.8629 - policy_head_loss: 1.2403 - value_head_loss: 0.0647
  Epoch 9/10
  118/118 [==============================] - 2s 14ms/step - loss: 0.8068 - policy_head_loss: 1.1344 - value_head_loss: 0.0583
  Epoch 10/10
  118/118 [==============================] - 2s 13ms/step - loss: 0.7335 - policy_head_loss: 0.9922 - value_head_loss: 0.0538
  I0916 14:24:38.595780  7116 trainingonly1NN.py:232] Values for empty board: new network: P : [0.20057818 0.11068489 0.15129891 0.20042823 0.13117987 0.04705378
   0.15877616]  v: 0
  
  ```
  
  **Tags:** neural-networks, deep-learning, architecture, overfitting
  
  > **Comments:**
  > **username_1**: You misunderstood about the learning rate. Your network is diverging. Use a **low** learning rate (and lots of iterations) instead.  
  > **username_0**: But both losses I am tracking (policy and value, these are output units) reach 0, how can the overall loss go up?   I wonder if there is a weight initiliazation issue or something  
  > **username_1**: Sorry, I missed the combined nature of the losses. I'm not sure what is going on. However your premise "I used a high learning rate to make sure the data in this example is sufficient on the training" is wrong in general. High learning rates will often lead to diverging loss values. What happens when you try a lower learning rate and more iterations?  
  > **username_0**: Then P does get updated correctly,  but v stays at 0. This is what is so odd to me. I mean it stays exactly 0, not 0.0001 or -0.0001 or something, it always stays at 0. and that is pretty bad  
  
  ---
  
  ## 0 answers
  
  