title: Why does the Keras Early Stopping function not stop the training, although the monitored value is increasing?
body: <p>I am trying to train a neural Network for a regression problem and I implemented the Keras Early Stopping Function to avoid overfitting.</p>
  
  <p>Now the early stopping function stops the programm almost directly when I montior "val_loss" and the result is a useless NN, but when I monitor "val_mse" the training goes on and on without stopping, although i can see "val_mse" increasing throught the training and i set patience = 0.</p>
  
  <p>I seem to have misunderstood the Early Stopping Callback since i thought that it does monitor the value and stop the training as soon as the value starts increasing again.</p>
  
  <pre><code>np.random.seed(7)
  
  #Define Input
  tf_features_64 = np.load("IN_2.npy")
  tf_labels_64 = np.load("OUT_2.npy")
  tf_features_32 = tf_features_64.astype(np.float32)
  tf_labels_32 = tf_labels_64.astype(np.float32)
  
  X = tf_features_32
  Y = tf_labels_32[0:10680, 4:8]
  #Define Callback
  tbCallBack = TensorBoard(log_dir='./Graph{}', histogram_freq=0, write_graph=True, write_images=True) #TensorBoard Monitoring
  esCallback = EarlyStopping(monitor='val_mse',
                             min_delta=0,
                             patience=0,
                             verbose=1,
                             mode='min')
  
  #create Layers
  visible = Input(shape=(33,))
  x = Dropout(.1)(visible)
  #x = Dense(63)(x)
  #x = Dropout(.4)(x)
  output = Dense(4)(x)  
  
  Optimizer = optimizers.Adam(lr=0.001
                              #amsgrad = True)
  
  model = Model(inputs=visible, outputs = output)
  model.compile(optimizer=Optimizer,
                loss=['mse'],
                metrics=['mae', 'mse']
                )
  model.fit(X, Y, epochs=8000, batch_size=20, shuffle=True, validation_split=0.35, callbacks=[tbCallBack, esCallback])
  </code></pre>
  
  <p>As an example, i get the following output where i can clearly see, that the val_mse increases over the epochs.</p>
  
  <pre><code>  20/6942 [..............................] - ETA: 0s - loss: 0.0022 - mean_absolute_error: 0.0373 - mean_squared_error: 0.0022
  1620/6942 [======&gt;.......................] - ETA: 0s - loss: 0.0011 - mean_absolute_error: 0.0251 - mean_squared_error: 0.0011
  3260/6942 [=============&gt;................] - ETA: 0s - loss: 0.0015 - mean_absolute_error: 0.0290 - mean_squared_error: 0.0015
  4900/6942 [====================&gt;.........] - ETA: 0s - loss: 0.0017 - mean_absolute_error: 0.0301 - mean_squared_error: 0.0017
  6500/6942 [===========================&gt;..] - ETA: 0s - loss: 0.0016 - mean_absolute_error: 0.0301 - mean_squared_error: 0.0016
  6942/6942 [==============================] - 0s 37us/step - loss: 0.0016 - mean_absolute_error: 0.0294 - mean_squared_error: 0.0016 - val_loss: 0.0011 - val_mean_absolute_error: 0.0240 - **val_mean_squared_error: 0.0011**
  **Epoch 334/8000**
  
    20/6942 [..............................] - ETA: 0s - loss: 0.0025 - mean_absolute_error: 0.0367 - mean_squared_error: 0.0025
  1620/6942 [======&gt;.......................] - ETA: 0s - loss: 0.0012 - mean_absolute_error: 0.0257 - mean_squared_error: 0.0012
  3260/6942 [=============&gt;................] - ETA: 0s - loss: 0.0014 - mean_absolute_error: 0.0274 - mean_squared_error: 0.0014
  4860/6942 [====================&gt;.........] - ETA: 0s - loss: 0.0014 - mean_absolute_error: 0.0268 - mean_squared_error: 0.0014
  6400/6942 [==========================&gt;...] - ETA: 0s - loss: 0.0012 - mean_absolute_error: 0.0254 - mean_squared_error: 0.0012
  6942/6942 [==============================] - 0s 39us/step - loss: 0.0012 - mean_absolute_error: 0.0249 - mean_squared_error: 0.0012 - val_loss: 0.0032 - val_mean_absolute_error: 0.0393 - **val_mean_squared_error: 0.0032**
  **Epoch 335/8000**
  
    20/6942 [..............................] - ETA: 0s - loss: 9.5175e-04 - mean_absolute_error: 0.0243 - mean_squared_error: 9.5175e-04
  1620/6942 [======&gt;.......................] - ETA: 0s - loss: 0.0017 - mean_absolute_error: 0.0312 - mean_squared_error: 0.0017        
  3260/6942 [=============&gt;................] - ETA: 0s - loss: 0.0013 - mean_absolute_error: 0.0271 - mean_squared_error: 0.0013
  4860/6942 [====================&gt;.........] - ETA: 0s - loss: 0.0014 - mean_absolute_error: 0.0277 - mean_squared_error: 0.0014
  6460/6942 [==========================&gt;...] - ETA: 0s - loss: 0.0013 - mean_absolute_error: 0.0266 - mean_squared_error: 0.0013
  6942/6942 [==============================] - 0s 38us/step - loss: 0.0013 - mean_absolute_error: 0.0268 - mean_squared_error: 0.0013 - val_loss: 0.0046 - val_mean_absolute_error: 0.0491 - **val_mean_squared_error: 0.0046**
  **Epoch 336/8000**
  </code></pre>