Michael_Sidorov:

Just to note: I’m aware of the fact that each time the weights are randomly initialized, but the behavior described above is rather consistent over 10 runs, which renders this possibility quite not likely.


You can improve the reproducibility with:
  
      

      keras.io
  

  
    

Keras documentation: Keras FAQ



  

  
    
    
  

  


Also I don’t see the relu activations in the first sequential model.