Solved by vdw in post #2 
              
              
                Since you only posted the model but not the code for training, it’s difficult to say what’s going on. Here just some pointers that strike me as odd: 


Are you using the whole data set as a single batch? Usually training is done in mini-batches, it’s typically better for training. 


According to yo…