Solved by ptrblck in post #5 
              
              
                You could start by overfitting a small dataset (e.g. just 10 samples) by playing around with the hyperparameters (optimizer, learning rate etc.). 
If that doesn’t help, I would suggest to simplify the model. 
Once your can come up with a model and hyperparameters, which overfit this small dataset, y…