Turn off all the regularization effects. The model will overfit then. Romeve dropout, use batch-size = 1, Learning rate should be small (you can get there in steps). Further try using wide and shallow network.