You could try to scale down the problem a bit, use a small data samples (e.g. only 10 samples), and try to overfit the model on this dataset by playing around with the hyperparameters, architecture, optimizer etc.
Once your model successfully predicts the small dataset, you could then try to scale it up again.