I would scale down the problem to just a single input and try to overfit your model.
If that’s not possible, your architecture, the hyperparameters or the training routine might have a bug or are not suitable for the problem.