Have you tried the previous suggestion of starting with a very small training set and seeing if your model can perfectly overfit it? That should help narrow down the issue (e.g., if there is a correctness issue in the training pipeline it would prevent the model from training properly at this stage, vs. a hyperparameter issue which would likely still allow the model to overfit a small training set).