When the model is not converged, first I think the model is not enough to predict the target, and second the optimizer cannot work well for the current setting or the dataset. For example, the convergence problem of the Adam optimizer has recently been pointed out in  On the Convergence of Adam and Beyond. If you think the gradient computation fails, you should point out the error of the gradient concretely.