If you are concerned about model underfitting, you should try to overfit your model on a small mini-batch and see it the accuracy goes to 100%. If the model is not able to overfit then, you are underfitting.