In theory the error during training is called bias.
Ways to reduce bias


Increase the number of hidden layers


change the NN architecture


Train NN for longer


MAYBE! Threshold you are using for dropout might be too high, which are causing more units/neurons to turn off as you train longer.