You could observe the loss of your validation data and stop the training at the lowest point, i.e. when the model starts to overfit to your dataset. This could stabilize your probabilities a bit.
OK, let me know, if you get stuck computing the confusion matrix.