Hi,



 luanpham:

The validation loss is increasing when validation accuracy is increasing, too. So, the best model should be the lowest loss or highest accuracy?


From little experience I have, I tend to take the best model as the one with the highest accuracy, if there is no overfitting (not your case).



 luanpham:

If we choose the highest accuracy as the best model, then if we look at the losses, easy to see the overfitting scenarios (low training loss and high validation loss).


Yes, you are right.



 luanpham:

If we choose the lowest loss as the best model. Why sometimes the loss is high with the accuracy, it seems like have true predict but low confidence (aka. probability, I use cross-entropy loss). Then, is the high confidence is always good?


Yes, a model with high confidence in its predictions is preferable. It is also why the loss is often a better metric to look at when you want to determine if the model is overfitting (e.g. the accuracy does not differentiate from a prediction at probability 0.5 or 0.9, both can predict the correct class, but your loss does).
In your situation, I would try to modify the architecture of the model to get rid of the overfitting that you have.
Good luck!