Thanks much for your answer.
It’s just that my model was evaluated by confusion matrix, while in training I have to choose something else to minimize.
I was wondering if there is a way to minimize directly the confusion matrix, now I know that the answer is No.