Confusion matrix is used to evaluate my model, while I trained it using some cost functions such as MSE or Cross Entropy. It is not good from mathematics point of view, since the model is trying to minimizing a value which is not the value used to evaluate it.
So I was wondering if there is a cost function which directly minimize a value in confusion matrix, so seem like the answer is a No.