@sgugger WAIT!  You are telling me that after having trained the two models separately, on the same dataset where I only change the labels to be predicted, I just need to average the values obtained from the metrics calculated by the compute_metrics function (i.e. accuracy_score, precision, recall, F1) during prediction on test dataset and is it as if I had “combined the two models and made the two predictions at the same time”?
If so, you’ve solved a huge problem for me.