Is it acceptable if my model has a very high amount of true negatives and a precision score of 100%? I currently have a model that's tasked with multilabel classification with an extra "no answer" option for negative predictions. The metrics that I'm using are precision, recall, and the F1 score. The be more specific, it's a relation classification task where the model is provided text and pairs of entities and is tasked with classifying the relation for each pair.
To roughly outline the evaluation pipeline I have:

*

*Extract all positive predictions from all of the predictions (i.e., samples where the model didn't predict "no answer"). The total number of positive predictions is the denominator for our precision (i.e., TP + FP).

*Among these positive predictions, compare with the ground truth dev/test set and see how many (sample, label) instances overlap. The number of overlapping instances will be our true positive count.

*The total number of samples in the dev/test set will be the denominator for recall (i.e., TP + FN).

*Calculate precision, recall, and F1 as usual.

First off there are a ton of negative predictions. Among the (roughly) 100,000 samples, I think around 80-90% are labeled as "no answer." I suppose that's not exactly outrageous, but what's also concerning me is that my precision is 1.0 and recall is 0.53, leading to an overall F1 score of around 0.70, which I've personally never seen before.
How should I interpret whether this perfect precision/mediocre recall scenario is acceptable or not? Any thoughts are appreciated, thanks.


my precision is 1.0 and recall is 0.53

If you think it in terms of binary classification (positives and negatives), this means that of all the actual positives, your model predicted only 53% of them correctly as True positives (recall) (bad thing), at the same time, those 53% that your model predicted as True positives were in fact positives (precision) (good thing).
In general sense, this doesn't seem a good thing.
You should look at class-wise precision-recall values to better analyze your model's performance w.r.t each class.
