Combine accuracy, precision, and recall I am working on a classification problem. Several models are produced and all have accuracy, precision and recall metrics on test data. I need to pick the best model among the alternatives.
What I can think of immediately is to combine precision and recall using F1-measure and use this as a decision metric to pick the best model.
However the requirement I am given is that accuracy should also be part of the decision metric or I should prove that combining F1-measure and accuracy will not improve the decision metric. Does anybody have any idea how to do either?

This approach is questionable.  Instead use establish statistical principles as detailed here.  Here are a few.

*

*Turn the problem into a prediction problem instead of a classification problem, by estimating outcome probabilities.  This allows for close calls, gray zones, etc., and does not require artificial "data amputation" by balancing.  Imbalance is not a problem.

*Use the gold standard log likelihood or penalized log likelihood (or Bayesian posterior) measure which captures much more information than the discontinuous measures you mentioned.

*In addition to using the log likelihood as the optimality criterion, unbiasedly and flexibly estimate the calibration curve to show that the predicted risks have good absolute accuracy.  This allows the predictions to be used for decision making.

