Should I use a one-sided or a two-sided test when deciding to deploy new system? In the following situation, should I use a one-sided or a two-sided significance test?
Situation: Suppose I have a face recognition system A that recognizes faces with a certain accuracy. Now I want to test if my new system B has significantly better accuracy. If it does, I will deploy B. If B is just the same or even worse than A then I will not deploy B.
Related question: I have a related question about two-sided tests, specifically: Rejecting the null hypothesis here would mean that the new system B different from A, which means it could be significantly worse or significantly better -- the two-sided test does not say which it is. Correct? If that's correct, what could be the value of a two-sided test in my situation above, where I want to avoid deploying a system that is worse than the existing system?

I think this is actually a perfect use of a one-tailed test. The risk of one-tailed tests is the potential to capitalize on chance, but this is not an issue here since you have explicitly defined the important region of the parameter space a priori, and done so in a practically justified way. As long as you truly take no new action as a result of a large effect in the opposite direction, there seems to be nothing wrong with a one-tailed test in this instance.
The p-value of a two-sided test doesn't tell you the direction of the effect, but the effect estimate does. If the mean for B is greater than the mean for A and the two-sided p-value is small enough to reject the null hypothesis that the true means are the same, then you can say that the true mean for B is greater than the true mean for A, the same conclusion you would reach with a significant one-sided test in the right direction.
