Why does $\mu > 0$ (or even $\mu > \epsilon$) "seem easier” to substantiate than $\mu \neq 0$? Consider a random variable $X$ following a normal distribution $N(\mu,\sigma^2)$. Suppose that we have drawn iid samples of $X$, obtaining a data set with a sample mean $\bar{x}>0$.
We want to test whether the data supports $\mu >0$ with a significance level $\alpha \in (0,1)$. Setting
\begin{equation*}
H_0: \mu \le 0, 
\quad H_1: \mu > 0, 
\end{equation*}
we examine the $p$-value
\begin{equation*}
p \;=\; \Pr(\overline{X} \;\ge\; \bar{x} \mathrel{|} H_0) \;=\; \Pr(\overline{X} \ge \bar{x}\mathrel|{\mu \le 0}) \;\le\; \Pr(\overline{X} \ge \bar{x} \mathrel{|} \mu = 0).
\end{equation*}
[N.B.: As kindly pointed out by @dimitriy, in practice we often set the null hypothesis simply to $\mu=0$, for which the $p$-value would be directly the RHS of the above inequality.]
As in textbooks, the data provides a sufficient evidence to substantiate $\mu > 0$ if $p \le \alpha$.
Now, we change our mind and test whether the data supports $\mu \neq 0$, with
\begin{equation*}
\hat{H}_0: \mu = 0, 
\quad \hat{H}_1: \mu \neq 0,
\end{equation*}
and the $p$-value being
\begin{equation*}
\hat{p} \;=\; \Pr(|\overline{X}| \ge \bar{x} \mathrel{|} \hat{H}_0) \;=\; \Pr(|\overline{X}| \ge \bar{x} \mathrel|{\mu = 0}).
\end{equation*}
Again, textbooks tell us that the data provides sufficient evidence to substantiate $\mu \neq 0$ if $\hat{p} \le \alpha$, with $\alpha$ being the aforementioned significance level.
Here comes the part that confuses me. Since
$$\hat{p} \ge 2p,$$
it turns out "easier" to substantiate $\mu >0$ than $\mu \neq 0$, in the sense that the data may strongly support $\mu >0$, but does not constitute sufficient evidence for $\mu \neq 0$; equivalently, the data may strongly suggest rejecting $\mu \le 0$ without being able to reject $\mu = 0$. For instance, imagine that $p \le 0.6\%$, $\hat{p} = 1.2\%$, and $\alpha = 1\%$.
[Update: We can strengthen the above observation a bit: there exists an $\epsilon >0$ that it “seems easier” to substantiate $\mu > \epsilon$ than doing the same for $\mu \neq 0$.]
So what is the logic? What have I overlooked or misunderstood? I believe the calculation is trivial to everyone here, but there seems to be something to say about the interpretation. What is it?
In my textbook, hypothesis testing is compared to a court trial, with the null hypothesis being the suspect is innocent. Following this comparison, a probably quite poor metaphor for the paradox elaborated above is that we may have strong evidence to support that "A killed B by stabbing a dagger into B's chest", but not enough justification to substantiate that "A killed B". No, I must have made some silly mistake.
Thank you very much.
Background: Being not a statistician by any means, I am asked to teach a course that involves a part of statistics. The question occurs to me when I am preparing my notes.


the data may strongly support $\mu > 0$, but does not constitute sufficient evidence for $\mu \neq 0$

I don't know if this reasoning helps or if it is 100% correct... You accept to be wrong $\alpha$-percent of the times in hypothetical repeats of the test; for example, you accept to be wrong 5% of the times in rejecting the null.
In a two-tailed test you spread this 5% in the left-tail and in the right-tail of the test distribution. In the one-tailed test, you still accept the be wrong with the same frequency (say, 5%) but always in the same direction so you put all the 5% in the same tail.
In fact, I wouldn't say that in the one-tailed test it is easier to reject, but rather that you need less extreme values. You need less extreme values compared to the two-tailed test in order to keep the overall error rate at the same $\alpha$ level.
Your trial metaphor is interesting but perhaps is not appropriate because A killed B is not more (or less) extreme than A killed B by stabbing.
