Why does $\mu > 0$ (or even $\mu > \epsilon$) "seem easier” to substantiate than $\mu \neq 0$? Consider a random variable $X$ following a normal distribution $N(\mu,\sigma^2)$. Suppose that we have drawn iid samples of $X$, obtaining a data set with a sample mean $\bar{x}>0$.
We want to test whether the data supports $\mu >0$ with a significance level $\alpha \in (0,1)$. Setting
\begin{equation*}
H_0: \mu \le 0, 
\quad H_1: \mu > 0, 
\end{equation*}
we examine the $p$-value
\begin{equation*}
p \;=\; \Pr(\overline{X} \;\ge\; \bar{x} \mathrel{|} H_0) \;=\; \Pr(\overline{X} \ge \bar{x}\mathrel|{\mu \le 0}) \;\le\; \Pr(\overline{X} \ge \bar{x} \mathrel{|} \mu = 0).
\end{equation*}
[N.B.: As kindly pointed out by @dimitriy, in practice we often set the null hypothesis simply to $\mu=0$, for which the $p$-value would be directly the RHS of the above inequality.]
As in textbooks, the data provides a sufficient evidence to substantiate $\mu > 0$ if $p \le \alpha$.
Now, we change our mind and test whether the data supports $\mu \neq 0$, with
\begin{equation*}
\hat{H}_0: \mu = 0, 
\quad \hat{H}_1: \mu \neq 0,
\end{equation*}
and the $p$-value being
\begin{equation*}
\hat{p} \;=\; \Pr(|\overline{X}| \ge \bar{x} \mathrel{|} \hat{H}_0) \;=\; \Pr(|\overline{X}| \ge \bar{x} \mathrel|{\mu = 0}).
\end{equation*}
Again, textbooks tell us that the data provides sufficient evidence to substantiate $\mu \neq 0$ if $\hat{p} \le \alpha$, with $\alpha$ being the aforementioned significance level.
Here comes the part that confuses me. Since
$$\hat{p} \ge 2p,$$
it turns out "easier" to substantiate $\mu >0$ than $\mu \neq 0$, in the sense that the data may strongly support $\mu >0$, but does not constitute sufficient evidence for $\mu \neq 0$; equivalently, the data may strongly suggest rejecting $\mu \le 0$ without being able to reject $\mu = 0$. For instance, imagine that $p \le 0.6\%$, $\hat{p} = 1.2\%$, and $\alpha = 1\%$.
[Update: We can strengthen the above observation a bit: there exists an $\epsilon >0$ that it “seems easier” to substantiate $\mu > \epsilon$ than doing the same for $\mu \neq 0$.]
So what is the logic? What have I overlooked or misunderstood? I believe the calculation is trivial to everyone here, but there seems to be something to say about the interpretation. What is it?
In my textbook, hypothesis testing is compared to a court trial, with the null hypothesis being the suspect is innocent. Following this comparison, a probably quite poor metaphor for the paradox elaborated above is that we may have strong evidence to support that "A killed B by stabbing a dagger into B's chest", but not enough justification to substantiate that "A killed B". No, I must have made some silly mistake.
Thank you very much.
Background: Being not a statistician by any means, I am asked to teach a course that involves a part of statistics. The question occurs to me when I am preparing my notes.

There are some misconceptions in your question that I need to clear up before getting to the answer.
The null hypothesis $H_0$ in a statistical test is always the claim you want to argue against. The alternative hypothesis $H_1$ is the claim you hope to be true.
The null and the alternative need to be

*

*mutually exclusive (no overlap)

*collectively exhaustive (partition the parameter space)

*the equality sign ($=$, $\ge$, or $\le$) almost always appears in the null.

So your first test should have $H_0: \mu \le 0$ and $H_1: \mu > 0$.
Your second test should have $H_0: \mu = 0$ and $H_1: \mu \ne 0$.
The p-value is the probability of seeing the observed mean (or something even more extreme) if the null hypothesis was true. Then we apply the rule “reject the null when the p-value is small.” The basic idea is that if seeing a big mean is unlikely if the null was true, the null is likely to be false.
There is a slight complication in the first test. The null is a composite one: it's an interval rather than a single point. So we will have to calculate the probability when $\mu=0$, then when $\mu =-1$, and also everywhere else below zero, since all those points are inside the null. But that’s an infinite number of points! What we do instead is to calculate the probability at the most extreme point of the null hypothesis, closest to alternative parameter space, which is at $\mu = 0$. This means that the p-value is exact only for $\mu=0$. If $\mu<0$, then our p-value is just a conservative bound on the type I error rate (the error being finding a negative effect when there is none). In other words, if the true effect is negative, then finding a false positive result is even less likely than 5% (or whatever value of $\alpha$ your question requires). This is also the reason why statistics packages will express the one-sided null as $\mu=0$ rather than $\mu \le 0$, which is technically correct, but confusing notation.
Now for your question. For both one-sided and two-sided tests, we calculate the p-value with $\mu=0$. Suppose you observe a mean of $u>0$. With a two-sided test, you need to calculate $\Pr(\bar X \ge u \vert \mu=0)$ and $\Pr(\bar X
\le -u \vert \mu=0)$, since both kinds of extreme values constitute evidence against that null.
With a one-sided test, seeing a mean that’s less than $-k$ doesn’t count as evidence against the null, so we only calculate $\Pr(\bar X \ge u \vert \mu=0)$. This is why the p-value is larger in the two-sided case, which means it’s easier to reject in the one-sided case.
Another way to put this is that a two-sided test is just two one-sided tests cobbled together (a superiority and an inferiority one).
