Why does the value function induce a partial order and not a total order? In Sutton and Barto's book: Reinforcement Learning: An Introduction, they write

Value functions deﬁne a partial ordering over policies.

My questions is why the value function is not a total order?
I would have thought: $\pi_1 \geq \pi_2$ if and only if $V_{\pi_1} (s_0) \geq V_{\pi_2} (s_0)$, where $s_0$ is the initial state distribution.

The follow sentence says:

A policy $\pi$ is defined to be better than or equal to a policy
$\pi'$ if its expected return is greater than or equal to that of
$\pi'$ for all states. In other words, $\pi \geq \pi'$ if and only if $V^\pi(s) \geq V^{\pi'}(s)$ for all $s \in \mathcal{S}$.

Of course, it's possible to define an ordering with respect to some state distribution, as you've suggested, but the state independent version is probably more useful.
