Assumptions of Mann-Whitney test for at least ordinal data I am reading article by Divine et al. about using Mann-Whitney test for data that is at least ordinal (i.e. it may be discrete with many ties). It says the following (in section 2.3):

That is, it (the Mann-Whitney test) generally does not depend upon any particular distributional form (or parameters) in order to generate the test statistic and p-value. In fact, it is the whole distributions that are being compared, rather than any sample-specific summary statistic(s). However, the procedure does depend upon some assumptions about those distributions. For instance, one important assumption is that the variances of the two distributions should be the same (Pratt 1964).

And in section 5.1 this paper recommends to use the Brunner-Munzel test instead of the Mann-Whitney test if the variances are unequal (as well as scipy.stats.brunnermunzel manual):

Although the basic WMW test may be invalid with unequal
variances (especially with unequal sample sizes), the Brunner–Munzel variation should work if the minimum sample size is at least 30 and the variance discordance is not too extreme. For a sample size (or sizes) below 30 and/or when one or more large clumps of ties are present, an exact/permutation WMW test (available in SAS and R) should be considered.

The hypotheses in this article are formulated as follows (in the two-sided alternative case; $X_1 \sim F, X_2 \sim G$):

*

*$H_0: ~ P(X_1 \gt X_2) + \frac{1}{2} P(X_1 = X_2) = \frac{1}{2}.$

*$H_1: ~ P(X_1 \gt X_2) + \frac{1}{2} P(X_1 = X_2) \neq \frac{1}{2}.$
I am wondering what are the other assumptions of such Mann-Whitney test? (besides equality of variances and independence of samples; if we want to use this test for some at least ordinal data, i.e. not necessarily continuous)

In the famous article by Fay and Proschan (2010) there is a very similar formalization (perspective) of the Mann-Whitney test which is given for continuous data:

where $\Psi_C$ is the set of all continuous distributions, $H_3$ is the null and $K_3$ is the alternative, $\mathrm{P} = H_3 \sqcup K_3$ is the full set of allowed distributions.
The assumption of equal variances (which I mentioned earlier, see the beginning of this post) is one of the requirements which are introduced to guarantee that $\mathrm{P}$ will not contain distributions with both $\phi(F,G) = 1/2$ and $F \neq G$. And I want to know what are the other assumptions (besides equality of variances) we need to guarantee that.
Indeed, according to article by Karch (2021), "The assumptions for the different perspectives are all a special case of the Mann-Whitney test’s core assumption, exchangeability. In the Mann-Whitney test setting, exchangeability reduces to if the null hypothesis is true, the two population distributions must be identical." In other words, different perspectives have different null hypotheses but in each case the full set of the allowed distributions $\mathrm{P}$ shouldn't contain distributions $(F,G)$ for which it is possible to have $F \neq G$ under the null. That's why for each perspective we have different set of assumptions (i.e. restrictions on $\mathrm{P}$) to guarantee that.
Fay and Proschan require continuous distributions here (although they defined $\phi(F,G)$ both for discrete and continuous distributions). I guess that they require this because the consistency of Mann-Whitney test is strictly proved only for continuous distributions. However, the article by Divine et al. shows that the aforementioned  formalization of Mann-Whitney test (it is given at the beginning of my post as well as hyperlink to the article) is perfectly valid for discrete data (which possibly contain many ties).

The null hypothesis of the MW-test, under which the distribution of the test statistic is computed, is that $H_0:\ F=G$, the two distributions are the same. This obviously implies that their variances are the same, but the latter "assumption" doesn't actually add anything (see below though). It is also assumed that data are i.i.d.
I think the confusion about ties comes from imprecision of what is actually meant when referring to the MW-test, just the test statistic, or also the distribution under the $H_0$. If there are ties, both asymptotically and for finite samples, the distribution under the $H_0$ that is used for testing has to be modified. This can be done (so the test can be applied), however the test can be seen as invalid if this is not done.
Now how about the "equal variances" assumption? I have mentioned the null hypothesis, however one can state that a valid test does not only require that the distribution under $H_0$ is correctly specified, but also that it has some properties under the alternative. Something of a minimal requirement is that the test should be unbiased, i.e., that the probability to reject under any distribution in the alternative should not be smaller than $\alpha$, the probability to reject under the $H_0$. Unbiasedness follows easily for the alternative that I have learnt (and that is one of the possibilities mentioned in Fay and Proschan), which is that $F$ is stochastically larger than $G$ (i.e., the cdf of $F$ is everywhere smaller or equal than that if $G$, and somewhere smaller). This does not require equal variances, and neither does "Perspective 3" as cited above from Fay and Proschan. Although there are examples of pairs of distributions with unequal variances with $F\neq G$ and $P(X_1>X_2)+\frac{1}{2}P(X_1=X_2)=\frac{1}{2}$ (I believe though I haven't checked that this holds for two Gaussian distributions with equal mean and different variances), I don't think it makes sense to say that the MW-test "assumes equal variances". Computation of the distribution of the test statistic under $H_0$ assumes even more than that, and the valid alternatives stated above against which the test in unbiased contain many pairs of distributions with unequal variances.
In fact one could state that using the first Alternative given in the question (which amounts to Fay and Proschan's Perspective 3) there is no further assumption beyond i.i.d. at all, as this contains all distributions. But Julian Karch (see his answer) has shown that the MW-test is not generally unbiased against this alternative. If you are really interested in this alternative, he recommends the Brunner-Munzel test.
However there may be assumptions implied by certain interpretations that are given to the test result, so this is something to be careful about. If for example a rejection of the null hypothesis is taken as evidence that $F$ is stochastically larger than $G$, one should know that the test also is unbiased against some alternatives for which this isn't the case, and it is implicitly assumed that these do not obtain (one such possibility would be Gaussian distributions with different means and different variances - this belongs to the "Perspective 3" alternative as far as I can see, but not to the "stochastically larger" alternative). Also, as Fay and Proschan mention, there are distributions for which $F\neq G$ and $P(X_1>X_2)+\frac{1}{2}P(X_1=X_2)=\frac{1}{2}$, which cannot be detected by the MW-test (although it is not so clear whether the user in such a case rather would want to reject, or whether they'd be happy to say that there is no evidence that one distribution tends to be larger than the other).
The MW-test can be safely used to test $F=G$ against the "stochastically larger"-alternative, which is how I think most people would interpret the test result, i.e., $F$ tends to produce systematically larger (or smaller) observations than $G$. The issue here is that not everything that is possible is covered, i.e., in reality it may be the case that $F\neq G$ but none of them is stochastically larger than the other, for example $F$ may produce more very large and more very small observations than $G$. In a real application I'd therefore look at visualisation such as boxplots and histograms to see whether this might be the case, and interpret results with caution.
Summarising, Fay and Proschan's distinction of different "perspectives" is important, because in fact different perspectives make different implicit assumptions when interpreting the test result, and not being aware of this may lead to misinterpretation. One could say that running the test itself, mathematically, does not require such assumptions (one can just take as null hypothesis all distributions that have rejection probability $\le\alpha$ and as alternative all those for which the rejection probability is larger), but making sense of the result does.
