Correct way to compare groups of ratings generated by same expert Suppose I ask an expert to assign Likert-scale ratings to sets of 100 objects from 2 different populations. Rating distributions are very far away from normality. I now want to determine whether there is a statistically significant difference between the mean scores of ratings of population 1 and population 2. Now, the first thing that comes to mind is to use a Mann-Whitney test (I know technically it does not compare averages, but you get the point). However, I am confusing myself due to the fact that the subject generating sets of ratings is the same. Does this setup violate Mann-Whitney's assumption of independent samples? Should I look into a different test?

You consider the scores conditioned on them coming from the same expert. That makes them independent. If you had scores from several experts, then the marginalized scores of two objects could be dependent.
