Can you help me determine if these two groups are statistically different? I have two groups, A & B. Let's say A are the users who wrote a review for place A, and B for place B.
For each user $u$ I have recorded his total number of reviews $r_u$. Since we're talking about number of reviews, which is a very skewed variable, I thought it would be better to use the natural logarithm $\log(r_u) = \hat{r_u}$.
Now, I suspect that the users who write for A are "fake" or at least "green" accounts with 1 or very few reviews, and I want to declare whether these users are statistically significant from those who write for B. From a visual inspection, it seems clearn that something is off (black trace = A; green trace = B).

Still, I would like to formalize this difference. I need to compare $R_A = \{\hat{r_u} \ \forall \ u \in A \}$ and $R_B = \{\hat{r_u} \ \forall \ u \in B \}$ somehow, but I am unsure on how to do it properly from a statistical point of view.
Some doubts and questions I have in mind (I am fully aware that it's not one specific question, I'm sorry about that):

*

*is it appropriate to test for difference of means? I mean, should I really compare means to show what I want to show? What about a given percentile? Or maybe compare proportions of 1-review users in the two groups? What would you go for? Would you go for more than one?

*if I compare means I'll go for an independent group t-test but I think it is clear that in my case the assumption of normality doesn't hold. Would you go with a non-parametric, e.g. the Mann-Whitney test?

*what's a good sample size? and how much do samples' sizes matter? For example, now I have gathered $|R_A| \approx 400$ and $|R_B| \approx 700$ and I am not really sure what to do with that (because I could even gather more for both). Should I sample $\approx$ 400 records from $R_B$ to level out the groups?

Thank you for your help.

*

*What you want to test depends on your substantive question. Based on your explanation, it looks to me like a test for whether the proportions of 1-review users differs between the groups might be helpful. You can simply set up a 2-by-2 contingency table and use Pearson's $\chi^2$ test. Alternatively, you could test "low numbers of reviews", e.g., 1-2 vs. 3 or more reviews.


*If you want to test differences in means, the t-test can be used even if the distributions are heavily non-normal as long as the sample size is "reasonably large". Your 400-700 is definitely fine.


*More data always gives us more information. Just be aware that tiny observed differences will be statistically significant if your sample size is large. I would say that you can probably be reasonably confident in your results based on your current sample size. Just be sure to also report the effect size ("50% of the reviewers for A had only a single review to their name, against 10% of the reviewers for B, and the difference is statistically significant, $p=\dots$").
