Why is ANOVA not p-hacking? Say we have some data with many parameters. As an example let's say I'm an not-so-ethical journalist working for a food website and I'm looking to write some clickbait article "backed by science" about how some food or lifestyle is good/bad for you.
My data might contain a set of thousands of people, their socioeconomic status, what they had for breakfast, whether they're vegetarian, whether they prefer tea or coffee, etc. and also what their scores are for an IQ test.
If I split the dataset into many (100+) groups, where group one might split vegetarians/meat eaters, group two might split coffee drinkers and tea drinkers, group 3 might split female coffee drinkers and male tea drinkers etc. If then use ANOVA to compare all groups to see if there's a statistically significant difference, is this not effectively p-hacking?
I understand that ANOVA won't tell us which groups are different, but it seems like it would be a quick way to confirm that there IS a (false) positive in there. And then it's a case of finding out which groups are different.

It is not p-hacking. Multiple groups are compared but only a single hypothesis is tested.
ANOVA computes a ratio of variances and a p-value can be computed for that ratio based on a single hypothesis.
Possibly the idea of 'anova = p-hacking' arises due to the confusing aspect that a hypothesis test is often not used to test a null hypothesis, but instead to give prove/justification for the alternative hypothesis (which can be many at once).
Note that ANOVA has the following properties

*

*ANOVA doesn't tell which of the many groups are different, but only that they are not the same.

*ANOVA is less powerfull than seperate t-test for all sort of combinations of groups.


If I split the dataset into many (100+) groups, where group one might split vegetarians/meat eaters, group two might split coffee drinkers and tea drinkers, group 3 might split female coffee drinkers and male tea drinkers etc. If then use ANOVA to compare all groups to see if there's a statistically significant difference, is this not effectively p-hacking?

It is not clear how you are gonna do this splitting and ANOVA. With ANOVA you can have multiple groups, but these should not be overlapping. You might do something like a linear model with all those variables, but then each additional group/variable is gonna decrease the degrees of freedom and make the ANOVA test less sensitive/powerful.
