Is Average Accuracy An Ill-Defined Metric? Five people each took a 100 question yes/no question test. I have their individual accuracy scores and was planning to report their average.
However, I was informed that "You cannot take an average of percentages derived from categorical variables" and I am trying to understand the full scope of that statement as I've never heard push-back from studies reporting the average accuracy.

It reports exactly what it sounds like, but that’s perhaps not what you want. Imagine getting $1/2$ the first time and $94/98$ the second time. Combined, you are $95/100$, but the average accuracy is $\dfrac{\dfrac{94}{98}+\dfrac{1}{2}}{2}\approx0.73$.
While you’re quite not wrong to say that the average accuracy is $0.73$, you decide which is more representative of your data. I’d go with the $95\%$, and I would expect most people to interpret “average accuracy” this way.
