I see your point but I don’t really agree. For example, in pytorch I can get the mean and variance of a binomial distribution. However, I could make the same argument: that the support of the binomial distribution, while being represented with some subset of the natural numbers {0, 1, 2, …, n}, is actually defined over a finite set of things that have no inherent order. Thus in this case taking the mean doesn’t make sense either. I get that this is a silly way to think about binomial distributions, but it’s consistent with what you said. I think that it would make sense to have the mean of the categorical distribution defined as sum_x {x*p(x)}. We assume and ordering on the support for all of the other distributions, it makes sense to me to do it for the categorical distribution too.