  A variety of problems in random-effects meta-analysis arise from the
conventional $Q$ statistic, which uses estimated inverse-variance (IV) weights.
In previous work on standardized mean difference and log-odds-ratio, we found
superior performance with an estimator of the overall effect whose weights use
only group-level sample sizes. The $Q$ statistic with those weights has the
form proposed by DerSimonian and Kacker. The distribution of this $Q$ and the
$Q$ with IV weights must generally be approximated. We investigate
approximations for those distributions, as a basis for testing and estimating
the between-study variance ($\tau^2$). Some approximations require the variance
and third moment of $Q$, which we derive. We describe the design and results of
a simulation study, with mean difference as the effect measure, which provides
a framework for assessing accuracy of the approximations, level and power of
the tests, and bias in estimating $\tau^2$. Use of $Q$ with sample-size-based
weights and its exact distribution (available for mean difference and evaluated
by Farebrother's algorithm) provides precise levels even for very small and
unbalanced sample sizes. The corresponding estimator of $\tau^2$ is almost
unbiased for 10 or more small studies. Under these circumstances this
performance compares favorably with the extremely liberal behavior of the
standard tests of heterogeneity and the largely biased estimators based on
inverse-variance weights.

Methodology