  In this article I recommend a better point estimator for Krippendorff's Alpha
agreement coefficient, and develop a jackknife variance estimator that leads to
much better interval estimation than does the customary bootstrap procedure or
an alternative bootstrap procedure. Having developed the new methodology, I
analyze nominal data previously analyzed by Krippendorff, and two
experimentally observed datasets: (1) ordinal data from an imaging study of
congenital diaphragmatic hernia, and (2) United States Environmental Protection
Agency air pollution data for the Philadelphia, Pennsylvania area. The latter
two applications are novel. The proposed methodology is now supported in
version 2.0 of my open source R package, krippendorffsalpha, which supports
common and user-defined distance functions, and can accommodate any number of
units, any number of coders, and missingness. Interval computation can be
parallelized.

Methodology