What statistics are good for measuring the tail? We're working with latency data and people are concerned with the worst case latencies. They're using the 99th and 99.9th percentile as KPIs (key performance indicators). We know that an unbiased estimator of even the median doesn't exist: An unbiased estimate of the median. And the bias issue becomes much worse for the high percentiles than it is for the median. In contrast, the sample mean is an unbiased estimator. To an extent, the sample mean will catch changes in the tail as well. I was wondering if there are any alternate KPI's that do better on the bias and yet are also focused on the tail. The conditional sample mean (conditional on the samples being larger than some number) was one measure that came to mind. But then the question is, what should the conditional threshold be? Should it be some static number or again a percentile? Any other descriptive statistics that focus on the tail and yet do better on the bias issue?

Probably this falls into topic of risk measures as mentioned by Glen_b. Expected Shortfall could be used. But it isn't a coherent risk measure. Entropic value at risk (EVaR) would be an option for coherent measure, if coherency axiom's are valid/important for latency measurement.
