Method of using auxiliary information for better results
In statistics, benchmarking is a method of using auxiliary information to adjust the sampling weights used in an estimation process, in order to yield more accurate estimates of totals.
Suppose we have a population where each unit formula_1 has a "value" formula_2 associated with it. For example, formula_2 could be a wage of an employee formula_1, or the cost of an item formula_1. Suppose we want to estimate the sum formula_6 of all the formula_2. So we take a sample of the formula_1, get a sampling weight "W"("k") for all sampled formula_1, and then sum up formula_10 for all sampled formula_1.
One property usually common to the weights formula_12 described here is that if we sum them over all sampled formula_1, then this sum is an estimate of the total number of units formula_1 in the population (for example, the total employment, or the total number of items). Because we have a sample, this estimate of the total number of units in the population will differ from the true population total. Similarly, the estimate of total formula_6 (where we sum formula_10 for all sampled formula_1) will also differ from true population total.
We do not know what the true population total formula_6 value is (if we did, there would be no point in sampling!). Yet often we do know what the sum of the formula_12 are over all units in the population. For example, we may not know the total earnings of the population or the total cost of the population, but often we know the total employment or total volume of sales. And even if we don't know these exactly, there often are surveys done by other organizations or at earlier times, with very accurate estimates of these auxiliary quantities. One important function of a population census is to provide data that can be used for benchmarking smaller surveys.
The benchmarking procedure begins by first breaking the population into benchmarking cells. Cells are formed by grouping units together that share common characteristics, for example, similar formula_2, yet anything can be used that enhances the accuracy of the final estimates. For each cell formula_21, we let formula_22 be the sum of all formula_12, where the sum is taken over all sampled formula_1 in the cell formula_21. For each cell formula_21, we let formula_27 be the auxiliary value for cell formula_21, which is commonly called the "benchmark target" for cell formula_21. Next, we compute a benchmark factor formula_30. Then, we adjust all weights formula_12 by multiplying it by its benchmark factor formula_32, for its cell formula_21. The net result is that the estimated formula_34 [formed by summing formula_35] will now equal the benchmark target total formula_36. But the more important benefit is that the estimate of the total of formula_6 [formed by summing formula_38] will tend to be more accurate.
Relationship to stratified sampling.
Benchmarking is sometimes referred to as 'post-stratification' because of its similarities to stratified sampling. The difference between the two is that in stratified sampling, we decide "in advance" how many units will be sampled from each stratum (equivalent to benchmarking cells); in benchmarking, we select units from the broader population, and the number chosen from each cell is a matter of chance.
The advantage of stratified sampling is that the sample numbers in each stratum can be controlled for desired accuracy outcomes. Without this control, we may end up with too much sample in one stratum and not enough in another – indeed, it's possible that a sample will contain "no" members from a certain cell, in which case benchmarking fails because formula_39, leading to a divide-by-zero problem. In such cases, it is necessary to 'collapse' cells together so that each remaining cell has an adequate sample size.
For this reason, benchmarking is generally used in situations where stratified sampling is impractical. For instance, when selecting people from a telephone directory, we can't tell what age they are so we can't easily stratify the sample by age. However, we can collect this information from the people sampled, allowing us to benchmark against demographic information.