2012 Spring Forecast Experiment: Forecast Verification Metrics 1.) Traditional, Dichotomous (2-category) Evaluation (excerpted from the WWRP/WGNE Group on Forecast Verification Research website on Forecast Verificaiton: Issues, Methods and FAQ (http://www.cawcr.gov.au/projects/verification/) For dichotomous variables (e.g., precipitation/reflectivity amount above or below a threshold) on a grid, typically the forecasts are evaluated using a diagram like the one shown in Fig. 1. In this diagram, the area represents the intersection between the forecast and observed “H” areas, or the area of Hits; represents the observed area that was missed by the forecast area, “M” or the and represents the part of the forecast that did not overlap an area of “Misses”; “F” observed precipitation, or the Alarm” area. A fourth area is the area outside both the “False forecast and observed regions, which is often called the area of Nulls” or “Correct “Correct Rejections”. This situation can also be represented in a table” like the one shown in Table “contingency 1. In this table the entries in each represent the counts of hit, misses, false alarms, and “cell” correct rejections. The counts in this table can be used to compute a variety of traditional verification measures, described in the following sub-sections. Table 1. Contingency table illustrating the counts used in verification statistics for dichotomous (e.g., Yes/No) forecasts and observations. The values in parentheses illustrate the combination of forecast value (first digit) and observed value. For example, YN signifies a Yes forecast and and a No observation. Observed Forecast Yes No Yes Hits (YY) alarms (YN) YY + YN No Misses (NY) Correct rejections (NN) NY + NN YY + NY YN + NN = YY + YN + NY + NN Critical Success (CSI) Also known as Threat Score (TS). CSI=TS=Hits/(Hits+Misses+False alarms)
2012 Spring Forecast Experiment: Forecast Verification Metrics 1.) Traditional, Dichotomous (2-category) Evaluation (excerpted from the WWRP/WGNE Group on Forecast Verification Research website on Forecast Verificaiton: Issues, Methods and FAQ (http://www.cawcr.gov.au/projects/verification/) For dichotomous variables (e.g., precipitation/reflectivity amount above or below a threshold) on a grid, typically the forecasts are evaluated using a diagram like the one shown in Fig. 1. In this diagram, the area “H” represents the intersection between the forecast and observed areas, or the area of Hits; “M” represents the observed area that was missed by the forecast area, or the “Misses”; and “F” represents the part of the forecast that did not overlap an area of observed precipitation, or the “False Alarm” area. A fourth area is the area outside both the forecast and observed regions, which is often called the area of “Correct Nulls” or “Correct Rejections”. This situation can also be represented in a “contingency table” like the one shown in Table 1. In this table the entries in each “cell” represent the counts of hit, misses, false alarms, and correct rejections. The counts in this table can be used to compute a variety of traditional verification measures, described in the following sub-sections. Table 1. Contingency table illustrating the counts used in verification statistics for dichotomous (e.g., Yes/No) forecasts and observations. The values in parentheses illustrate the combination of forecast value (first digit) and observed value. For example, YN signifies a Yes forecast and and a No observation. Forecast Observed Yes No Yes Hits (YY) alarms (YN) YY + YN No Misses (NY) Correct rejections (NN) NY + NN YY + NY YN + NN = YY + YN + NY + NN Critical Success (CSI) Also known as Threat Score (TS). CSI=TS=Hits/(Hits+Misses+False alarms)
SkillScoresDescriptionSE2012.docx