Introduction The Office of Forecasting, Research, and Analysis (OFRA) produces two caseload forecasts each year, one in the spring and one in the fall. The purpose of these forecasts is to estimate the number of clients anticipated to access DHS/OHA services in order to budget accurately, inform program management, and support oversight. This annual report analyzes the quality of the historical forecasts by program and provides information about consistency, trends and areas for improvement. This information should be useful to individuals who use the caseload forecasts and want to know how reliable they have been in the past. It also provides feedback to the forecasters themselves about how accurate they have been and where adjustments may be appropriate. FORECAST QUALITY MEASURES This report uses two indicators of forecast quality: statistical bias and accuracy. Other desirable characteristics of forecast quality such as the efficiency with which a forecaster uses available information are harder to assess. indicates the tendency of a forecast to over- or under-estimate real outcomes. To Statistical bias measure statistical bias, OFRA uses the - the arithmetic average Mean Percentage Error (MPE) of the forecasting errors. A small MPE can be produced by either small errors or errors (even large ones) which are offset by errors in the opposite direction. MPE imperfectly measures the quality of a forecast. However, for budgeting purposes, MPE is probably the best measure of quality since what matters is total spending over time, and savings from one month are frequently used to offset excess costs in another month. The accuracy of a forecast is the degree to which its values are narrowly dispersed around actual outcomes. To measure accuracy OFRA uses the the Mean Absolute Percentage Error (MAPE) – average of the forecasts’ errors without regard to arithmetic sign. From a technical perspective, MAPE is usually considered a better measure of quality since it measures errors in all directions. Both quality measures (MPE and MAPE) are calculated for 14 sequential forecast cycles (Spring 2008 through Fall 2014) for the first 6, 12, and 24 months of each forecast. Theoretically, and in practice, the quality of a forecast diminishes the further out the forecast projects. In an environment of changing policies and economic uncertainty it is hard to produce precise term forecasts. To address this challenge, OFRA produces a new forecast twice each year - in spring and in fall. Each forecast produces caseload estimates on a month-by-month basis for the remainder of the current biennium and the full upcoming biennium.
Introduction The Office of Forecasting, Research, and Analysis (OFRA) produces two caseload forecasts each year, one in the spring and one in the fall. The purpose of these forecasts is to estimate the number of clients anticipated to access DHS/OHA services in order to budget accurately, inform program management, and support oversight. This annual report analyzes the quality of the historical forecasts by program and provides information about consistency, trends and areas for improvement. This information should be useful to individuals who use the caseload forecasts and want to know how reliable they have been in the past. It also provides feedback to the forecasters themselves about how accurate they have been and where adjustments may be appropriate. FORECAST QUALITY MEASURES This report uses two indicators of forecast quality: statistical bias and accuracy. Other desirable characteristics of forecast quality such as the efficiency with which a forecaster uses available information are harder to assess. Statistical bias indicates the tendency of a forecast to over- or under-estimate real outcomes. To measure statistical bias, OFRA uses the Mean Percentage Error (MPE) - the arithmetic average of the forecasting errors. A small MPE can be produced by either small errors or errors (even large ones) which are offset by errors in the opposite direction. MPE imperfectly measures the quality of a forecast. However, for budgeting purposes, MPE is probably the best measure of quality since what matters is total spending over time, and savings from one month are frequently used to offset excess costs in another month. The accuracy of a forecast is the degree to which its values are narrowly dispersed around actual outcomes. To measure accuracy OFRA uses the Mean Absolute Percentage Error (MAPE) – the average of the forecasts’ errors without regard to arithmetic sign. From a technical perspective, MAPE is usually considered a better measure of quality since it measures errors in all directions. Both quality measures (MPE and MAPE) are calculated for 14 sequential forecast cycles (Spring 2008 through Fall 2014) for the first 6, 12, and 24 months of each forecast. Theoretically, and in practice, the quality of a forecast diminishes the further out the forecast projects. In an environment of changing policies and economic uncertainty it is hard to produce precise forecasts. To address this challenge, OFRA produces a new forecast twice each year - in spring and in fall. Each forecast produces caseload estimates on a month-by-month basis for the remainder of the current biennium and the full upcoming biennium. term
Annual%20Forecast%20Quality%20Report%20-%202015.docx