National Institutes of Health Pathways to Prevention Workshop: Methods for Evaluating Natural Experiments in Obesity
The rapid increase in overweight and obesity poses a significant and worsening public health threat. About 70% of U.S. adults are overweight or obese; racial/ethnic minorities and persons from low socioeconomic backgrounds are disproportionately affected. The complexity of obesity prevention and control requires addressing its multilevel and interrelated drivers. Evidence on effectiveness is needed to guide interventions to reduce overweight and obesity. Structured, public healthoriented frameworks have considered all influences on energy balance, including individual factors, behavioral settings, community and systems level influences, and social norms and values. A 2012 Institute of Medicine report (1) that focused on accelerating progress in obesity control recommended a systems approach to target critical environments for change. Policy actions or programs that are designed to change real world environments and influence obesity related outcomes can be evaluated as natural experiments. As defined by the Medical Research Council, natural experiments examine natural or unplanned variation in exposures by using design and analytic features that can support causal inference (2) Natural experiments include laws or policies (such as taxes on sugar sweetened beverages) programs implemented at a community level (such as park based physical activity programs) and environmental change (for example, construction of dedicated bicycle lanes) On 5 and 6 December 2017, the National Institutes of Health (NIH) convened the Pathways to Prevention (P2P) Workshop: Methods for Evaluating Natural Experiments in Obesity to identify the status of methods used to assess natural experiments to reduce obesity, areas in which these methods could be improved, and research needs for advancing the field. The workshop was cosponsored by the NIH Office of Disease Prevention; the National Cancer Institute; the National Institute of Diabetes and Digestive and Kidney Diseases; and the National Heart, Lung, and Blood Institute. A multidisciplinary working group developed the agenda, and the Johns Hopkins Evidence based Practice Center prepared a systematic evidence review (EPC report) (3) that addressed methods used for evaluating natural experiments in obesity. During the workshop, experts and stakeholders gave presentations, panel members asked questions of the experts, and on site and online participants commented during open discussions. After considering the EPC report findings and the workshop proceedings, an independent panel prepared a draft report for public comment and then revised it on the basis of the comments received. When formulating this report, the panel acknowledged the temporally and spatially dynamic nature of the obesity epidemic, which varies in severity across populations and contexts. For example, the epidemic of childhood obesity varies widely across groups defined by race, ethnicity, and income, and its rise predates the sedentary behavior associated with video game use and screen time. This heterogeneity is an overarching challenge in translating research findings into action. The goal of obesity research is to produce actionable evidence leading to interventions that will slow and ultimately end an epidemic that is predicted to continue well into the 21st century. To generate the requisite evidence, an integrated, networked, and contextually responsive research enterprise is needed. Creating and sustaining that broad enterprise should be a goal for the NIH, other funders, and stakeholders. This panel report summarizes the workshop, identifies research gaps, and provides recommendations to enhance the methodological rigor of natural experiments. The recommendations are organized around 4 issues: population based data sources and data integration, measurement of obesity related outcomes, study design issues, and cross cutting issues that transcend the key questions of the EPC report and are essential for achieving the goal of generating useful evidence for decision making in an ongoing fashion. Population Based Data Sources and Data Integration Summary of Findings The EPC report found a lack of shared comprehensive surveillance and research databases. Most available databases contain information on schools and communities, but few have a formal or detailed data dictionary that would permit efficient shared use. Presenters noted that many existing data systems are inadequate for evaluating the complex socioecological and biological factors driving the obesity epidemic at the level of granularity needed. Primary and secondary data sources differed in the instruments and approaches they used for data collection and in their follow up time frames. Data linkage and integration of multiple data sources could provide measurements across multiple levels. However, such integrated data systems are unavailable for widespread use. It was also noted that data development efforts are disparate, siloed, and often duplicative because of limited cross sector collaboration. Workshop presenters and participants conveyed a need for collaborative platforms and partnerships for more efficient evaluations of natural experiments with both retrospective and prospective data. They also emphasized the need to improve communication between researchers and policymakers, enhance engagement of policymakers, and prioritize availability of timely data to drive decisions. Recommendations to Increase Integration of Population Based Data Sources Table 1 summarizes recommendations to improve data sources and data integration. To accelerate progress and catalyze innovation, funding agencies should invest in a data ecosystem that lessens the burden of regulatory, privacy, and proprietary concerns that restrict access to and sharing of individual level data and/or geospatial data at lower levels of aggregation than ZIP codes. Building on existing infrastructure (such as the National Collaborative on Childhood Obesity Research) would help support integration of data systems on objective obesity related measures across the life span. The panel also recommends enhancing linkage of electronic health record data with community level data by leveraging advances in interoperability and health information exchanges. Table 1. Recommendations for Improving Data Systems The panel recommends fostering ongoing publicprivate partnerships to harness and coalesce the combined power of population level data in both the private (for example, Google) and public (for example, crime and school data) sectors, with the goal of establishing new and robust surveillance platforms that will enable dynamic and longer term monitoring of natural experiments. The panel urges recognition of relevant data sources outside the health sector, such as those related to agriculture, transportation, and city planning, and the importance of multisector collaboration. The integration of high dimensional data from mobile devices equipped with positioning systems offers promise for capturing dynamic data, which may improve the efficiency and reduce the cost of data collection. Measurement of Obesity Related Outcomes Summary of Findings Studies typically measure body composition, dietary intake, or physical activity. Gold standard measures of these outcomes, such as underwater weighing or doubly labeled water, are rarely used in natural experiments given their high costs and limited feasibility in community settings. Studies reviewed in the EPC report typically measured body composition using body mass index Z score or percentile in children (with adjustment for age and sex) and body mass index in adults. Children's height and weight were typically measured by trained staff, and studies of adults largely used self report. Other measures of body composition, such as waist to hip ratio and skinfold thickness, were not used. The EPC report found that dietary intake was typically measured as change in total daily caloric intake or intake of fruits and vegetables, sugar sweetened beverages, fast food, or fiber. Studies used 24 hour recall, food frequency questionnaires, short dietary questionnaires, record logs, or food diaries. Questionnaires varied widely across studies, limiting comparability and complicating pooling of data. Presenters highlighted the need to validate questionnaires in diverse populations and to develop Web based registries of standardized, valid, and reliable obesity related measures to encourage use of common measures. The National Collaborative on Childhood Obesity Research Measures Registry (www.nccor.org) lists assessment instruments for individual diet, food environment, individual physical activity, and physical activity environment in children. No similar resources exist for measures designed for adults. A workshop presenter discussed the need for more accurate measures of food intake, such as plate waste. Promising new technologies exist and others are being developed to improve the efficiency and accuracy of collection of dietary measures, such as assessment of dietary intake from digital images of meals, digital images accompanied by a voice recording describing the food, and smartphone diary applications. Studies in the EPC report generally assessed single aspects of physical activity, such as step counts, rather than multidimensional aspects of total physical activity. Assessment strategies included questionnaires, electronic monitoring, records or logs, and direct observation. Electronic monitoring and mobile devices (such as video cameras and tracking devices) are increasingly used to improve efficiency and accuracy of data collection. The EPC report found that some studies evaluated co outcomes, including food purchasing behavior, commuting behavior, food environment, and physical activity environment. Recommendations for Improving Measurement of Obesity Related Outcomes Table 2 summarizes recommendations to improve measurement of obesity related outcomes. The p
Annals of Internal Medicine