***TASK*** the task is to summarize an input biomedical literature in six sentences 
    ***INPUT*** the input is a biomedical literature 
    ***OUTPUT*** the output is the summary of an input biomedical literature in six sentences 
    ***DOCUMENTATION*** 
    ***EXAMPLES*** 
    Input: Introduction A number of breast cancer risk prediction models have been developed to provide insight into a woman 's individual breast cancer risk . Although circulating levels of estradiol in postmenopausal women predict subsequent breast cancer risk , whether the addition of estradiol levels adds significantly to a model 's predictive power has not previously been evaluated . Methods Using linear regression , the authors developed an imputed estradiol score using measured estradiol levels ( the outcome ) and both case status and risk factor data ( for example , body mass index ) from a nested case-control study conducted within a large prospect i ve cohort study and used multiple imputation methods to develop an overall risk model including both risk factor data from the main cohort and estradiol levels from the nested case-control study . Results The authors evaluated the addition of imputed estradiol level to the previously published Rosner and Colditz log-incidence model for breast cancer risk prediction within the larger Nurses ' Health Study cohort . The follow-up was from 1980 to 2000 ; during this time , 1,559 invasive estrogen receptor-positive breast cancer cases were confirmed . The addition of imputed estradiol levels significantly improved risk prediction ; the age-specific concordance statistic increased from 0.635 ± 0.007 to 0.645 ± 0.007 ( P < 0.001 ) after the addition of imputed estradiol . Conclusion Circulating estradiol levels in postmenopausal women appear to add to other lifestyle factors in predicting a woman 's individual risk of breast cancer Key Summary Points Risk prediction models are statistical models used to predict the probability of an outcome on the basis of the values of 1 or more risk factors ( markers ) . The accuracy of the model 's predictions is typically summarized with statistics that describe the model 's discrimination and calibration . Risk stratification tables are a more informative way to assess and compare the models . The tables illustrate the distribution of predictions across risk categories . That illustration allows users to assess 3 key measures of the models ' value for guiding medical decisions : the models ' calibration , ability to stratify people into clinical ly relevant risk categories , and accuracy at classifying patients into higher- and lower-risk categories . This information is contained in the margins of the risk stratification table rather than in its cells . The tables should only be used to compare risk prediction models when one of the models contains all of the markers that are contained in the other ( nested models ) ; they should not be used to compare models with different sets of markers ( nonnested models ) . The table predictions require corrections when casecontrol data are used . The recent epidemiologic and clinical literature is filled with studies evaluating statistical models that predict risk for disease or some other adverse event ( 15 ) . Because risk prediction models are intended to help patients and clinicians make decisions , evaluation of these models requires methods that differ from those used to assess models describing disease etiology . This is because the characteristics of the models are less important than their value for guiding decisions . Cook and colleagues ( 1 , 6 ) recently proposed a new approach to evaluate risk prediction models : a risk stratification table . This methodology appropriately focuses on the key purpose of a risk prediction model , which is to classify individuals into clinical ly relevant risk categories , and it has therefore been widely adopted in the literature ( 24 ) . We examine the risk stratification approach in detail in this article , identifying the relevant information that can be abstract ed from a risk stratification table and caution ing against misuses of the method that frequently occur in practice . We use a recently published study of a breast cancer risk prediction model by Tice and colleagues ( 2 ) to illustrate the concepts . Background A risk prediction marker is any measure that is used to predict a person 's risk for an event . It may be a quantitative measure , such as high-density lipoprotein cholesterol level , or a qualitative measure , such as family history of disease . Risk predictors are also risk factors , in the sense that they will necessarily be strongly associated with the risk for disease . But a large , significant association does not assure that the marker has value in predicting risk for many people . A risk prediction model is a statistical model that combines information from several markers . Common types include logistic regression models , Cox proportional hazard models , and classification trees . Each type of model produces a predicted risk for each person by using information in the model . Consider , for example , a model predicting breast cancer risk that includes age as the only predictor . The result ing risk prediction for a woman of a given age is simply the proportion of women her age who develop breast cancer . The woman 's predicted risk will change if more information is included in the model . For instance , if family history information is added , her predicted risk will be the proportion of women her age and with her family history who develop breast cancer . The purpose of a risk prediction model is to accurately stratify individuals into clinical ly relevant risk categories . This risk information can be used to guide clinical or policy decisions , for example , about preventive interventions for persons or disease screening for sub population s identified as high risk , or to select persons for inclusion in clinical trials . The value of a risk prediction model for guiding these kinds of decisions can be judged by the extent to which the risk calculated from the model reflects the fraction of persons in the population with actual events ( its calibration ) ; the proportions in which the population is stratified into clinical ly relevant risk categories ( its stratification capacity ) ; and the extent to which participants with events are assigned to high-risk categories and those without events are assigned to low-risk categories ( its classification accuracy ) . Risk prediction models are commonly evaluated by using the receiver-operating characteristic ( ROC ) curve ( 4 , 7 ) , which is a st and ard tool for evaluating the discriminatory accuracy of diagnostic or screening markers . This curve shows the true-positive rate plotted against the false-positive rate for rules that classify persons by using risk thresholds that vary over allpossible values . Receiver-operating characteristic curves are generally not helpful for evaluating risk prediction models because they do not provide information about the actual risks that the models predict or about the proportion of participants who have high or low risk values . Moreover , when comparing ROC curves for 2 risk prediction models , the models are aligned according to their false-positive rates ( that is , different risk thresholds are applied to the 2 models to achieve the same false-positive rate ) . This is clearly inappropriate . In addition , the area under the ROC curve or c-statistic , a commonly reported summary measure that can be interpreted as the probability that the predicted risk for a participant with an event is higher than that for a participant without an event , has little direct clinical relevance . Clinicians are never asked to compare risks for a pair of patients one who will eventually have the event and one who will not . Neither the ROC curve nor the c-statistic relates to the practical task of predicting risks for clinical decision making . Cook and colleagues ( 1 , 6 ) propose using risk stratification tables to evaluate the incremental value of a new marker , or the benefit of adding a new marker ( for example , C-reactive protein ) , to an established set of risk predictors ( for example , Framingham risk predictors , such as age , diabetes , cholesterol level , smoking , and low-density lipoprotein cholesterol levels ) . In these stratification tables , risks calculated from models with and without the new marker are cross-tabulated . This approach represents a substantial improvement over the use of ROC methodology because it displays the risks calculated by use of the model and the proportions of individuals in the population who are stratified into the risk groups . We will provide an example of this approach and show how information about model calibration , stratification capacity , and classification accuracy can be derived from a risk stratification table and used to assess the added value of a marker for clinical and health care policy decisions . Example Tice and colleagues ( 2 ) published a study that builds and evaluates a model for predicting breast cancer risk by using data from 1095484 women in a prospect i ve cohort and incidence data from the Surveillance , Epidemiology , and End Results data base . Age , race or ethnicity , family history , and history of breast biopsy were used to model risk with a Cox proportional hazard model . The study focused on the benefit of adding breast density information to the model . The hazard ratio for breast density in the multivariate model ( extremely dense vs. almost entirely fat ) was estimated as 4.2 for women younger than age 65 years and 2.2 for women age 65 years or older . This suggests that breast density is strongly associated with disease riskthat is , that breast cancer rates are higher among women with higher breast density . However , it does not describe the value of breast density for helping women make informed clinical decisions , which requires knowledge of the frequency distribution of breast density in the population . To evaluate the added value of breast density , Tice and colleagues defined 5-year breast cancer risk categories as low ( # # lt##1 % ) , low to intermediate ( 1 % to 1.66 % ) , intermediate to high ( 1.67 % to 2.5 % ) , and high ( # # gt##2.5 % ) . The 1.67 % cutoff for intermediate risk was presumably chosen on the basis of recommendations by the American Society of Clinical Oncology ( 8) and the Canadian Task Force on Preventive Health Care ( 9 ) to counsel women with 5-year risks greater than this threshold about considering tamoxifen for breast cancer prevention . Tice and colleagues used a risk stratification table ( Table 1 ) to compare risk prediction models with and without breast density . Table 1 . Five-Year Risks for Breast Cancer as Predicted by Models That Do and Do Not Include Breast Density Calibration Assessing model calibration is an important first step in evaluating any risk prediction model . Good calibration is essential ; it means that the model-predicted probability of an event for a person with specified predictor values is the same as or very close to the proportion of all persons in the population with those same predictor values who experience the event ( 10 ) . With many predictors , and especially with continuous predictors , we can not evaluate calibration at each possible predictor value because there are too few participants with exactly those values . Instead , the st and ard approach is to place persons within categories of predicted risk and to compare the category values with the observed event rates for participants in each category . The calibration of the risk prediction models for breast cancer can be assessed by comparing the proportions of events in the margins of Table 1 with the corresponding row and column labels . For the model without breast density , the proportions of observed events within each risk category are in the far-right Total column and they generally agree Objectives To develop a predictive model for pre-eclampsia based on clinical risk factors for nulliparous women and to identify a subgroup at increased risk , in whom specialist referral might be indicated . Design Prospect i ve multicentre cohort . Setting Five centres in Auckl and , New Zeal and ; Adelaide , Australia ; Manchester and London , United Kingdom ; and Cork , Republic of Irel and . Participants 3572 “ healthy ” nulliparous women with a singleton pregnancy from a large international study ; data on pregnancy outcome were available for 3529 ( 99 % ) . Main outcome measure Pre-eclampsia defined as ≥140 mm Hg or diastolic blood pressure ≥90 mm Hg , or both , on at least two occasions four hours apart after 20 weeks ’ gestation but before the onset of labour , or postpartum , with either proteinuria or any multisystem complication .  Preterm pre-eclampsia was defined as women with pre-eclampsia delivered before 37 + 0 weeks ’ gestation . In the stepwise logistic regression the comparison group was women without pre-eclampsia . Results Of the 3529 women , 186 ( 5.3 % ) developed pre-eclampsia , including 47 ( 1.3 % ) with preterm pre-eclampsia . Clinical risk factors at 14 - 16 weeks ’ gestation were age , mean arterial blood pressure , body mass index ( BMI ) , family history of pre-eclampsia , family history of coronary heart disease , maternal birth weight , and vaginal bleeding for at least five days . Factors associated with reduced risk were a previous single miscarriage with the same partner , taking at least 12 months to conceive , high intake of fruit , cigarette smoking , and alcohol use in the first trimester . The area under the receiver operating characteristics curve ( AUC ) , under internal validation , was 0.71 . Addition of uterine artery Doppler indices did not improve performance ( internal validation AUC 0.71 ) . A framework for specialist referral was developed based on a probability of pre-eclampsia generated by the model of at least 15 % or an abnormal uterine artery Doppler waveform in a subset of women with single risk factors . Nine per cent of nulliparous women would be referred for a specialist opinion , of whom 21 % would develop pre-eclampsia . The relative risk for developing pre-eclampsia and preterm pre-eclampsia in women referred to a specialist compared with st and ard care was 5.5 and 12.2 , respectively . Conclusions The ability to predict pre-eclampsia in healthy nulliparous women using clinical phenotype is modest and requires external validation in other population s. If vali date d , it could provide a personalised clinical risk profile for nulliparous women to which biomarkers could be added . Trial registration ACTRN12607000551493 The analytical effect of the number of events per variable ( EPV ) in a proportional hazards regression analysis was evaluated using Monte Carlo simulation techniques for data from a r and omized trial containing 673 patients and 252 deaths , in which seven predictor variables had an original significance level of p < 0.10 . The 252 deaths and 7 variables correspond to 36 events per variable analyzed in the full data set . Five hundred simulated analyses were conducted for these seven variables at EPVs of 2 , 5 , 10 , 15 , 20 , and 25 . For each simulation , a r and om exponential survival time was generated for each of the 673 patients , and the simulated results were compared with their original counterparts . As EPV decreased , the regression coefficients became more biased relative to the true value ; the 90 % confidence limits about the simulated values did not have a coverage of 90 % for the original value ; large sample properties did not hold for variance estimates from the proportional hazards model , and the Z statistics used to test the significance of the regression coefficients lost validity under the null hypothesis . Although a single boundary level for avoiding problems is not easy to choose , the value of EPV = 
    Output:
    
Most models yielded relatively poor discrimination in both internal and external validation .
This poor discriminatory accuracy of existing models might be because of a lack of knowledge about risk factors , heterogeneous subtypes of breast cancer , and different distributions of risk factors across population s. In addition the concordance statistic itself is insensitive to measure the improvement of discrimination .