question: Facing the rapidly spreading novel coronavirus disease (COVID-19), evidence to inform decision-making at both the clinical and policy-making level is highly needed. Based on the results of a study by Petrilli et al, we have developed a calculator using patient data at admission to predict the risk of critical illness (intensive care unit admission, use of mechanical ventilation, discharge to hospice, or death). We report a retrospective validation of the risk calculator on 145 consecutive patients admitted with COVID-19 to a single hospital in Israel. Of the 18 patients with critical illness, 17 were correctly identified by the model(sensitivity: 94.4%, 95% CI, 72.7% to 99.9%; specificity: 81.9%, 95% CI, 74.1% to 88.2%). Of the 127 patients with non-critical illness, 104 were correctly identified. This, despite considerable differences between the original and validation study populations. Our results show that data from published knowledge can be used to provide reliable, patient level, automated risk assessment, potentially reducing the cognitive burden on physicians and helping policy makers better prepare for future needs.