Fact-checking the death toll estimates from Hurricane Maria in Puerto Rico
By Amy Sherman June 5, 2018
Many news reports latched on to the number 4,645 in a new Harvard University study about the death toll in Puerto Rico from Hurricane Maria — more than 70 times above the government's official count.
While the number grabbed headlines, it requires explanation about the methodology and what the study actually found. The number is an estimate, not a specific count of documented deaths.
Death counts after a disaster are important, because they fuel recovery efforts and planning for the future. For months, researchers have said that the government's official count of 64 from the September storm was an undercount. The Harvard Study, published in the New England Journal of Medicine a few days before the start of the June 1 hurricane season, has attracted a lot of attention.
But it's not the only study that's worked to come up with a more accurate death count; there are actually several studies that have tried to create their own estimates.
"There is growing body of literature and growing scientific consensus that it is not 64," said Mario Marazzi-Santiago, director of the Puerto Rico Institute of Statistics. "The next question is what is the right number, and that will take some time."
The challenge of death estimates
Part of a death toll is a count of direct deaths from a storm, such as being struck by a falling tree or drowning in a flood.
The more complicated count comes from indirect deaths caused by unsafe or unhealthy conditions, such as a lack of electricity cutting off a dialysis machine to a kidney patient.
Usually, the death toll of an event such as Hurricane Maria is determined through an exam by a medical examiner. But Puerto Rico officials did not have the proper resources to effectively conduct forensic examinations.
Without complete medical records, researchers were left with other ways to try to come up with an estimate.
Researchers from Harvard T.H. Chan School of Public Health relied on interviewing people and asking them about causes of death in their households. Researchers selected 3,300 randomly chosen households and found 38 deaths after the hurricane, including three from direct causes and 12 from interruption of necessary medical services. The Harvard researchers extrapolated based on that data about the number of "excess deaths" (the number of deaths compared with the same period during the previous year) and found a 62 percent increase in the mortality rate.
The researchers concluded that there was a range of 793 to 8,498 deaths with a confidence interval of 95 percent. But it was that midpoint number of 4,645 "excess deaths" that drew most of the media attention.
In a question and answer document, the researchers emphasized that there is uncertainty associated with the estimate. The researchers said that they didn't say that 4,645 died: "We provide a 95 percent confidence interval of 793 to 8,498, and 4,645 falls in the middle of this range." (Marazzi-Santiago of the Puerto Rico Institute of Statistics said that the confidence interval means that if samples were taken over and over, 95 percent of the samples would lead to an estimate within the range of about 800 to 8,500.)
We searched the Nexis news database and found more than 100 news articles with the figure of 4,645 or 4,600 in the headline, but far fewer explained that the study actually cited a range.
University of Puerto Rico statistician Roberto Rivera, who along with colleague Wolfgang Rolke used death certificates to estimate a much lower death count, said that indirect estimates should be interpreted with care.
"Note that according to the study the true number of deaths due to Maria can be any number between 793 and 8,498: 4,645 is not more likely than any other value in the range," Rivera said.
Rivera published an analysis in February concluding that in the first six weeks after Hurricane Maria, the death count was between 605 and 1,039.
Following the Harvard study, the government in Puerto Rico released additional data showing there were at least 1,400 additional deaths on the island in the months after the hurricane. But the government didn't explain how many were attributed to the hurricane.
Other researchers have also estimated a death toll in the ballpark of 1,000, including the New York Times.
Alexis R. Santos-Lozada of the Pennsylvania State University was part of the team of demographers in 2017 that found that there were 1,085 added deaths. Santos-Lozada told PolitiFact that his finding doesn't conflict with the Harvard study when examining what it said in full: an estimated range between 800 and 8,000.
"The Harvard study has a range that includes my value within it," he said.
Donald Berry, a biostatistics professor at the University of Texas MD Anderson Cancer Center in Houston, has been one of the biggest critics of the Harvard study. He said that the study compares apples to oranges by comparing actual death rate data in 2016 with data based on interviews in 2017.
"The big thing is the methodology is so completely different, you don't know what you are dealing with," he said. "What you end up with is garbage."
Months ago, the government in Puerto Rico commissioned a study to examine the death toll by the George Washington University Milken Institute School of Public Health. That study is ongoing.
While the Harvard study was based on interviews with households, the Milken Institute study will examine death certificate and other mortality data from September 2017 to the end of February 2018. In a statement, it said it expected its study to produce "a narrower range of uncertainty."
The first phase of the study is expected to be done this summer.
On June 1, Gov. Ricardo Rosselló told CNN's Anderson Cooper he welcomed the results of the Harvard study and does not stand by the official death toll. He emphasized the importance of understanding that that original count was far too low.
"We had established that this number was going to be much, much higher than what we had as an official tally," he said.
UPDATE Sept. 13, 2018: Since this story posted, another study attempting to approximate the death toll was commissioned by the government in Puerto Rico and published by the George Washington University Milken Institute School of Public Health. The study released in August 2018 analyzed death certificates and other mortality data and found an estimated 2,975 excess deaths between September 2017 through the end of February 2018. The team compared the total number of deaths during that time to the expected number based on historical patterns and found that the number was 22 percent higher than would have been expected.
New England Journal of Medicine, "Mortality in Puerto Rico after Hurricane Maria," May 29, 2018
Nishant Kishore, Hurricane Maria Mortality Study FAQ, May 31, 2018
New York Times, "Puerto Rico's Hurricane Maria Death Toll Could Exceed 4,000, New Study Estimates," May 29, 2018
New York Times, "Protesters Demand Audit of Hurricane Maria Death Toll in Puerto Rico," June 2, 2018
New York Times, "Why Are the Death Tolls in Puerto Rico From Hurricane Maria So Different?" June 2, 2018
Washington Post The Fact Checker, "Did exactly 4,645 people die in Hurricane Maria? Nope," June 1, 2018
Los Angeles Times, "Hurricane Maria's death toll was 70 times higher than Puerto Rican officials have reported, study says," May 29, 2018
CNN, "Puerto Rico's governor says there will be 'hell to pay' if officials withheld mortality data," June 1, 2018
AP, "Puerto Rico agency sues government to obtain death data," June 1, 2018
Buzzfeed, "Puerto Rico's Government Just Admitted 911 People Died After The Hurricane — Of 'Natural Causes,'" Oct. 17, 2017
The Conversation, "Why Puerto Rico's death toll from Hurricane Maria is so much higher than officials thought," Jan. 3, 2018
George Washington University, Milken Institute School of Public Health Statement on New Study to Estimate Excess Mortality in Puerto Rico Tied to Hurricane Maria, May 30, 2018
George Washington University, Milken Institute SPH Project Will Estimate the Excess Mortality in Puerto Rico Tied to Hurricane Maria, Feb. 22, 2018
Roberto Rivera and Wolfgang Rolke, "Estimates of excess deaths in Puerto Rico following Hurricane María," February 2018
Alexis Santos-Lozada and Jeffrey Howard, "Estimates of excess deaths in Puerto Rico following Hurricane Maria," November 21, 2017
Interview, Hugh E. Willoughby, Florida International University, Department of Earth and Environment, June 4, 2018
Interview, Hugh Gladwin, Florida International University, anthropologist and associate professor in the department of Global and Sociocultural Studies, June 4, 2018
Interview, William Booher, FEMA spokesman, June 4, 2018
Interview, Dr. Mario Marazzi-Santiago, Executive Director of the Puerto Rico Institute of Statistics, June 4, 2018
Interview, Roberto Rivera, College of Business University of Puerto Rico, June 4, 2018
Interview, Alexis Santos-Lozada, Penn State assistant Teaching Professor, Department of Sociology and Criminology, June 4, 2018
Interview, Donald Berry, professor, Department of Biostatistics, The University of Texas MD Anderson Cancer Center and Adjunct Professor, Department of Department of Statistics, Rice University, Houston, June 4, 2018
Interview, Nishant Kishore, Harvard Phd candidate in epidemiology who previously worked for the U.S. Center for Disease Control & Prevention (CDC) as a Data Systems Developer, June 4, 2018
President Donald Trump, Tweet and Tweet, Sept. 13, 2018
George Washington University Milken Institute School of Public Health, "Ascertainment of the estimated excess mortality from Hurricane Maria in Puerto Rico," Aug. 28, 2018
Browse the Truth-O-Meter
More by Amy Sherman
stated on January 15, 2021 a headline:
"Inauguration Cancelled, Trump Remains in Office."
By Amy Sherman • January 16, 2021
stated on January 10, 2021 an interview on Fox:
"Republicans have no way to communicate" after tech companies cut off Parler.
stated on January 6, 2021 a U.S. Senate speech:
"Last year, Pennsylvania elected officials passed a whole new law that allows universal mail-in balloting, and did it irregardless of what the Pennsylvania Constitution said."
By Amy Sherman • January 8, 2021
stated on January 7, 2021 a speech:
In more than 60 cases, judges "looked at the allegations that Trump was making and determined they were without any merit."
stated on December 20, 2020 a Facebook post:
"'Simple math' shows Biden claims 13 million more votes than there were eligible voters who voted in 2020 election."
By Amy Sherman • December 23, 2020
President Donald Trump will hold his inauguration for a second term at Mar-a-Lago
stated on December 9, 2020 a tweet:
"No candidate has ever won both Florida and Ohio and lost."
stated on December 2, 2020 a speech:
"No state in the country verifies United States citizenship as a condition for voting in federal elections."
By Amy Sherman • December 7, 2020
stated on December 1, 2020 a Facebook post:
A Dominion representative in Gwinnett County improperly downloaded election data to "manipulate the data."
Ron DeSantis
stated on November 30, 2020 a press conference:
"The states that are locked down are increasing at twice the rate we are" in Florida
By Louis Jacobson • December 2, 2020
stated on November 18, 2020 a roundtable with healthcare workers:
The law governing the transition of presidential power "doesn't require there to be an absolute winner. It says the 'apparent' winner."
By Louis Jacobson • November 19, 2020
stated on November 13, 2020 a Facebook post:
Says Sen. Kelly Loeffler said on Parler that "Black people shouldn't be allowed to vote."
By Amy Sherman • November 16, 2020
stated on November 5, 2020 a White House speech:
"If you count the legal votes, I easily win. If you count the illegal votes, they can try to steal the election from us."
By Amy Sherman • November 6, 2020
stated on November 4, 2020 a Facebook post:
"118-year-old William Bradley voted via an absentee ballot in Wayne County, Michigan this year. William died in 1984. They're trying to steal this election."
Detroit has "2,503 dead people registered" including "one voter born in 1823."
stated on January 15, 2021 in a headline:
stated on January 15, 2021 in a Facebook post:
Says a photo shows armed soldiers protecting Joe Biden's inauguration.
By Miriam Valverde • January 16, 2021
stated on January 12, 2021 in an Instagram post:
Says Joe Biden stayed silent about violence at Black Lives Matter protests.
By Tom Kertscher • January 16, 2021
Jon Rosenthal
stated on January 5, 2021 in a tweet:
Says that "the way to reach 'herd immunity' is for >70% of the population to get vaccinated."
By Brandon Mulder • January 15, 2021
Amanda Chase
stated on January 7, 2021 in Facebook post:
"Antifa is the culprit" in the Capitol riot.
Devin LeMahieu
stated on January 10, 2021 in a TV interview:
"It takes quite a while on Election Day to load those ballots, which is why we have the 1 a.m. or 3 a.m. ballot dumping in Milwaukee."
By Eric Litke • January 15, 2021
stated on January 6, 2021 in a Facebook post:
"Coronavirus vaccinations seem to be causing 50 times the adverse events of flu vaccinations after just the first of two shots."
Viral image
A photo shows Democratic lawmakers wearing clothing and masks with the Nazi swastika.
By Samantha Putterman • January 15, 2021
Sarah Godlewski
stated on January 9, 2021 in Twitter:
"It wasn't all women that lost jobs (in the December jobs report), it was mostly Black and Latina women. In fact, white women gained employment."
By Nusaiba Mizan • January 15, 2021
Photo shows Rep. Lauren Boebert with Capitol rioters before she gave them a tour of the building "one day before the insurrection."