How can you speak from authority if the data you rely on is flawed? It’s a problem for policymakers around the world who have been relying on a scientific model that projects the number of dead from the coronavirus. The White House has been relying on the model to formulate policy and develop plans to reopen the economy.
But epidemiologists are saying that the model is hopelessly flawed and gives misleading results.
“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington.
Others experts, including some colleagues of the model-makers, are even harsher. “That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, home to several of the researchers who created the model, and who has served on a search committee for IHME. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”
Why does this matter?
The IHME projections were used by the Trump administration in developing national guidelines to mitigate the outbreak. Now, they are reportedly influencing White House thinking on how and when to “re-open” the country, as President Trump announced a blueprint for on Thursday.
The problem is that the model initially showed a U.S. death toll of up to 240,000 but now projects fewer than 70,000 deaths. Has social distancing really been the key to keeping the death rate low? The researchers argue that policymakers may believe that policies well short of the draconian measures taken by China in Wuhan prevented the horrific loss of life when, in fact, the model giving those numbers in the first place was fatally flawed.
The IHME projections are based on data from U.S. cities being compared to cities in other nations. But that methodology is flawed.
According to a critique by researchers at the London School of Hygiene & Tropical Medicine and Imperial College London, published this week in Annals of Internal Medicine, the IHME projections are based “on a statistical model with no epidemiologic basis.”
“Statistical model” refers to putting U.S. data onto the graph of other countries’ Covid-19 deaths over time under the assumption that the U.S. epidemic will mimic that in those countries. But countries’ countermeasures differ significantly. As the epidemic curve in the U.S. changes due to countermeasures that were weaker or later than, say, China’s, the IHME modelers adjust the curve to match the new reality.
There are other, more technical reasons to distrust the IHME model, but the bottom line is that it misinforms national leaders.
“This appearance of certainty is seductive when the world is desperate to know what lies ahead,” Britta Jewell of Imperial College and her colleagues wrote in their Annals paper. But the IHME model “rests on the likely incorrect assumption that effects of social distancing policies are the same everywhere.” Because U.S. policies are looser than those elsewhere, largely due to inconsistency between states, U.S. deaths could remain at higher levels longer than they did in China, in particular.
Models are only as good as the data that’s input. You can have the best-designed model in the world and still come to wrong conclusions because the data you put in is wrong.
The scary reality is that all we can do is hope for the best. We hope that social distancing will keep the death toll down, but we won’t know for sure until history unfolds and tells us whether we were right or wrong.
Editor’s Note: Want to support PJ Media so we can keep telling the truth about China and the virus they unleashed on the world? Join PJ Media VIP and use the promo code WUHAN to get 25% off your VIP membership.
Join the conversation as a VIP Member