By estimating that there are some 655,000 “excess deaths” due to the Iraq war, Johns Hopkins University professor Gilbert Burnham touched off a firestorm of criticism across the blogosphere.
While opinions rocketed back and forth, no one tracked down the Lancet study’s principal author and asked about counter-arguments and methodological niggles. So I did.
THE INTERVIEW:
In the course of several cordial e-mail exchanges, I put a series a questions to Burnham based on the comments made by Pajamas Media readers, in the comments section of a recent article by Iraq the Model. Read carefully and see if you can find your own comment embedded in a question or two.
Most online objections fall into one of three categories: timing of publication (or motivation), methodology, and results. I asked him about the last two, in the course of reporting for another publication.
(I decided to ignore the first category, timing. While a lot of bloggers pointed out that Burnham’s study, published in the Lancet, a British medical journal, appeared just weeks before the 2006 congressional elections and that the Lancet had published a similar study just weeks before the 2004 presidential elections, I thought, so what? Anyone remotely clever schedules the publication of his work at a time when people might actually care enough to read it and discuss it. Besides, it is natural in an election year to consider the death toll in Iraq.)
PajamasMedia: Professor Burnham, below is a transcript of a CNN reporter questioning President Bush about your survey.
SUZANNE MALVEAUX, CNN: A group of American and Iraqi health officials today released a report saying that 655,000 Iraqis have died since the Iraq war. That figure is 20 times the figure that you cited in December at 30,000. Do you care to amend or update your figure and do you consider this a credible report?
PRESIDENT BUSH: No, I don’t consider it a credible report, neither does General Casey and neither do Iraqi officials. I do know that a lot of innocent people have died and it troubles me and grieves me. And I applaud the Iraqis for their courage in the face of violence. I am, you know, amazed that this is a society which so wants to be free that they’re willing to ? you know, that there’s a level of violence that they tolerate.
Do you have any thoughts or response to Bush’s answer?
Burnham: I agree with the President that we should all be concerned with the loss of human life in Iraq.
The primary motive in our work is to assess the human burden of the conflict on the people of Iraq, and use this information to promote measures to better protect civilians caught up in conflict–not just in Iraq but in the future wars which of the 21st century. The methods we used are standard methods used world wide; the US Government financially cluster sample surveys as a way of providing health and other information in many countries which are assisted by the USA.
PajamasMedia: The Iraqi government is also questioning your survey results. The Manchester (U.K.) Guardian reported:
The Iraqi government said the report’s death toll was “inflated” and “far from the truth”, but did not give its own figure for the deaths. An Iraqi government spokesman, Ali al-Dabbagh, said in a statement that the toll in the report “exceeds the reality in an unreasonable way.”
Any thoughts on this?
Burnham: The numbers are large, and they surprised us . Yet it is only through population-based sampling that information such as the impact of conflict can be assessed. From isolated numbers taken from morgues and hospitals one can not arrive at a national figure. Not possible.
PajamasMedia: In 2004 the Lancet study found 100,000 excess deaths due to the war. By 2006, the number climbed to 655,000. In the past two years of conflict we see five times as many deaths as resulted from a period that included large-scale combat and massive air strikes. How do you account for the sharp increase?
Burnham: The news reports and reports from organizations which count bodies or accounts of deaths have already provided this information on the worsening violence. Sectarian violence is now a major problem which was only beginning in 2004.
PajamasMedia: In the Lancet article, you write: “Thus, the data presented here validates our 2004 study, which conservatively estimated an excess mortality of nearly 100,000 as of September 2004.” Does this mean you used the same procedure in 2006 that you did in 2004? Why do the results show a sharply higher death toll? Has there war gotten that much worse, or are there differences in methodology that make it difficult to compare the results of the two studies?
Burnham: We built on what we learned in the 2004 survey to design the 2006 survey. But in both surveys we started at 1 January 2002, thus giving us a chance this time to confirm the findings from 2004, though we visited none of the same neighborhoods in 2006. The methodology was the same, though we increased the sample size and the number of clusters [from 33 to 47]. The 2006 data from January 2002 through Sept 2004 (end of the 2004 study) were virtually identical to the 2004 studies. The death rates are rising, because the violence is getting worse.
PajamasMedia: The Lancet study uses a baseline mortality rate (the rate during Saddam years) of 5.5/1000 – almost half of the mortality rate of Europe. The mortality rate in the EU is 10.10/1000. Given Europe’s excellent health care, public health infrastructure and, lack of war in the past 60 years, how is it possible that Iraq’s baseline is half that of the EU? Are you simply relying on pre-war publications or was the baseline itself generated by interviews with random clusters?
Burnham: This was a ‘cohort’ study, which means we compared household deaths after the invasion with deaths before the invasion in the same households. The death rates for these comparison households was 5.5/1000/yr.
What we did find for the households as a pre-invasion death rate was essential the same number as we found in 2004, the same number as the CIA gives and the estimate for Iraq by the US Census Bureau.
Death rates are a function of many things–not just health of the population. One of the most important factors in the death rate is the number of elderly in the population. Iraq has few, and a death rate of 5.5/1000/yr in our calculation (5.3 for the CIA), the USA is 8 and Sweden is 11. This is an indication of how important the population structures are in determining death rates. (You might Google ‘population pyramid’ and look at the census bureau site–fascinating stuff.)
PajamasMedia: During the same period, Iraq is at war with Iran and itself. Public-health infrastructure was poor, although perhaps not as poor as today. Does it seem plausible to you that the baseline (or pre-war) mortality rate is accurate?
Burnham: Yes as above. Yes as being the right number, and Yes as what we need it for–comparisons in the same households before the war.
PajamasMedia: The Lancet Study comes up with a post-war mortality rate almost double that Saddam’s Iraq. In fact, it is roughly equivalent to the mortality rate in Hungary is 13/1000. Does that rate seem plausible, given Hungary’s superior infrastructure and almost 50 continous years of peace? Is it possible that both the pre- and post-war mortality rates are too low? Why not?
Burnham: There are many old people in Hungary , 40% are over age 55 vs. 9.3% in Iraq over 55. That’s the difference.
PajamasMedia: You conducted interviews in 47 clusters, 12 of them were in Baghdad, 2 were in Basra, and 3 were in Anbar. Approximately 25% of the estimate comes from Baghdad (only 21% of the population in Baghdad). This seems disproportionate. Is it possible that you over-sampled “hot zones” relative to population?
Burnham: We used the 2004 UNDP/MoP estimates for governorates. We divided the total population by the number of clusters, and then moved on a systematic way through the population assigning clusters proportionate to the population numbers we were using.
Obviously a cluster near the “edge” could fall either in or out of a governate. As we could not split clusters to be 1/2 in one governorate and 1/2 in another, it might look like there was an intentional oversampling. In the end we are satisfied by the representativeness of the sample.
Baghdad, by the way, was not the most violent area, being in an area of mid-level death rates. There are always chances that sampling was done in more hot spots, but there is an equal chance that, with a natural human tendancy to self-preservation might cause sampling to be the other way to unconsciously sample in cool spots where one might be safer.
PajamasMedia: All of their interviews came from populated areas where they could interview people (so word-of-mouth of their purpose would help keep the interviewers safe), but also where the violence is concentrated. Would overlooking rural areas create a bias? Or have you controlled for that factor in some way?
Burnham: We sampled from all administrative units selected. Using the standard nearest-front-door approach, the teams would not likely select isolated homesteads which were many miles away from the area being sampled.
PajamasMedia: One of your findings in the Lancet article: “Across Iraq, deaths and injuries from violent causes were concentrated in adolescent to middle age men.” That is, males of combat age. Does this suggest that collateral damage is unlikely to be the cause of death?
Burnham: It may will be that some or even many of the males could have been combatants. It is not possible to judge the motives of the dead through surveys. We could record only what households told us.
PajamasMedia: You write that an active survey is more accurate than a “passive” system of counting media reports, morgue reports or other lists of the dead, which are often grossly incomplete in a war zone. This seems reasonable. To make sure people weren’t making things up, you teams received death certificates some 80% of the time. Also reasonable. So why are the active death figures an order of magnitude higher than the passive counts?
Burnham: The difference depends on the proportion of the the passive-reporting facilities whose reports on death tolls reach some central tabulating body. Our information is that not many facilities are reporting, and what is being reported is often being manipulated.
PajamasMedia: According to this report 14% of the 655,000 people died as a result of suicide bombers, which would be about 91,700 people. This is far greater than the media estimates and suicide attacks tend to be well-reported, as opposed to shooting or roadside bombings. Do you have any specific information about where these 97,000 people were killed?
Burnham: We did not report on suicide bombers.
PajamasMedia: Your study attributes a third or more than 200,000 would have died in air raids. That is more than Hiroshima and Nagasaki combined. And air raids have basically ceased in cities since 2003. Is it possible that your teams are counting something else aside from aerial bombardment in this category? If so, what would that be? Could it be a translation error?
Burnham: Overall, 13% of deaths were attributed to airstrikes, not 200,000. We reported what households said; the data were recorded in English, as all interviewers spoke English.
PajamasMedia: Historical comparisons might be helpful here. 650,000 violent deaths is about 150,000 more than the number of soldiers who died (violently and by disease) during the American Civil War, a conflict which involved a population larger than Iraq’s, and lasted a year than the current conflict has been going on. There is nothing in Iraq that looks like Shiloh, Antietam, Gettysburg, Cold Harbor, etc. What makes you believe that Iraq is deadlier than the American Civil War?
Burnham: What we are reporting is cumulative deaths over a 40 month period throughout an area of 26.1 million, not a 1-2 day battle field event.
PajamasMedia: Perhaps those interviewed related the number of deaths per extended family (and not “household”). If the researchers interviewed members of the same extended family (but who were not in the same household), then they might have double-counted. EXAMPLE: If I lost someone in my family on 9-11, but not in my household and a researcher asked me if I lost a family member on 9-11, I would answer yes – as would members of several households. All for the same solitary death. Aside from death certificates, how did you control for over-counting?
Burnham: The death had to be of someone eating and sleeping in the selected household (our definition of a household) for 3 months before the death, in order to be included. We reviewed each cluster and each written death entry individually. We are satisfied that there was no double counting.
——-
Return to Pajamas Media homepage
Join the conversation as a VIP Member