The Truth About Polling: Yes, Romney Is Probably Tied or Winning
Knowing the nuts and bolts of polling science leads to this rational conclusion.
September 27, 2012 - 7:49 am
First: when you read a poll, you’ll always see it stated as something like “53 percent Democratic, margin of error of plus or minus 3.5 percent.” Here’s how you should read that: “The polling company believes that if the election were taken today, there is 1 chance in 20 that the actual result will be more than 56.5 percent Democrat or less than 49.5 percent Democrat.”
You see, that’s what the “margin of error” really means: by statistical methods, they believe that 19 out of 20 times — or 95 percent of the time — the real value would come out between 49.5 percent and 56.5 percent.
Second, and this is where the controversy is coming now: the quality of those results depends on the accuracy of that original model.
So, when a Quinnipiac poll comes out and says that Ohio is now 53 percent Obama, what should you do? Ready, class?
First, we restate that. I can’t find the margin of error for this poll readily (this should make you suspicious at the start), but if you can’t find one, you’ll always be close to right if you say it’s 3 percent plus or minus. So we re-read this as “they’re saying there’s 1 chance in 20 it will come out either above 56 or below 50.”
Second, we look for the polls’ “internals” — in other words, that turnout model. Now, here’s where this poll gets interesting. Hugh Hewitt interviewed Peter Brown, the head of Quinnipiac, and challenged him directly on the turnout model:
HH: I want to start with the models, which are creating quite a lot of controversy. In Florida, the model that Quinnipiac used gave Democrats a nine-point edge in turnout. In Ohio, the sample had an eight-point Democratic advantage. What’s the reasoning behind those models?
PB: Well, what is important to understand is that the way Quinnipiac and most other major polls do their sampling is we do not wait for party ID. We ask voters, or the people we interview, do they consider themselves a Democrat, a Republican, an independent or a member of a minor party. And that’s different than asking them what their party registration is. What you’re comparing it to is party registration. In other words, when someone starts as a voter, they have the opportunity of, in most states, of being a Republican, a Democrat, or a member of a minor party or unaffiliated … (emphasis added)
A little later, he says:
HH: Why would guys run a poll with nine percent more Democrats than Republicans when that percentage advantage, I mean, if you’re trying to tell people how the state is going to go, I don’t think this is particularly helpful, because you’ve oversampled Democrats, right?
PB: But we didn’t set out to oversample Democrats. We did our normal, random digit dial way of calling people. And there were, these are likely voters. They had to pass a screen. Because it’s a presidential year, it’s not a particularly heavy screen.
In other words, the way Quinnipiac does it is they assume that the thousand or so people they poll on a certain day really do represent the population, and if that comes out with a D+8 distribution, that’s the way they report it.