Polling Theory: When Polling Is Useless
Do current conditions make polling meaningless?
October 5, 2012 - 12:00 am
Today, we’re going to perform a little thought experiment. Einstein was big on them, so they have an honorable history. I want you to imagine a portable cement mixer – my Gods, they have them on Amazon! The one at the link holds 3.5 cubic feet of mix, a nice handy size for, say, pouring a pad for a barbecue or mortar for a flagstone walk. But instead of filling it with Portland cement and gravel, we’re going to fill it with a cubic foot of red BBs and another cubic foot of blue BBs. (It turns out that there are about 585,000 BBs to the cubic foot, so that makes a nice number.)
We switch it on, and it makes a horrible noise as all those BBs tumble about. After a few minutes, we figure the BBs are thoroughly mixed, and we have a bit of a headache.
The thing is, we were kind of sloppy filling the buckets. Just about then, the boss walks by and wants to know the proportions of red and blue BBs.
“Well, roughly half and half.”
“Roughly isn’t good enough — I want a good estimate, and I want to know how much error there may be.”
Now he tells us.
This is basically the problem that pollsters always have. They have a pretty good idea of how many voters there are in a state or in the whole country; what they want to know is the number of Republican voters versus the number of Democrat voters. (Notice how I artistically chose red and blue BBs?)
Now, it happens we know statistics, even if we’re not great at measuring BBs, so we know we can estimate the proportions by just counting a relative few of the BBs, as long as we assume they really are well-mixed, and that we take a sample randomly. In fact, it turns out that as long as the number of things we’re looking at is big and our sample is small, we can determine the margin of error just from the size of the sample. I’m not going to explain why here — thank me later — but you can read the Wikipedia article on it if you’re so inclined. The fact is, though, that for a 95 percent confidence level — the same one chance in 20 of being wrong we’ve talked about in previous polling articles — the margin of error for a sample of size n will be just about 1/√n. So if we choose only 10 BBs, the margin of error is just about plus or minus 30 percent. Not very satisfactory. Let’s look at some other values:
|Sample size||Margin of error (pct)|
By the way, this rule works for any random sample, whether it’s BBs, babies born in Baghdad, or Biden supporters, as long as the size of the whole population is really large.