So What’s Wrong with Research 2000’s Polling?
Daily Kos founder Markos Moulitsas Zúniga is suing the polling company for fabricating results. What were the tip-offs that led to R-2000's downfall?
July 2, 2010 - 1:17 pm
After the elections in 2008, Daily Kos, the largest liberal community blog out there, contracted pollster Research 2000 to conduct a large number of race-specific and “State of the Nation” polls. The results of Research 2000’s surveys have come under increasingly intense fire lately, and on July 1, Daily Kos founder Markos Moulitsas Zúniga filed suit against Research 2000 and its owner and CEO Del Ali. As a poll compilator who has used Research 2000’s polls in my projections, I am keenly interested in the beef that the uber-leftist Kos has with them.
Polling is an inexact science, and the numbers that polling firms produce are constantly scrutinized by political observers of all ideological stripes. Sometimes, when results seem out of line with others — or they don’t match one’s preferred outcomes — pollsters can be accused of driving a political agenda. But rarely, if ever, do you hear of a lawsuit brought against a pollster for outright fabrication. Simply oversampling one demographic or another, or asking respondents leading questions, is no grounds for legal action. Moulitsas must have seen something much more significant — and damaging — in Research 2000’s work than just methodological discrepancies
In fact, what he alleges is quite serious, indeed. The lawsuit states that Moulitsas “was approached by a number of independent statistical analysts with regards to Research 2000’s polling for Daily Kos. Their analysis of the published data revealed a number of statistical anomalies regarding the results which revealed that Research 2000 had almost certainly falsified the results in whole or in part.” As Greg Sargent of the Washington Post affirms, “this is a big, big deal. Research 2000 polls have been widely cited by many news organizations, and have helped shape the national political conversation.”
So what exactly left Kos feeling defrauded by Ali and Research 2000? Sifting through the lawsuit itself and comments made by others on both the right and the left of the political spectrum, I’ve identified three main factors which I believe drove him to sue.
Nate Silver, founder of fivethirtyeight.com, is partly to blame for the disillusion with Research 2000. He recently revised his extensive analysis of pollster performance and published the updated results. Using a massive database of thousands of polls from hundreds of polling firms and numerous weighted metrics, Silver constructed a quantitative method for ranking poll-taking skill. The end of all this number crunching is a value he dubs “pollster-introduced error.” Essentially, the lower that number, the better the polling firm.
At the top of the list of 64 firms with at least 10 polls published during the sample period were such familiar names as The Field Poll and SurveyUSA. At the bottom? You guessed it. Research 2000’s pollster-introduced error was one of the five highest. In fact, no other polling firm releasing as many polls fared worse. Undoubtedly, when Kos saw the polls tied to his website falling so far down the accuracy scale, he must have felt he was not getting his money’s worth. Yet Silver’s calculations, while certainly disconcerting, were just the beginning of the rift between Daily Kos and Research 2000.