The tools of the political pundit's trade include polls, reams of uncorrelated data, more than a passing knowledge of how elections work, and, of course, the "gift of gab," as my Irish grandmother used to say. Actually, my grandmother was just being polite. A pundit has to have the ability to pass along convincing bulls**t.
And no other tool in a pundit's toolbox is more beholden to bulls**t than the ubiquitous "election forecast."
Soothsaying is more accurate. Flipping a coin is equally factual. Reading the entrails of a newt or having an astrologer weigh in can't be beaten compared to an election forecast.
"The truth is we don’t have nearly enough data to know whether these models are any good at making presidential prognostications. And the data we do have suggests these models may have real-world negative consequences in terms of driving down turnout," writes political scientist Justin Grimmer in Politico.
Bad for democracy, bad for accuracy, but really, really good for a pundit if you can BS your way through a 1,200-word article on "Who's gonna win in November?"
It's important to differentiate between polls (snapshots) and forecasts (three guesses and the first two don't count). Forecasts aggregate (combine) polls into a gigantic data soup from which they ladle out the wisdom of the ages or their best guess for who's going to win.
Election forecasts have a long history in political science, but they entered the political mainstream because of Silver’s accurate predictions in the 2008 and 2012 elections. Now, many news outlets offer probabilistic forecasts and use those models to declare that candidates have an expected number of Electoral College votes and probability of winning the election. See ABC News’ 538, The Economist and Silver Bulletin, among others.
Are these calculated probabilities any good? Right now, we simply don’t know. In a new paper I’ve co-authored with the University of Pennsylvania’s Dean Knox and Dartmouth College’s Sean Westwood, we show that even under assumptions very favorable to forecasters, we wouldn’t know the answer for decades, centuries, or maybe even millenia.
Nate Silver became the first celebrity political prognosticator. Was he lucky or good? As Grimmer points out, forecasting presidential races doesn't have much of a track record so for the moment, we'll have to say that Silver was lucky.
In fact, there have been only 60 presidential elections in all of U.S. history. That's not much of a data set to predict anything accurately.
Compare the information available when forecasting presidential elections to the amount of information used when predicting stock prices, forecasting the weather or targeting online advertising. In those settings, forecasters commonly use millions of observations, which might be collected almost continuously. Given the difference, it isn’t surprising that forecasters in other settings are more easily able to identify the best performing model.
"Models" are the key. Pollsters use statistical models based on years of history as well as reams of U.S. census data. But forecasters have a paucity of information, so they use "educated guesses" based on their education and soothsaying abilities.
The soothsaying comes in when a forecaster tries to assign how much "weight" to give each poll used in the forecast. Is Morning Consult better or worse than Emerson or Quinnipiac? Some polls are more accurate regionally. Some are less accurate nationally.
Statistical models do offer the chance for forecasters to be clear about these assumptions, whereas pundits’ assumptions are often unstated or difficult to determine. But without data to evaluate how the assumptions affect calibration or accuracy, the public simply does not know whether the modeling decisions of one forecaster are better than the modeling decisions of the other.
There are also real-world calibrations that need to be taken into account. How much weight should be given to economic indicators?
Unemployment is rising slightly, inflation is cooling off, and the Fed is considering cutting interest rates. Which is the best indicator of how the economy may affect the election? The site 538 uses 11 economic indicators in its model. Meanwhile, 62 percent of voters in a recent poll said they thought the economy was bad.
Related: Dead 20 Years, Ronald Reagan Can Still Trigger the Left
Suffice it to say that there's no rational reason to write about or think about or talk about political forecasts except that they give us all something to write about, think about, and talk about politics at cocktail parties.
Or whatever the 2024 equivalent is.
Join the conversation as a VIP Member