Obama and the Swan

It’s interesting to think about Barack Obama’s trip to South Asia and the Middle East after watching Philip Tetlock’s wonderful January 2007 video on the efficacy of forecasting. Tetlock asked himself why pundits never lost their reputations by making bad predictions. Jonathan Schell, in his famous book the Fate of the Earth predicted Ronald Reagan’s policies would increase the probability of a nuclear war. At that time most analysts in the CIA were predicting that the Soviet Union would last forever. Neither came true. Yet Johnathan Schell is still selling books and one of the analysts who felt confident the USSR would last a long time is now Secretary of Defense, Robert Gates. Tetlock came to the conclusion that political punditry was so unaccountable because no one was keeping score.

So to find out its real value he began to compile a sample of 28,000 predictions by 284 experts over 18 years to see which of them came true. The results were disappointing. Expert predictions barely outperformed simple statistical prediction schemes such as those which assumed no change or that the latest rate of change would continue. In other words experts could not predict the future with any clarity but we consulted them anyway because of the need to appear in control of even future events. Indeed in Tetlock’s early 2007 video illustrates the weaknesses of expert prediction perfectly; with its references to “regional experts” who were very confident that the Surge would fail. Maybe it will, if we wait long enough. But one type of expert was clearly worse than others; the kind he termed the “hedgehog”. They made the least accurate predictions of all. “Hedgehog” in this context denotes someone who bets on extreme outcomes based on a theoretical construct, such an ideological position. The term is taken from a poem by the ancient Greek Archilocus who wrote “the fox knows many things, but the hedgehog knows one big thing”. Hedgehogs are those who “just know” what is going to happen with the certainty of a true believer. They are guided by the “big thing”.

It was better Tetlock found, for predictors to be consciously aware of the unkown; to be informed not only by knowledge but anti-knowledge — what we don’t know. The experts who did that, who were open to the idea that the world was messy, full random and complex influences — he called the “foxes”. Normally they were more boring than the hedgehogs. Hedgehogs tended to categorically tell people what was going to happen or not happen, while foxes were often only willing to offer probabilities and forecast over short horizons. Tetlock argued that while the foxes predicted things better, the public was much more willing to listen to the hedgehogs, especially when they could tell a good story.

Even more entertaining than Tetlock is the video lecture by Nasim Nicholas Taleb, the author of the Black Swan. Taleb’s attitude toward life was changed by his discovery of how little he could predict and set about discovering the bounds of knowledge and anti-knowledge. Taleb in his lecture describes his now famous view that events in the world can be categorized into the categories of mediocristan and extremistan. Mediocristan is populated by events in which individual outcomes do not change the aggregate result by much. In this category phenomena can be easily mathematized and classic statistical predictions rule. Some domains of physics are like this.

But the other place — which by far encompasses most of the things that affect us — is dominated by a class of events that is not so easily mathematized. He calls this Extremistan. Here the rare, high impact event rules and individual events can have disproportionate effects on final outcomes. His example is the Black Swan. For most of human history all swans were believed to be white because all were observed to be white. But when Captain Cook arrived in Australia the first Black Swan they encountered was enough to invalidate a multitude of prior observations. We went from a world in which all swans were white to one in which they might be of a different color by a single observation.

It is not surprising that Taleb and Tetlock are friends. In common their work has highlighted the importance of what we don’t know. Taleb’s Black Swan idea partially explains why Tetlock’s “foxes” do better than is “hedgehogs”. “Foxes” understand that they don’t know the answers and are open to the existence of Black Swans and consequently they incorporate information which a “hedgehog” might throw away. Since hedgehogs already understand the future they are less likely to see what doesn’t fit and are consequently much more likely to be caught off guard by unexpected developments.

It is tempting to characterize Obama’s approach to international relations as one resembling that of Tetlock’s “hedgehog”. Although he is going to Afghanistan and the Middle East ostensibly to examine the conditions on the ground, it is not for the purpose of reworking his policies. He already knows what those are. They are given. Rather, he is there to discover what obstacles might obstruct their implementation. Obama already knows what he is going to do. It is this clarity of vision that makes him so attractive to his supporters. But it is also the source of the greatest danger to his policies. What happens if Obama says, “Yes we can” and reality says, “No you can’t”? What happens if the hedgehog meets the Black Swan?

Tip Jar.