Outfoxing the Pundits

Outfoxing the Pundits

A groundbreaking study sheds light on how experts make predictions—and why they often are so calamitously wrong. 

In his groundbreaking study Expert Political Judgment: How Good Is It? How Can We Know?, Philip Tetlock employs the research done by psychologists, neuroscientists and game theorists to show how we make political decisions. The fundamental properties of the human mind—preference for simplicity, aversion to ambiguity and dissonance, belief in a controllable world, misunderstanding of probability theory—lead us to rely on narratives, the simple story lines that integrate and interpret information, both making sense of complex realities and trying to predict the future. Understanding how such narratives are used in politics sheds light on the biggest foreign-policy calamities of our times.

Not Outfoxed

There is no direct correlation between the intelligence and knowledge of the political expert and the quality of his analyses and forecasts, argues Tetlock, a political psychologist at UC Berkeley. What the pundit was thinking mattered less than how he was thinking.

To prove his point, Tetlock gathered and analyzed more than eighty thousand forecasts by academics and journalists about a variety of global issues, including the collapse of the Soviet Union and the end of the apartheid system in South Africa, discovering—surprise, surprise—that the experts were wrong more often than blind chance.

But occasionally Tetlock encountered an analyst who got it right more times than he got it wrong. And contrary to what the conventional wisdom would suggest, getting it right on, say, whether the Soviet Union would collapse or not, had very little to do with whether one had received a PhD with distinction in Russian Studies from Harvard. In fact, knowing too much about a subject and having strong personal commitment to the issue could be a major obstacle to getting it right. Or to put it differently, great minds don't necessarily make for great pundits.

Utilizing a typology of classifying political thinking and behavior introduced by the late British political philosopher Isaiah Berlin, Tetlock demonstrated the usefulness of sorting political experts along a rough cognitive-style continuum anchored at one hand by his prototypical “hedgehog” and at the other by his prototypical “fox.”

Hedgehogs are intellectuals who “know one big thing,” one Big Idea that they tend to apply in dealing with policy issues. They integrate the many pieces of information into a mega narrative that reflects a long-held theory and ideology, dismissing facts that don’t fit into their dogma. Foxes, on the other hand, know many things. They tend to examine the many facts before them, and try to draw the outline of their policy analyses based on the reality on the ground and not on a grand theory that may sound very profound but doesn’t provide an explanation that could produce workable solutions to concrete challenges.

The intellectually aggressive hedgehog is an ideologue who seeks, under the banner of parsimony, to expand the explanatory power of that Big Idea to “cover” new cases. He toils devotedly within one tradition and imposes formulaic solutions on ill-defined problems.

The more eclectic foxes are empiricists who are content to improvise with ad hoc solutions to keep pace with a rapidly changing world. Drawing from various and sometimes contradictory ideas and traditions, she is better able to improvise in response to changing events and more successful in predicting the future than the hedgehog.

Paradoxically, Tetlock noted, there was a perversely inverse relationship between the scientific indicators of good judgment and the qualities that the media most prized in pundits—an “expertise” in a specific field and the single-minded determination required to prevail in ideological combat. Hence, when it comes to the op-ed pages and the television news shows, it's the hedgehog that rises to the top.

The public and the elites, including the media, tend to be attracted to the assertive hedgehogs and their Big Ideas, which make for good copy and sound bites that make it easier to employ a battle cry to mobilize people to “war” on this or that problem. The foxes sound a bit hesitant and irresolute and they do a lot of zigzags, use a lot of “on one hand” and “on the other hand,” which doesn’t sound so great on television.

Rise of the Hedgehogs?

Life has been good for the hedgehogs since Tetlock completed his study. Consider our contemporary media environment, in which thousands of Internet sites and blogs, Facebook posts and tweets, and even the surviving representatives of the mainstream media are competing for our attention—and providing us 24/7 with instant narratives.

This media environment is more conducive to the kind of simplistic narratives that reflect the one-dimensional thinking of the hedgehogs. These narratives can accommodate very complicated foreign-policy issues—often involving many players, interests, geographic locales and cultures—into a brief blog post or a catchy tweet. These hedgehog narratives thus transform an intricate global crisis into a morality play, a suspenseful plot dominated by the a clash between “good guys” and “bad guys,” a cosmic struggle between grandiose ideas and great civilizations.

The specter of the United States and the West facing the threat of an Axis of Evil or Islamofascism is exactly the kind of narrative that most Americans—who are not familiar with the differences between Sunnis or Shia, or between Pashtuns and Tajiks—find easy to understand and digest. At the same time, creating expectations about the global spread of liberal democracy under U.S. leadership helps construct narratives that intellectuals committed to the Enlightenment project find attractive.

Political leaders can employ these narratives to mobilize public support to deal with this or that global threat. Indeed, just consider the many instant narratives that were promoted—through government spinning and media events, in columns and blogs, journal articles and books—to justify the decision to go to war in Iraq and to continue fighting there for a decade: Iraq is part of the Axis of Evil; Saddam Hussein was involved in the 9/11 attacks; Saddam is Hitler; Iraq has weapons of mass destruction (“mushroom clouds”); the war will be short and costless (“cakewalk”); the war will pay for itself; a democratic Iraq is part of the Freedom Agenda; Iraq is like post-WWII Germany and Japan; the road to Jerusalem runs through Baghdad; there is no insurgency in Iraq; the insurgents are terrorists; “We have to fight in Iraq so we don’t have to fight at home; “We are turning the corner”; many “tipping points” like the fall of Saddam’s statue or the surge; “We need to leave with honor.”

More recently, the political turmoil in the Arab World has helped produce many and ever-changing story lines. There is the notion that there was an Arab Spring à la Eastern Europe in 1989, which under the inspiration of cool Westernized bloggers and Facebook would bring about a flourishing of liberal democratic systems “like in Turkey” that would be “good for America and Israel.” Then there was the conflicting narrative warning that the Arab Middle East would be ruled by radical and anti-Western regimes and be transformed into a theocracy “like in Iran,” which would be “bad for America and Israel”—or, alternatively, that these developments would result in a war between Sunnis and Shia.

Political pundits should acquire a sense of humility and recognize that the world is too complex to unravel. Even under the best conditions, their minds are not equipped for complete understanding. Failing this, the public should not take narratives too seriously and should be a bit suspicious when politicians or pundits try to sell Big Ideas to them.

There is some good news. The reality in Iraq has exposed the faulty analyses and forecasts of one variety of hedgehog—the neoconservatives—and helped strengthen the hands of Washington’s foxes, who call for continual readjustment and flexibility: Democracy is a great idea—but perhaps it won't work in Iraq and the Middle East. There is a need for U.S. leadership in the world—but that's very different than U.S. hegemony and unilateralism. There are many unsavory regimes in the world—but sometimes you have to talk with them. Military force is an important tool in international affairs—but only as the last resort.

Those who recognize these facts are tempted to ask—are we all foxes now?

Leon Hadar, senior analyst at Wikistrat, a geostrategic consulting group, is the author of Sandstorm: Policy Failure in the Middle East (Palgrave Macmillan, 2005).