Reading Tarot on K Street

August 25, 2009 Topics: Society Regions: Americas

Reading Tarot on K Street

Mini Teaser: Who doesn’t want to know whether the Dow will close above ten thousand at year’s end, whether the Saudis can maintain their oil production, whether China will rise and Russia will fall, or whether a new dictator lurks in the Middle East?

by Author(s): Philip E. Tetlock

From the September/October 2009 issue of The National Interest.


Ian Bremmer and  Preston Keat , The Fat Tail: The Power of Political Knowledge for Strategic Investing (New York: Oxford University Press, 2009), 272 pp., $27.95.

Bruce Bueno de Mesquita , The Predictioneer's Game: Using the Logic of Brazen Self-Interest to See and Shape the Future (New York: Random House, 2009), 272 pp., $27.00.

George Friedman , The Next 100 Years: A Forecast for the 21st Century (New York: Doubleday, 2009), 272 pp., $25.95.


[amazon 0767923057 full] WE ARE awash in political-forecasting products: books, reports, updates, outlooks, data points, extrapolations, scenarios . . . and who doesn't want to have a glimpse into the future? Who doesn't want to know whether the Dow will close above ten thousand at year's end, whether Russia will bully more of its neighbors, whether North Korea will do something really "crazy," whether the Saudis can maintain their oil production . . . and who among us is not curious whether 2045 will be the year of the "singularity" when, according to some futurists, artificial intelligence surpasses human minds, technological advances skyrocket under the control of machine-human hybrids and those of us who can hang on long enough have our own shot at immortality? But if wishes were horses, beggars would ride. There is a paucity of evidence-peer-reviewed scientific evidence-that forecasters know how to deliver the goods: reliably accurate political, economic and technological predictions. In fact, when I have staged competitions, many forecasters fail to outperform the proverbial dart-throwing chimpanzee-and most cannot outperform extrapolation algorithms that simply predict "more of the same."


KARL MARX hit the nail on the head when he quipped that, when the train of history hits a curve, the intellectuals fall off. But there is still great potential for mischief in the political-forecasting business. The demand for accurate predictions is insatiable. Reliable suppliers are few and far between. And this gap between demand and supply creates opportunities for unscrupulous suppliers to fill the void by gulling desperate customers into thinking they are getting something no one else knows how to provide.

On top of this, the laws of probability tell us that at least a few superstar forecasters will arise just by chance if enough pundits make enough predictions. According to the extreme skeptics, we should just wise up and give up waiting for our forecasting savior. Even such fabled figures as Warren Buffet, the sage of Omaha, and George Soros, the man who broke the Bank of England, may just be lucky-the human equivalents of coins that, if tossed long enough, eventually yield surprisingly long streaks of "heads" or "tails."

But extreme skepticism is too bitter a pill for most mortals to swallow, implying as it does that, when it comes to the big decisions-whether to reassure rather than deter Iranian ayatollahs or North Korean dictators or bail out or let sink mega-financial institutions-we do as well in the long run by flipping coins as consulting experts. That is why, in part, I classify myself as a moderate skeptic. Hope springs eternal-and I cannot bring myself to write off the forecasting business completely. When new products pop up in the marketplace of ideas, I cannot help but wonder whether any of them might somehow fill the void.

I am not however a moderate skeptic for just desperate wish-fulfillment reasons. A good deal of research indicates that some ways of thinking ("cognitive styles") do translate into somewhat more correct forecasts. When we score the accuracy of thousands of predictions from hundreds of experts across dozens of countries over twenty years, we find the best forecasters tend to be modest about their forecasting skills, eclectic in their ideological and theoretical tastes, and self-critical in their analytical styles. 1 Borrowing from philosopher Isaiah Berlin, I call them foxes-experts who know many things and are not finicky about where they get good ideas. Paraphrasing Deng Xiaoping, they do not care if the cat is white or black, only that it catches mice.

Contrast this with what I call hedgehogs-experts who know one big thing from which likely future trends can be more or less directly deduced. The big thing might be any of a variety of theories: Marxist faith in the class struggle as the driver of history or libertarian faith in the self-correcting power of free markets, or a realist faith in balance-of-power politics or an institutionalist faith in the capacity of the international community to make world politics less ruthlessly anarchic, or an eco-doomster faith in the impending apocalypse or a techno-boomster faith in our ability to make cost-effective substitutes for pretty much anything we might run out of.

What experts think -where they fall along the Left-Right spectrum-is a weak predictor of accuracy. But how experts think is a surprisingly consistent predictor. Relative to foxes who are less encumbered by loyalties to an all-encompassing worldview, hedgehogs offer bolder forecasts and, although they hit occasional grand slams, they strike out a lot and wind up with decidedly poorer batting averages.


[amazon 0195328558 full] READERS NOW know my biases: a deep belief in the need for independent, objective scoring rules for gauging expert accuracy and a deep skepticism of big-idea hedgehogs. Nothing in the books under review-merely the latest tomes to sit on the forecasting shelves-has altered the first belief but, as we shall see, one of the books has shaken the second.

The authors are all entrepreneurial futurists, but each offers a strikingly distinctive approach to prediction. I organize these approaches under three headings: the superpundit model in which readers take it, more or less on faith, that the forecaster has a pipeline into the future not available to ordinary mortals (a category into which I place George Friedman's The Next 100 Years ); the technocratic-pluralism model in which the authors never get around to making falsifiable predictions of their own but do offer readers a pretty comprehensive survey of forecasting mistakes and an inventory of tools for avoiding them (a category into which I place Ian Bremmer and Preston Keat's The Fat Tail ); and the scientific-reductionist model in which the author embraces a particular theory from the social sciences and shows how, if you apply that theory thoughtfully to real-world contexts, you can derive surprisingly accurate forecasts (a category into which I place Bruce Bueno de Mesquita's The Predictioneer's Game ).

[amazon 1400067871 full] Reading these three books, it is easy to feel like a frustrated shopper wandering aimlessly down the forecasting aisle in the supermarket of ideas. The products on offer are packaged well-but we have no objective benchmarks, no trusted Consumer Reports , against which to gauge performance. We have no idea whether we would be better-off paying one of these consultancies gobs of money for their proprietary forecasts or simply downloading the latest odds from a high-profile prediction market that culls individual bets on world events such as Tradesport. Indeed, would we do as well relying on the dart-throwing chimps or mindless extrapolation rules, like "Predict the most recent rate of change"? So, caveat emptor.


FIRST, THE superpundit model.

George Friedman, the author of The Next 100 Years , is also the founder and CEO of STRATFOR, "the world's leading private intelligence and forecasting company." Of the three books, his was the least persuasive. But Friedman does his readers at least one good turn. He repeatedly warns them of how easy it is for the conventional wisdom, anchored as it is in the status quo, to underestimate cumulative change over multidecade spans of time. How many in 1945 would have predicted that Japan and Germany would in thirty-five years be the second- and third-most powerful economies in the world, or that China in 1970-still in the convulsive grip of Mao's Great Proletarian Cultural Revolution-would be the fourth-biggest economy in 2007 and moving up the rankings fast? The answer is "extremely few." Yet these outcomes now have the whiff of inevitability about them. Appreciating the power of hindsight bias makes us think harder about how we think: if we can "explain" the past so easily and predict the future so poorly, perhaps we should reconsider how we go about doing both.

Essay Types: Book Review