Defending Defense’s Dubious Data

March 11, 2011 Topic: EconomicsDefense Region: China Blog Brand: The Skeptics

Defending Defense’s Dubious Data

Think tanks should be focused on good scholarship, not advocacy.

I am continually amazed by the shenanigans people get away with in the think tank community. Take, as one recent case, a sentence from the recent “Defending Defense” report on China’s military buildup:

If one uses the purchasing power parity (PPP) exchange rate, which accounts for [differences between personnel and manufacturing costs in China and the United States], the Chinese core military budget may well approach $300 billion, making it the second largest in the world. (emphasis mine)

That sentence is tied to an endnote which reads, “Calculating defense purchase power is inherently an imprecise science. The $300 billion figure represents the conclusion of no one study, but rather a rough average of several, including those sponsored by the U.S.-China Economic and Security Review Commission.”

Wednesday afternoon, I emailed the only person whose name was listed on the report, Richard Cleary, an AEI research assistant, asking which studies the authors used to arrive at this figure, because it is wildly out of step with every serious estimate of Chinese military spending I have ever seen. I have received no response from Cleary.

I am forced to conclude that Defending Defense has chosen not to defend the $300 billion figure because they know it is indefensible.

I have dealt with wild claims about Chinese military spending before. In 2007, John J. Tkacik, Jr., then a senior research fellow at the Heritage Foundation, and Heritage President Ed Feulner both alleged that Chinese spending amounted to roughly $450 billion. (Heritage appears to have reorganized their website, so the link to Tkacik in my original piece goes to a different article. Another version of Tkacik making the claim is here.)

The problem with what Tkacik and Feulner were doing is that they were using extremely high-end estimates of Chinese military expenditures at market exchange rates (MER) and then blanketing that total figure with the PPP deflator, yielding a figure around $450 billion US. As I noted at the time, you can’t do that. I realize that defense economics is boring, but this stuff is important. To make a complex story simpler, you use PPP for defense expenditures that are indigenously produced (labor, for instance) and MER for goods and services that touch world markets.

I have long considered the International Institute for Strategic Studies’ Military Balance to be the lodestar for working on this problem. As they have long made clear, and as they note in their current report,

No specific PPP rate exists for the military sector, and its use for this purpose should be treated with caution. Furthermore, there is no definitive guide as to which elements of military spending should be calculated using the limited PPP rates available. The figures presented here are only intended to illustrate a range of possible outcomes depending on which input variables are used.

This is a very conscientious attitude toward a very tricky problem. The current IISS estimate places Chinese expenditures at approximately $166 billion, using what they view as an appropriate mixture of PPP and MER for calculations. (The latest IISS figure that was available when the Defending Defense report was being prepared was $114 billion.)

It is entirely possible that the IISS researchers are wrong, but are they wrong by a factor of two? And if Defending Defense wishes to allege that they are, shouldn’t someone be willing to make the argument why? If Defending Defense wants a reputation for producing good scholarship, rather than being an advocacy group, someone needs to step up and defend their numbers.

Until and unless that happens, I simply note that people would not be allowed to get away with this sort of thing in the academy, and we should not allow it in think tanks, either.