Revising the two RRs

Wow!  What a storm in mainstream economics has been caused by the revelations of a new paper undermining the data and thus the conclusions of one of the most famous studies on the relationship between the size of public debt and economic growth.  In 2010, economists Carmen Reinhart and Kenneth Rogoff released a paper, “Growth in a Time of Debt.”  Their “main result is that…median growth rates for countries with public debt over 90 percent of GDP are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower.”  Indeed, countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate.

And this is a big deal because politicians around the world have used this finding from R&R to justify austerity measures.  By austerity measures, we mean that cutting government spending, eliminating budget deficits and reducing levels of public debt are necessary in order to restore long-term economic growth.  More borrowing and increasing spending would be disastrous, so the austerity argument goes.   In the US, many politicians have pointed to R&R’s work as justification for deficit reduction.  Paul Ryan, the Republican chair of the House Budget committee who has pushed for rapid fiscal tightening in the US, cited the Reinhart-Rogoff study as “conclusive empirical evidence that total debt exceeding 90 per cent of the economy has a significant negative effect on economic growth”. The Washington Post claimed it was economic consensus view that if the “debt-to-GDP could keep rising and stick dangerously near the 90 percent mark economists regard that as a threat to sustainable economic growth.”  In Europe, R&R’s work has also been used to justify austerity policies.

Now, in a new paper, “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff,” Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst, using R&R’s own data, find that Reinhart and Rogoff selectively exclude years of high debt and average growth. They use a debatable method to weight the countries. And amazingly, there also appears to be a basic coding error in their excel spreadsheets that excludes high-debt and average-growth countries.  Reinhart-Rogoff use 1946-2009 as their period, with the main difference among countries being their starting year. In their data set, there are 110 years of data available for countries that have a debt/GDP over 90 percent, but they only use 96 of those years. The R&R paper didn’t disclose which years they excluded or why. Herndon-Ash-Pollin find that they exclude Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). This has consequences, as these countries have high-debt and solid growth. Canada had debt-to-GDP over 90 percent during this period and 3 percent growth. New Zealand had a debt/GDP over 90 percent from 1946-1951. If you use the average growth rate across all those years it is 2.58 percent. If you only use the last year, as Reinhart-Rogoff does, it has a growth rate of -7.6 percent. That’s a big difference, especially considering how they weigh the countries.

Also Reinhart-Rogoff divide country years into debt-to-GDP buckets. They then take the average real growth for each country within the buckets.  So the growth rate of the 19 years that U.K. is above 90 percent debt-to-GDP are averaged into one number. These country numbers are then averaged, equally by country, to calculate the average real GDP growth weight.  The U.K. has 19 years (1946-1964) above 90 percent debt-to-GDP with an average 2.4 percent growth rate. New Zealand has one year in their sample above 90 percent debt-to-GDP with a growth rate of -7.6. These two numbers, 2.4 and -7.6 percent, are given equal weight in the final calculation, as they average the countries equally. Even though there are 19 times as many data points for U.K.   This weighting significantly reduces the average; if you weight by the number of years you find a higher growth rate above 90 percent.

Finally, As Herndon-Ash-Pollin puts it: “A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49…This spreadsheet error…is responsible for a -0.3 percentage-point error in RR’s published average real GDP growth in the highest public debt/GDP category.”  Belgium, in particular, has 26 years with debt-to-GDP above 90 percent, with an average growth rate of 2.6 percent (though this is only counted as one total point due to the weighting above).  And this error is needed to get the results they published, and it would go a long way to explaining why it has been impossible for others to replicate these results.  So what do Herndon-Ash-Pollin conclude? They find “the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim].” And they are unable to find a break point where growth falls quickly and significantly.

R&R responded quickly to this attack on their work: “We literally just received this draft comment and will review it in due course,” they said.  But they argued that “The weight of the evidence to date – including this latest comment – seems entirely consistent with our original interpretation of the data”   Really?  As Paul Krugman put it in his blog (http://krugman.blogs.nytimes.com/2013/04/16/reinhart-rogoff-continued/), “R&R’s “response to the new critique is really, really bad.  First, they argue that another measure — median growth — isn’t that different from the Herndon et al results. But that is, first of all, an apples-and-oranges comparison — the fact is that when you compare the results head to head, R-R looks very off. Something went very wrong, and pointing to your other results isn’t a good defense. Second, they say that they like to emphasize the median results, which are much milder than the mean results; but what everyone using their work likes to cite is the strong result, and if R-R have made a major effort to disabuse people of the notion that debt has huge negative effects on growth, I haven’t noticed it   So this is really disappointing; they’re basically evading the critique. And that’s a terrible thing when so much is at stake.”  

Two eminent mainstream economists from the most prestigious and conservative Harvard University whose studies on debt have been used by everyone as a defence of the policies of austerity (although not openly by R&R themselves) have been found to distort, mislead and make basic errors in their stats!  It does not make you very confident about mainstream economics, if you ever were.

In several previous posts, I have called on R&R’s data and conclusions to throw doubt on the arguments of Keynesians like Paul Krugman and others who have argue dthat that rising and high debt does not matter. For example, this is what I said in March 2010 “Recent research by two leading capitalist economists, Carmen Reinhart and Kenneth Rogoff (see economics.harvard.edu/faculty/…/This_Time_Is_Different.pdf) , has shown that when in any country, the public debt ratio reaches 90% of GDP, that will reduce its economic growth rate by 1.0-1.5% pts a year.  So an economy that used to grow on average at 3% a year will now grow at only 1.5%.  If that is borne out, it will really hit the ability of capitalism to provide jobs, services and wage rises.” And again in May 2012,  I said  “In their very latest report, the historians of debt, the Reinharts and Kenneth Rogoff confirm the relationship between debt and growth under capitalism (see Carmen M. Reinhart, Vincent R. Reinhart, and Kenneth S. Rogoff, Debt Overhangs: Past and Present, NBER Working Paper No. 18015 (April 2012)).  They looked at 26 episodes of public debt overhangs (defined as where the public debt ratio was above 90%) and found that on 23 occasions, real GDP growth is lowered by an average of 1.2% points a year.  And GDP is about 25% lower than it would have been at the end of the period of overhang.”  So the revelation that the data and methods of  R&R are riddled with errors and doubtful manipulations is disconcerting.

R&R have now come back with a more considered response to the new paper’s criticisms (http://blogs.ft.com/ftdata/2013/04/17/the-reinhart-rogoff-response-i/). R&R accept that there is a coding error on their data but reject the other criticisms.  They conclude “So do where does this leave matters on debt and growth? Do Herndon et al. get dramatically different results on the relatively short post war sample they focus on? Not really. They, too, find lower growth associated with periods when debt is over 90 per cent. Put differently, growth at high debt levels is a little more than half of the growth rate at the lowest levels of debt. They ignore the fact that these results are close to what we get in our Table 1 of our AER paper they critique, and not far from the median results in Figure 2 despite its coding error. And they are not very different from what we report in our 2012 Journal of Economic Perspectives paper with Vincent Reinhart—where the average is 2.4 per cent for high debt versus 3.5 per cent for below 90 per cent.”

But before we write off all of R&R’s work, I’d make the following points.  First, there is other research that makes similar claims, including more recent work by Reinhardt and Rogoff.  A R&R put it in their reply to the attack on their work: “our 2012 JEP paper cites papers from the BIS, IMF and OECD (among others) which virtually all find very similar conclusions to original findings, albeit with slight differences in threshold, and many nuances of alternative interpretation. These later papers, by the way, use a variety of methodologies for dealing with non-linearity and also for trying to determine causation. Of course much further research is needed as the data we developed and is being used in these studies is new. Nevertheless, the weight of the evidence to date –including this latest comment — seems entirely consistent with our original interpretation of the data in our 2010 AER paper.”

For example, the BIS by Stephen G Cecchetti, M S Mohanty and Fabrizio Zampolli’s, The Real Effects of Debt, published in September 2011, analysed OECD countries between 1980 and 2010. It also found that, beyond a certain level, debt is a drag on growth. They conclude that for government debt, the threshold is around 85 per cent of GDP; for corporate debt it is 90 per cent of GDP and for household debt it is also around 85 per cent.   The McKinsey Institute also published a very illuminating study arguing that deleveraging usually took four to seven years after a crisis breaks (see Debt and deleveraging, mckinsey.com/mgi/…/debt_and_deleveraging_full_report.pdf).  And in a recent World Economic Outlook, April 2012, IMF researchers outlined the evidence that debt does make a difference both to the depth of the crisis and the strength of the recovery, concluding that: “recessions preceded by economy-wide credit booms tend to be deeper and more protracted than other recessions”  (p96) and “housing busts preceded by larger run-ups in gross household debt are associated with deeper slumps, weaker recoveries and more pronounced household deleveraging“, p115).   Also, Owen Zidar pulled data from R&R’s AER paper on debt to GDP and economic growth to see if there is any evidence of a break at a 90 (http://owenzidar.wordpress.com/2013/04/16/debt-to-gdp-future-economic-growth/). He found that persistently 90+ Public Debt to GDP countries tend to grow less quickly over next 5 years, but the distribution of historical outcomes suggests that disaster scenarios are much less common than just modestly slower 5 year growth for these countries.  And the data show that there’s no special break at a debt to GDP ratio of 90.

And second, even if there is still evidence of a good correlation between high debt levels and low growth rates, the issue of causality remains.  Is it high debt that causes low growth or low growth that leads to high debt?  Countries could have high debt-to-GDP ratios because they are having serious economic problems.  The years of high debt in the US are to be found at the end of the 1939-45 war and it quickly came down as the economic boom began. In the past debate on causality, R&R have dismissed this criticism as wishful thinking. “We’re quite aware that you have causality going in both directions,” said Reinhart. “But please point out to me what episodes from 1800 to the present have we had advanced economies who carried high levels of debt growing as rapidly or more rapidly than the norm.” Belgium after World War I, she says, fits the bill, but that’s basically it. “It’s not about some exotic magic threshold where you cross the Rubicon,” she says. “But high debt levels are like a weak immune system.”