Part 4—The New York Times lets them pretend: In early 2010, two highly-regarded Harvard professors authored a bungled study.
Their paper proved to be quite influential, in some destructive ways. But the paper included some gruesome mistakes—and for reasons no one has yet explained, it took three years for the world to find out.
On April 19, Paul Krugman described “the story so far” in his New York Times column. Near the start of his piece, he described the paper’s key finding. He also described the acclaim the paper received:
KRUGMAN (4/19/13): At the beginning of 2010, two Harvard economists, Carmen Reinhart and Kenneth Rogoff, circulated a paper, ''Growth in a Time of Debt,'' that purported to identify a critical ''threshold,'' a tipping point, for government indebtedness. Once debt exceeds 90 percent of gross domestic product, they claimed, economic growth drops off sharply.According to Krugman, the professors had claimed that a nation’s economic growth drops sharply once that nation’s debt exceeds 90 percent of GDP. Krugman put the word “threshold” inside quotes, suggesting that the professors had used that term to describe the tipping-point their paper claimed to establish.
Ms. Reinhart and Mr. Rogoff had credibility thanks to a widely admired earlier book on the history of financial crises, and their timing was impeccable. The paper came out just after Greece went into crisis and played right into the desire of many officials to ''pivot'' from stimulus to austerity. As a result, the paper instantly became famous; it was, and is, surely the most influential economic analysis of recent years.
In fact, Reinhart-Rogoff quickly achieved almost sacred status among self-proclaimed guardians of fiscal responsibility; their tipping-point claim was treated not as a disputed hypothesis but as unquestioned fact.
As it turned out, the professors had made at least one embarrassing error in their now-famous study. (Krugman claims other mistakes.) Three years after their paper appeared, three graduate students checked their work, then reported the gruesome error.
Since the paper was so influential, why hadn’t anyone else ever checked the professors’ work? We still don’t know the answer to that. But one week after Krugman’s column appeared, the professors did what they must:
They published their own op-ed column in the New York Times! They did acknowledge one gruesome mistake, but they said the “main finding” of their paper had not been overturned by the graduate students. They also rejected Krugman’s account of what their main finding was—although they only said they were contradicting unnamed “conservative politicians.”
This left us with several questions:
What was the “main finding” of Reinhart and Rogoff’s original paper? How had this finding been so misunderstood, not only by those conservative pols, but also by the progressive Krugman, whose account of their work went unmentioned in their op-ed column?
We thought Reinhart and Rogoff’s piece was rather fuzzy on these key points, and that the Times had enabled the fuzz. Let’s review what the famous professors said in their op-ed column.
As they started, the famous professors defined their paper’s “main finding.” In our book, what follows is rather fuzzy. Do you understand what they said?
REINHART AND ROGOFF (4/26/13): In May 2010, we published an academic paper, ''Growth in a Time of Debt.'' Its main finding, drawing on data from 44 countries over 200 years, was that in both rich and developing countries, high levels of government debt—specifically, gross public debt equaling 90 percent or more of the nation's annual economic output—was [sic] associated with notably lower rates of growth.According to the professors, their paper’s main finding was the following: Over the past 200 years, “high levels of government debt” have been “associated with notably lower rates of growth.”
Given debates occurring across the industrialized world, from Washington to London to Brussels to Tokyo, about the best way to recover from the Great Recession, that paper, along with other research we have published, has frequently been cited—and, often, exaggerated or misrepresented—by politicians, commentators and activists across the political spectrum.
Such a finding is perfectly possible, of course. But what specific “high levels of government debt” did the professors have in mind? Specifically, they were talking about “gross public debt equaling 90 percent or more of the nation's annual economic output,” they said. Such debt levels have been “associated with notably lower rates of growth.”
Can we talk? To our ear, it almost sounded like the pair were describing some sort of threshold or tipping-point! If we interpret that formulation literally, it sounds like a country would be OK if its debt level was 89 percent of GDP—but that notably lower rates of growth have been associated with a debt level of 90 percent!
That seems unlikely, but reading literally, that seems to be what the professors said. But hold on! A bit later, they criticized the work of those graduate students—and they described their own paper’s “fundamental finding” again:
REINHART AND ROGOFF: Our research, and even our credentials and integrity, have been furiously attacked in newspapers and on television. Each of us has received hate-filled, even threatening, e-mail messages, some of them blaming us for layoffs of public employees, cutbacks in government services and tax increases. As career academic economists (our only senior public service has been in the research department at the International Monetary Fund) we find these attacks a sad commentary on the politicization of social science research. But our feelings are not what's important here.Must that highlighted passage mean what it seems to say? Read literally, that passage does seem to describe a threshold for lower growth, or a tipping-point.
The authors of the paper released last week—Thomas Herndon, Michael Ash and Robert Pollin—say our ''findings have served as an intellectual bulwark in support of austerity politics'' and urge policy makers to ''reassess the austerity agenda itself in both Europe and the United States.''
A sober reassessment of austerity is the responsible course for policy makers, but not for the reasons these authors suggest. Their conclusions are less dramatic than they would have you believe. Our 2010 paper found that, over the long term, growth is about 1 percentage point lower when debt is 90 percent or more of gross domestic product. The University of Massachusetts researchers do not overturn this fundamental finding, which several researchers have elaborated upon.
“Over the long term, growth is about 1 percentage point lower when debt is 90 percent or more of gross domestic product,” the professors say, describing their paper’s fundamental finding. Read literally, would this not imply that growth would be one point lower at 90 percent than at 89 percent?
That seems like a dramatic jump. But isn’t that what the professors seem to say in this rather fuzzy passage?
Can we talk? The professors do seem to be describing some sort of tipping-point! Perhaps they don’t mean to do that, but that’s the way it sounds. But hold on! As they continued, the professors almost seemed to offer a third account of their famous finding.
Warning! They are now describing what they said in a paper they wrote last year! But it almost sounds like they’re fleshing out their initial “main finding:”
REINHART AND ROGOFF (continuing directly): The academic literature on debt and growth has for some time been focused on identifying causality. Does high debt merely reflect weaker tax revenues and slower growth? Or does high debt undermine growth?The professors are now describing a paper from 2012. With that in mind, please note the non-denial denial they have now authored:
Our view has always been that causality runs in both directions, and that there is no rule that applies across all times and places. In a paper published last year with Vincent R. Reinhart, we looked at virtually all episodes of sustained high debt in the advanced economies since 1800. Nowhere did we assert that 90 percent was a magic threshold that transforms outcomes, as conservative politicians have suggested.
We did find that episodes of high debt (90 percent or more) were rare, long and costly. There were just 26 cases where the ratio of debt to G.D.P. exceeded 90 percent for five years or more; the average high-debt spell was 23 years. In 23 of the 26 cases, average growth was slower during the high-debt period than in periods of lower debt levels. Indeed, economies grew at an average annual rate of roughly 3.5 percent, when the ratio was under 90 percent, but at only a 2.3 percent rate, on average, at higher relative debt levels.
(In 2012, the ratio of debt to gross domestic product was 106 percent in the United States, 82 percent in Germany and 90 percent in Britain—in Japan, the figure is 238 percent, but Japan is somewhat exceptional because its debt is held almost entirely by domestic residents and it is a creditor to the rest of the world.)
In last year’s paper, the professors say, they didn’t “assert that 90 percent was a magic threshold that transforms outcomes, as conservative politicians have suggested.” But various people have said that they did make some such assertion in their earlier 2010 paper—in the paper which has been under review, the paper which was influential.
And of course, it wasn’t just “conservative politicians” who made this claim about their earlier paper. Paul Krugman said the same thing on April 19, just one week before the professors wrote their own op-ed column! (Krugman didn’t use the word “magic,” of course. The professors used that term to burlesque the claims of their critics.)
At this point, we’re going to stop, perhaps from a sense of embarrassment. The professors have now shifted their field; they are now describing what they said in a paper they published last year, not in the original paper which has come under such criticism. As they pull this bait-and-switch, they describe a less than earth-shattering finding: On average, countries with debt levels less than 90 percent have stronger growth, by about one point, than countries with debt levels higher than 90 percent.
Clearly, this involves no “magic threshold.” Nor is the claim all that shocking, since they’re talking about averages for two groups of countries, some of which may have very high or very low debt levels. In this passage, the threshold or tipping-point seems to be gone—but the professors are no longer describing the paper which has been under review!
Did the professors describe a “threshold,” a tipping-point, in their original paper? As recently as April 19, Krugman said they did. One week later, the professors responded—or pretended to respond.
What happened when the professors responded? In their first two attempts to describe their “main finding,” they did seem to describe something resembling a threshold. Then, they issued a classic non-denial denial—they denied that they said such a thing in a totally different paper! They blamed “conservative politicians” for misconstruing their paper’s main finding, but they failed to say that Krugman had said the very same thing just one week before.
This was a horrible column—and we're not even graduate students! But this sort of thing often occurs when the nation’s professors turn to the Times, seeking the great paper’s succor.
For years, the Times has been involved in a love affair with the nation’s often useless professors. This has produced a horrible culture at the newspaper, even perhaps in the groves of our massively failed academe.
Tomorrow: Bad love affair, horrible culture
Thankѕ on your marvelouѕ рostіng!ReplyDelete
I truly еnjoyеd reading it, уou will be a great author.
I will bе sure to bookmark your blοg and wіll evеntuаlly comе back from
now οn. I want tо encourаgе you tо cοntinue
your great writing, havе а niсe
Reѵieω my blоg post :: zootoo.com
R&R were content to sit quietly while their work was widely bruited as showing that there was a 90% debt/GDP threshold, above which countries would experience "notably lower rates of growth" than countries not exceeding that threshold.ReplyDelete
This 2010 paper of R&R's was quite broadly cited as proving that US debt must be reined in below 90% of GDP or US the US economy would suffer that low growth.
While this usage of their work (work now known to be deeply flawed, with both outright errors and biased selection and omission of data) continued, R&R did not speak up to say "NO no, that's not what we said."
R&R were darlings of the austerity crowd (a crowd that encompassed many "liberals" and "centrists" as well as "conservatives"). And R&R were to all appearances quite content to be celebrated and widely cited in that role.
Now their house of cards has fallen in on them. Their errors (and selection biases) have been pointed out.
NOW, finally, they object to the "misuse" of their work. Too little, too late. And they obfuscate and prevaricate even now, pretending their 2010 work still has some value.
A minor quibble with your otherwise excellent post:ReplyDelete
Michael Herndon's co-author's are actually Professors at University of Massachusetts, not fellow graduate students. I believe the study that Herndon performed was a project for a graduate level economics class. Professors Ash and Pollin were his advisers.
It's not a minor point.Delete
Somerby has been reminded of this misstatement previously by numerous commenters and done nothing to correct it. Having to concede that professors contributed to discrediting the 2010 R & R dissembling is simply not in Somerby's DNA.
This is the third or fourth post in which TDH has made that "graduate students" claim. It seems to be a deliberate lie. I suppose he has some kind of theory why it isn't a lie, but he hasn't shared it with the rest of us, and he keeps repeating the claim even though the error has been identified in comments from the beginning.ReplyDelete
Therefore I'm not sure we should treat it as a minor quibble.
We chuckle. (To use Somerby's “we” affectation.)ReplyDelete
Here we have a blog devoted to documenting and denouncing the outrageous and forever uncorrected false narratives created by the media over the past decade or two, and yet the Exalted Impresario of this blog, Somerby, leaves uncorrected his own false narrative, despite being informed by numerous commenters of his error. Can you not just laugh? Of course, at this point in my life I still laugh at such human behavior. But so did Mark Twain at a comparable stage of his life--before he developed full-blown misanthropy towards the end and began spewing pure bile. So, urban legend, I may yet find myself someday sharing your more harshly condemnatory attitude towards Somerby.
I would disagree with you, though, when you call it a “deliberate lie”. To me, it's far more plausible that Somerby made a simple mistake—but feels acknowledging it would somehow illegitimize his life's work of ridiculing and decrying other members of the media for their mistakes. And even though any reader is free to peruse these comments and learn about Bob's errors, he's counting on the fact that probably few people do so. So he's made a simple calculation: correct the mistake in his next blog post on this subject--thereby, in his mind, hurting his credibility with ALL his readers, or pretend he's unaware of his mistake (I say PRETEND he's unaware because I can't believe at this point he still thinks all three are grad students) and preserve his image of omniscience with most readers while making himself appear profoundly pathetic in the eyes of those few who grasp what's going on. I can see his choosing the first option on the basis of Somerby Utilitarianism, an almost mathematical calculation of the greatest good—for Bob Somerby.
Of course, urban legend, you might respond, with some persuasive force, that a knowingly uncorrected mistake is tantamount to a deliberate lie, but to me, though both might equally mislead people, and both might be equally understood by their author to be misleading people, the matter of prior intent (malice aforethought) differentiates one from the other and puts Somerby's offense into a less egregious category.
"Malice aforethought" would occur each time the false statement is repeated if at any time along the way it is known to be false. My question is, why the hell would he keep doing it? It's a completely unnecessary distraction -- consumes all the comments -- from an otherwise useful analysis, especially grating when it's from someone whose criticism at minimum borders on the self-righteous.Delete
Urban legend, morally it's a delicate matter differentiating what Somerby is doing in repeating something he NOW knows to be false because he can't bear to acknowledge error, and what he would have been doing if had knowingly made a false statement initially. In my eyes, a “DELIBERATE lie” connotes a malevolent plan of which lying is an inherent part right from the start. It's your word “deliberate” that I take exception to, because it powerfully suggests that he had premeditatedly embarked on a course of deception. I believe Somerby is lying, to be sure, but belatedly, and defensively, because he foolishly thinks he must to preserve his credibility.Delete
You might argue that “deliberate” merely means “intentional” and since what Somerby is doing is clearly intentional then you would conclude that you're correct in terming it a deliberate lie. But here's why that argument fails. A merely false statement is not necessarily a lie—in order to qualify as a lie it must be knowingly false: in other words it must be intentional. Therefore if “deliberate” in this context merely meant “intentional” it would be redundant. So when used in the phrase “deliberate lie” the word “deliberate” takes on a special meaning, beyond merely “intentional”--the one that I described in the first paragraph. In any event, this is my own “feel” for how the phrase “deliberate lie” is used.
As to “why the hell he would keep doing it”: that's a fascinating question that I can't answer right now because the midnight hour is approaching where I am, and I must get to sleep. But tomorrow I'll take a stab at it.
I don't think Bob has been reading the comments lately. I'm going to say he is just making and perpetuating a mistake about the "three graduate students" He strikes me as too intellectual honest to be intentionally doing this. Then again, 2 professors and their grad student doesn't have quite the same oomph. BTW many influential papers were written when the authors were graduate students, and usually published after they completed their Ph.D.'s. The fact that a Grad student made this discovery is not that important.ReplyDelete
I made the point re the other two authors being Herndon's professors in an email to Bob. So, he is either not reading or ignoring his emails as well.Delete
It would be completely understandable that a grad student would appreciate the additional credibility and support that would come from two full professors. In an interview, Herndon said that he felt scared when he could not replicate the results and when he "discovered" the causes after reviewing the original spreadsheet that R&R provided after numerous (three?) emails.
[BTW, Dean Baker, Beat The Press, always responds to comments and emails]
To the second Anonymous, I think that the professors provided more than just credibility and support. I believe that they turned the prospective paper from a mere expose of the Excel error into a more thorough-going critique of R-R's methodology, which Paul Krugman feels was shockingly bad on multiple fronts.Delete
By the way, it's good to know you sent an email to Somerby alerting him to his error. So my supposition that he almost certainly has become of aware of it by now, one way or the other, seems valid.
Oops, I meant I can see Somerby NOT choosing the first option on the basis of Somerby utilitarianism.ReplyDelete