Paul Romer on mathiness

Error message

  • Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in _menu_load_objects() (line 579 of /var/www/drupal-7.x/includes/menu.inc).
  • Deprecated function: implode(): Passing glue string after array is deprecated. Swap the parameters in drupal_get_feeds() (line 394 of /var/www/drupal-7.x/includes/common.inc).
Published by Anonymous (not verified) on Sun, 17/05/2015 - 2:02am


Top growth theorist Paul Romer has an essay in the AER Papers & Proceedings, in which he comes down harshly "mathiness" in growth theory. "Mathiness" is his term for when people (allegedly) use math in a sloppy way, to support their preferred theories. Romer warns direly that the culture of econ theory has become a lot more tolerant of mathiness:

If mathiness were used infrequently,...it would do localized, temporary damage. Unfortunately...as the quantity increases, mathiness could do permanent damage because it takes costly effort to distinguish mathiness from mathematical theory. 

The market for mathematical theory can survive a few...articles filled with mathiness. Readers will put a small discount on any article with mathematical symbols, but will still find it worth their while to work through and verify that the formal arguments are correct, that the connection between the symbols and the words is tight, and that the theoretical concepts have implications for measurement and observation. But after readers have been disappointed too often by mathiness that wastes their time, they will stop taking seriously any paper that contains mathematical symbols. In response, authors will stop doing the hard work that it takes to supply real mathematical theory. If no one is putting in the work to distinguish between mathiness and mathematical theory, why not cut a few corners and take advantage of the slippage that mathiness allows? The market for mathematical theory will collapse. Only mathiness will be left. It will be worth little, but cheap to produce, so it might survive as entertainment. 

[I]n the new equilibrium: empirical work is science; theory is entertainment. Presenting a model is like doing a card trick. Everybody knows that there will be some sleight of hand. There is no intent to deceive because no one takes it seriously. Perhaps our norms will soon be like those in professional magic; it will be impolite, perhaps even an ethical breach, to reveal how someone’s trick works. 

When I learned mathematical economics, a different equilibrium prevailed. Not universally, but much more so than today, when economic theorists used math to explore abstractions, it was a point of pride to do so with clarity, precision, and rigor...If we have already reached the lemons market equilibrium where only mathiness is on offer, future generations of economists will suffer.

Romer has now joined the chorus of old famous guys - Krugman, Solow, Stiglitz, Farmer - who are very vocally mad about the way mainstream economics theory is done.
Romer is not afraid to name names. Interestingly, although he's talking only about growth theory and not about business cycle theory, most of the people he's mad at are the same guys that the Keynesians are mad at - Robert Lucas, Ed Prescott, and David K. Levine. He also calls out Thomas Piketty.
Romer gives specific examples of what he calls mathiness (links are to working-paper versions):
1. Prescott and McGrattan (2010): Romer says that this paper includes a term that the authors label "location," but that doesn't correspond to any real measure of location.
2. Boldrin and Levine (2008): Romer criticizes this paper for assuming that a monopolist would also be a price-taker, and for making various hand-wavey arguments.
3. Lucas (2009): Romer criticizes this paper for making a hand-wavey argument to dismiss the idea that investment in embodied technology (books, blueprints, etc.) can be a source of sustained growth, when there are well-known models in which it can. Romer also points out a random math error in the paper, and uses this to argue that reviewers don't pay close attention to math.
4. Lucas and Moll (2014): Romer criticizes this paper especially harshly. Lucas and Moll claim that their model, in which there is no creation of new knowledge, is "observationally equivalent" to models in which new knowledge arrives very slowly. Romer shows that the truth of this claim depends on which order you use when taking a double limit. He reveals that he told the authors about the problem, but that they ignored him and left it in the paper.
5. Piketty and Zucman (2014): Romer points out the by now well-known "gross vs. net" problem in Piketty and Zucman's definition of savings.

All in all, this seems like a pretty loose collection of criticisms. Hand-wavey arguments, dubious definitions, bad assumptions, and math errors are all very different things. So this essay at first can seem like a grab-bag of gripes that Romer has with individual rivals' papers.
But I think Romer is on to something about the culture of econ theory, at least in the "macro"-ish realms of growth, business cycle, macro-labor, macro-trade, and macro-tax theory (I don't know nearly as much about the culture of the "micro"-ish fields like game theory, decision theory, I/O, etc.; and I know that finance theory has a very different culture). In these "macro"-ish fields, people seem to view math more as a tool for stylized description of ideas than as a tool for quantitative prediction of observables. 
Romer's examples of "mathiness" are all very recent examples. But going back to earlier models, I don't really see much more tight connection of variables to observables. Yes, in a Solow model you can tie capital K to observable things like structures and machines and vehicles. But you'll be left with a big residual, A. Then you can break A down and extract another term for human capital, H. Can you really measure human capital? Human capital can't be bought and sold on a market; you have to bundle it with other goods. So it's very difficult to get a clean measurement of the value of the existing stock of human capital, the way you could get a clean measurement of the existing stock of delivery trucks. Romer cites human capital as a good example of non-"mathiness", but I don't really see a huge difference between that and the "location" used by Prescott and McGrattan (2010). Maybe a minor difference, but not a huge one. As for the remaining A, there's not really any quantitative way to measure the stock of ideas except as a residual. And as for goofy assumptions, well, any growth model is going to have at least one or two assumptions that would make a newcomer to the econ field throw up her hands in disbelief.
Mathiness isn't anything new, it's just the way these econ fields work. The math is there as a storytelling aid (and possibly as a signal of intellectual ability). I think Karthik Athreya said it best:

My view is that a part of what we do is "organized storytelling, in which we use extremely systematic tools of data analysis and reasoning, sometimes along with more extra-economic means, to persuade others of the usefulness of our assumptions and, hence, of our conclusions...This is perhaps not how one might describe "hard sciences".

Do the guys Romer calls out play a little faster and looser with definitions and rely more on hand-wavey arguments? Oh, I'm sure they do - but that's because they're famous old guys. Writing down hand-wavey stuff is a privilege afforded to famous old guys in every academic discipline I know of. In econ, it gets politely published in top journals, but all the hotshot young people just sort of shake their heads anyway, and the only net effect is to pad out the length of the journals. Are the guys Romer calls out more political than the average economist? Maybe.
But in general, the whole discipline of macro theory - in the general sense, including growth and parts of labor, trade, and tax theory - is chock full of mathiness. Even most of the best models ("best" being a highly relative term, of course). The original Solow model seems to me like a rare exception, not a typical example of the Good Old Stuff.

But in any case, I highly recommend the Romer piece, which is a master class in catching errors in models, as well as a fascinating window into the Byzantine world of academic politics.

Update: Brad DeLong has a follow-up post explaining his view of some of the history behind the argument in the field of growth economics. Basically, the idea is that George Stigler didn't like people using models with imperfect competition, since this might open up a window for government intervention. DeLong thinks that Lucas and other "freshwater" types inherited this anti-imperfect-competition bias, causing them to be too down on Romer's models. This is interesting history that I didn't really know about before.

Update 2: Paul Romer has a response on his blog. Excerpts:

Noah Smith asks for more evidence that the theory in the McGrattan-Prescott paper that I cite is any worse than the theory I compare it to by Robert Solow and Gary Becker... 

There is no such thing as the perfect map. This does not mean that the incoherent scribbling of McGrattan and Prescott are on a par with the coherent, low-resolution Solow map that is so simple that all economists have memorized it. Nor with the Becker map that has become part of the everyday mental model of people inside and outside of economics... 

Noah’s jaded question–Is the theory of McGrattan-Prescott really any worse than the theory of Solow and Becker?–may be indicative of what many economists feel after years of being bullied by bad theory. And as I note in the paper, this resignation may be why empirically minded economists like Piketty and Zucman stay as far away from theory as possible... 

For specific purposes, some maps are better than others. Sometimes a subway map is better than a topographical map. Sometimes it is the other way around. Starting with any good map, we can always increase the resolution and add detail. 

No map is perfect, but this does not mean that all maps are equal. It certainly does not mean that an internally consistent map that with so little detail that you can memorize it is on a par with incoherent scribbling.

In the rest of the post, he goes into depth about why he thinks the McGrattan and Prescott paper constitutes "incoherent scribbling." But he also notes that the other papers he goes after in his "mathiness" piece should not be let off the hook:

Noah also notes that I go into more detail about the problems in the Lucas and Moll (2014) paper. Just to be clear, this is not because it is worse than the papers by McGrattan and Prescott or Boldrin and Levine. Honestly, I’d be hard pressed to say which is the worst. They all display the sloppy mixture of words and symbols that I’m calling mathiness. Each is awful in its own special way.

He also has another short post about a Lucas paper.

It appears that Prescott, Lucas, Levine, and others of the unofficial "freshwater" club have annoyed more high-level colleagues than just Paul Krugman. Only a few months ago, Roger Farmer took to his blog to unleash an anti-Prescott blast. Romer and Farmer are not politically-minded media-engaged types like Krugman and DeLong, but their aggravation with the Lucas/Prescott school is, if anything, even more intense.

And yes, Romer is right that I'm jaded. Is it so obvious? *takes swig from hip flask, rubs beard stubble*