Friday, January 10, 2014

The most damning critique of DSGE



If DSGE models work, why don't people use them to get rich?

When I studied macroeconomics in grad school, I was told something along these lines:

"DSGE models are useful for policy advice because they (hopefully) pass the Lucas Critique. If all you want to do is forecast the economy, you don't need to pass the Lucas Critique, so you don't need a DSGE model."

This is usually what I hear academic macroeconomists say when asked to explain the fact that essentially no one in the private sector uses DSGE models. Private-sector people can't set economic policy, the argument goes, so they don't need Lucas Critique-robust models.

The problem is, this argument is wrong. If you have a model that both A) satisfies the Lucas Critique and B) is a decent model of the economy, you can make huge amounts of money. This is because although any old spreadsheet can be used to make unconditional forecasts of the economy, you need Lucas-robust models to make good policy-conditional forecasts.

Let me explain. An unconditional forecast is when you say "GDP growth will be 2.4% next year", or "inflation will be 1.7% next quarter". For this kind of thing, any old spreadsheet will do.

A policy-conditional forecast is when you say "If the Fed tapers, inflation will fall by 0.5% next year." To get these forecasts as good as possible, you need to know how policy affects the economy. and if your model is not Lucas-robust, then you will not be able to know how policy affects the economy, so you will react sub-optimally to a policy change.

For example, suppose the Fed suddenly lowers interest rates substantially. Most people, using their silly spreadsheets with their 70s-vintage Phillips Curves, will forecast a rise in GDP growth, so they will pay a lot for stocks, expecting higher profits from the increased growth. But wise DSGE modelers, using the Nobel-winning and ostensibly Lucas-robust Kydland-Prescott 1982 model, know that the Phillips Curve is not structural. They know that the promised growth will not occur, so as soon as stocks become overpriced, they short the S&P. When the hoped-for growth does not materialize and stocks fall, the DSGE modelers reap a huge profit at the expense of the spreadsheet modelers.

Now that's a bit of an old example, so let's take a more modern one. Suppose the Fed launches a new program of QE. Clever DSGE modelers, armed with Steve Williamson's 2013 QE paper, know that QE will be deflationary rather than inflationary (as most people think). This allows them to take other investors, who are armed only with spreadsheets, for a ride, shorting TIPS and buying Treasuries. Voila - instant riches. Williamson himself endorses this idea, writing:
[I]f it does anything, QE will lower the inflation rate over the long run. And the long run comes sooner than you might think, i.e. if QE gives you a short-run increase in inflation, then if it's like typical monetary easing, then that effect lasts only a year or two. More to the point, there are other forces post-financial crisis that will cause the real interest rate on safe assets to rise, and inflation to fall further, so long as the Fed keeps short nominal rates at or near the zero lower bound. And there are good reasons to think that the Fed will be stuck at the zero lower bound indefinitely. Conclusion: expect less inflation rather than more. That has to matter for your portfolio choices. (emphasis mine)
So as we see, a Lucas-robust DSGE model has the potential to make its wielders a LOT of money. This is especially true in the current environment, where correlations are high and macro events have become much more important to investors' performance.

But not necessarily. Being Lucas-robust is necessary for making optimal policy-contingent forecasts, but it is not sufficient. You also need the model to be a good model of the economy. If your parameters are all structural, but you've assumed the wrong microfoundations, then your model will make bad predictions even though it's Lucas-robust.

So now let's get to the point of this post. As far as I'm aware, private-sector firms don't hire anyone to make DSGE models, implement DSGE models, or even scan the DSGE literature. There are a lot of firms that make macro bets in the finance industry - investment banks, macro hedge funds, bond funds. To my knowledge, none of these firms spends one thin dime on DSGE. I've called and emailed everyone I could think of who knows what financial-industry macroeconomists do, and they're all unanimous - they've never heard of anyone in finance using a DSGE model.

If you know someone who does, please reply in the comments. I'm sure there's someone out there. But even if there is, they haven't soared to fame and fortune on the back of their DSGE model.

So maybe they're just using the wrong DSGE models? Maybe they're using Williamson (2012) instead of Williamson (2013). I mean, after all, there is a huge, vast, unending array of DSGE models out there, most of which purport to explain the entire macroeconomy, and most of which are thus mutually exclusive at any point in time. Maybe two or three of them are right at any given point in time, but maybe this set switches around as conditions change. Perhaps finance-industry people are simple unable to pick out the right DSGE model to use on any given day.

But if finance-industry people can't know which DSGE model to use, how can policymakers or policy advisors?

In other words, DSGE models (not just "Freshwater" models, I mean the whole kit & caboodle) have failed a key test of usefulness. Their main selling point - satisfying the Lucas critique - should make them very valuable to industry. But industry shuns them.

Many economic technologies pass the industry test. Companies pay people lots of money to use auction theory. Companies pay people lots of money to use vector autoregressions. Companies pay people lots of money to use matching models. But companies do not, as far as I can tell, pay people lots of money to use DSGE to predict the effects of government policy. Maybe this will change someday, but it's been 32 years, and no one's touching these things.

As I see it, this is currently the most damning critique of the whole DSGE paradigm.


Updates:

Matt Yglesias comments.

Tony Yates comments, at greater length.

The anonymous denizens of EJMR discuss DSGE and the private sector. Interesting post:
Insider in a small, boutique private equity firm. We do not use DSGE models, but we do use DSGE models as a screening device, in the same spirit as graduate programs use real analysis. It turns out that we can get these guys cheaper than MBAs, and they have similar levels of firepower.
So DSGE does function as a kind of signaling - the ability to make DSGE models is valuable to companies even if the models themselves aren't, because they indicate general intelligence, computer skills, creativity, and/or work ethic. That makes sense.

Chris House, my old first-year macro instructor and a prof at the University of Michigan, has A) started a blog, which you should read, and B) attempted to rebut this post. Check it out! Fun fact: The argument in quotes at the top of this post is actually paraphrased from stuff Chris House said to me, which is very similar to what he now writes in his post. I must say, though, I think Chris' post actually reiterates and strengthens the point I make in my post. The difference between policy-conditional and other forecasts is key here. If the average policymaker can use a DSGE model to improve her predictions of the results of her policies, then the average financial trader can use the same model to improve her predictions of the results of the policymaker's policies.

Tyler Cowen thinks I'm too quick to dismiss DSGE, and that instead, critiques like this one should merely make us "downgrade" DSGE. This is a very Bayesian approach, and I like it. But notice that Bayesian beliefs depend on your priors. If you started out thinking that DSGE was totally awesome, then critiques like this one should make you temper your enthusiasm. But if you started out thinking that DSGE was hocus-pocus, then critiques like this should tend to strengthen your belief somewhat.

134 comments:

  1. I can confirm the expectation, Australian investment banks and hedge funds do not use DSGE models. Very few who are employed to forecast would know what they are.

    ReplyDelete
  2. But.. markets are efficient so wise DSGE modellers will not make money. The DSGE paradigm wins again!

    ReplyDelete
  3. Anonymous3:23 AM

    DSGE models historically have completely ignored finance. Finance is simply replying in kind.

    ReplyDelete
  4. Aren't DSGE models primarily used in policy institutions in order to provide a narrative as to what has been occuring within the economy? Mainly because at somepoint economists need to talk to non-economists who desire to have some form of structured/causal explanation of why things are the way they are?

    Out of interest what macro models do financial institutions use? VARs?

    ReplyDelete
  5. Sometimes I bore myself. In fact when I read "Lucas critique" I usually bore myself. I will bore you too. I fear I can't resist discussing the meaning of "Lucas robust". It is quite different from the meaning of "hopefully Lucas robust".

    We can go back to the source of the Lucas critique to avoid the word "Lucas". That source was named Jacob Marshack http://en.wikipedia.org/wiki/Jacob_Marshak (Lucas cited him). He was also a minister in a Menshevik government once. He introduced the word "structural." Ah that's the ticket. They phrase "Lucas-robust" can be translated to "your parameters are all structural."

    OK so what does "structural" mean in this context ? It is currently often used to mean "having something to do with optimizing" but it originally meant policy invariant. Now it is not necessary to write down a model in which agents optimize to have policy invariant parameters -- the Newtonian model gravity gives policy invariant close enough to be useful even if we do not discuss the utility function of planets.

    Similarly the fact that one writes down a model in which a parameter appears in a utility function does not make empirical estimates of that parameter policy invariant. that would be true if the model were reality. The fact that the model fits available data OK tells us no more than it would if no techniques of optimization were not used in solving the model. The relationship between our conditional forecasts and conditional probability distributions can't depend on whether we or not we consider the regression coefficient of consumption growth on the real interest rate the estimate of a parameter of a utility function or just an ad hoc regression coefficient.

    The argument that models with micro foundations are Lucas-robust makes no sense at all. I think it has precisely exactly zero merit of any kind.

    ReplyDelete
    Replies
    1. Sure. But I say *even granting* Lucas-robustness to a subset of dsge models, they fail.the market test.

      Delete
  6. By the way, I bore myself but you don't bore me. This is a brilliant post. The examples are very convincing.

    ReplyDelete
  7. I can't help noticing that HSBC London is currently looking to hire a DSGE expert.

    ReplyDelete
    Replies
    1. Anonymous5:33 AM

      I believe they are also used in BBVA
      (see, eg. http://serviciodeestudios.bbva.com/KETD/fbin/mult/Europe_Economic_Watch_190711_tcm348-263401.pdf?ts=1122011 )

      Delete
    2. That is good info! Thanks!!

      Delete
  8. Anonymous5:30 AM

    I would point to an alternative explanation. People use VARs in part because there are software packages that do almost all the estimation work for them (Eviews, etc.). While DSGE models have existed for 32yrs it is only during the last 10yrs that software to solve them has emerged (I'm thinking of Dynare), and only during the last 5 years or so that the software has been capable of estimating them.

    I would therefore argue that it is only now that they are beginning to be a realistic alternative to VAR for "real-time" analysis of the macroeconomy. Since "real-time" analysis is where money is to be made it is not surprising that DSGE models are not yet widely used in macro-finance. It will however be bad news for DSGE if they do not begin to be used in macro-finance over the next decade.

    (In short, until a tool is easy to use it will not be widely used, regardless of whether or not it might be a good tool)

    ReplyDelete
    Replies
    1. foosion6:08 AM

      The value of a better model is high enough to many financial institutions that it would be worthwhile developing a software package.

      Delete
    2. Right foosion. I was paid to code up my company's own proprietary option model, even though we could have just bought over-the-counter stuff with a standard model.

      Delete
  9. Jonathan5:33 AM

    While private sector economists may not use DSGE models to forecast (at least, not yet) *understanding* such models may well be advantageous to them:

    1/ Central Banks clearly take DSGE seriously (whether they should or not is another question). So understanding DSGE models may help your understanding of a Central Bank's reaction function. A private sector economist in 2008 who had read and understood Eggertson and Woodford (2003) would have been much better able to predict the Fed's actions once the fed funds rate hit zero.

    For example, see Gavyn Davis (former chief economist at GS) discuss the papers of Janet Yellen and the importance of having done the same with Bernanke:
    http://blogs.ft.com/gavyndavies/2013/10/13/the-economics-of-janet-l-yellen/

    2/ If not DSGE, then what? Relying on empirics without theory can be dangerous - just ask an inflation forecaster in the 1970s. If you accept this (I know many might not, but just for the sake of argument) - the question is what theory?

    As you yourself note here:

    http://noahpinionblog.blogspot.co.uk/2013/04/a-world-without-macroeconomists.html

    There are many crazy theories out there (hopped onto zerohedge recently?) and DSGE, while not perfect, is a disciplining device that helps you get rid of the crazy ones.

    The common theme here is thinking in terms of modern macro - forward thinking, general equilibrium etc is important for private sector economists, even if they aren't programming up model economies on Dynare just yet.

    ReplyDelete
  10. Well, according to most DSGEs, private sector people already have rational expectations, so they don't need to study DSGEs. Problem solved!

    More seriously - you raise some good points, but here's a possible counterargument: At this stage, DSGE models are not very good at making precise quantitative forecasts (in the conditional sense you described), simply because macro is tough - so there would be no point in constructing a private DSGE model to get precise QE-conditional profit-maximizing portfolio weights, or whatever.

    But models can still provide qualitative insights - if X happens and assumptions A,B,C are approximately true, then Y will increase/decrease/stay the same. At the same time, there's a natural division of labor - academic economists generate new insights with DSGE models, industry economists apply these insights (passed to them through textbooks or survey papers) when analyzing specific questions they face, according to their best judgement about which insights are most relevant for their situation.

    Do industry people use modern macroeconomic theory in this way? I don't know. If they don't, there could be several reasons - maybe DSGE are worthless. But in that case, what kind of theory people *do* use? Old-style Keynesian models? Intuition? Reciting pages from Mises? That would be interesting to know.

    On the other hand, maybe it just takes time for theory to trickle down from research frontier to textbooks, and people will use them more in the future. Or maybe most DSGE models don't actually bring that much new insights - e.g. the whole New-Keynesian research program seems to be mostly about deriving standard undergraduate Keynesian macro with intertemporal optimization (which I think is valuable, but it won't change people's decision very much).

    ReplyDelete
    Replies
    1. I agree with this - economics should not run before it can walk, and explaining the general, stylised behaviour of the economy is surely the best starting point we can hope for.

      Obviously there's still the issue of whether DSGE does this sufficiently. I think it still suffers from the ex-ante choice of model problem the Noah references here.

      Delete
    2. The problem with this argument is that some DSGE models have very different qualitative prediction from the market movements. In other words, simple directional trades should make you money, but they don't.

      Delete
    3. "Reciting pages from Mises?"

      :-) Called the "Peter Schiff methodology."

      Delete
  11. A view from a finance guy in London. No one I've met uses DSGE or even has it on their radar. Only interest would be from banks (like HSBC mentioned above) trying to understand central bank's decision making process.

    This is obviously not an endorsement: they would try to understand Scottish rainfall models better (say) if they thought this was part of the central bank process.

    ReplyDelete
  12. Chris Sims:
    Sims, Christopher A. “Macroeconomics and Reality.” Econometrica 48, no. 1 (January 1980): 1–48. doi:10.2307/1912017.

    Until this is satisfactorily answered (reply here if you think it is), this issue should have been settled in 1980. Remember Sims, Doan, and Litterman were at Minnesota, either the U or the Bank, or both.

    ReplyDelete
  13. More generally the industry doesn't hire new Keynesians, freshwater people or market monetarists. Otoh they pay big bucks for MMT-related analysis (McCulley at Pimco, retired rich, Hatzius at GS).

    ReplyDelete
    Replies
    1. Wonks Anonymous1:10 PM

      I thought Hatzius was supposed to be rather mainstream in his economics, rather than having any association with MMT.

      Delete
    2. Funny, Hatzius endorsed MM a year and a half ago.

      Delete
    3. PIMCO never used MMT in its bond trading decisions. McCullley only loosely started discussing some Post-Keynesian elements of economics in the latter years of his career.

      And Jan Hatzius sometimes uses Wynne Godley's sector financial balances. He doesn't "do MMT" in any meaningful sense and is actually much more sympathetic to Market Monetarism than MMT.

      Delete
    4. @HankB

      http://www.economonitor.com/lrwray/2012/06/25/paul-mcculley-mmt-won-declare-victory-but-be-magnanimous-about-it/

      @Krzys,
      Funny, but he didn't. He likes NGDP targeting a la Woodford who said... he never heard of Market Monetarists and doesn't read their blogs. Hatzius also doesn't believe in standard MM idiocies like monetary policy always offsetting fiscal.

      Delete
    5. Hi PeterP,

      I think that 2011 MMT victory lap was a bit premature. Some MMT economists predicted the Euro was "doomed' and would collapse.

      http://www.levyinstitute.org/pubs/hili_113a.pdf

      http://bilbo.economicoutlook.net/blog/?p=19247

      But Europe is now seeing economic activity at 36 month highs and the ECB seems to have contained the crisis. It looks like MMT's dooms day predictions were actually wrong.

      Delete
    6. Peter,

      Given that NGDP targeting is the central idea of MM, who reads whose blogs is irrelevant.

      Delete
    7. Anonymous6:38 AM

      @ HankB,

      MMT was correct in predicting the demise of the Euro. It has already failed and is continuing to fail. The consequences of mass unemployment, output lost forever and a youth undergoing hysteresis will waste an entire generation. If that doesn't qualify as failure I'd like to know what does.

      It is my belief that the Euro will be continued out of blind political dogmatism. It was born as a political construct and the sooner it is formally abandoned the better.

      bill40

      Delete
    8. Anonymous8:25 AM

      Even the out-going Ben Bernanke reveals himself as an MMT Wannabe:-

      http://www.theguardian.com/commentisfree/2013/dec/19/ben-bernanke-legacy-calling-out-congress

      which kind of brings him into line with the Jan Hatzius Godleyian view of the world and of course Sectoral Balances Analysis are core to MMT:-

      http://www.businessinsider.com/goldmans-jan-hatzius-on-sectoral-balances-2012-12#ixzz2RKT3ON4I

      http://www.realclearmarkets.com/docs/2012/12/112812_usecs.pdf

      http://www.businessinsider.com/the-bull-case-for-2014-2013-12

      Delete
  14. Anonymous9:02 AM

    Well, a DSGE proponent would simply counter that the industry is run by a bunch of herd-following idiots, surely?

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  15. Anonymous9:10 AM

    I trade with econometric models. I have tried. Or rather, my research associate (fresh out of a school), tried hard. He is convinced that DSGE models are superior. But backtesting proved to be a rather frustrating exercise for him. Not that I was surprised. But sometimes the only way to learn is to fail yourself.

    This brings up another issue. If the FOMC were to look at their models as something they were trading with, (trade unemployment and inflation), they would see they have terrible real time results. That should tell them something.

    ReplyDelete
    Replies
    1. This is great info. Thanks!

      Delete
  16. Daniel Bachman9:12 AM

    I've been doing forecasting in the private sector since 1992. I've worked on teams that sold forecasting models and forecasts to lots of major industrial and financial companies.Had to relearn what I thought I had learned in grad school in the 1980s. There no...repeat no...competition to traditional Klein-type macroeconometric modeling. Never has been. (It's not done in spreadsheets, by the way, it's done in EViews or something similar).

    This is also true of the public sector. Don't let the central bank thing fool you. They may look at DSGEs, but when it comes to to create the forecast that the Open Market Committtee looks at, the staff goes to FRB/US, which is more or less an updated traditional model. And they use addfactors when necessary. Just like in 1969. Meanwhile, CBO and CEA/OMB mainly use traditional models for their forecasts--and to answer questions from policy principals. So even in the policy world, DSGE is really more of a curiosity than a serious tool. Oh, and Europeans actually use more structural modeling and know more about it than in we do in the U.S.

    Incidently, I've never heard of anybody using VARs in a commercial context. Too difficult to explain to decision makers. Maybe some finance people do this, but I haven't seen it.

    I wrote a paper about this long ago, but never could figure out where to publish it. http://www.colby.edu/economics/faculty/mrdonihu/mcs/bachman.pdf.

    ReplyDelete
    Replies
    1. Finance people use VECMs, which are based on VARs...

      Delete
    2. Anonymous9:43 AM

      I have seen VARS integrated into a neo-classical synthesis type model to account for some financial variables. But it was 1 (3 or 4 variable) VAR appended to a Klein-style model of 800+ equations.

      I have alos see a number of industry models and can confirm what Daniel says from my more limited exposure.

      Delete
    3. I took a forecasting class in undergrad where we used VARs and VECMs (among other tools), *and* used Eviews, which is hella ugly. Couldn't find a job in forecasting, so maybe there's more to Mr. Bachman's statement after all...

      Delete
    4. Noah, in my experience, if someone is providing some projection to management (which is perhaps different from a trading situation), they have to explain it in a way that one manager can explain it to his boss and so on up the later. You have to be able to say what's driving things. There is a clear preference away from VARs since it is hard to say what is driving what. While I have a preference for VECM vs. VAR, it doesn't solve the underlying communication issue.

      Delete
    5. I also work in industry producing forecasts. I agree with the sentiment that standard Klein-type macro models are still the bread and butter of our trade. There's relatively simple and have a long track record, and in the daily grind of dealing with clients, data, etc., you don't have a lot of time left to keep refining and trying new models.

      That said, I think some of the attitudes here about DSGEs are too dismissive. No, I don't calibrate or estimate them myself, but we still look at the modern macro literature to get a good handle on the current thinking in terms of policy analysis, how the macroeconomy behaves, etc. While doing our forecasts, we certainly pay some attention to academic work (often based on DSGEs), and we'll manipulate our forecasts using add factors to reflect that.

      In short, we don't directly use DSGEs to forecast, but we do pay attention to what they tell us about macro and incorporate that information into our traditional forecasting tools using add factors and overrides. Just because we don't use the latest tools to produce forecasts doesn't mean we're stuck in the 70's in our thinking and subjective outlook.

      Delete
    6. Daniel Bachman1:43 PM

      Brian, I agree that current Klein-type models have adopted quite a bit from macro research of the past 40 years. And, yes, you can learn useful things from the recent literature.

      However, the question is really tied up in how the academic world views business and government forecasting. Lucas and Sargent predicted that Klein style models were on their way out (in the 1970s) and that didn't happen. Would be worth understanding why that forecast didn't pan out, no? But the usual view in academia is that this reflects a flaw on the part of those of us in the industry rather than a flaw in the academic research agenda.

      Just as an example, right now I am dealing with the fact that it is impossible to find somebody trained at school in using Klein-style models. The only way you can learn this is via apprenticeship. Yet these models remain the world's main tools for using economics in business (and government, too). I have to untrain people before I can train them. Why isn't there any academic program that takes producing this type of forecast seriously?

      Delete
  17. Anonymous9:31 AM

    Ask Steve Williamson to trade with his own models. Maybe we will pair him up with Taleb. Should be a killer combination to raise AUM. And hedging each other's tails.

    ReplyDelete
    Replies
    1. Anonymous9:51 AM

      Call it the Dynamic Macro BS Partners LLP

      Delete
  18. I propose the following a corollary "VECM (or SVAR or whatever finance people actually use) models are unconditionally better for forecasting than DSGE models"

    I still think that's not the same thing as saying that DSGE models don't work.

    One possibility is that VAR is as good or better than DSGE for forecasting as long as things are mostly normal (ie no financial crisis and the monetary policy rule hasn't changed much). I think that describes most of the last 32 years.


    ReplyDelete
  19. Though no fan of DSGE models in their current form, I think you have two basic facts wrong.

    First, business economists who use macro models are not using them primarily to speculate on asset markets. The main task is for individual businesses to plan investment, inventories, etc. So they really want plain vanilla unconditional forecasts, "what's your best guess of GDP next quarter?" People who want to make money in asset markets don't use macro models at all.

    Second, you are assuming that better policy conditional forecasts will produce better unconditional forecasts. You make money off unconditonal forecasts in the end. But the bitter truth of economic modeling is that this simply is not true. Models that make good policy predictions are necessarily clean, transparent, simplified, and thus don't exploit all the little correlations in the data to make better unconditional forecasts. And vice versa, models that do exploit correlations in the data fall apart on making conditional forecasts -- the big lesson of the shifting phillips curves of the 1970s.

    You are wishing this fact away. Surely good conditional forecasts should help us to construct some sort of melange model that does both, you suggest. But so far that is a wish not a fact. The fact is, unstructural use of correlations produces better unconditional short term forecasts, and simple micro founded models produce better conditional forecasts.

    By the way we are not alone. Pure "structural" weather models don't do as well in making actual forecasts as statistical approaches which combine different structural models, simple correlations in the data, and forecaster judgement.

    ReplyDelete
    Replies
    1. Anonymous11:48 AM

      Professor - it would be really interesting to see you expand on why you aren't a fan of DSGE models in their current form, and if you have any thoughts on how the profession could try to remedy the failings you're thinking of.

      I know you're busy, but perhaps something to consider for a future blog post?

      Delete
    2. On your first point: there are plenty of economist types employed at macro hedge funds and bond funds who would love to have an edge over the market on the impact of fed policy on GDP, inflation etc. There is a lot of money at stake here (the Pimco Total Return Bond fund's AUM >$250bn!) and these people (to my knowledge) ignore DGSE so for me Noah's point stands.

      Delete
    3. Wonks Anonymous1:13 PM

      Readers might be interested to know that Cochrane previously discussed conditional vs unconditional forecasts at Cato Unbound, touching on Tetlock's "foxes vs hedgehogs": http://www.cato-unbound.org/2011/07/15/john-h-cochrane/defense-hedgehogs

      Delete
    4. Hey, John!

      Though no fan of DSGE models in their current form

      I'd say that itself deserves a Grumpy Economist post... ;-)

      First, business economists who use macro models are not using them primarily to speculate on asset markets.

      I know, that's my point! My point is that if DSGE models worked, firms could be doing this. The fact that they're not is very informative.

      You are wishing this fact away. Surely good conditional forecasts should help us to construct some sort of melange model that does both, you suggest. But so far that is a wish not a fact.

      I'm not really wishing, I'm saying that, OK, suppose there's a Fed policy change, and buy-side firms want to know how to bet on the effects. They base their bet on a model that is aX+bY+cZ, where X is a weighted combination of all available reduced-form models, Y is a weighted combination of all available non-DSGE ostensibly-structural models (such as sVAR's), and Z is a weighted combination of all available DSGE models.

      We observe c=0. You'd think that if DSGE models as a class had any validity, we'd see c>0, however small.

      Delete
    5. Seconding Noah here. No one is "Wishing the fact away" (I'd rather like to know, "why is this fact the case?").

      "So they really want plain vanilla unconditional forecasts"

      Maybe I'm missing something here, but why would business owner assume that a GDP forecast should be policy invariant? If the best forecast is conditional, wouldn't everyone prefer that forecast, ceteris paribus?

      "People who want to make money in asset markets don't use macro models at all."

      And we're back where we started.

      Delete
    6. You can easily enhance returns by using conditional forecats in entry and exit. If the models can't do it, they are simply false.

      Delete
    7. "People who want to make money in asset markets don't use macro models at all."

      I don't agree with this. Depends on which model you use. For example the simple static IS/LM model predicted no inflation and therefore falling yields (real and nominal). People who used this insight made a lot of money.

      The Fed is trying to affect the real economy through wealth effects. Does this not imply that getting the model right give one better information about positioning on assets?

      Delete
    8. John H. Cochrane sez: " Pure "structural" weather models don't do as well in making actual forecasts as statistical approaches which combine different structural models, simple correlations in the data, and forecaster judgement. " - but taking your analogy one step further: the really important stuff for planet Earth is in fact 'structural weather' models of CLIMATE as opposed to 'whether it is going to rain this weekend' WEATHER forecasts.

      Delete
  20. Anonymous11:48 AM

    Professor - it would be really interesting to see you expand on why you aren't a fan of DSGE models in their current form, and if you have any thoughts on how the profession could try to remedy the failings you're thinking of.

    I know you're busy, but perhaps something to consider for a future blog post?

    ReplyDelete
    Replies
    1. Anonymous11:49 AM

      Sorry, this was for Prof Cochrane (above).

      Delete
  21. By Noah's own standard, then, seat pants versions of neo-Austrian theories of the business cycle have passed the market test, given their popularity among his quantitative finance colleagues, and among non-academic Wall Street economists.

    I mean, jeez, is it really so hard to believe that government committments to purchase assets and peg interest rates could induce and/or exacerbate strategic complementarities in financial asset markets?

    ReplyDelete
    Replies
    1. Assumption: "neo-Austrian" theories are popular on Wall Street.

      They just bark loudly.

      Delete
    2. By Noah's own standard, then, seat pants versions of neo-Austrian theories of the business cycle have passed the market test

      Yes, but don't fall for the fallacy of the converse.

      validity --> use
      thus !use --> !validity

      But use !--> validity

      Delete
    3. Hmmm, whilst I admire your mastery of freshman symbolic logic, you better let Larry Summers in on that whole fallacy of the converse thing:

      "But a growth strategy that relies on interest rates significantly below growth rates for long periods virtually ensures the emergence of substantial financial bubbles and dangerous buildups in leverage."

      http://www.washingtonpost.com/opinions/lawrence-summers-strategies-for-sustainable-growth/2014/01/05/9143313c-74b9-11e3-8b3f-b1666705ca3b_story.html

      Not to mention John Landon-Lane:

      We show that “loose” monetary policy – that is having an interest rate below the target rate or having a growth rate of money above the target growth rate – does positively impact asset prices and this correspondence is heightened during periods when asset prices grew quickly and then subsequently suffered a significant correction.

      http://www.nber.org/papers/w19585

      Delete
  22. Anonymous1:50 PM

    I'd suggest several reasons for this. Firstly, financial companies are run by people who don't have a very good intuition for (macro)economics. For example, a lot of hedge funds have started assuming that interest rates are not very volatile. Anyone with a modicum of economics knowledge knows that this is an artefact of the ZLB: max([natural rate],0%) is always going to be less volatile than [natural rate] near 0%. Once we leave the ZLB these companies will be a bit stuffed.

    DSGEs will only really be accepted if they match these managers' intuitions, which will only happen if they are also broken and useless.

    Secondly, on the other hand, DSGEs tend to ignore a lot of concepts like "hot money" that are demonstrably very significant from a financial perspective. For example a lot of the current boom in the BRICs is partially driven by highly-liquid investors from Europe chasing yield. The flow of hot money into and back out of third-world economies is a major cause of their crashiness.

    Thirdly, academic DSGEs are normally set up as toy systems that only handle one or two economies and a handful of economic indicators. Re-tooling them as fully-fledged world models is possible but non-trivial.

    Fourthly, and relatedly, with DSGEs I get the impression we're still in a situation where the maths dictates the model. Finding closed-form solutions to DSGEs... well, everyone's got to have a hobby, but it really shouldn't be necessary to make progress, and models with no closed-form solution shouldn't be excluded from consideration.

    Also, for some reason, the "global macro" community of hedgies seems to be dominated by crazy-ass technical analysis types. Maybe there's a gap in the market here?

    ReplyDelete
  23. Hi Noah,

    That's not what those models were built for. It's well-known that you do not need economic theory to forecast. For the CEO of General Motors, it's possible to set up an in-house forecasting group, but it's lower cost, and probably a better product, to simply buy the service from Macro Advisors (a fine firm, located here in Clayton MO). But I would not use the Macro Advisors model for policy analysis. That's what the "DSGE" models are intended for. Whether any models currently in use at the Fed, the ECB, the CBO, or whatever, are any good is another question.

    ReplyDelete
    Replies
    1. That's not what those models were built for.

      Of course not. Who cares? As you yourself pointed out, they should have ramifications for portfolio choices.

      It's well-known that you do not need economic theory to forecast.

      Unconditional forecasts, yes. Policy-conditional forecasts, NO. That is the point of my post.

      Delete
    2. It seems the point of the post is that "DSGE" models are no good, because private industry doesn't use them. So, I'm saying it's clear why private industry doesn't use them. That's because the models were not designed to sell to private industry. No one is suggesting that those people should find them useful. We design the models for policy analysis, and for pure scientific pursuits - let's try to understand how the world works.

      Maybe you could write us a post about how a goldfish is no good to take on a duck hunt.

      Delete
    3. Daniel Bachman3:28 PM

      Ah, but Treasury and CEA do use Macro Advisors for policy analysis. (CBO uses its own in-house structural model). Because there's no real alternative when the Treasury Secretary or President want an answer. You might not want that, but the point of the original post is that---fact---DSGE models aren't used much in the "practical" world. My challenge to the academic world is to ask why instead of simply asumming that we business/government types are wrong.

      And it's not true that business forecasting doesn't need economic theory. Much of it takes place in the medium-term, nonfinancial business world, where you need theory to explain (e.g.) your auto or housing forecast. That's theory.

      Stephen, you are welcome to set up a company to compete with the Klein type models. Klein did it, after all. So did Larry Meyer. Why has nobody from the RE world ever come up with any competition in this market?

      Delete
    4. If the industry doesn't use them, but the models actually are better then the currently used in forming conditional forecasts, then it's a clear business opportunity. The DSGE modeled should form their little hedge fund and produce superior returns for a while. As always, they need to put money where their mouths are.

      Delete
    5. So, I'm saying it's clear why private industry doesn't use them. That's because the models were not designed to sell to private industry. No one is suggesting that those people should find them useful. We design the models for policy analysis, and for pure scientific pursuits - let's try to understand how the world works.

      The point of my post is that if DSGE models are indeed useful for policy, they must be useful for trading as well, even if that is not what they were designed for.

      See?

      And the Steve Williamson quote I posted says much the same thing! Read what you wrote!

      Delete
    6. Daniel Bachman4:11 PM

      "Those people"? I missed that. Stephen, may I suggest that you might not know that much about how the private sector use economics if you haven't worked in the industry. My job is not really to supply a bunch of numbers--it is "to try to understand how the world works" so I can explain it to decision-makers. The model and forecast are tools to that end.

      So if DSGE's really helped me understand better "how the world works" I would use them. I don't use them. Which is a fact. And I think that's an important point to understand.

      Delete
    7. I think we're running into a problem here as to what we mean by "DSGE."

      1. Noah seems to think that what I do is somehow representative of "DSGE," but I'm a monetary theorist. In my academic work, I don't construct large quantitative models that I fit to data. That said, with reference to the quote from my blog above, some of my theoretical work - and things I blog about - are suggestive of successful portfolio strategies. Indeed, if I were working on Wall St., I think I could help someone make some money.

      2. What Macro Advisers does is just fine. People want stories, and that stuff can be used to tell a story, even if it means absolutely nothing. But there is a market for stories, apparently.

      3. "So if DSGE's really helped me understand better "how the world works" I would use them. I don't use them. Which is a fact. And I think that's an important point to understand."

      So, you don't use them. But can you tell me what they are?

      Delete
    8. Here's another observation. Gary Gorton makes, and has made, a lot of money on Wall Street. But I don't think Gary knows any more than I do (ask him what he thinks, if you want). Therefore, I think I could get paid very well to work on Wall St. Larry Christiano, who I think we could more accurately call a DSGE guy, has a very different skill set from Gary and me. I'm willing to assert that he could also make a lot of money on Wall St. if he wanted to.

      So, I think people schooled in modern economics have marketable skills. Even as academics, we get paid excellent salaries relative to what most academics get (and they're always complaining about it). So, that's the market test. What would you be doing with a Physics PhD, Noah?

      Delete
    9. Noah seems to think that what I do is somehow representative of "DSGE," but I'm a monetary theorist. In my academic work, I don't construct large quantitative models that I fit to data.

      Yes, yes, I know. Technically those models aren't dynamic and stochastic. But I think the same argument applies to all of the ostensibly-Lucas-robust microfounded modern macro models in use today, and "DSGE" is just a term people recognize. It's a buzzword, sure.

      Larry Christiano, who I think we could more accurately call a DSGE guy, has a very different skill set from Gary and me. I'm willing to assert that he could also make a lot of money on Wall St. if he wanted to.

      I'm sure he probably could, being a smart guy. But could he do it using the same kind of model that he makes to try to answer policy questions (i.e. the kind we saw him present at St. Louis)? That's really the question.

      So, I think people schooled in modern economics have marketable skills.

      No question about that; this post wasn't about job markets for DSGE-trained economists.

      What would you be doing with a Physics PhD, Noah?

      That's a question I've often asked myself, but I guess we'll never know now!

      Delete
    10. Yes, some economists can make some money on the street, mainly by talking to other economists, called regulators, or to customers selling them stories. It's mainly PR and litigation hedging. No academic c economist would be allowed close to a trading book (or even a strategy team).

      Delete
    11. Stephen,
      Some of your results suggest portfolio strategies? You don't even realize how funny that sounds. You remind me of a classic example of a Wall Street quant who thinks they understand the market/economy because they understand ( their) model best. Now, it happens extremely infrequently that said quant is actually right. They do know something better. Yet, it's extremely unlikely.

      If you want to see how little of the economy you understand, try and put some trades on. Hey try even paper trading, but not ex post but going forward. You will very quickly discover how little you know.

      I think every wanna be regulator, central banker and so on, should be forced to actually to test their suppositions by actually making decisions, I.e. Putting trades on in whatever their field/market is. That will teach them humility really quick.

      Delete
    12. "It's mainly PR and litigation hedging."

      You don't realize how funny that sounds. I know plenty of economists who have been full-time academics, and who have worked full time on Wall St., at Goldman Sachs, Bear Stearns, AIG. Others are full-time academics and have Wall St. retainers or short-term consulting contracts. Those people aren't just there for window dressing, they are doing heavy lifting.

      Delete
    13. "Yes, yes, I know. Technically those models aren't dynamic and stochastic. But I think the same argument applies to all of the ostensibly-Lucas-robust microfounded modern macro models in use today, and "DSGE" is just a term people recognize. It's a buzzword, sure."

      Actually, I'm not sure what you know. My models indeed are dynamic and stochastic, though there's not always aggregate uncertainty in there. We could talk about the "GE" too. Some of the models I work with are not GE in the sense of competitive equilibrium, as there can be strategic components in there. So, when you say "ostensibly Lucas-robust micro-founded," it seems you're talking about the whole field of macroeconomics. So it's not clear where your argument is going. You want to argue that all that stuff is useless because some people in the private sector don't want to think about it?

      Delete
    14. Yes, all. My critique is of the whole paradigm. There's so damn *much* of this stuff out there that it's quite probable that a few of the models would in fact be very useful to industry. But industry cannon find them. Why not? Because the macro profession has no way of judging models invalid and throwing them out. So whatever works - if any of the models work - gets lost in the massive junk pile, and we keep just adding and adding to the junk pile.

      *That's* what I'm criticizing. It's really not even about the models as much as it's about the scientific standards of the profession that allows all these models to live. I will write another big post about this fairly soon...

      Delete
    15. "... it's about the scientific standards of the profession that allows all these models to live..."

      You're expecting something of economic science that it can never deliver. This is just the nature of the beast. I've learned to love it, but obviously you have a hard time with it. First, in macro, there will never be one model for every problem. We're trying to understand the world, so the models have to be simple. You have to break down this massive problem into little parts so that we can come to grips with what is going on. So, there are going to be a lot of models. That's very democratic, don't you think? We can even have very different models of the same phenomenon, and one might be as useful as another. I can like Mozart, and I can like Radiohead too.

      The models are all wrong. Many people have emphasized that point (most recently Lars Hansen, though I can't remember where I read it). That's what makes economics different. These things are all invalid, but we work with wrong models because they are simple, and - of course - because they are useful.

      Delete
    16. You're expecting something of economic science that it can never deliver. This is just the nature of the beast. I've learned to love it, but obviously you have a hard time with it.

      I guess that's true.

      I can like Mozart, and I can like Radiohead too.

      A) OK, but this isn't art. You just pointed out that we (i.e. taxpayers) pay macroeconomists relatively generous amounts of money to do this stuff. If it's all a matter of taste, what are the taxpayers getting for that money?

      B) Weirdly, I don't like either Mozart or Radiohead. I know, I'm strange.

      The models are all wrong...That's what makes economics different.

      I hope you don't hate me if I don't exactly find that fact encouraging...

      These things are all invalid, but we work with wrong models because they are simple, and - of course - because they are useful.

      Useful for what?

      And aren't simple models, like little OLG models, more useful for storytelling than these big infinitely-forward-looking monstrosities?

      Delete
    17. FTFY

      B) Weirdly, I don't like either Mozart or Radiohead. I know, I'm strange.^H^H^H^H^H^H^H wrong

      Delete
    18. They are not window dressing? They are doing "heavy lifting"? Meaning they direct trading strategies and get a cut of a trading book? Let me assure you that pretty much never happens. Yes, some ex Econ academics do work on the street, but that has nothing to do with their Econ background. They are many more ex math or physics academics than Econ ones working close to front office. Even they are mainly window dressing. I hired lots of PhDs over the years, and as a rule I would not hire Econ PhDs. Their skills were simply wrong for front office work.

      Anyway, don't take my word for it. Try and trade on your models. You will quickly discover how little your models are worth.

      Delete
    19. By the way. Let me mention that basic economics is actually quite useful for understanding certain basc regularities in the markets. However, the analytical richness is pretty much exhausted once you go beyond some basic undergrad con. The moment you start playing with more complex/ sophisticated models they overfit data almost instantaneously and you end up playing with noise.

      Delete
    20. Not sure I buy that, Ksrzyksrykz. McFadden's models famously predicted BART ridership, and there's a lot of stuff based on those, right?

      Delete
    21. "And aren't simple models, like little OLG models, more useful for storytelling than these big infinitely-forward-looking monstrosities?"

      Now you're actually getting down to business. If you claim to be condemning all of modern macro, then that's essentially damning all of modern economics to hell, as it's now impossible to say where micro stops and macro begins. So now I think you're saying that "storytelling" might be useful. That seems a bit on the pejorative side, but to put a nice spin on it, we could broaden our notion of storytelling by saying that we have some little models that give us some sparks of insight into how the world works.

      Possibly the area of agreement we can find here is the following. I find various models useful, sometimes for quite different reasons. Maybe they tell me something about how monetary policy works; maybe they provide some insight into why some people in the world are so poor and some are so rich; maybe this just provides me with another tool - I'm going to learn how this thing works, and I'm going to file that away for later use when I'm struggling with some problem.

      So, I have a toolkit - it's some array of neoclassical growth model/search and matching/mechanism design/information economics/OG model/Lagos-Wright/etc. etc. And then I use that for everything I do, whether I'm writing academic papers, blogging, writing textbooks, giving advice to people who make policy.

      But there is a class of monster models, which started with 1960s Lawrence Klein type 400-equation macroeconometric models. Those were the models Lucas was critiquing. And there's a tendency for more contemporary models like what came out of Schmets-Wouters and Christiano-Eichenbaum-Evans to grow into monsters that really look much like the Lawrence Klein models. And those monsters are really quite useless, I think, though the people who construct those things and work with them will argue otherwise, of course.

      Delete
    22. Krzys,

      I'm not sure that we're even discussing the same thing. Two questions for you:

      1. What exactly is the line of business you're in.
      2. Why are you reading Noah's blog, anyway?

      Delete
    23. B) Weirdly, I don't like either Mozart or Radiohead. I know, I'm strange.^H^H^H^H^H^H^H wrong

      Didn't say I liked them these characters either. I assure you that my tastes are soooo cool that no one likes what I do.

      Delete
    24. If you claim to be condemning all of modern macro, then that's essentially damning all of modern economics to hell, as it's now impossible to say where micro stops and macro begins.

      No. False. Not even slightly true. First of all, even if the techniques used are similar, the questions they try to answer are different and the data and empirical methods they use to validate their theories are completely different.

      So now I think you're saying that "storytelling" might be useful.

      People do enjoy stories! I'd rather models make usable quantitatively accurate predictions about stuff, but stories are better than nothing. Heck, that's all cosmologists and a lot of evolutionary biologists get from their models.

      And there's a tendency for more contemporary models like what came out of Schmets-Wouters and Christiano-Eichenbaum-Evans to grow into monsters that really look much like the Lawrence Klein models. And those monsters are really quite useless, I think, though the people who construct those things and work with them will argue otherwise, of course.

      I suspect you're probably right. You should write this blog post! "Monster models" is a great title. I will link approvingly. :-)

      Delete
    25. Also:

      2. Why are you reading Noah's blog, anyway?

      Well gee, thanks! ;-)

      I assure you that my tastes are soooo cool that no one likes what I do.

      NOW I MUST KNOW

      Delete
    26. Stephen,

      1. I do quant commodity trading.
      2. I like Econ for its qualitative insights. It's always a useful check. Plus I always look around for stuff I might be missing.

      Delete
    27. Noah,

      Thank you, although you forgot a couple of z's.
      Anyway, the cases you quote correspond to situations in which the economic structure is very transparent, the optimization does not interact with the (estimation) noise and the underlying DGP is stationary and very stable and you clearly control the rules of the game. That's why mechanism design application (like auctions) work well. In actual markets, where the fundamentals are not very transparent, your ability to filter noise away are very limited, econ models will typically fail.

      Let me give you a perfect example from commodities. Around 10-15 years ago, GE was marketing a great product: GE maps. It was great since it perfectly captured the fundamentals in the local electricity market in Pennsylvania: PJM. How perfectly? GE maps was used by the system operator to actually dispatch the system in real time. So, they knew the underlying supply and demand structure pretty much with perfect precision. Yet, they couldn't predict or even explain past prices. Neither levels nor volatilities. Consider how strange that is. Electricity is the simplest commodity. No storage, hence (almost) no inter-temporal optimization in the underlying production decision. Fundamentals as transparent as could be. Yet, no success.

      Delete
    28. That's why mechanism design application (like auctions) work well.

      Yep.

      So of course I was being extremely tongue-in-cheek with my examples...naturally you can't do what I described, which is why people don't. ;-)

      Delete
  24. Anonymous2:47 PM

    I feel as though the post already went over the ground being covered in several places above. Beginning with:

    "DSGE models are useful for policy advice because they (hopefully) pass the Lucas Critique. If all you want to do is forecast the economy, you don't need to pass the Lucas Critique, so you don't need a DSGE model. . . . The problem is, this argument is wrong. . . . Let me explain."

    It would be interesting to hear some more in-depth responses to the explanations offered in the post.

    ReplyDelete
  25. Anonymous3:15 PM

    I think a factor has got to be the difference between the ethos of the finance sector and that of academia. Obviously, there is often quite a divergence of opinion between academic economists and financiers (think of figures like Peter Schiff). Every industry and occupation comes with its own ideological baggage, determined in large part by the kind of people it attracts

    ReplyDelete
    Replies
    1. Daniel Bachman3:33 PM

      Sorry to be repetitive here, but an awful lot of business forecasting takes place in the nonfinancial sector. It might be more fruitful to think of how a corporate planner in an auto or capital goods company uses economic forecasts.

      Delete
    2. Anonymous4:28 PM

      Okay, but I think the same still applies

      Delete
  26. Hi Noah,
    Why is the Lucas Critique (I admit I have not read his original) right? Is it something like the Heisenberg Principle? Could it possibly be that a consistent theoretical model simply has time varying parameters instead?

    ReplyDelete
    Replies
    1. Lucas Critique is right because there is no reason to believe that the parameters of any model you decide to write down are fixed parameters. They could change when policy changes.

      Delete
    2. Ok I see the point (and as much as I thought about models having time varying parameters).

      Its interesting and perhaps the Critique does not apply universally (only to macro policy models perhaps?). The reason I say this is for example we kow that the Black-Scholes model with constant volatility parameter does not do a great job. Which is why implied vols are constantly updated and traded. But that hardly refutes a model based on the no-arbitrage conditions.

      Delete
    3. Some form of the critique applies to most things.

      Delete
  27. Noah, does anyone really "use" economics to get rich? How useful is economics in actual market application? Does anyone actually utilize a macroeconomic approach to making money in the markets? I work for a large hedge fund and I don't know anyone who relies solely on economists and economic analysis for their market decisions.

    Thoughts?

    ReplyDelete
    Replies
    1. Depends on what you define as "economics". A lot of firms use time-series statistics to get rich. Some time-series techniques were developed by economists.

      Also, there are lots of other industries where people use game theory to get rich...that's why all those tech startups are hiring auction theorists like crazy.

      Delete
    2. Anonymous4:42 PM

      Does anybody in the business use macro models to inform their trading decisions?

      Delete
    3. Anonymous4:46 PM

      In other words, does your statement that "if DSGE models are indeed useful for policy, they must be useful for trading as well," not also apply to other types of models as well, and do they all fail the market test?

      Delete
    4. Yes. Bridgewater Associates supposedly does this a lot.

      Delete
    5. Sure, some other models fail the market test (as I'd expect, since I believe humanity is mostly ignorant about macro due to poor data), but macro hedge funds and banks use a lot of non-DSGE models.

      Delete
  28. This comment has been removed by the author.

    ReplyDelete
  29. i need to understant that,has individual weaken the environment due to industrialization?adam's smith freemarket theory played any role in it,i tried to analyse it in very short in my blog...please comment in it,though this is completely different topic but i would love to recieve comments
    http://syedmazenquddusi.blogspot.in/2014/01/side-effect-of-capitalistmy-view.html

    ReplyDelete
  30. While it's true as John Cochrane says that traders don't rely on or usually even much understand models, they generally have analyst support who do understand and use models. And it does mean something that those guys never adopted DSGE, though maybe not as much as Noah thinks it means.

    It's not a vote of confidence in traditional models. It's more a case of why change if the newer is more complicated without being better. Neither type of model produces actionable predictions. If you dig into Bloomberg you can find data on how often the average forecast of the economists they survey get right the direction of this or that macro parameter, and the track records are all very close to 50%. Some particular economist beat that, others do worse, but that's generally viewed as the natural dispersion of the random.

    But every research team has to take a stand on macro, and clients don't like to hear that you're just kind of guessing. If old models are as good as any, why toss them out?

    So the more pompous promises made for DSGE indeed fell through, big surprise. But the finance industry's "meh" at DSGE was more about convenience of the process than the quality of the results.

    ReplyDelete
    Replies
    1. Same thing as always. The beauty of the street is that if you think you have an edge that other traders are ignoring, then you trade against them and take their money. If you think they are lazy, well, simple solution, don't be lazy yourself and form the hedge fund already. Untold riches await.

      Delete
  31. I was an economists at two banks way back, in 1987-89, and in 1994-5. Economists were used for PR, talking about econ stats, and sometimes setting the table for discussions about the economy. The real decisions were made by guys winging it about macro forecasts. So, no DSGE, but really no macro at all, in terms of any graduate level stuff from any thread.

    Note they also don't use Stochastic Discount Factors....

    ReplyDelete
    Replies
    1. Nowadays there are lots of people doing formal macro modeling at big banks, right? Just not structural, microfounded, DSGE-type stuff.

      Delete
    2. The apex of private sector economists was the 1970s, when everyone was optimistic about large scale macro models. By the time I joined a bank in 1987 our big bank had about 14 economists, and they would get misty remembering the good old days when there was 50+ professionals devoted to economics. After demonstrating they didn't have any value, their ranks slowly dwindled, so when I was at a bank in 1994 we had one economist and me his assistant, and he was just to respond to press requests are interpreting macro releases: the bank liked getting their name in the press.

      When I was at Moody's no one inside listened to the company economist though again, he was often quoted in the press and interviewed, and so remained valuable for that reason. Needless to say, these PR economists have Keynesian causal mechanisms because they are more intuitive than vague supply shocks, regardless of whether they were empirically better, and the whole point is to be popular with Joe six-pack.

      Once, back in 1994, I was sent to Lodi Ohio, and was speaking to a room full of bank clients about our (ie, my bosses) macro forecast. I was basically a prop for sales pitches by local bankers to their clients to sell services. I had a lot of reasons why interest rates would rise, inflation would rise, etc. Of course, if that was really good info (it wasn't), they would have wanted it at our Asset and Liability Committee meeting (ALCO), and perhaps not want to share it. Our ALCO committee couldn't care less what our economist said. I realized this was dumb, and got into risk management (forecasting portfolio volatility), and later portfolio management, because when done well, they are actually useful. None of this uses any macro models.

      Delete
  32. If WIkipedia is to be believed, a DSGE model requires a production function and a utility function. That is, you have to have a theory about what people are able to sell and what people want to buy. That seems to be problem. If you can develop either of those two functions, or even better, both, you can make money without invoking DSGE. It's sort of like putting the bell on the cat.

    The other problem is the E, equilibrium. The way you make your money in the financial sector is when the equilibrium changes, and when the equilibrium changes the market is, by definition, not in equilibrium. All told, I'm not sure just what DSGE adds to a financial actor's usable knowledge.

    ReplyDelete
  33. This isn't limited to macroeconomics. Physics based models in space and solar physics also tend to be far worse than historical based (regressions, lookup-tables, neural nets, etc) models. However, the physics based models allow us to learn about the subject by exploring what is otherwise an opaque system.

    ReplyDelete
    Replies
    1. But that's because you know the physics itself is right. In econ, if we had good microfoundations - i.e., a good understanding of how consumers and firms make decisions - we could do something similar to what you're talking about.

      Delete
  34. Two additions in two comments; first addition:

    About 20 years ago, Geweke* had a piece on the scale of innaccuracy caused by 1) modeling with the Lucas' critique but having an unaddressed aggregation problem built into your model, versus 2) not modeling with the Lucas' critique but avoiding the aggregation problem (essentially a Klein model). He concluded that the scale of the innaccuracies from the two problems was about equal.

    This is consistent with the view of some in this thread that private sector forecasters may not use DSGE models because that would require an investment of time and effort without any improvement in outcomes.

    * I think it was Geweke (in the AERP&P). I couldn't find it online on Saturday morning. If the thread keeps up, I can find the cite in my hard copy in my office on Monday.

    ReplyDelete
  35. Second addition:

    Some of the discussion here brings to mind Canner, Mankiw and Weil's 1997 "An Asset Allocation Puzzle". They argued that investment advice about how to split up your portfolio was inconsistent with how theory said it should be done. The end result was that the advice that deviated from theory wasn't very costly, so it could be regarded as near-rational. Essentially, portfolio managers could engage in cheap talk that deviated from the theory without doing much harm.

    I wonder if this is what is going on with the of macro models in the private sector. We all know that the differences in performance between a Klein model, a VAR/VECM, and a DSGE (while there) just aren't that large: they all miss so much stuff that their bottom-line results aren't that different. In this case, there isn't much cost in using something that you've already got (like a Klein model) instead of something that might be better (a DSGE). So perhaps what we are observing is that the latitude available for cheap talk is very large.

    ReplyDelete
  36. Noah, when you talk about 32 years that is not fair to dsge models. We learned how to efficiently estimate them maybe 10 years ago or even less. Just compare the dates when VAR spftware became available and when dynare became available.
    I think one big reason for focus on VARs is theyve been aorund much longer and theyare simpler. DSGe will get there as well, youll see.
    Not that I disagree with your other points.

    ReplyDelete
    Replies
    1. What, calibration isn't good enough? ;-)

      Delete
  37. The whole article and almost all the comments are above my pay grade?

    Two questions:

    If DSGE model is not usable (in financial and investment decisions), why spend so much ink on it?

    Why are so many people cognizant of it?

    ReplyDelete
    Replies
    1. Same as with string theory in physics: the internal logic takes over, the bureaucratic logic assures replication, r and people will always find a good reason not to confront their models with data to keep their jobs.

      Delete
  38. From what I've seen, investment decisions are based on a plausible narrative of what will happen and why. If you can't tell that story within 10 minutes and 3-pages, then it won't fly at the top levels. And that narrative had better fit the political view, personal experience, and rule of thumb of the top guy.

    Complex models tend to obscure the narrative, so no one is going to stake their fortune on it. Nor should they, since too many programming mistakes can be made and too many variables aren't measured timely or precisely. Once the narrative has been chosen, then maybe models will be used to quantify the magnitude and timing of the expected outcome. The complex models are often used for showmanship rather than strategic decisions.

    Investors' expectations of inflation for the past 4-years were simply because MBA's didn't know the difference between the monetary base and the money supply, and that the latter was created by commercial bank lending more than central banks. It was a confusion of terms, they didn't really rely even on simple spreadsheet models.

    ReplyDelete
  39. Noah, I think a lot of this is a case of big mental adjustment costs. Private sector macro to me still seems dominated by people who got most of their training in the 70's and 80's, at a time when DSGE models were rudimentary. The state of the art DSGE modeling with everything from financial sectors, to search frictions to all sorts of imperfect information/learning are probably too recent for many people to be aware of outside academia and central banks. And I think switching from the simultaneous equations model with all variables observables, to a state space framework where you aknowledge the importance of latent,unobserved state variables and try to measure them is a big improvement in perspective about the macroeconomy. But things like Kalman filtering are still not part of the standard training of most Masters in economics students I think, so the technical baggage required for using DSGE models may be missing sometimes (though it's probably there in some quant/finance applications- e.g for stochastic volatility modeling.).
    Macro forecasting is not exactly a business with a great reputation for accuracy either in conditional or unconditional forecasting, so while you can't clearly prove their forecasts and scenario analysis would improve with using DSGE models, it's hard to argue that they couldn't improve if they diversified their toolkit.
    A lot of DSGE models are about giving insight, sharpening intuition by taking a more theoretical perspective, which then you can complement via your judgement/add factors +statistical models like VAR's to get a final forecast. It's not meant to be the single forecast producer, and neither should a large old style model (forecast combination over several models almost always dominates individual model forecasts). And much of the insight can also be obtained by going over all sorts of DSGE models/papers, without necessarily using one as a key forecasting tool.
    Otherwise, I suspect things like Bayesian VAR's or dynamic factor models or sometimes just univariat ARIMA's will always dominate in pure forecasting at 1-2 years horizon. The more relevant question is whether DSGE models can be useful in risk/scenario analysis. You seem to suggest no, because the microfoundations are wrong, but that's a bit of a straw man. After all, it's not like the typical consumption or investment function in a big old style models are true either. They're also surely missing many variables (that must be the case since they're limited to thinking only of observable variables or using very rough proxies for unobservable things like potential output), using the wrong linear functional form, too hasty in assuming error terms are IID and don't contain richer dynamics, and in general often seeming to ignore key supply/demand considerations that should matter beyond the shortest forecasting horizon. And the notion that they take into account bounded rationality isn't very serious to me beyond the fact that they include lags of variables (but so do policy oriented DSGE models, and it's hard to know you've really separately identified habit formation, from adjustment costs, from adaptive expectations when you add lags of consumption for example). And I don't see anything reflecting prospect theory or other state of the art psychological theory in old style macro models either. So any model you use is false, but we can disagree about whether it contains enough reality to be useful (I'd say a modified version of the permanent income consumption model that allows for some demographic factors, a proportion of households who just consume all their income and some discount factor shocks isn't so bad. You'd probably disagree).

    ReplyDelete
    Replies
    1. The point about people in the private sector being old is precisely the red herring that academics have clung to since the RE revolution. I was trained in the 1980s in RE/real business cycles and had to relearn quite a bit to become an effective business analyst in the 90s. This is still the case. It's not a question of age, it really is a question of DSGE's not being useful in the business environment.

      The point of the original post is that the private sector doesn't use DSGE's (or, for that matter much of the structure of post-1960s macroeconomics). And this is also true for government by the way...when push comes to shove the discussion will be understandable by anybody who took intermediate undergraduate macroeconomics in the 1970s.

      This is a fact, which has been driving people crazy for a long time. It doesn't necessarily invalidate DSGE's or real business cycles as a tool for analysis, but it does raise some serious questions. It also suggests that the experience of analysts in the government/business world might be useful for developing theoretical approaches. But instead I typically see some variant of "business economists are ignorant."

      Sorry for the mini-rant. I quite agree that a toolkit is useful, and, of course, nobody sane gives a decision-maker a purely model-based forecast. Which is the type of thing that should give us all pause. Except (I have to get this in) the Klein-type model system recognizes this formally, with addfactors, while the DSGE approach...well I don't see it in the literature.

      Same thing with risk. If DSGE's are better for scenario analysis, why don't Macro Advisors and Moody's Economy.com have competition from this direction in supplying tools for stress testing. Because I'm not aware of anybody supplying a DSGE based tool for stress testing. It does seem like a potentially lucratuve market niche.

      Delete
    2. Actually I think Moody's has started looking at DSGE models as part of their credit risk scenarios modeling recently. 1 possible advantage with DSGE models, is that with current software such as dynare it's quite easy to do for example a 3rd order Taylor approximation of the dynamics, so you can better capture the important nonlinearities that may matter if you're trying to simulate a more extreme scenario. I'm not aware of anything equivalent in the Klein/Cowles comission style models.
      I'm not implying business economists are ignorant, though there is a general lack of confidence in business economists' forecasts (exemplified for example in the chapter on economic forecasting in Nate Silver's recent book the "Signa and the Noise")- to be fair forecasting the economy either as a point forecast or with scenarios/well calibrated confidence bands is going to be hard whatever you do.
      In academic applications, you never see add factors, because that's not part of what you want to do in a journal article where the goal is to see how far a model or theory can go before failing without add factors (I doubt Tobin or Solow or Modigliani used add factors in any way when writing an academic article using some IS/LM model). But in policy environments, DSGE models use variants of add factors quite a lot either through the shock processes or via explicit model combination (treat the DSGE model as a core model, then add extra dynamics/adjustment via a non-core block), and presumably you'd do the same in private sector macro consulting. In some cases, the add factors/fixes from the perspective of one DSGE model can be justified as compensating for missing factors/mechanisms that are present in other auxiliary DSGE models that are richer in some dimensions but may be harder to work with. The best exposition I know of is the recent BoE explanation of their forecasting framework,
      http://www.bankofengland.co.uk/research/Documents/workingpapers/2013/wp471.pdf
      .
      The core/non core approach is also given a good exposition by the BoE as part of their previous model,
      http://www.bankofengland.co.uk/publications/Documents/other/beqm/beqmfull.pdf
      .
      Other central banks may be using similar procedures, but are less transparent.

      Delete
  40. Noah: "But notice that Bayesian beliefs depend on your priors"

    It may be so, but your priors may be wrong and you should punish yourself more the more confident you were in these mistaken priors. I had this discussion with Scott Sumner and god, it is frustrating.

    Ok, lets have this example: you have 3 theories that may explain some phenomena. You need to be more than 50% confident in order for you to publicly subscribe to a theory. Imagine that your priors are as follows: 51% for theory A, 48.99999999999 for theory B and 0.00000000001 % for odd theory C that you deem almost impossible.

    Now new evidence comes up that makes you update your priors as in the following 2 scenarios:

    1) New evidence supporting theory B appears so that the posterior probabilities are as follows: A - 49% ; B - 51.99999999999% and C - 0.00000000001 %

    Since it is now theory B that passed the threshold you make public announcement that effective now you are "completely" changing your opinion and now support theory B. However from Bayesian point of view it is not complete change of opinion. You reduced probability of theory A and B by approximately 4%. We may say that it is not a large change. You made a mistake, but not a large one.

    2) New evidence supporting theory C appears. Now it looks like this: A - 51% ; B - 46.99999999999% and C - 2.00000000001 %

    Now your public "opinion" did not change much. No bold statements, you still support theory A, right? But this is HUGE, really HUGE from bayesian point of view. You just elevated something from being labeled as "really, really incredibly not likely" to just "not likely". You made a huge change in your bayesian scoring scale. You should really berate yourself for having such terrible priors.

    So Bayesian is something else from what you think. The most useful concept here is that of information entropy, a concept from information theory. This concept can help you measure how "surprising" a change in probability was, how much new information you gained with a new evidence. The point being that the lower the probability of a surprise the larger the information value it has. Promoting low probability even by orders of magnitude means a lot of information was gained. And by this account the scenario 2 has a much larger impact then scenario 1.

    ReplyDelete
  41. Daniel Bachman8:33 AM

    This is interesting. I quite agree about the lack of confidence in business economists forecast--this is well demonstrated in the literature, as well as by Nate Silver. At this point, nobody in the economics world really expects much ability in forecasting turning points (educating our customers is another matter). This is, of course, where we really need a good tool. I don't believe DSGE's did that well in the last downtown. But how could any econometric model predict that the Lehman would fail when it did?

    So the question is whether DSGEs provide another useful tool. As the original post points out, so far this has not proven to be the case. (Again keep in mind that while Central banks have DSGEs, they also use Klein-type models. The Bank of England, as far as I can tell from what they show on their web site, haven't replaced the Quarterly Model with a DSGE.)

    We had a session on models a couple of years ago at AEA (sponsored by NABE, so it was relegated to an inconvenient location). One of the Fed people at that session made an interesting point: That central banks (and especially the Fed Board) have a lot more resources than the private sector, so they simply buy more of all types of forecasting, without necessarily making strict cost-benefit analysis of what they are buying. A nicer way to put it might be that central banks are more willing to buy new, risky technology than the private sector.

    In any event, I welcome the possibility that DSGE models could provide a useful tool. I am currently a purchaser of a traditional model, but would welcome a sales call from somebody who could convince me that their DSGE model would be a better tool.

    Any takers? If not, I think the original post stands.

    ReplyDelete
    Replies
    1. I don't get what you're saying about about the BoE? The core of their old forecasting framework was a DSGE model. The difference relative to a VECM is that instead of just having convergence to some long run static EQ, their error correction terms are responding to the deviation between a dynamic equilibrium model (a DSGE model, or maybe rather a DGE since it seems the BEQM core is non stochastic). Their new core model is a DSGE model similar to an open economy version of Smets and Wouters' model+some credit constrained households.
      The post says this has not proven to be the case so far very frequently in the private sector. But I don't know that Klein type models were useful in the private sector in the 1950's either, which didn't mean they're not useful today. If I understand correctly the leaders in using large macro models were a macro consulting firm established by Klein himself and the Fed through Modigliani's MPS model. At the time it must have been much easier probably to gain clients just by showing up with a model when there was nothing comparable. Nowadays, the market is still oligopolistic but more crowded, and it would be harder for someone like Larry Christiano to open up a macro consultancy business at the same level as say Oxford Economics or IHS: it wouldn't be enough for him to quit his university job, he'd need to find dozens of other like minded economists, coordinate them, hire secretaries and business development managers...in short there are barriers to entry into the field, and like in other (semi) scientific fields it's not always obvious to convince non expert clients that a new method can do better than something that's been around for decades and seemed to provide a satisfactory though highly imperfect service. But who knows, maybe Bernanke and Gertler will get bored with academia (not sure what Bernanke's plans are now) and they'll team up to provide macroeconomic advice based on their New Keynesian DSGE+Financial accelerator model? I personally wouldn't mind listening to Bernanke's macro advice.
      By the way, Nate Silver and my comment weren't about how come the point forecast from business economists didn't predict Lehman brothers or other recessions. The point is that typical private sector forecasts seem to be badly calibrated: their confidence/uncertainty bands around the forecast (e.g in the SPF density forecasts) are too narrow and underestimate the risks facing the economy (and it's true this was also an issue for DSGE models estimated on great moderation samples when the financial crisis hit) - and why is everyone so fixated on the point forecast when you know it's an (almost) probability zero event? With all the uncertainty around, it's the confidence bands and the scenarios which should be more important.

      Delete
    2. Daniel Bachman5:00 PM

      Daniels:

      First, I just want to thank you for having such a reasoned discussion. I appreciate learning from you about some aspects of DSGE's I didn't know.

      On the BOE: I got this from looking at their website, so maybe I am wrong? But it does seem to imply that they still use their quarterly model, which looked a lot to me like a small-scale traditional model. I know for a fact that, at the Fed Board, the staff's main tool remains FRB/US (I have a chapter on Forecasting in the Federal Government coming out in a book soon(?), so I've spoken to people there about it). Not that DSGE's aren't used, but when it comes to organizing all the pieces of the toolkit, the Klein-type model is still at the core.

      I don't think you are correct about the IO characteristics of the economic forecasting industry. There have certainly been new entries over the years---Dismal Scientist/Economic.com came out of nowhere to pick up a significant market share in the 1990s, for example. And the big firms are always updating their models anyway. If there was a market advantage to be gained from using a new methodology, I think it would have been adopted. I know this because I've been there (and pushed successfully for the adoption of error correction in estimation, among other things). In fact, I heard that one of the large firms hired a DSGE modeler at one point, but they parted ways after a couple of years when it became evident that the firm's investment wasn't worthwhile.

      It doesn't take "dozens" of economists. Three Ph.D.'s and maybe one full-time manager could probably get it started. There's plenty of turnover among business economists, so lots of opportunity to find people who are ready for something new and different that they can sell to their users as more up-to-date than what others are using.

      Agree about the calibration issue--and especially the point forecasts. But that leads to the difficulty of getting users to see beyond the point forecast. Having been there a lot, I can assure you that it's non-trivial.

      You see, I would like to get beyond the idea that DSGE's aren't accepted because of market failures, or ignorance, or whatever, and look more closely at how and why decision-makers use information--and how economics supplies it to them. I think that's a really worthwhile area of thought that ought to play a role in driving the research agenda. And I think the survival of Klein-style models is a really interesting data point in that context.

      Delete
  42. Anonymous10:35 AM

    I can speak from the model based macro hedge trading perspective.
    1) Most of the biggest macro funds are fundamental, NOT technical. And I mean the ones that are running models.
    2) This is a case of having no faith in the DSGE or Fed models. Speaking from personal experience (over 20 years), I can say that a lot of the traditional theory is suspect. Recent examples being - no balance sheet or debt view of the economy has led to inflation and growth being lower than forecast. Interest rates do not work in the real economy as advertised in traditional models. Why should I use them if they do not work? It is clear to us in the real world that the world does not work like the Fed describes.
    3) Dont even get me started on currency trading. The fact that academic models have such a bad track record should tell the macro guys something.
    4) Ultimately, the continual denial by the Fed that asset prices do not swing between overpriced and underpriced AND that they have nothing to do with it just plain does not make any sense.

    ReplyDelete
    Replies
    1. This is consistent with everything other people have told me!

      Delete
  43. A consultancy in New Zealand called BERL runs a CGE model. I am not aware of anyone who takes their work seriously. I am aware of many people who think they are muppets.

    ReplyDelete
    Replies
    1. Jim,

      CGE models have been around for a long time and have been used by quite a few private consulting companies. However, they are not DSGE models. They are micro models used for looking at sectoral shifts or implications of policies.

      Barkley Rosser

      Delete
  44. I have posted on this topic now, a bit late I guess, at Econospeak, econospeak.blogspot.com/21014/01/are-people-being-mean-to-those-doing.html , although I must note, Noah, that I am somehow finding myself unable to link to your posts, although this is now the second time I have tried to do so and failed. Not sure what is up.

    ReplyDelete
  45. Someone just clicked a link here which sent him or her to my blog (thanks). So I re read the post. It is brilliant -- absolutely wonderful

    ReplyDelete