Slashdot Mirror


The Formula That Killed Wall Street

We recently discussed the perspective that the harrowing of Wall Street was caused by over-reliance on computer models that produced a single number to characterize risk. Wired has a piece profiling David X. Li, the quant behind the formula that enabled the creation of such simple risk models. "For five years, Li's formula, known as a Gaussian copula function, looked like an unambiguously positive breakthrough, a piece of financial technology that allowed hugely complex risks to be modeled with more ease and accuracy than ever before. With his brilliant spark of mathematical legerdemain, Li made it possible for traders to sell vast quantities of new securities, expanding financial markets to unimaginable levels. His method was adopted by everybody from bond investors and Wall Street banks to ratings agencies and regulators. ... [T]he real danger was created not because any given trader adopted it but because every trader did. In financial markets, everybody doing the same thing is the classic recipe for a bubble and inevitable bust."

24 of 561 comments (clear)

  1. the formula that killed wall street: by Shakrai · · Score: 5, Insightful

    G+R+E+E+D

    --
    I want peace on earth and goodwill toward man.
    We are the United States Government! We don't do that sort of thing.
    1. Re:the formula that killed wall street: by Shakrai · · Score: 5, Insightful

      actually, greed is good. it's the great motivator. really, it's the only motivator.

      It's a good motivator when it's tempered with wisdom. It's a bad motivator when you blinds you to the long term consequences of your actions. I've been saying for years that it seems like our entire economic system has been tailored to next quarters results at the expense of building/investing for the long term. Who cares if this quarter has record profits if you paid for those record profits with the future viability of your enterprise?

      --
      I want peace on earth and goodwill toward man.
      We are the United States Government! We don't do that sort of thing.
  2. Nothing wrong with models. by gravos · · Score: 5, Insightful

    There is nothing wrong with using a model. Models are good. They help us simplify the world so that we can understand it. For example, we have hundreds of competing climate change models that explain what is going on and predict what we should expect. We model the weather for forecasts. And so on.

    But. And it is a big but. You must know the limitations of your model. By definition, a model is a simplification of a complex phenomenon. That does not make it flawed: that makes it a model. Overreliance on the model is your fault, not the fault of the model.

    1. Re:Nothing wrong with models. by morgan_greywolf · · Score: 5, Insightful

      However, there are some models that are just bad. If we take your climate change model example, simply going outside and measuring the temperature, and then comparing it to a temperature you took one the same day three years in a row and then plotting the statistical trend is a very poor model. Using that model, one might assume that we have drastic global cooling going on. It doesn't matter how much you rely on that model, if you rely on it all, you're going to be dead flat wrong.

    2. Re:Nothing wrong with models. by wjh31 · · Score: 4, Insightful

      Even more important that the limitations of a model are the assumtions taken in developing the model and/or feeding the data into the model, these should always be made clear to whomever the user of the model is, and it is then up to the user to decide if those assumtions are reasonable for their use of it.

    3. Re:Nothing wrong with models. by BrokenHalo · · Score: 5, Insightful

      You don't need an MBA to know there are bust-boom cycles.

      You also don't need an MBA to know that there is a limit to the number of balls a juggler can keep in the air at any time before he drops one. And when one ball drops, the whole thing falls apart. As the truism goes, those who don't learn from history are doomed to repeat it...

    4. Re:Nothing wrong with models. by maraist · · Score: 5, Insightful

      I disagree. A model defines a static or pseduo-static system. It takes the non-linearities out of a system to make them as close to a linear, 1st order or 2nd order system as much as possible, such that you can produce matrices of inputs to outputs. All models also are accompanied by regions of legitimacy.. Namely the non-linear (or super non-linear) components press close to zero in these regions. Outside the regions, those non-linearities become too much 'error' for the model to be valid. Ideally, you can use separate models for different regions, and you have a nice continuum. But for that, you need to be able to first measure a region parameter.

      The problem here is that you're talking about a model for an investment strategy that is inherently non-deterministic, non-linear and more importantly recursively adaptive. The region you're operating in, is part of the outcome variables.

      Consider 3 investors each with equivalent information systems (including risk modeling, present-valuation, and product-viability forecast, whatever).

      In a vaccum, a model, assuming a static system might be appropriate. Balance-sheets, due-dilligence, market trends, geo-politics, etc. are all valid. But consider that the other two investors have the power to effect the system. Consider that they can manipulate, propping up an industry, or willfully collapsing it (over-buying, or short-selling). By acting irrationally in the short term, they can sufficiently distort all the measureable parameters to your equation to force you to act inappropriately.

      Thus by taking a short-term hit, one of your competitors can gain a much greater long term advantage.

      Thus, KNOWING that you use certain models, allows your competitors to game the system.. Note they need to have significant resources in which to do this.. But the old addage that you need money to make money exactly applies here. Why would a wealthy person only accept 3% to 15% ROI when they can control certain markets and earn 500%.

      Now explicit market manipulation is illegal. But there is nothing illegal about gambling (sadly). Thus betting against the 'known wisdom' is perfectly legal.

      So now you have two camps.. Conservatives that trust their models (blind to the fact that people can manipulate them in the long-run). And advanced speculators who bet against the market. Over time, if one is considered unbalanced, then more and more itchy investors will switch from one side to another.. Until an equilibrium is reached where any and all metrics become meaningless - An equal proportion of investors will honor measureable data as there are people betting against the data. The raw data therefore has no material impact as to the future valuation of an asset. Note, as such a system evolves, the 'measureable' data will change over time. Namely instead of measuring the viability of a company, you measure the prospects for news and bet based on historical trends of the news outlets, not whether the news is good or not.

      This can only happen if you have a gaussian distribution of strategies. Namely a massive pool of investors operating independently with an equal liklihood of choosing one of an infinite number of strategies, such that an equal ration of buy/sell decisions could be produced.

      You can think of it as the classic "Is the poison in your drink or mine" attempt at gaming the system. Any number of strategies can be employed to decide which action is best, but the more you employ, the greater resemblance to random-decisions is created.

      The short is, no formula can adequately valuate a market that is based on such a recursively adaptive system. Determining the risk of a car accident, a plane accident, a flood, etc. These are deterministic to a large degree (short of global warming and legalizing pot). But the college that first advocated investment strategies based on such finite metrics should be unaccredited in my view. A car owner isn't trying to game the insurance market, but a stock holder or company seeking stock value is.

      --
      -Michael
    5. Re:Nothing wrong with models. by spun · · Score: 4, Insightful

      As most of the people responsible walked away with a fat profit, one could posit that they did learn from history. Why should they care that they screwed over the rest of us? Our economic system ignores externalities on that scale. Burn down a house, go to jail. Burn down the economy, get a fat bonus.

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    6. Re:Nothing wrong with models. by OeLeWaPpErKe · · Score: 4, Insightful

      Actually the assumption is this. "Lousy" mortgages are mortgages with 12% chance of default (and this was extremely high, most were something like 2% or so).

      So let's calculate what happens :
      1000 "lousy" mortgages, $200000/house, profit if fully paid back = $20000, over 20 years, profit in case of default = value of house (which rose about 1% per year on average). Default occurs "on average" after 1 year.

      So let's see :

      Initial investment : 1000 * -$200000 = -200000000
      Income from defaults : 120 * ( $200000 + 1% * $200000) = $24240000
      Income from success : 880 * ( $200000 + $20000 ) = $24240000

      Total profit : $24240000

      I dare you to find the flaw in these numbers. Yet there was a flaw. Without an education in statistics you will not see this coming, and even if you do, it's not trivial at all :

      That 12% failure rate was non-uniformly distributed. So what happened ? First 10 to 15 years nobody defaulted, resulting in a house price hike that was a lot higher than anticipated (and therefore further increasing the profits and decreasing the risk to the banks), and then, after 16 years of tiny amounts of failure, huge amounts of people started defaulting all at once, causing a massive decrease in house prices.

      This is where the theory of "let's just support the banks for one year and they'll pull through" theory comes from. IF default rates drop massively it will work.

      So why did this break the banks ? Well due to hedge funds. Hedge funds are the equivalent of using borrowed money in a casino. If you win, it multiplies your winnings. If you lose, it multiplies your losses. When a small loss accumulated, it was massively multiplied (as in a factor of 1000 was not rare) by hedge fund managers. Whether the "smaller" original loss would have been enough to topple banks, we'll never know. One thing is for certain, all these "oh I predicted it" hedge fund managers are like the devil laughing at his victims burning ... they're not innocent of the crash, at all. They're the maffia bosses who massively profit by making loans they KNEW the debtors wouldn't be able to repay.

      And now they act all "I told you so".

      They saw a disaster coming, and by worsening the disaster, they made their fortune. These people should be hanged from the highest available flagpole, not celebrated.

      Fantastic flash explanation

    7. Re:Nothing wrong with models. by Archangel+Michael · · Score: 4, Insightful

      Burn down the economy get a big fat check is about right.

      The problem with slow justice, is that it gives the appearance of no justice. Bernie Madoff will, in all likelyhood, die without spending a day in PMITA Prison.

      And his family will keep the benefit of all the Billions that disappeared, because all the assets in trusts and such are untouchable by the law, even if they are ill gained. That is what a trust was created exactly to protect. Trusts are nothing more than legal money laundering.

      And there is nothing wrong with making a profit. The problem isn't profit, or even greed. The problem is that people will use the rules in place to screw others and hide the loot. It doesn't matter what the rules are, and the more complex the rules, the easier it is to hide malfeasance. More rules don't stop it. People will do evil regardless of the rules.

      Bad people don't follow the rules, and will use the rules as an excuse to do bad. They use the rules to take advantage of others. Once people realize that rules are for the law abiding, not the law breakers, then we can get rid of all the stupid rules which don't prevent anything.

      --
      Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  3. One word by DigiShaman · · Score: 5, Insightful

    Diversity.

    --
    Life is not for the lazy.
  4. Or there's my financial formulae by Rosco+P.+Coltrane · · Score: 5, Insightful

    - Don't spend the money you don't have
    - Don't do credit unless you absolutely have to

    I know I know, Wall Street are these big finance hotshots who do complicated things that have nothing to do with personal finances, but what is it they do apart from speculating and playing with money they don't have, or other people's money? They just hide that simple fact under abconce financial constructs, but that's all they do in the end.

    Bring back some morals sanity in the credit business and there won't be anymore crisis of this magnitude. No need for math here...

    --
    "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
  5. Not Wall Street. Us. by computersareevil · · Score: 5, Insightful

    It isn't killing Wall Street. Those jokers are getting $billions$ in free money.

    It's killing us, the people who work for a living and have to provide all those $billions$ or suffer the inflationary consequences when the Feds just print it.

  6. Re:Not Wall Street. Us. by Notquitecajun · · Score: 5, Insightful

    A BIG part of the problem is Washington's tendency to reward economic losers at the expense of the people who know what they're doing, and I'm NOT just talking about the poor. There are plenty of the high-salary types who have some sort of governmental loophole or backing that saves them when they screw a big company up.

    It's one reason we don't need to be bailing out bad companies, and instead rewarding or backing up the good ones with incentives and tax cuts so that they can really succeed and push forward.

  7. Thou shall not calculate behaviour by Anonymous Coward · · Score: 5, Insightful

    This brings me to the crucial issue. Unlike the position that exists in the physical sciences, in economics and other disciplines that deal with essentially complex phenomena, the aspects of the events to be accounted for about which we can get quantitative data are necessarily limited and may not include the important ones. While in the physical sciences it is generally assumed, probably with good reason, that any important factor which determines the observed events will itself be directly observable and measurable, in the study of such complex phenomena as the market, which depend on the actions of many individuals, all the circumstances which will determine the outcome of a process, for reasons which I shall explain later, will hardly ever be fully known or measurable. And while in the physical sciences the investigator will be able to measure what, on the basis of a prima facie theory, he thinks important, in the social sciences often that is treated as important which happens to be accessible to measurement. This is sometimes carried to the point where it is demanded that our theories must be formulated in such terms that they refer only to measurable magnitudes.
    It can hardly be denied that such a demand quite arbitrarily limits the facts which are to be admitted as possible causes of the events which occur in the real world. This view, which is often quite naively accepted as required by scientific procedure, has some rather paradoxical consequences. We know: of course, with regard to the market and similar social structures, a great many facts which we cannot measure and on which indeed we have only some very imprecise and general information. And because the effects of these facts in any particular instance cannot be confirmed by quantitative evidence, they are simply disregarded by those sworn to admit only what they regard as scientific evidence: they thereupon happily proceed on the fiction that the factors which they can measure are the only ones that are relevant.

    Hayek. Nobel Prize Lecture, 1974.

  8. Don't Blame the Equation by mothlos · · Score: 4, Insightful

    This seems to be a popular story for the past few weeks, but it is a mistake to blame the statistical method used. The problem wasn't that they were all using the equaton, it is that they were all mis-using the equation. All statistical tools can fail to be sensitive to certain aspects which may be critical to an application.

    People in finance applied these statistical tools believing that they would be able to master risk with them. Unfortunately, they made assumptions that certain things would continue to be the same in the future, plugged the information into the equation, and now science was telling them that everything would be alright. If everybody on Wall Street was making decisions based on the Magic 8 Ball would we blame the ball or the foolishness of those misapplying it?

  9. Happy square root day! by mcgrew · · Score: 4, Insightful

    The love of money is the square root of all evil.

    This formula may have and probably did help crash the world's stock markets (yesterday's Dow Jones was HALF of its worth at its high last June), but the reality is that high energy prices drained everyone's wallets.

    When Bush took office, gasoiline here in Springfield was $1 per gallon. At Wall Street's high last summer it was nearly $4.50, over four times as high. We talk about elders living on a "fixed income" but the fact is almost all wage earners' incomes are fixed. We can't demand raises or overtime and have to live within our means. But when that $20 per week gasoline budget quadruples to $80 per week, with heating and electric costs going up as well, that takes money out of other aspects of the economy. Sooner or later people are over their heads and behind on bills, and things spiral out of control.

    The result of that and other factors is what you see now.

    Happy square root day, everyone.

  10. Re:Citation, please by Dunbal · · Score: 4, Insightful

    With respect, classical economics and Austrian economics are not quite the same thing

          Sorry, I'm not an economist. Therefore if I said something incorrect through ignorance I apologize. I merely wished to emphasize that truly we live in interesting times. I think it's when the world (and especially the consumer intensive US) finds out we've bumped into the limits of our resources on this planet. We can't all have an SUV. We can't all waste electricity. We can't all have a worry free life, and independence, and a nice house, and a big screen tv, and eat in good restaurants, etc. The boom in commodity prices - in part fueled by massive demand from the BRICIT countries that are also expanding their middle classes and trying to adopt an "American" standard of living - has another side to it. Not only was demand increased - but supply is at or near maximum. There IS no more copper, there IS no more gold, platinum WILL run out in 20 years or so, etc.

          Therefore commodities (including petroleum) priced themselves right out of the market. This triggered, and is triggering, financial default from everyone who was living "the dream" on credit. And now the cards keep tumbling. Oh, we will reach a new equilibrium some day - but our population keeps expanding, and those resources keep getting more scarce.

    --
    Seven puppies were harmed during the making of this post.
  11. This Is Their Bullshit Cover Story by Doc+Ruby · · Score: 4, Insightful

    Yeah, "complex mathematical model". Tell it to the judge.

    They did indeed use this model, and the work of many other PhD mathematicians, physicists, and other geniuses. But any of the bankers could have looked at this whole class of derivatives from mortgages and seen the basics that make the model a joke. They sold millions of mortgages and other loans to people using artificially low initial interest, to get people to take the loans, but which ballooned to rates they couldn't afford, so they'd have to default. Inevitably, a large percentage would certainly default. A losing bet overall for banks holding those loans. Meanwhile, each bad loan was "good" because the banks could sell many times the number of derivatives on it. Which was "good" because they got paid for the derivatives they sold, but was much more "bad" because the derivatives would cost the issuing bank many times more when it came due. The derivatives came due when the mortgages defaulted. Which was inevitable.

    So whatever "gaussian copula" model they use to convince each other it was good, basic business sense would have insisted that the business was bad, horribly bad. These bankers don't get paid for discovering new math, they get paid for their years of experience and business sense. So they should have laughed this model out of the boardroom, even if they didn't understand why it was wrong. They should have known it was wrong, as the past few years proved beyond any doubt. But they embraced it instead, and centuries old banks like Lehman Brothers have gone down, taking us with them (and no end in sight).

    Because ultimately, the model was a way to delay the costs of a business that paid some fat revenue up front. Since bankers are paid in huge bonuses for the initial year of revenue, and then leave before the bills come due , they got paid to make those bad deals, because they paid off up front, before costing many times more their benefit a few years later. By which time the bankers are gone with their early bonuses. Which have a lot more buying power when the economy collapses, and everyone else is holding merely the debt they created.

    Nice work, if you can get it. Since they ruined the banking system and everything else, no one can get any work at all.

    These people are holding the money. Their bonuses often equal the losses that destroy their bank. The government should take back that money to pay for fixing and repairing some of the mess they made. "Fiduciary responsibility" is a requirement of bank execs, and these violated that by the $TRILLIONS. Make them pay for what they did. That's a simple model anyone can understand. Not just a complex conjob to hide behind.

    --

    --
    make install -not war

  12. Greed by Arthur+B. · · Score: 4, Insightful

    Blaming greed for a financial crisis is like blaming gravity in a plane crash.

    --
    \u262D = \u5350
  13. Preposterous! by Comboman · · Score: 5, Insightful
    nature of bubbles is that they burst when they reach physical limits of the stuff of which they are made. In our case it was human gullibility.

    Preposterous! Human gullibility is one of the few things that has no limits.

    --
    Support Right To Repair Legislation.
  14. It wasn't li's fault because money is broken. by Colin+Smith · · Score: 4, Insightful

    It has been broken since 1694.

    Credit is an exponential function. Go check the national debt (in any country) for the last couple of centuries. It's an exponential growth curve. Credit has an exponential function built in to to it. When credit is created, it is created with an equivalent amount of debt attached, which pays interest.

    So you have : credit on one side | debt + interest on the other.

    So in order to work AT ALL, the supply of credit must grow exponentially every year to pay the interest on the previous year's debt. If it doesn't, there is a monetary collapse as the debt consumes the credit.

    Li's function simply allowed the process to continue until they ran out of people to lend money to. The problem has been there as long as money lenders.

     

    --
    Deleted
  15. Re:It wasn't Li's fault. by smellsofbikes · · Score: 4, Insightful

    >The managers making the decisions didn't know what it all meant and the guys using the model didn't adequately explain the model's limitations.

    Or the managers didn't understand their explanations -- or more likely yet, didn't *want* to understand their explanations.
    This doesn't look fundamentally different than the Challenger explosion: the technical staff knows there's a problem, keeps saying that there's a problem, but their upper management is invested in there not being a problem. It's really difficult to explain something to someone whose job depends on ideas that conflict with what you're explaining.

    --
    Nostalgia's not what it used to be.
  16. Re:Picking up pennies in front of bulldozers by Chris+Burke · · Score: 5, Insightful

    If they are random then why do they predict economic change with 100% accuracy?

    HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA!

    *gasp gasp*

    HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA!

    *pant pant*

    HAHHAHAHA, oh God that's rich. Seriously, you meant that? HAHAHAHAHAHAAHAHAHAHAHAHAHA!

    *wipes tears from eyes*

    No other reaction is possible for such a statement. Is this a delayed posting from 2007? Not that it wouldn't be equally laughable, but at least it was conceivable to maintain the self-delusion that it isn't. Today? You're saying the economy is 100% predictable, and you're saying this today.

    HAHAHAHAHAHAHAHAHAHAHAHAHAHAHA!

    --

    The enemies of Democracy are