Slashdot Mirror
The Perils of Simplifying Risk To a Single Number
A few weeks back we discussed the perspective that the economic meltdown could be viewed as a global computer crash. In the NYTimes magazine, Joe Nocera writes in much more depth about one aspect of the over-reliance on computer models in the ongoing unpleasantness: the use of a single number to assess risk. Reader theodp writes: "Relying on Value at Risk (VaR) and other mathematical models to manage risk was a no-brainer for the Wall Street crowd, at least until it became obvious that the risks taken by the largest banks and investment firms were so excessive and foolhardy that they threatened to bring down the financial system itself. Nocera explores the age-old debate between those who assert that the best decisions are based on quantification and numbers, and those who base their decisions on more subjective degrees of belief about the uncertain future. Reliance on models created a 'false sense of security among senior managers and watchdogs,' argues Nassim Nicholas Taleb, who likens VaR to 'an air bag that works all the time, except when you have a car accident.'"
For an EXCELLENT article about this, read Malcolm Gladwell's "Blowing up", which you can find online for free:
http://www.gladwell.com/2002/2002_04_29_a_blowingup.htm
Watch the Teaser Trailer for "The Lightning Thief" Her
http://www.nytimes.com/2009/01/04/magazine/04risk-t.html?pagewanted=all
Hmmmm. Math or "subjective degrees of belief about the uncertain future".
I've always operated on the principle that they were all lying, thieving, immoral, unethical, and greedy fucking bastards that were ready to bend you over for a nickel. Seems my supposition is being proven correct more and more each day.
Until recently, it was the smaller guys in the stock market that were getting screwed and the whole system kept the thievery down to a manageable level. Now from the largest, to smallest, they all seem to be getting destroyed, American in ruins, and the previously rich and powerful with outstretched hands at the Feds.
Of course maybe that is too cynical, but I always saw the stock market as rigged from the beginning. What do I know though? :)
So what's the VaR of using VaR? :)
When you don't have a leg to stand on, don't even get up.
I don't think that the problem is a single number it is connectivity. You might think that if you have three investments with a 10% risk of losing £1,000,000 the chances of all three of them losing £1,000,000 is 0.1*0.1*0.1 = 0.001 or 0.1%. The thing is if one loses that much then the markets may lose confidence meaning the others go down too - they are not independent probabilities.
The problem in the financial world is one of thinking there's a single factor called "risk". In fact there are many, interlinked factors: The risk the business will go bust is one - however from that sprout a whole range of subsidiary risks: from losing all your investment to getting back 95% of it.
Similarly with mortgage risk and any other type of investment. What the financial markets need is a better understanding of the causal links between risks and to price the returns on investments accordingly.
That will be a *big* job, and one that will take years or decades to iron the bugs out of.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Indeed we did. And I think we came to the consensus that it was a load of bollocks.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Nocera explores the age-old debate between those who assert that the best decisions are based on quantification and numbers, and those who base their decisions on more subjective degrees of belief about the uncertain future.
Why is anyone still making this distinction, as we now know that the only self-consistent numerical representation of risk follows directly from our subjective degrees of belief about the uncertain future? Furthermore, we have known this for over a generation... isn't it about time that the knowledge start filtering into the popular discourse?
While Bayesian methods are not always all that useful for practical problems (I use them on occasion in my work) the conceptual foundations and deeper understanding of the nature of plausible reasoning and its relation to probability theory needs to be more widely understood.
One of the big take-home messages from the Bayesian revolution is that probability theory is nothing but quantification of what we do subjectively, insofar as our subjective impressions are self-consistent, so the only people who are still debating quantitative vs subjective approaches as such are people who do not understand the question.
Blasphemy is a human right. Blasphemophobia kills.
Is here any roleplayer that does NOT know how using an artificial value to describe "real" problems automatically leads to some people "playing the system" instead of playing the game?
Nobody here ever had a munchkin in his troupe? A powergamer? A minmaxer? Someone who learned the rules and immediately started to look for loopholes, how to play by the rules without actually taking them serious?
Now why did anyone think this would be different when real money is involved, and thus the incentive to abuse the rules way higher?
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Risk models are largely irrelevant because the only risk anyone in the financial sector is really interested in minimizing is the risk that they will get fired. The way to do that is to do almost exactly the same thing as everybody else, no matter how mind blowing stupid it is. Plenty of people realized that banks etc were not nearly as sound as commonly believed years ago. Those that tried to act on this were fired long ago since they weren't making as high a ROI as those willing to invest in dodgy hedgefunds etc. Rational market my ass.
http://rareformnewmedia.com/
If the Government becomes a lawbreaker, it breeds contempt for law;
Objective or subjective models don't mean anything to people who only care about short-term performance. Whether the investment is good or bad in the long term doesn't matter to an investment manager who stands to get a seven figure bonus based on the current year's numbers. So what if the company fails next year? Not his problem.
Liberal economics -- not liberal politics, quite the opposite most of the time -- explicitly derives its conclusions from three assumptions: that individuals make rational decisions, that they have access to information, and that they are free to buy/sell.
Those are pretty reasonable assumptions, and, when they hold, the conclusions tend to hold.
The difference with physics is that when physicists start saying "assuming that this body is of negligible mass and at non-relativistic speeds" they don't end their exposé with "thus we have a solution to the three body problem for three super massive black holes at 0.999 c"
Social psychology has shown repeated instances where rationality is seriously impaired. For example, social proof can make us all really stupid. And cognitive dissonance is a bitch. What do those words mean? When a million idiots do something stupid, you're very likely to think it's a very good idea, too. And the longer you've been doing something stupid without negative consequences, the less likely you are to stop.
Add to that the fact that those "investment vehicles" were designed to conceal information, specifically financial risk, and right here you have two out of three pillars of classic economic theory missing. Is it any wonder the whole thing went down?
Finally, I wonder if any free marketer / libertarian types actually read any Adam Smith. I remember reading a quizz, which unfortunately I can't find anymore, Marx vs. Smith, in which you were asked to identify whom had written what. Very hard to tell them apart in some cases.
No, the problem was reducing to a single number, you yourself say that just looking at 95% VaR (2 to 3 times occurence daily over one year) is not enough. You're right they need to consider 95% and 99% VaR, among other levels of risk tolerance...and I know many firms do and have been. I believe the bigger problem was the faulty assumptions in calculating VaR, primarily assuming a standard distribution bell curve. Many portfolios do not have symmetric profit. Also, when prices start to soar or plummet, volatility increases. Furthermore, a random walk doesn't correctly articulate the complex actions of market participants (prone to fear, marriage to a position, etc.) and IMO underestimates the outer reaches of the bell curve. According to this flawed modeling, it wasn't a once in 30 year event being ignored, it was a once in 30 million year event. Obviously it is extremely probable that was an incorrect estimation of the risk, but it wasn't an error of only looking at 95% VaR.
The problem with using a single number is simple: It is easily gamed and there's lots of incentive to do so.
So people will sell you worthless junk that technically has a high number rating because if you're relying on the number you'll pay them for their worthless junk.
Fanatically anti-fanatical
As an example, suppose that the distribution suggests the chance of losing 50 million dollars is +3 sigma for some measure. The problem is that there is a subtle effect - say panic, herd effect or some interaction of derivative models - which only becomes significant around the 3 sigma mark. The result could be that the exposure at a 4 sigma event is billions of dollars. A proper risk model would need to take this into account
My conclusion based on what I have read so far is that the physicists (in particular) involved in developing quantitative models would have benefited from a lot more exposure to real world experiment. They would then have had more of a clue about the unreliability of data away from the mean, scatter, and the importance of the fact that in physics subtle errors turn out to be signs that the model is wrong - e.g. relativistic effects only become important at a significant fraction of c.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
Economics is not a science.
Science is the application of the scientific method. When's the last time you saw an economist perform an experiment where only one variable was at play?
If he explores all forms and substances Straight homeward to their symbol-essences; He shall not die.
A friend of mine is a risk assessment quant who was working at Lehman right up to the point where they declared bankruptcy. I asked him about this article the other day. He said that their models started telling them something was very wrong back in 2007. The problem was that Fuld (the CEO) refused to believe what the models were saying.
The most accurate model in the world won't help if you don't pay atention to the results it produces.
There's also apparently an issue with the classical VaR models depending on transparent pricing, which these real estate instruments lack. So some of the most troublesome assets apparently weren't in the model.
You might think that if you have three investments with a 10% risk of losing £1,000,000 the chances of all three of them losing £1,000,000 is 0.1*0.1*0.1 = 0.001 or 0.1%.
No, no-one who actually calculates and uses VaR thinks that. Anyone who has done any statistics, like all finance quants, will correctly take into account covariances. The actual problem is the interpretation of the "correct" VaR, and relying on it too heavily.
I'll give you the actual definition of VaR. If you calculate the VaR(10 day, 5%) to be $100,000, this means that there is a 5% chance that the loss on your portfolio over a 10 day period will be larger than $100,000, or that your profit will be larger than $100,000 assuming a symmetric distribution. It's when people think "Oh that's great, we can ONLY lose $100,000" when you have a problem. The actual loss could be ANY value larger than $100,000.
It's hardly a perfect statistic, since there are still many assumptions involved. However, it's still a decent estimator and it's better than making a wild guess based on gut feelings. Despite what most people currently believe, a lot of brainpower has gone into developing financial theories and some stuff is pretty damn good. The financial industry deserves some bashing, but it frustrates me when people spread incorrect information; at least complain about the right things.
Commonly used analogy in derivative trading: "Picking up nickels in front of a steamroller" (sometimes bulldozer).
Modest returns, low rate of failure, but really messy when you do fail...
Reality's an untamed beast
That's difficult to master,
But models are quite docile
And give you answer faster.
From a pome I saw in a computer book from the 70's, can be found online here http://www.langston.com/Fun_People/1993/1993AFE.html
vi +
Even if it's a complex number? Or would that make it too imaginary?
You'd have to provide some evidence that most foreclosures are investment properties. More likely everyone believed that they'd be able to refinance out of their ARM on their primary (read:only) house, because "home values all ways go up." When that wasn't the case you get what we see now.
There's plenty of blame to be spread around, from the builders who overbuilt saturating the market to the bankers who financed every subdivision to come along, to the home buyers who thought they wouldn't really have to pay the higher rate in x years, to the mortgage brokers who sold loans to people who couldn't afford them on commission, to the banks who bought, them bundled, them broke them apart, and took out CDSs on them, to the hedge funds, retirement pensions, and private investors and anyone else who didn't bother to divide median home price by median income to see if their investments were reasonable.
What you appear to be trying to describe is the neo-liberal paradigm. That's not really what I'm talking about, although it is my opinion that it is complete bollocks, but that's just my opinion.
My point is that you can't take liberal economic theory, keep the conclusions and expect them to hold when you've clearly removed the starting assumptions.
On top of that, what you write isn't even logical:
Or maybe, the assumptions are:
- that individuals make decisions which are more rational than if someone else makes it for them
What does "more rational" mean? Classical economic theory assumes that someone is rational in that it will buy something at a lesser price if it can, and will attempt to sell the least of something it's got (good, service, labor ...) for as much money as it can. That's it. How can you be less rational?
In any case, if there is government intervention, which is I suppose what you are against, it's got nothing to do with the rational part of the argument, it has to do with the freedom part of it. And I haven't talked about this.
- that they have access to better information
Again, what does that mean? If gov't regulation forces companies to be more transparent (a la Sarbannes-Oxley), it means less freedom for the company but more information for the market as a whole. It's once more an impact on the third assumption but clearly not on the second.
I disagree, accurate information is required for the market to choose the most efficient price/product.
"In the long run we're all dead" Keynes famously said; and in that case it means that, sure, if we wait long enough, people will stop trusting liars and crooks and they will be weeded out, but by that time damage will have been done, and the market will not have functioned optimally in the mean time.
Furthermore, engineered systems have two separate control systems: normal operating controls and independant safety controls. Never the twain shall meet, for often the normal controls exacerbate the situation and must be pre-empted by the safety controls. The more advanced the normal controls (optimization), the more advanced the safeties have to be.
None of this is present in finance. VaR may be all well and good as a normal operating measure, but does nothing in the tail which will blowup. I do not see anything as a tail safety measure institutionalized. What measures are taken are done on "gut feel".
Risk, in financial terms, is a measure of the variability of returns, i.e. the standard deviation of the returns. A well-diversified portfolio generally reduces the variability due to the individual risks of investments being uncorrelated. Harry Markowitz, the father of portfolio theory, pointed out that these quants all assumed that a basket of mortgages is highly uncorrelated and thus well diversified. However, in a broad real estate downturn, they all become very highly correlated. Therefore, if your standard deviation WAS 10%, it suddenly becomes 50% or more, which rapidly changes your VaR from a handful of millions to several billion overnight. VaR, being an oversimplification, didn't take that into account and all the big investment firms suddenly had billions of dollars at risk and billions of dollars of losses without realizing it. It's simply a matter of garbage-in, garbage-out, something my Portfolio Analysis prof drilled into our head and hopefully gets drilled into the heads of Wall Street CEOs.
The biggest problem with those creating financial instruments from home loans is that no one tested their models with systemic housing price decreases.
Economist Arnold Kling said that many years ago, Freddie Mac actually did "stress testing" of their portfolios under a 20% systemic real estate market downturn, but during the early 2000's they abandoned this technique.
CDOs did a good job of reducing the risk of early repayment and "random" defaults on mortgages. However it ended up concentrating the risk for a systemic market downturn.
Unfortunately, I am sure that some time in the future there will be another huge systemic risk that both government and the private sector will miss and we'll get hit again. The only thing we can do is keep economic freedom high in the period in-between to allow the economy to restructure (less jobs in building, more in health care, for example) in order to return to growth.
Probability of something going wrong: 1.0
Rethinking email
It's pretty clear what they (classical liberal economists) mean by rational, information and freedom. The definition is part of the theory.
And since this is a theoretical model, it is also understood that nothing in reality fits perfectly.
When people are rational most of the time, are reasonably informed, and have some freedom to buy/sell, market will work for the greater good. That's the theory.
I'm just saying that here people weren't informed, and weren't being rational due to social proof + commitment; and that there's no need to invoke the dreaded loss of freedom to realize that the whole system couldn't work according to freemarket fundies' theories.
Access to information is never perfect -- being subject to scarcity like all other goods
Really? That's a very peculiar statement to make in this day and age, and on this particular site.
In essence, you seem to be treating freedom as an independent (even insignificant) aspect of economics, when in practice you cannot assume either rationality or optimal access to information without it.
I get it, you're a libertarian. You defend your opinions, if only just by parroting your usual lines.
Me, I'm just looking at the underlying theory. Rationality, information, freedom. Three conditions. Two of them are missing; whether the third is present or not is moot.
What's so hard about it?
Disclaimer: IAAMFPHDS (I am a mathematical finance PhD student).
While quants could accurately gauge the historical covariance of different assets in a portfolio, what they failed to take into account is that there is correlation in the tails of the distribution.
An example of this is that, back in the good old days, there was a degree of correlation between the Dow and the FTSE 100. If the FTSE 100 went up, it was a decent indicator that the Dow would also be up, but by no means a sure thing. However, during the crisis, the two indices practically moved in lockstep.
The moral of the story is that in the rare event that things get bad, correlation tends to spike. The models failed to take account of this, which is part of the reason we're in this mess.
Slashdot: news for Apple. Stuff that Apple.
The problem with VAR is not the measure itself, which is assuredly useful if one understands the limitations.
The problem is that once any risk measure (that is say, 95%+ 'reliable') becomes institutionalized as the gold standard, catastrophic failure of the financial system is inevitable (at least according to the general black swan theory).
Why? Because any firm that doesn't optimize profit against the risk criteria is going to have a lower P/R, and will lose capital to firms who are more 'efficient' at investing as long as things are 'normal'. This will result in the firm either folding, or being acquired.
If the firm does optimize to the risk criteria however, they stick a ton of risk into the tails of the risk distribution, which isn't measured, and so they'll get taken out when the black swan hits (ie. a rare event occurs, and all that hidden risk smacks them upside the head).
(and yes, I know this is a very simplified explanation).
Standardized, regulated exchanges usually come about when a market already existed before, but it would be desirable to have a more transparent, reliable market clearinghouse. With stocks, people invented shares long before anyone opened an actual stock-exchange: you write out a contract on paper agreeing that, in return for $x, so-and-so now owns 1 share of your company, and you have a contract somewhere specifying how many total shares there are, when/if new shares can be issued, etc. It's basically what happens when you try to expand a small partnership to more than a few people and bring in investors.
Once you get enough of these fractional-ownership certificates floating around, each with slightly different rules, and disputes start arising about who owns what and what that means, the logical next step is to agree on some relatively standardized method of fractional ownership, and a central clearinghouse to trade the certificates. Which is what stock markets are.
On the other hand, moving that sort of business to stock markets also increases the number of market participants and frequency of transactions by reducing entry and transaction costs---it's impossible, for example, to "day-trade" paper stock certificates in person directly with their owners thousands of times per day. That has positive and negative effects---positive in that it increases available capital and gives retail investors more parity of access compared to large investors, and negative in that it makes the whole system more volatile and sensitive to chaotic-systems effects.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Wow. It looks like most people don't get it.
If Mr A gave Mr B billions of dollars of The Public's Money to play at a casino and both Mr A and Mr B got filthy rich when times were good, and when it blows up all that happens is Mr B loses his job and Mr A keeps his job by blaming Mr B or saying BS like "perfect storm/everyone was doing it".
Why then should Mr A and Mr B be doing things differently?
After all, in the following year, Mr A passes billions to Mr C who does pretty much the same thing as Mr B. And Mr B? He's hired by Mr D who wants Mr B to make him richer (just like he did for Mr A).
AFAIK, not long after LTCM blew up, its founder John Meriwether still managed to get hundreds of millions of dollars to start a hedge fund.
What I see are individuals making pretty rational decisions, those decisions sometimes just happen to be bad for a lot of other people. But why should those individuals care?
Their conscience should bother them? The last I checked the Economists leave the conscience stuff to "The Invisible Hand". People laugh at the religious, guess who really has even less of a clue on how things work? At least the religious have some idea about the "Invisible Hand" sort of stuff.
It's hilarious that you have all those people saying/writing stuff like "When Genius Failed".
That's like the sheep saying the wolves have failed just because the wolves dropped 95% of a billion sheep over a cliff, whilst "only" managing to stuff themselves to the brim with 1% of the billion sheep. I'm sure the wolves were a bit upset about the whole thing, but hey there are billions more sheep...
Yeah I see failure alright. Go figure where.
You want to reduce the risk of stuff blowing up, and how big they blow up? It has nothing to do with creating better financial models or better economic theories.
It has to do with making and enforcing rules like: if too many sheep die, we shoot and skin the wolves responsible. Simple as that.
All that transparency and regulation is worthless if at the end of the day the wolves get away.
> After seeing the rampant fraud committed by the global financial elite, I'm very inclined to agree with you. What we need isn't just a number that quantifies risk, but also a number that quantifies trust.
Won't they just game that number, too? Once you fix the rules for any system, people will start to attack them. And, based on what I've seen in online games, they'll find a way to break them. Especially when there's real money at stake, not just virtual gold and items (though even those can be converted to cash these days...).
This is related to the paradox of high standards: once you set your standards too high, the only people who "meet" them are the worst cheaters.