Slashdot Mirror


Computer Glitch Causes Havoc and Losses on Nasdaq

goombah99 writes "In an illustration of how fragile the electronic stock market system is the NY Times is reporting how a tiny computer glitch rippled through the Stock Markets with buyers who bought low and sold high taking huge losses. An erroneous large sell order was entered. Many people bought at this low price, then signed options contracts to sell these at higher prices, locking in a profit. Or so they thought utill the erroneous low sell order was removed. Now to honor their options they had to buy the stock at a higher price. Since exchanges trust each other's trade prices it rippled throughout the system. There does not seem to be any way to gracefully undo such errors."

36 of 324 comments (clear)

  1. Poor slobs. by Highlander · · Score: 2, Insightful

    I wonder how many people got fired because of it.

    H.

  2. Trading has its risks by stomv · · Score: 5, Insightful

    and while the SEC and others do their best to proteect traders, mistakes do happen. This is part of the random process of the markets, and must be accounted for when making a trade, even on options markets.

    If you lost money, sorry. Unless the SEC/others can prove that somebody is liable for the initial mistaken order, you lose. Tough. Trading is risky, and sometimes the risks are completely unforseen.

    1. Re:Trading has its risks by wizrd_nml · · Score: 5, Insightful
      That's pure nonsense.

      The markets are meant for people to invest their money in businesses they feel will make a decent return for them. Investment risk consists of inherent risk of the industry, currency risk, political risk, etc. Nowhere in that equation is there EVER risk of a glitch in the computing system factored in.

      Mistakes happen because people are unethical, criminal, or just dumb managers. But mistakes should never ever happen because the system that you gave an order to buy or sell for you decided to have a glitch.

      Someone IS liable. NASDAQ is liable! NASDAQ is a company and it will be sued for the losses that it caused other people. It's as simple as that.

    2. Re:Trading has its risks by wizrd_nml · · Score: 2, Insightful
      Nasdaq can make all the rules it wants. But not all rules/disclaimers are legally binding. For example, you can't as a company say I'm going to sell you a car but I'm not liable if it doesn't run. Even if that rule was in the contract, a court wouldn't allow it.

      It would be interesting to see what happens next.

    3. Re:Trading has its risks by fermion · · Score: 4, Insightful
      The markets are meant for people to invest their money in businesses they feel will make a decent return for them. Investment risk consists of inherent risk of the industry, currency risk, political risk, etc. Nowhere in that equation is there EVER risk of a glitch in the computing system factored in.

      The investor chooses the risk and the reaction to short term changes. The low risk, long term investor would not likely be affected by this mistake. Such investors seldom keep track of minute to minute prices. Such an investor might notice something funny happened the previous day, and check on the details, but when nothing was found go on with life.

      It the high risk trader that is going to be burned by this scenario. This trader has chosen the high risk levels, and should know the consequences. And, your fantasy world notwithstanding, information is sometimes wrong. (In fact this case is not about computers but information dispersal) Sometimes you lose money. Sometimes you can sue for damages. But these traders are big boys and girl. The smart ones double check information if it seems out of the ordinary (and wrong quotes come by more than you might believe, often direcly from the exchange). They do not run home to mommy and daddy complaining that a 5th grader sold his pubes for $10, and you now want your money back.

      --
      "She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
    4. Re:Trading has its risks by bagsc · · Score: 5, Insightful

      You're at least half right. But this isn't a question of information about prices - if it were, NASDAQ and the ECN would be off the hook. This is a question of whether the ECN executed orders that should not have been executed and NASDAQ didn't cancel them all. That's when risk minimizers are hit.

      If I market buy because the ticker says a $50 stock is selling at $40, it goes through between 10:46 and 10:58, then NASDAQ is right to cancel at 12:30: no problem. If my option straddle executes on the volatility on both sides, one before and one after 10:58, but NASDAQ cancels the options in-the-money (on Instinet) but not the options out-the-money (on another ECN), then its a problem. If I'm an idiot, and leave those options open unchecked through a halt, then its my fault for engaging in a risky behavior and getting slammed in the ensuing short-squeeze.

      Other stocks in the sector were off by 10%, so it was not stupidity to think that a 20% move was legitimate. NASDAQ halted Instinet, but not other ECNs. Archipeligo already announced intentions to file suit with the SEC on the matter. And that won't be the last suit filed on it.

      --
      http://www.accountkiller.com/removal-requested
  3. Bound to happen. by dolo666 · · Score: 4, Insightful

    This reminds me of a nugget.

    The stock market is frail, and a fool's playpen. I remember hearing a story about a huge media barron, before the stock market crash that led to the great depression. The mogul was standing in this elevator and overheard busboys talking about how they were going to start playing the stocks. The millionaire immediately sold everything when he got to the office. His reasoning was that if two people who had no money were playing stocks, that they were a sign that the whole system was at fault and doomed. I forget who this person was, so if anyone remembers... hehe feel free to say.

    The guy's logic is correct even to this day, imho. The big companies that go public hope that an infusion of cash will make them more profitable, but it usually ends up that they get to take a break on stockholder's money for a while until it's deadline time again and they have to scramble to make product/service X work.

    The whole system is wrong.

    Look at all the ads for investing these days. They all suggest that you trust them to make you money, and they have as selling points, how easy it is to make money. The easier it is, the more moot it is, imho.

    There is no easier way than hard work.

    Glitches are bound to happen. Remember when the grid went down this past summer? I would have suspected major losses then, but somehow it wasn't that bad?

    1. Re:Bound to happen. by darkov · · Score: 4, Insightful

      There is no easier way than hard work.

      I wonder how old you are, because you sound like my father.

      The fact is that there is no harder work than taking a risk. Things like the stock market allow people to take measured risks in return for a greater reward. It also provides cheap capital for businesses and liquidity in trading businesses. Without it the economy would be less efficient and your "hard work" would buy you a whole lot less.

    2. Re:Bound to happen. by the+eric+conspiracy · · Score: 3, Insightful

      The stock market is frail, and a fool's playpen.

      Actually the stock market has been the best place to invest one's money over the last 70 years. No other investment actually outpaces inflation.

      Remember when the grid went down this past summer? I would have suspected major losses then, but somehow it wasn't that bad?

      When your predictions don't come true, it's time to re-evaluate your assumptions.

      Glitches are bound to happen.

      Yup. The question is what happens afterwards - do they get corrected and people move on, or do not get corrected and people lose confidence?

      The former seems to have happened here showing that the system has a degree of failure tolerance.

      Look at all the ads for investing these days. They all suggest that you trust them to make you money, and they have as selling points, how easy it is to make money. The easier it is, the more moot it is, imho.

      As you well know, advertisements and reality are two different things. Making money by investing it is hard work, requires intelligence, wisdom and perserverence. If you let yourself be driven by fads and ads, well you are one of the people PT Barnum was talking about.

  4. Hyper-transactional databases? by mcrbids · · Score: 5, Insightful

    My first thought when reading the summary above was that this would be an easy problem if managed by a central, relational database system.

    Simply "roll back" the transaction that failed, and the dependencies would cancel themselves out. But, then I realized that the current RDBMS model only allows for a single transaction - you can't nest them.

    Also, transactions are private only - you cannot transact with data in the middle of another transaction.

    Thus, you might have ACID compliance, but only with one level of "undo".

    How hard would it be to create an RDBMS that supports infinite levels of "undo" or transaction/rollback.

    Such that you commit transaction A, which affects rows 1,2,3, and 11. Then, another transaction B which affects (further) rows 2, 3, and 12.

    Then, if you roll back transaction A, transaction B would be similarly affected. I dunno - the depencies may get rediculous - but it seems that this could and should be done at some point.

    Bright idea? Or another noise from an unpleasant orifice?

    Let me know what you think!

    --
    I have no problem with your religion until you decide it's reason to deprive others of the truth.
  5. Not always possible by Pac · · Score: 2, Insightful

    Think about it: people who bought and sold the erroneously priced stock can undo their sells and buys. Now the people who bought from those first generation buyers must be allowed to undo also. In the second generation a new problem arrises: these people have no reason to undo nor do they have done anything wrong - you bought a car, paid $1000 instead of $10000 because the clerk at the store made a mistake and sold me the car for $5000: why should be forced to give you the car back when the store come to collect the rest of its money?. Now interact a little more - the future market works at a very fast pace, hundreds, thousands of trades may happen in a minute. Somewhere down the line things may get really messy, bith logistically and legally.

    1. Re:Not always possible by gertsenl · · Score: 5, Insightful

      I say there's a real simple way to solve this, no logistic or legal mess. Make them make good on the original sell order. They, in turn, want to sue the software developer? Let them handle that on their own time and out of THEIR pockets.

      --
      --Leo
    2. Re:Not always possible by Sancho · · Score: 2, Insightful

      If you're going to scam a store, it's best to do it with cash. Then they can't even call you, and they certainly can't try to charge you for the device after-the-fact. I'm surprised that didn't happen in this case.

    3. Re:Not always possible by gilroy · · Score: 3, Insightful
      Blockquoth the poster:

      Of course my friend didn't do it

      It disturbs me that no one seems to question this "of course". Your friend was wrong. He made an unjustifiable profit on a mistake made in good faith by the clerk -- a mistake he clearly recognized as such. What he did might have been legal but it was wrong. And yes, I'm the kind of guy who goes back to a store if I discover I've received too much change.
    4. Re:Not always possible by ameoba · · Score: 4, Insightful

      You ever try going back to a store and telling them you got too little change?

      --
      my sig's at the bottom of the page.
  6. Why undo such errors? by A+non+moose+cow · · Score: 4, Insightful

    "There does not seem to be any way to gracefully undo such errors"

    They wouldn't have to be gracefully undone, if there was a simple check to gracefully prevent them from being made.

  7. Warning: Instant big profits never happen... by LostCluster · · Score: 4, Insightful

    I think the people left holding the bag here are exactly the right ones: The ones who thought they were gonna make instant big profits.

    Not only did think they had bought something something at far below its value, they then signed options contracts to sell what they had just bought at slightly below its regular price. They should have known something was fishy... why would anybody want to pay close to the normal price to them if the price had just plumeted? Why would anybody want to sell to them at far below the usual price?

    The should have known that the rules of the game allowed for their trade to be undone, yet they committed to an options contract that couldn't be undone because if they had hesitated, they risked their "instant profits" going away... their fault.

    1. Re:Warning: Instant big profits never happen... by otherwhere · · Score: 2, Insightful

      You're half right. Every one of the, uncancellable, option trades had to be covered by a market maker in the option market on which it was sold. (A market maker is a firm that is always willing to buy at some price and sell at a slightly higher price, so that there is always a market.) The market maker doesn't just assume all the risk -- it covers it self by buying and selling the stock underlying the option, from the equity market, in this case nasdaq. These market makers, at least one of whom, per exchange, is obligated _always_ to provide a market for the stock, get their equity trades rolled back, also, even thought they calculated the price of the options based on the erroneous underlying -- if the stock is trading at $40 the right to sell it next month @ 45 is worth at least $5, a little more since there is time value -- it could go up and make more money, but it can exercise right now and get that $5 profit.

      So the market makers are fucked because they sold all the options at bad prices, had the equity trades they used to offset the risk they are obligated to take removed, and now must scramble to price the options correctly, since the historical data used to price an option, i.e. volatility of the nderlying stock price is now hosed.

      For every angle shooter, there was (at least) one situationally innocent player getting the same reaming.

  8. Maybe Not Fixable, But Preventable by DrunkenTerror · · Score: 3, Insightful

    /. write up:

    There does not seem to be any way to gracefully undo such errors.

    From the article:

    Such losses would have been prevented if the markets had not resumed trading until a decision was made on which trades, if any, should be canceled. But with markets intensely competitive, trading resumed before officials had made their decisions. The losers were traders who were not responsible for the errors or the slow decision making.

    But I guess hindsight is 20-20, right?

  9. Re:Open Sourced by LostCluster · · Score: 2, Insightful

    Even open source software is at risk to GIGO flaws. Garbage data in, garbage data out...

  10. there goes my confidence by TheSHAD0W · · Score: 2, Insightful

    How can I have confidence in a stock market where trades can be suddenly reversed if someone cries, "but it was a mistake!" That's like my buying a car and while driving out of the lot someone jumps in front of me and demands an extra $5,000 because they made a mistake. Sorry, I've got my receipt, the car is mine buddy.

  11. Insurance by squashed · · Score: 2, Insightful

    Graceful way to undo? Of course not. However, it's not to say that a financial market could not operate with these types of glitches anticipated in the system. For example, there could be introduced explicit insurance against the risk of such infrastructure failures. Notably, large players -- typically making the most sweeping "hedging" buys of the kind described here -- would not buy such an insurance product. Instead, they would self-insure, adopting internal practices that would factor in the risk of infrastructure failure, and spread the risk across all its operations. In other words, in the absence of explicit insurance, it's just another example of games that only large institutions should play. In fact, it is often neither advisable nor desirable to make markets "safe" for small players. It's just too expensive, in terms of additional overhead and lost market efficiencies.

  12. Re:Oversimplification by dolo666 · · Score: 2, Insightful

    There is no better ROI than genius.

    You could be like the guys who invented ICQ.

    Or you could create a business, that slowly churns in the coin.

    Or you could make a movie like Blair Witch, using about $5 of pocket change and some cigarettes, a scrap of tent and some gasoline, and make way more money than you'd ever make measuring stocks. Just don't make a sequel. :P

    Stocks are mere gambling, imho. The house always wins...

  13. Its not a glitch by t0ny · · Score: 1, Insightful
    Since when is human error considered a 'glitch'? Somebody entered an incorrect trade, people purchased based on that price, etc. Sounds like the system worked exactly as it was designed.

    Maybe they need to put more checks in the system to prevent human error. But its certainly not a computer glitch.

    --

    Manipulate the moderator system! Mod someone as "overrated" today.

    1. Re:Its not a glitch by Anonymous Coward · · Score: 1, Insightful

      Since when is human error considered a 'glitch'?

      Since middle management. They must cover their fat asses or they might lose their platinum card and conference room time.

    2. Re:Its not a glitch by Wolfrider · · Score: 3, Insightful

      --You have to give them some props:

      > The price plunged, falling from $57.50 at 10:46 a.m. to a low of $39.25 at 10:54 a.m. Mr. Goldman said that Gr8Trade officials noticed the trading and notified Nasdaq of a possible problem. A Nasdaq official, who declined to be quoted by name, said Nasdaq contacted the company and was told there was no news to explain the move. It halted trading at 10:58 a.m.

      --Twelve-minute response time. That's better than I would ever have thought! Could have been a lot worse.

      --
      .
      == WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??
  14. Re:The cancel probably shouldn't have happened by LostCluster · · Score: 5, Insightful

    However, Archipelago was the first to make the decision to resume trading, so most of the people who got burned did so there. NASDAQ then was caught in a no-mans-land of decision making... their investigation hadn't yet returned an explanation, but Archipelago's actions indicated that they had already made a decision that the trades were going to stick. For a trading halt to be effective, there has to be a trading halt everywhere. The markets should have seperate regulatory divisions, but they all should be coming to the same decisons at about the same time. Archipelago clearly didn't do a good investigation here... that's the question that needs further investigation.

  15. None. by Anonymous Coward · · Score: 2, Insightful

    As reported early, the "new" it is a compendium of "experts" each with little to no responsibility for the system as a whole.

    Any those may get fired surely will NOT be the responsible parties.

    Who's responsible? The people that allow sub-standard organizational theories to hold sway. The people who focused on minimizing expenses over insuring appropriate accuracy.

    In short, every manager and technologist in the place, except for those already unemployed because they argued against the policy.

  16. Re:allocation of bad luck by Anonymous Coward · · Score: 1, Insightful

    If you built the orange? I surely would not consider that "bad luck", except perhaps, in that you lost a gamble you played at my risk.

    You built the bad orange, you wilfully cut various corners to your immediate advantage and my ultimate detriment, knowing it might make me sick, even bankrupt me, and...

    you sold it to me anyway.

  17. Re:A right to compensation. by Bob+Gelumph · · Score: 4, Insightful

    You don't seem to understand...
    A better analogy is that you sold the orange for 20c, then after someone was locked into buying it, because they have already sold it to someone else, you tell them that the orange is only available for $1, because the orange you originally offered did not exist.

    --
    I'm gonna need a spec.
  18. Moral of the story... by mabhatter654 · · Score: 4, Insightful

    ...is for buyers and sellers to all SLOW DOWN and pay attention to long term performance rather than minute-by-minute numbers which aren't real meaningful statistics anyway. Frankly anything outside the offical quarterly reports is speculation anyway! Simply allowing only 1 trade per 24 hour period per stock would fix many, many issues with the market right now. The "day traders" should be restricted to playing "numbers" with Magic:TG cards and Ty Beanie Babies....rather than mucking with our financial backbone.

    1. Re:Moral of the story... by skaffen42 · · Score: 2, Insightful

      Nope. That is absolutely the worst thing you could do to a market. The day traders actually REMOVE risk from the market. If it wasn't for them liquidity would drop and the market would become less efficient.

      Have you noticed that spreads keep on getting smaller? Think for a minute about why that happens...

      --
      People couldn't type. We realized: Death would eventually take care of this.
  19. This is capitalism, buddy. by Anonymous Coward · · Score: 2, Insightful

    If that corporation noticed it had accidentally overcharged the customer, there is no way in hell it would notify the customer and say "Here's your money back! Sorry!".

    The corporations go out of their way to fool and trick people into things; there's no fucking way you should let them off the hook when they fuck up in your favor.

    Maybe if we had a more rational economic system where the corporations didn't exist solely to fleece people out of their money, your solution would be allowable, but there's no way in hell anyone should give an american corporation the benefit of the doubt. They wouldn't think twice about raping you up the ass when given the chance.

  20. Re:Hmm.. by PleaseDontBeTaken · · Score: 2, Insightful

    You sound like you know what you are talking about, but it still doesn't make that much sense.

    Was the client long one million shares of the stock, and they went to send 1000 orders to trade out of the stock, all at the same time, rather than 1000 orders for 1000 shares each spaced two minutes apart (would take days).

    Or was the error that the client was only long 1000 shares, and then they autobailed their 1000 shares but the execution for some reason didn't feed back to their inventory, so it keep trying to sell the "same" 1000 shares over and over (and would have until the stock hit zero).

    If it was more the former, that would fit what you explained. If it was more the latter, that wouldn't explain why the sales never flattened their inventory and thereby (mathematically) stopped the autobail process.

    --
    --
  21. Wrong: NASDAQ got it 100% wrong by PleaseDontBeTaken · · Score: 2, Insightful

    Cancelling trades, even real errors, is the worst policy for NASDAQ. Here's why:

    #1 It destroys faith in the markets. A lot of the people who bought stock to provide liquidity had their buys cancelled, but not their sales. Therefore they lost money. This is a rare, but not the first time, this has happened. All those liquidity providers will be a little slower to stabilize the markets in the future because of the risk someone comes and cancels their trades later. Than means when you go to sell your 1000 shares of SCOX when the market is down and SCOX is down more, but you need the money to buy your house, that bid you are counting on might not be there, or it might be a lower price that it would otherwise be. And this happens thousands of times compared to the occasional "real" error, so the cost of this "what if my trade gets cancelled worrying" is very high and very real.

    #2 Negates personal (and corporate) responsibility by the people who caused the problem (which turned out to be a problem for a lot of people, not just them). People should think a bit before they hook up hundreds of millions of dollars to an automated machine, especially one that does things like "sell if I'm down 5%." Then they should think again. And if they handle that much money, we should be able to rely that they are sophisticated entities that should absorb their own errors. If not, someone needs to take their computers away and give them back their Nintendos to play with instead.

    #3 Nasdaq only cancelled orders executed through certain systems under their direct purview, not all trades done during that time regardless of system. Half a solution really is worse than none at all. Pretending the rest of the market doesn't exist (it's not part of us, so it's not our problem) is not a high quality solution.

    If you want serious reliability, it is possible. You have to think, and you have to be willing to pay for it, one way or another, if it's really that important.

    I tried to find a good link to Jay Forrester's Reliability of Components article but the only thing I could find was the IEEE which wanted me to buy it (again--I can't find it). If anyone knows of a valid link, or has the pdf, please respond or email me.

    --
    --
  22. Re:Hmm.. by schon · · Score: 4, Insightful

    the feature which would auto bail-out of a losing position which was in place for client X was discovered and used by client Y, who wasn't even supposed to know about it.

    Ahh, security through obscurity.

    I hope that appropriate action is taken against whoever decided that was the best way to prevent the feature from being used by an unauthorized party.