Slashdot Mirror


Computer Glitch Causes Havoc and Losses on Nasdaq

goombah99 writes "In an illustration of how fragile the electronic stock market system is the NY Times is reporting how a tiny computer glitch rippled through the Stock Markets with buyers who bought low and sold high taking huge losses. An erroneous large sell order was entered. Many people bought at this low price, then signed options contracts to sell these at higher prices, locking in a profit. Or so they thought utill the erroneous low sell order was removed. Now to honor their options they had to buy the stock at a higher price. Since exchanges trust each other's trade prices it rippled throughout the system. There does not seem to be any way to gracefully undo such errors."

27 of 324 comments (clear)

  1. Hmm.. by Wigfield · · Score: 5, Interesting

    What if some smart but malicious programmer rigged the system for his own profit? I know this example here is a glitch, but perhaps in the future...

  2. Interesting approach from Clancy by Chairboy · · Score: 5, Interesting

    In a Tom Clancy book (Debt of Honor, I think), a computer programmer takes advantage of a weakness in the stock market to induce a crash. After a week of the market shut down, they recover by resetting the prices to where they were the day before the glitch and instructing stock brokers on steps to avoid re-creating the crash.

    It doesn't apply to this situation, but the specifics of how they do it is interesting for anyone who might want to check out the book.

  3. 1987 by all+your+mwbassguy+a · · Score: 1, Interesting

    just think what would have happenned if it took any longer for that order to be removed.

  4. The cancel probably shouldn't have happened by 31415926535897 · · Score: 5, Interesting

    I work for a firm that writes software for options traders and clearing firms. Sometimes system glitches do happen (or more often than not, a user error, like entering in the wrong price). However, when this happens and a trade occurs, it sticks unless both parties agree to bust the trade.

    The fact that the trades were cancelled without permission from everybody involved in the trades is quite disturbing (because then it can set up precedence that any of your trades could be cancelled without you knowing about it, and that can really screw up your position).

    Some people lose money because of mistakes, and some people make money because of mistakes...that's part of how the market works, and you should be willing to accept that risk if you're going to trade.

    If it really was that bad (and a $20 difference is huge), and archipellago did screw up, they should take responsiblity and take the losses. If someone just entered in the wrong ask price, then that firm should take the responsibility. I know if our systems screw up our traders, then we mitigate those losses.

    I have a feeling there might be some lawsuits in the near future if there were a lot of shares traded.

    1. Re:The cancel probably shouldn't have happened by Gunzour · · Score: 4, Interesting

      It does not happen all the time. I believe this is the first time NASDAQ has ever cancelled trades *after* allowing a stock to resume trading. It is fundamentally unfair to allow a stock to be trading and then after the fact announce that some of those trades will be cancelled. That is why NASDAQ has the ability to halt a stock, and it should not have been resumed until after all decisions about cancelling trades were made and published.

  5. Re:Virus by Anonymous Coward · · Score: 2, Interesting

    They (mostly) run OpenVMS. Good luck with the virus.

    Anonymous Ex-Deccie coward

  6. Re:Hyper-transactional databases? by cduffy · · Score: 2, Interesting

    Even that is excessive. Tag each table entry with a timestamp, and never issue deletes or updates -- just a new insert which either declares an old value to be considered deleted or changed. Want to roll back to $GIVEN_DATE? Just delete everything added since that time.

    That said, this wouldn't be practical in this case, just because the amount of rearchitecting that would be needed to implement it.

  7. Only about the yelling part, I think by Pac · · Score: 2, Interesting

    given enough damage, it is not impossible for Nasdaq to consider voiding every deal since the glitch started...

  8. False start by Archipelago? by LostCluster · · Score: 5, Interesting

    The system actually seemed to have worked pretty well except for the actions of th Archipelago market. There's no way to prevent errant data from making its way to the financial markets, so the question is what are you gonna do about it once it gets there?

    What's supposed to happen is that everyone is supposed to stop trade in the stock while market officials try to sort out what happened. The NASDAQ did just that, and called the company involved to see if they had any news that would have justified the drop and they responded that there was no news. NASDAQ announced that their initial review indicated that there was errant trading going on, reserved the right to cancel the trades made before the halt, and released the stop. Within the hour, they confirmed the source of the problem, and revesed the errant trades.

    Yet, while trading was still halted on NASDAQ, Archipelago undid their halt without any announcement that anything was wrong. This is wrong on two levels... not only did it falsely convince other people that the drop was for real, but it also pressured NASDAQ's decision-makers to hurry up, otherwise NASDAQ would lose trading volume to Archipelago.

    So, the blame for this mess really belongs at Archipelago... they seem to have done an investigation that resulted in a verdict of no error, where in 20/20 hindsight we know there was an error on the play. Did Archipelago conduct a flawed investigation, or did they conduct any investigation at all? This was a case of the market's self-policing rules falling apart rather than any computer program...

  9. Re:Trading has its risks by LostCluster · · Score: 4, Interesting

    Actually, Archipelago's the one to blame if you're gonna blame an exchange. Archipelago released the hold on the stock first, so most of the people who thought they were making an instant-profit by buying when it was low and selling minutes later turned out to be the instant losers. Archipelago's actions seemed to indicate the morning trades were going to stick. When the NASDAQ released their hold, they did so with a warning that anybody who had bought low in the morning should stand by because it was likely their trades were gonna be undone, and within the hour the NASDAQ followed through with the cancelations.

    So, sorry money hungry lawyers... you'll just have to settle for suing a .com-like stock market out of existance, NASDAQ's hands are clean...

  10. Re:Not always possible by zeno_2 · · Score: 2, Interesting

    One of my old friends I used to work with went into staples, they had a coupon or somethin where one of the Sony Digital Mavica camera's was like 250 bucks. The coupon said "limit 2", but for some reason the clerk thought this meant, you get 2 for 250 bucks. My friend had paid with a credit card, and walked out with 2 cameras for 250 bucks. A day or so later he got a call from the Staples manager, the manager wanted him to come in and take one of the camera's back. Of course my friend didn't do it, but it was somewhat humerous that the manager called him up and asked him to bring one back..

  11. Re:Easy Fix by Gunzour · · Score: 5, Interesting

    Nobody knew it was a bug at the time. They simply saw the price dropping dramatically and decided to take a risk and bet on a price rebound. It was only after the halt and after the resume and *after* these people sold what they though they had bought that NASDAQ decided to cancel the orders.

    Daytraders often buy on dips, betting that the stock is being oversold. This is a decent strategy, since often the reaction to bad news is more extreme than the news warrants. So, the stock dips suddenly, then regains a lot of what it lost. This is risky, because sometimes it turns out that news was even worse than initially reported, and the stock goes down even more. Daytraders understand this risk and accept it.

    However, NASDAQ has introduced an entirely unprecedented risk -- that your buy order may be cancelled with no notice after you have already sold it forcing you into a short position that you did not intend.

    Take this scenario:

    10:55 AM - Investor sees huge dip in stock, enters BUY order for 1000 shares

    10:56 AM - Investor gets confirmation from broker that he bought 1000 shares at $40, total price $40,000 (plus fees)

    10:58 AM - Stock is halted

    11:19 AM - Stock resumes trading, price starts going back up

    11:55 AM - Investor puts in SELL order for 1000 shares

    11:56 AM - Investor gets confirmation from broker that he sold his 1000 shares at $50, paying $50,000 (minus fees). That's a profit of $10,000.

    12:28 PM - NASDAQ announces cancellation of all trades between 10:46 and 10:58 AM.

    12:30 PM - Broker adjusts Investor's account to remove cancelled BUY order from 10:55 AM. But the SELL order was not in the cancelled time frame. Investor now has -1000 shares of stock and must buy to cover the debt.

    12:35 PM - Investor enters BUY order to cover the 11:30 SELL.

    12:36 PM - Investor gets confirmation from broker of BUY at $55 per share, total cost $55,000. Since the shares were sold at $50/share, that's a loss of $5000, due to NASDAQ's cancelling after the fact.

    If NASDAQ had announced it was cancelling transactions before resuming the stock, the investor would not have entered the SELL order in the first place, and the whole thing would have been a wash. That would be the fair way to handle it.

  12. Re:Bound to happen. by Galvatron · · Score: 4, Interesting
    As someone else pointed out, this may be an urban legend; the version I'd heard of this involved shoeshine boys. The lesson to take away, however, is not "the whole system is wrong." The lesson is that all available sources of cash had been exhausted. The stock market will only rise so long as there are more buyers than sellers. The only way that happens is if there are people with cash that has not yet been invested in the market. During runaway bull markets, everyone wants in, so people sell bonds, cash out their savings accounts, etc. and dump it all in the stock market. Eventually, these sources of cash run dry, and the market crashes. The point of the original story is that if busboys are putting money in the market, then we're near the limit, there are no more new sources of cash for the market, and it's time for a crash.

    The same thing happened in the 90's. I read an article about how states which, during the great depression, had passed laws forbidding governmental organizations from putting money in the market were now repealing those prohibitions. Well, if Depression-era legal prohibitions were being repealed, then the market was due for a crash. Unfortuately, my prediction was a good two years eary, but oh well.

    --
    "The question of whether a computer can think is no more interesting than that of whether a submarine can swim" -EWD
  13. Australian Stock Exchange by Anonymous Coward · · Score: 1, Interesting
    Another example of how computers and stock exchanges don't always mix well: The ASX recently allowed a company to change its ticker code to 'AUX'.

    Now, try saving a file as AUX.txt (or anything) in Windows see what happens. (It's right up there with LPT, PRN, NUL, COM, etc..)

    Unfortunately many market analysis programs try to do just that when they store data! Kaboom!

  14. Re:Bound to happen. by cubicledrone · · Score: 4, Interesting

    There is a science to investment. You just have to know how to read numbers, know how a corporation works, understand how trading works and understand what you are buying.

    Stocks were never meant to be traded as speculation on the share price. Purchasing stock is purchasing a share of a company's future earnings in the form of dividends. This is why people who invest long term make more than people who buy expecting to sell with a profit in the same week.

    There are several stocks paying 5-10% real returns as dividends right now. Try getting that rate at the bank. The best CD right now might pay 2%, if that.

    --
    Business isn't willing to pay for products, innovation and careers, so we get brands, mortgage commercials and layoffs.
  15. Re:Trading has its risks by mkldev · · Score: 3, Interesting
    Rescinding sales after the fact may qualify as willful tortious misconduct. The requirement is that it must have been a direct action taken by NASDAQ (rescinding the stocks, in this case) with intent to harm a business (the day traders) rather than to promote its own purposes.

    If it can be shown that NASDAQ's systems caused this, rather than a glitch in someone else's systems, then their action was taken as an act of self-preservation, and would not be tortious. If it was caused by a problem on someone else's system, though, it probably would be.

    If, however, the failure was caused by NASDAQ's systems, the fact that safeguards were not put in place to detect such a runaway sell order qualifies as gross negligence on their part.

    Either way, if that were the contractual agreement between the day traders and NASDAQ, it seems likely that NASDAQ is screwed. However, the text you quote above is clearly web site boilerplate information, and has nothing to do with the contractual obligations NASDAQ has regarding an actual sale of stock. Without those contracts and a solid knowledge of contract law in the state of New York, we're all basically guessing here.

    That having been said, the odds do seem to lean in favor of NASDAQ being found to be negligent, and thus at fault, IMHO.

    --
    120 character sigs suck. Make it 250.
  16. Re:Hyper-transactional databases? by Unordained · · Score: 2, Interesting

    a) rdbms theory does not preclude nested transactions. in fact, CJ Date & H Darwen seem to rather think it should be part of the the requirement for a rdbms. current products don't all support nested transactions, but some do.

    b) you don't want this anyway. a trade is a contract, and like contracts usually are, undoable only if both parties agree to do so. in the case of trades like these, with a domino effect, the top-level traders who want to undo their trade (or at least one side does) need to ask the permission of everyone affected. that could be thousands, millions of people only a few minutes later. (especially in a case like this, where people pounce on a good deal.)

    not all decisions made based on the stock market stay in the stock market. while you would possibly undo all trades 'based' on the bad trade, you'd miss all the indirect dependencies within the stock market, as well as the business decisions made outside the stock market: deciding you can afford to buy something you wouldn't have otherwise, etc.

    i would personally ask for absolutely -no- ability to undo transactions of this sort automatically. no trades should be undone, regardless of how 'wrong' they seem, unless everyone involved agrees. if you sold lower than you meant to, and someone buys ... they're not likely to agree. and that's just too bad.

    if a computer glitch resulted in an incorrectly posted price, then the computer systems and their maintainers should be responsible, but the trades should continue without interruption. they provide a service, they failed to provide it correctly, and no remedy would be complete enough if done strictly by modification of the stock market. it's a case where they can't remedy their mistake by undoing it -- only by reimbursing the harmed parties.

    however, the -value- of the mistake is complicated. it's a time-based systems, with humans making decisions. can they prove what the price should have been, or how many people would have traded at that price? it's difficult to prove how much loss was incurred. that's a very judge/court-oriented thing to decide.

  17. Re:Trading has its risks by Gorobei · · Score: 2, Interesting
    How the hell can Archipelago be to blame? People wanted to buy and sell at specific prices - Archipelago enabled them to do it. If traders ignored the risk of exchange actions, that's their problem.


    Exchange/clearing interventions of various types have occured many times in the past (for example, the redefinition of futures contracts during the Hunt brother's Silver corner, the post-WWI coffee futures reset, the post-9/11 T+5 swap settlement change.)


    NASDAQ was the party making the exceptional change, not Archipelago.

  18. The issue is settlement. by Slashamatic · · Score: 4, Interesting
    The problem of trading back from a point in the past (as in more than one business day previously) is that you have to reverse settlements that have happened. Shares tend to stay on the depository system so they are easier to deal with (although this would need some hacking at the registrar as well), but cash gets transferred out of the control of market participants. For the shares, well generally, you are just a beneficiary name on a computer somewhere and the shares exist just as a global 'certificate' with the nominee set to the depositary account holder. Cash gets moved, generally very quickly and also internationally. For example, I sell GM for dollars at the NYSE, switch the dollars to Eoros and then use the proceeds to buy VW in Germany on Xetra. Two distinct markets, and a forex transaction.

    Clancy was a bit simplistic there - it would be a hell of a rollback.

  19. Re:Hyper-transactional databases? by mabhatter654 · · Score: 2, Interesting
    The only other issue is the "second-order" effects... i.e. I saw this "deal" so I sold another stock at an sub-optimal position to get in on the "deal" now that stock has changed positon and I want it back...

    Frankly, they should heavily penilize the errant broker...perhaps 1 month inelligibility to trade...and make the "day traders" live with their choices. Day Trading is a questionable, but legal practice anyway...like french fries & soda pop, too much will wreck the market...perhaps a few incidents like this will self-curb the trend or kick out a few sloppy brokers!

  20. Re:Trading has its risks by Gorobei · · Score: 2, Interesting
    You should prepare for that risk the day you open your brokerage account, buy a seat on the exchange, etc. You have no more reason to expect a trade to be immediately good than you have a right to mow down slow pedestrians if you have a green light: a big set of (fairly common-sense) rules apply.


    Wall Street rewards non-diversifiable risk taking, and one of those risks is that trades will be disputed. Your average investor has many ways to mitigate that risk (dollar cost averaging, investing in mutual funds, using a broker.) Don't cry for them that took some damage in this case: it's a risk they knew (or should have known) they were running. You don't see them volunteering to return previous profits on arbitrage trades that went off well, do you?

  21. lazyness/stupidity? by Anonymous Coward · · Score: 2, Interesting

    I wrote the front end to an options management system for a commodities market a few years ago. market managers could drag the volatility graph as they saw fit (with the real trades superimposed).

    it was a quiet market mainly because only about 3 people really knew how to deal these things so most traders venturing in there would be fleeced.

    anyhow, as the terminal would reside on the trading floor i thought some security would be in order. the user had to log in to the system (validated against the network credentials) then after 2 mins of nothing happing, it would log out.

    the annoyed the trading floor staff and i was instructed to remove it, despite vigerous protesting (or should i say, a stream of explitives from me). lo and behold within 2 weeks there was an investigation - in a quiet period a trader had gone in and rigged the end of day prices (to make his book look better).

    this taught the management a lesson. this also taught me some things

    #1 dont underestimate human lazyness
    #2 dont understimate the stupidity of managment that is 'customer focused'

    now i look at this it has bugger all to do with the topic, but really ive wanted to get this off my chest for like 7 years :)

  22. Re:Bound to happen. by system_trader · · Score: 2, Interesting

    Stocks were never meant to be traded as speculation on the share price.

    This is completely false. Investors would never purchase stock without the ability to sell it.

    Purchasing stock is purchasing a share of a company's future earnings in the form of dividends.

    Future earnings and dividends always have some uncertainty. Traders and investors trade or hold stock positions based on speculation on the future of earnings/dividends.

    This is why people who invest long term make more than people who buy expecting to sell with a profit in the same week.

    This is completely false. It is possible to be successful with both long term (investing style) and short term (trading style) speculation.

    The key to low risk returns is to have a reliable, statistically valid method for speculation. Some find success with a long term fundamentals-based approach, others with exotic neural networks, or other higher frequency analysis. Some can achieve profitability with hundreds of trades per day. It is wrong to assume that any one time frame or bias (eg. long positions only) is inherently more successful.

    There are several stocks paying 5-10% real returns as dividends right now.

    That's because the stock price has dropped relative to the dividend, while the dividend has remained constant. What most don't realize is that these stock prices AND their dividends are at more risk than 5 to 10%. What good is a 10% annual dividend if the stock drops 25% in the first year?

    Try getting that rate at the bank.

    Banks offer a very low risk return.

    In general:

    higher returns == higher risk
    lower returns == lower risk

  23. Re:Easy Fix by LostCluster · · Score: 2, Interesting

    Here's the problem with your timeline. At 11:19 AM, trading didn't restart at NASDAQ. It restarted at Archipelago. When trading re-started as NASDAQ, there was a simultanious warning that it was likely the pre-stoppage trades would be reversed.

    Sure, there was out of line behavior, but it happened at Archipelago... who is a completely different operation. The moral of the story is to trust NASDAQ for a fairly played market, and beware of Archipelago opening trading too soon on stocks that should remain halted.

  24. Re:Poor slobs. by LostCluster · · Score: 2, Interesting

    Maybe a whole company. Archipelago now has a lot of explaining to do about their regulatory practices. The stock market system depends on self-regulation by the exchanges, and this newcomer got caught making a very questionable early decloration that things were all clear, while NASDAQ took longer but got made the right call. Archipelago then got reduced to copying off of NASDAQ's homework, and leaving some of its own customers holding the bag.

  25. Re:Bound to happen. by the+eric+conspiracy · · Score: 2, Interesting

    From http://www.axaonline.com/rs/3p/sp/5015.html:

    The stock portfolio did not suffer a loss in any of the 685 separate 20-year holding periods. In every period, the annual rate of return for the stock portfolio was greater than the inflation rate. At the same time, the bond portfolio outpaced inflation in only 317 of the 685 20-year holding periods -- by a much lower margin."

    So in other words, you are wrong - in more than 1/2 of the baseline 20 year periods bonds DID NOT outpace inflation.

    Real Estate is even worse - it is illiquid to the extreme and subject to severe price fluctuations as a result of interest rate variations. The real estate bubble of the 90's pretty much put an end to the concept of real estate as a reasonable approach for personal investing. The upcoming retirement of the baby boomers along with sales of homes implicit in this will place a severe secular downward pressure on real estate.

  26. Re:Its not a glitch by Sivaram_Velauthapill · · Score: 2, Interesting

    Financial systems are the heart of capitalism. Therefore, they are extremely efficient. There are a ton of resources that go into it, and there is a ton of money that they make. Even in poor countries, the financial systems are very good.

    Sivaram Velauthapillai

    --
    Sivaram Velauthapillai
    Seeking the meaning of life... @slashdot of all places ;)