Slashdot Mirror


Computer Glitch Causes Havoc and Losses on Nasdaq

goombah99 writes "In an illustration of how fragile the electronic stock market system is the NY Times is reporting how a tiny computer glitch rippled through the Stock Markets with buyers who bought low and sold high taking huge losses. An erroneous large sell order was entered. Many people bought at this low price, then signed options contracts to sell these at higher prices, locking in a profit. Or so they thought utill the erroneous low sell order was removed. Now to honor their options they had to buy the stock at a higher price. Since exchanges trust each other's trade prices it rippled throughout the system. There does not seem to be any way to gracefully undo such errors."

22 of 324 comments (clear)

  1. Re:The cancel probably shouldn't have happened by treat · · Score: 4, Informative
    The fact that the trades were cancelled without permission from everybody involved in the trades is quite disturbing (because then it can set up precedence that any of your trades could be cancelled without you knowing about it, and that can really screw up your position).

    That precedent is already there, for example NASDAQ's "clearly erroneous" rule.

    Really this happens all the time and I don't know why this particular incident made /.

  2. Re:Interesting approach from Clancy by KD5YPT · · Score: 3, Informative

    Yes, its the Debt of Honor. But another thing that work in their advantage is that NO record was kept due to the glitch, therfore they could reset the entire stock market and basically declare all tradings done during those days were void.

    --
    In US, you can easily buy enough major firearms to wipe out your neighbourhood but a few little fireworks are banned.
  3. Re:Hyper-transactional databases? by solman · · Score: 2, Informative

    The problem is that there are many trading systems.

    You can't use two phase commit because it doesn't scale in the real world.

    This means that all trades would have to be conducted on a single system.

  4. Re:Bound to happen. by Preston+Pfarner · · Score: 2, Informative

    >> before the stock market crash that led to the great depression
    > I believe the mogul was Warren Buffett.

    Wow, I knew Warren Buffet was getting on in years, but it's quite surprising to learn that he was already a mogul 74 years ago!

    (actually, he was born in 1930, the year after the crash)

  5. Back when I was writing trading software... by ptomblin · · Score: 4, Informative

    It was unusual to see the spread between buy and sell markets be more than a few cents. And with the software that let you see the position on NASDAQ and all the other order books simultaneously, that spread was getting even smaller.

    So I find it puzzling that traders wouldn't realize something was amiss with a $20 spread on a stock. I'm sure they did realize it was amiss, and there was a strong possibility that NASDAQ would break the trade, but they figured they'd go ahead with the trade just in case they could make some money before it was broken. It was, they lost money, and now they're crying.

    BTW: Somebody asked what NASDAQ's software runs on. Mostly they use Suns, although there are some Windows systems, and possibly some SGIs.

    --
    The next Cmdr Taco duplicate will be ready soon, but subscribers can beat the rush and see it early!
  6. Re:Hyper-transactional databases? by Fnkmaster · · Score: 4, Informative
    Well, the first problem is that all trades are pretty much temporally dependent for a given instrument. So you basically have to back out all the bad trades made after a point in time. Which is essentially what was done in this case - the trades *were* all cancelled. Keeping a real transaction open would be prohibitive and silly, since you want to design a system where these kinds of fuckups are very rare and manual.


    Unfortunately, people don't seem to understand the real problem here. The problem is that people make offsetting trades in other markets, that are built on other systems, to lock in profits in the primary market. This story was about traders who sold options contracts to lock in profits on the stock itself. The trades on the stocks were busted by NASDAQ, but the options trades can't be backed out of, they are in a separate market. Thus the trader gets fucked. Having a transactional rollback capability on the NASDAQ wouldn't help here, it would have to encompass all the other markets people might trade in.


    Mind you, I would think there would be legal recourse here based on contract law. The buyer entered into an option sale contract with reasonable reliance on the NASDAQ's "promise" that they bought the stock at a low price. Promissory estoppel against the NASDAQ, or against Archipelago? I don't know, sounds to me like an interexchange issue that needs legal or regulatory collaboration more than it needs a technical solution.

  7. Re:Trading has its risks by kayen_telva · · Score: 2, Informative

    poppycock to your nonsense !

    you think they didnt think of this ??

    Disclaimer
    Because of the possibility of human and mechanical error as well as other factors, NASD and its affiliates are not responsible for any errors in or omissions from the information contained in or accessed through this Site. All such information is provided "as is" without warranty of any kind. NASD and its affiliates make no representations and disclaim all express, implied and statutory warranties of any kind to the user and/or any third party, including any warranties of accuracy, timeliness, completeness, merchantability and fitness for any particular purpose. Unless due to willful tortious misconduct or gross negligence, NASD and its affiliates shall have no tort, contract or any other liability to user and/or any third party. NASD and its affiliates shall under no circumstance be liable to user and/or any third party for any lost profits or lost opportunity, indirect, special, consequential, incidental or punitive damages whatsoever, even if NASD has been advised of the possibility of such damages. Some U.S. states and foreign countries provide rights in addition to those above or do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages. Therefore, the above limitations may not apply to you, or there may be state provisions which supersede the above. Any clause of this Disclaimer declared invalid by the appropriate authority shall be deemed severable and shall not affect the validity or enforceability of the remainder. The terms of this Disclaimer are governed by the laws of the State of New York and may only be amended in a writing signed by NASD.

  8. Re:It was Joe Kennedy by WryCoder · · Score: 2, Informative

    He got a tip from a shoeshine boy and figured that the last players were in the game and the market could only go down. He got out and preserved his fortune through the crash of 29.

  9. Re:Its not a glitch by carld · · Score: 3, Informative


    accordinging to the article it's not entirely clear what happened.

    .
    .
    .
    "There was some sort of system glitch," said Andrew Goldman, executive vice president of Instinet Group. "We are trying to figure out precisely what it was and who caused it. It appears that the result was an unintended effect on the stock in question."

    Other market officials said that the sell order apparently went into an electronic loop, endlessly repeating. Then automatic systems sprayed those orders throughout the market.
    .
    .
    .

  10. Re:Bound to happen. by bagsc · · Score: 4, Informative

    The big companies that go public hope that an infusion of cash will make them more profitable, but it usually ends up that they get to take a break on stockholder's money for a while until it's deadline time again and they have to scramble to make product/service X work.

    I've seen many types of correct critiques of the system, but this is just wrong. When a company receives money by issuing equity, they give up (through dilution of voting rights) partial control of the company. Management only authorizes stock issues when they expect to make money faster with the increased capital than their original equity could.
    An IPO or follow-on offering brings in money, but it doesn't make the issuer "rich." If those accelerated earnings don't materialize, the company will be worth just as much as it was worth before, only now the original owners have a smaller stake.

    Look at all the ads for investing these days. They all suggest that you trust them to make you money, and they have as selling points, how easy it is to make money.

    I challenge you to produce evidence of this. This is strictly illegal. Read up on securities law my friend, and you will notice that the regulations on investment advertising is pretty severe. If you promise someone to make them money and can't deliver, you are in violation of the law.
    The Securities Exchange Act of 1934 set out the general provisions, and the National Association of Securities Dealers (NASD) Advertising Rules have strict guidlines on what constitutes violations regarding investment advertising.

    And even if somehow this ad got past the NASD, it wouldn't get past the SEC. If you learn anything about investing, learn that SEC Rule 10b-5 is your friend.

    --
    http://www.accountkiller.com/removal-requested
  11. A right to compensation. by Anonymous Coward · · Score: 1, Informative

    Programmer, or not, the likes of NASDAQ are privite entities and should be 100% accountable for their actions and every consequence.

    If the "error" can't be undone, however it was allowed to happen, then NASDAQ needs to make it right, everywhere, or be terminated immediately as a uniquely harmful and unmigigated risk to the safety and welfare of the nation.

    With a couple honest actions like this, and we might find a few people actually giveing a damn about who they hire, proper practices, and making sure the wares they spew onto the rest of us aren't just so much "who cares" nonsense driven more by minimal expense outlays than a notion of delivering a real product.

    I have to pay for my mistakes, even to the point I end up in a box under the bridge. So should they.

  12. Re:Warning: Instant big profits never happen... by Herkum01 · · Score: 2, Informative

    Do you know what the market is going to do at any given month, how about any given week? Day? Hour? Minute? The market is set up under the principle that transactions are executed immediately and without prejudice. If the stock price goes down $2, who is to say that is wrong? You? Stock prices can go down drastically in a few hours, does anybody sit there deciding that they know the reason why it went down? Hell no, someone puts in a order to buy ABC at 40, and that is it. If the someone sells at 40 the transaction is completed. No moralizing.

    The point is, the stock market is setup to not worry about the specific reasons why something occurs, it is only to do transactions between buyer and seller. If the software fucks up, well the company who was using the software is responsible and can get sued for it.

    Saying that the buyer beware is just an excuse for unreliable software. You use the software you are responsible, in much the same way if you drive a car and you hit a person you are responsible. Does not matter if it was intentional or accidental.

  13. Re:Trading has its risks by treat · · Score: 4, Informative
    I believe nasdaq has a set of rules for those who trade on it, and one of those rules I'm sure has to do with obvious mistakes.

    This is NASDAQ's "clearly erroneous" rule. I'm sure a Google search will turn up plenty. All serious players are aware of this rule.

    When you as an individual do trades, you do them with a brokerage, not with nasdaq, no matter how transparent it looks, and liability in that case depends on your contract with your broker.

    It's not their fault, and the contract should cover this.

  14. Re:The cancel probably shouldn't have happened by Anonymous Coward · · Score: 1, Informative

    i'm a futures broker and this scenerio is common. traders sign their life away to trade electronicly.

    if you specificly try to speculate on an eroneous price, then you get what you ask for. if your buy gets busted after you've liquidated and now your short ... you learn quickly.

    the real risk and problem is this scenerio:

    You are long a market with a STOP order below the market. An bad tick triggers your stop executing your electronic stop order. Later, as they should, they come back and bust all the fills triggered by the bad tick. The trouble is your electronic trading platform does not replace the stop order. You will only find out that your order is not working when you figure it out by reading your orders page or your broker calls you.

    So your screwed if you aren't glued to your screen or if you miss your brokers call.

  15. This is not the first time this has happened by vandelais · · Score: 4, Informative

    On Oct 2 2002, someone at a brokerage firm Bear Stearns entered a 4 million dollar trade as a 4 billion dollar trade and it wasn't doublechecked and caused most market indices do go down about a half percent DURING NORMAL TRADING HOURS during the last hour of trading.

    This was widely reported in the financial press, and eventually the sell position was unwound.

    Since the order was a sell order tied to a diversified holding, it caused this decline to happen with both the electronic Nasdaq exchange and also the auction-based NYSE.

    "In October of last year, for example, a trader at Bear Stearns mistakenly entered an order to sell $4 billion in stocks instead of $4 million. And two years ago London's stock market collapsed after one hapless trader entered an extra zero into a sell order."

    See

    http://stacks.msnbc.com/news/945909.asp?0sl=-21& cp 1=1

    and

    http://news.bbc.co.uk/1/hi/business/2294525.stm

    for more details

    Previous errors

    Mistakes have been made in market trading before by other companies.

    In May last year, London's FTSE 100 index dropped by more than 2%, after a trader typed 300m, instead of 30m, while selling a parcel of shares.

    In 1998 a Salomon Brothers trader mistakenly sold 850m-worth of French government bonds by LEANING ON HIS KEYBOARD.

    And at the end of 2001, shares in Exodus, a bankrupt internet firm, jumped by 59,000% when a trader accidentally bid $100 for its shares, at a time when its value was 17 cents.

    --
    Game: Player 'Donald J Trump' now has AI skill level 'experimental'.
  16. Re:Hmm.. by Anonymous Coward · · Score: 5, Informative
    What if some smart but malicious programmer rigged the system for his own profit? I know this example here is a glitch, but perhaps in the future...

    I work for the company in question, and I assure you this is not the case. What basically happened is that the feature which would auto bail-out of a losing position which was in place for client X was discovered and used by client Y, who wasn't even supposed to know about it. Client Y added the setting to support autobail, but didn't include a lag time to send the orders.

    Thus, the order was sent thousands of times before the error was noticed.

    For it to have been malicious, the programmer would have had to have contact with the client in question, which is not the case. It was a multi-layered mistake, plain and simple.

  17. chronology of events by system_trader · · Score: 2, Informative

    According to the posts on elitetrader.com:

    "A guy in my office said it was a trader at Bear. He was supposed to send a market order for 9,000....accidentally sent it for 9,000,000."

    Apparently an erroneous sell order (offer) was placed for 9M shares at $42. About 2M shares executed before the remainder of the order was canceled.

    Read the real-time reaction of traders here:

    http://www.elitetrader.com/vb/showthread.php?s=&th readid=25431

    Everyone realizes the offer is out of line. Some declare "free money!" and purchase as many as they can, then sell at $47, $52, $55, etc. None seem to realize that trades will be busted (canceled) until it's officially announced.

  18. Add hysteresis to tackle glitches + bugs by Morgaine · · Score: 4, Informative

    Anyone that's done some control theory knows how to solve the problem --- just add some hysteresis into the feedback loop, ie. response delay in both directions.

    All forms of instability are reduced in their effect by this means, so it doesn't matter whether the instability stems from human error, bugs, or system glitches arising from other things.

    And exactly how would one do this? There's a ton of ways, and quite a few of them simply entail holding quoted prices steady for a mandated period, plus a few adornments.

    There are much more creative ones around though which could probably work even better, like allowing only audio readouts in trading rooms so that info comes in slowly like in tickertape days, or the one I like best, allowing traders to use no equipment other than the morning's financial newspaper, plus a pen and notepad. :-)

    --
    "The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
  19. As a trader I feel I can comment by Lawrence_Bird · · Score: 3, Informative

    1) Anybody who bought at those low prices knew something was
    either a) very wrong or b) very bad news was out. This was
    after all a $60 stock and they were now buying at $42.
    Without going into detail, I find it impossible they lost
    any money on options trades (no open exchange, prices were
    much higher when they did reopen).

    2) Similar things have happend at CBOT and CME on their
    electronic stock index products a few times this past year.
    Most recently, a trader was said to have entered a stop/loss
    order in the less liquid Dow Jones futures for a very large
    amount. As CBOT handles S/L orders natively (internally),
    their system proceded to hit bids until the order was
    filled. Those trading S&P futuers did notice what was going
    on and started selling there as well, perhaps fearing a
    terrorist or other news related item. The sell off was not
    as far (500 Dow pts), in part because of the greater market
    depth, and the fact nobody could figure out what was going
    on. Trades were subsequently broken, hours later, on the
    CBOT, but NONE were cancelled on the CME. This caused quite
    a bit of pain for people who had s/l orders open which were
    executed, for all intents erroneously. (no not me, but I do
    know one who was). CME refused to cancel as the original
    event did not occur in house.

    To this day, I do not believe the order was entered in
    error. I believe a hedge fund or other firm had a need to
    buy, and buy large. What better way to get filled then to
    trigger a large move in a closely correlated market (Dow
    futures) where you know the majority of the trades will be
    cancelled, and just sit on the bid in the other market
    (CME) as people panic sell. And do not rule out collusion.

  20. Re:The cancel probably shouldn't have happened by Gunzour · · Score: 3, Informative

    NASDAQ's own press release gives exact times for everything, except for what time they decided to cancel the erroneous trades. There are reports in the Yahoo message board for COCO of people who bought shares during the time period in question unable to get an answer from their brokers as to whether or not there buy was valid, even after trading resumed.

    To me, Archipelago seems to be the big problem here. They willfully resumed trading of the security even while NASDAQ continued the halt, and complained that Nasdaq "used this authority to attempt to impose on its competitors a trading halt in a situation that was unique to it."

    It would seem to me that if a stock is halted on one market, it should be halted on all markets, and I think it was irresponsible for Arch to resume trading. Clearly there was enormous confusion over this stock during the time frame in question, and the purpose of a halt, in my opinion at least, is to allow information to get out to eliminate the confusion in order to ensure a fair market. Arch's action of unilaterally resuming trading resulted in increased confusion and an unfair market situation.

  21. So, what's the problem? by Orne · · Score: 2, Informative

    I caught the various news reports on CNBC on friday evening... apparently NASDAQ is only allowed to halt a stock when they are under investigation for regulation violations. When NASDAQ froze it, they also asked the west coast & overseas markets to halt trading on the ticker, but there was no precedent for this; NASDAQ just broke procedure. The interviewees seemed really pissed.

    From the article, it appears that the software that communicated market orders went into a loop, and submitted a loop of Sell orders on this one stock. If it were just little old me selling stocks I don't own, it's called a Short, and I'd be liable for buying back any shares I don't own. If its really a computer error, then its up to the market providers to cancel the orders.

    At some point, the SEC needs to find out who was liable this this little adventure. Why does NASDAQ allow companies to submit raw sell commands w/out proofing them for validity? And what about this software company? If it were me playing with Ameritrade, and their software repeats my order 100 times, shouldn't the software company be liable for all of the (unsolicited) trades? If these trades should have been cancelled, why did some markets resume trading before they validated the orders? I wouldn't be surprised if there's another round of market rules that fall out of this, because obviously there's a big loophole here.

  22. Re:Bullshit by sigwinch · · Score: 2, Informative
    When you buy stock, and you get confirmation that your trade has successfully executed, it's a done deal.
    No, it isn't. Execution of a trade means merely that matching buy and sell orders were recorded by the exchange. There is no verification of account balances: you are putting blind faith in an unseen seller. The actual payment is made later, during settlement, which is where account balances are checked and adjusted. In the U.S., settlement occurs three business days after execution. In foreign markets and international transactions, settlement may take weeks or even months.

    When you use the result of an order execution before its settlement, you are borrowing from your broker. The SEC allows this because the vast majority of settlements are completed as expected, so it doesn't destabilize the market, and it keeps money from being tied up. But even this generosity has limits: if you buy a stock, sell it the same day, then buy it again the same day**, the SEC considers you to be engaged in "pattern day trading" and makes you capitalize your account just like you were an option trader.

    **Subject to some exceptions that don't apply to the average individual investor.

    The company that created the bogus sell orders should be accountable for actually making those sales, even if it means they now have to buy up stock at a higher price.
    No sale occurred. They should be held accountable for breaching the settlement contract. Exactly what that means depends on the fine details of the contracts and of contract law, of which I have little knowledge.
    You WILL base your future behavior on the premise that you now hold that stock.
    All transactions for unseen goods entail a degree of risk and blind faith. The wise trader learns the laws governing such trades, learns how common and likely the various failure modes are, and acts accordingly. I refer you to the Bloomberg Financial Glossary definition of settlement risk: "The risk that one party will deliver and the counterparty will not be able to pay and vice versa." Competent traders are well aware of this risk.

    To put it more bluntly, when you buy a bill of goods up the river, you owe yourself to verify the situation before you sell to somebody who likes breaking kneecaps.

    This kind of behavior will undermine the stock market (and at a time when there's already plenty of reason for potential traders to be wary.)
    Even the most honest and careful make mistakes. If every mistake entailed unlimited risk as a matter of policy, only the kings of finance and complete idiots would trade. That would not be an improvement.
    --

    --
    Kuro5hin.org: where the good times never end. ;-)