LSE Breaks World Record In Trade Speed With Linux

LingNoi writes with this excerpt from ComputerWorld UK: "The London Stock Exchange has said its new Linux-based system is delivering world record networking speed, with 126 microsecond trading times. The news comes ahead a major Linux-based switchover in twelve days, during which the open source system will replace Microsoft .Net technology on the group's main stock exchange. The LSE had long been criticised on speed and reliability, grappling with trading speeds of several hundred microseconds. The 126 microsecond speed is 'twice as fast' as its main international competitors, the London Stock Exchange said. BATS Europe and Chi-X, two dedicated electronic rivals to the LSE, are reported to have an average latency of 250 and 175 microseconds respectively. Neither company immediately provided details. But many of the LSE's older and more traditional rivals offer speeds of around 300 to 400 microseconds. Nevertheless, Linux is now standard in many exchanges, including the New York Stock Exchange."

Comments

Best for headshots

[Anonymous Coward]

Windows games: Plentiful and well developed. Mac games: Barely existent. Linux games: Lowest latency.

Re:Best for headshots

[ledow]

Strangely, that would be an enormous selling point to many gamers if it were true. I'm not sure it is, but it would sell, just like the expensive fans, super-duper mice, and other "oxygen-free-cable"-style shit of an "enthusiastic" gamer.

Re:Best for headshots

[kiddygrinder]

plentiful and well developed is a stretch, if linux had a port of the hl2 engine (possible, but not that likely... guessing about 25% chance) and a wow client (less likely but if valve did it i'd guess maybe 5%) it'd probably be an option for 95% of the market... and suddenly people would be pretty interested in latency.

Re:Best for headshots

[rtb61]

With various versions of mobile Linux becoming the dominate smart phone operating systems whether as Android, MeeGo or MobLin and the smart phone by becoming the dominant form of computer, creating applications for that environment becomes commercially important, especially when all those operating systems with a little tweaking can readily power smartbooks.

Re:Best for headshots

[baegucb]

WoW client works fine under wine (tad better than under Windows imho).

Re:Best for headshots

[Auroch]

WoW client works fine under wine (tad better than under Windows imho).

Really? And what is the major improvement/difference, besides running under wine/in linux?

Re:Best for headshots

[TheKidWho]

The expensive fans are more of an overclockers toy than a gamers. Like everything else in life, you get what you pay for. Cheap fans usually have terrible sleeve bearings, are very loud, and the sleeve bearings will die out within a year of usage. Also, the heatsink usually tends to be more expensive than the fan itself and is just as important if not moreso. You may be confusing the two.

As far as expensive mice go, it's an issue of ergonomics and precision. My space ball that I use for 3D CAD modelling cost more than any expensive gaming mice I can think of and that also has to deal with ergonomics and precision.

"Oxygen-free-cable" I have no idea you're referring to.

Re:Best for headshots

[the Haldanian]

WoW client works fine under wine (tad better than under Windows imho).

Really? And what is the major improvement/difference, besides running under wine/in linux?

Swap performance and VM handling, frankly. It's better at not putting things in swap it shouldn't, and quickly getting things off swap it needs. Generally less lumpy if you aren't swimming in memory like I'm not.

I also get the added benefit of instantly switching in and out of the game with virtual desktops, and being able to shut down all unnecessary systems and programs, and SIGSTOP/hibernate large things like firefox so they sink entirely out of memory until I call them back, and I have total scripting and file control over all my addons and logs and things.

Also, crashes won't take out the box, and if things get really gnarly, I log in behind my GUI remotely and shoot things. For some games that are going under due to game corruption, flicking to the CLI, Kill, restart, can get the game back before the server has given up on your connection. WAY faster than Windows.

Re:Best for headshots

[Dancindan84]

You're much less likely to get wife aggro.

Re:Best for headshots

[Stachybotris]

He's referencing Monster cable, such as this product, and making fun of the fact that 'audiophiles' will spend enormous amounts of money on gear that they have no solid way of proving is better.

Re:Best for headshots

[mcgrew]

Like everything else in life, you get what you pay for.

When anyone says that to me I hang on to my wallet for dear life. If you buy those expensive digital cables for stereo, you are NOT getting what you paid for -- you've been scammed. When you pay $100 for a pair of blue jeans, you're also being scammed. When you pay three times for a bottle of Alieve what you would pay for a bottle of generic naproxin sodium, you're getting scammed. In none of these instanced are you getting what you pay for.

You do, however, usually (but not always) pay for what you get. When you buy an overclocking fan, you are paying for what you get provided it really is a superior fan. When you buy Alieve you're paying for what you get, plus some. All you're getting is naproxin sodium, no different than the generic that costs 1/3 as much.

Re:Best for headshots

[ledow]

And if those fans / heatsinks really made economic sense, then every server room in the world (from little offices up to large datacenters) would be using them. If you're paying more than a handful of coins for them, however, you are wasting your money. The fact is that the replacement of a fan/heatsink with a item that isn't supplied by the systems manufacturer makes no sense and doesn't, actually, get you any worthwhile gain for the money you spend. You might save 2 degrees in temperature but for the money you spend on some of the systems I've seen you could do that by just cleaning the fans once a year.

A gain of reduced audio noise? I'll give you that. I myself own a fanless PSU (it's heatpipe-tech, not water-cooling or anything) that runs my server. It's absolutely 100% silent. Sadly my drives and other devices aren't, but I can see that in some ultra-specialist environments you need to make an almost-silent PC. However, putting your baseunit in a cupboard, or at least behind a bit of pretty wood that dampens sounds aimed towards the user is actually more effective and (nowadays) zero inconvenience (run your USB hub to your desk-top, done).

I use a silent PSU because it has no moving parts (however, it channels an awful lot of heat into the surrounding metal case which probably puts my motherboard temperature up by a degree or two, so I've not really "gained" anything there in terms of product life). It cost me £10 more than a standard PSU of the same wattage, back in the day, so it was an easy decision to make. But, to be honest, it's sitting in the same case as a processor fan that's been in the system since the same build, from new, and is still spinning after 5 years and including several months of inactivity somewhere in that. Fans do die but I've worked in schools using 10-year-old equipment that's on all day, every day, never cleaned and subject to the worse treatment (kids inserting Lego into the holes) and yet, when I took them to be scrapped, I used to remove their fans to add to my stock. They *all* still spun perfectly, and that was before I air-sprayed / oiled them. A fan alarm costs pence. I know, I have one that's independent of my motherboard and goes mad if the fans are slow or not spinning (like a UPS alarm, almost). The noise? I don't hear it when it's sitting inside it's not-acquired-for-that-purpose wooden cupboard (with the back chopped off to allow airflow).

So it's not that an expensive heatsink or fan will make my computer last longer (the temperature-failure link is dubious, as proved by many modern datacenters that don't bother with cooling and only see a slight increase in failures, and the cost of some of those setups is better poured into just taking better care of the damn thing), or that it will make it work faster (unless I overclock but already that's into the same dubious areas as played with your fuel mixture on your car - you rarely get an overall gain given some risk, reduced product life, etc.), or that it will make it use less energy (it's a pittance). Any of the reasons you can choose are VASTLY outweighed by the initial costs.

(I hate overclockers - I challenge any overclockers to ring up a significant number of CPU cycles gained by overclocking when you take into account their overall average computer usage. Yeah, you might get a handful of FPS better but the games aren't designed for that anyway so all you gain is stretched textures on a unnecessarily high resolution moving slightly faster than your perception would ever allow you to detect anyway).

Expensive fans, heatsinks, etc. do not make economic sense for 99.999% of users. That's ridiculously high a proportion. A 50p fan every year is actually much more sensible that spending the amounts of money that I've seen people drop on such stuff. Really. Now, expensive mouse... yes, I'll give you that. Expensive display. All the things that you actually use all the time and require to work with absolute precision - there is nothing more annoying that a slightly un

Re:Best for headshots

[Anonymous Coward]

As Microsoft strives to migrate their core technologies from the desktop onto the Web, so too is their propaganda machine migrating from the established press to the informal social web.

Microsoft shills are invading social web sites everywhere - in forums, discussion groups, comments to news items, edits to Wikipedia, manipulation of search engines, comments to blogs - posing as innocent participants to promote their agenda and counter wide spread complaints about their shady marketing practises. Even in the comments section of blogs by Microsoft employees on their own corporate site they employ sock puppets to say the things the author felt inappropriate to say directly. They race to place their shill postings at the top spot in the comments section of news and blogs, or perhaps they are given advance notice enabling them to do this where they are a sponsor.

The evidence is here on Slashdot for all to see, without embellishments from me. What I say here is amounts to only a digest of hundreds of postings by others. A careful investigator can see for himself the evolution of discussions on Microsoft related issues, especially those showing competitors in good light. As one reads from Slashdot's historical record on through to recent times, the evolution of Microsoft's efforts to pervert Slashdot's discussions becomes readily apparent. Microsoft's ambition is to twist internet discussions around a full 180 degrees until these discussions become a platform for propaganda from Microsoft's "Ministry of Truth". A study of the comments of the shills posted here can be cross-correlated with postings on other sites. Their pattern of saturating a discussion with shill postings, and the repeating of mindless memes becomes obvious. Their harassment, ridicule, and suppression of criticisms is designed to intimidated those who would speak out against them. They seek to establish and enforce a discipline of giving Microsoft "fair treatment" and their propaganda the same consideration and respect a real person would deserve.

In the process they are destroying the social web as we know it. This insidious attack on the infrastructure we rely upon to form our opinions in a complex world has both a direct and an inhibitory effect on free speech as a side effect.

We must stop this while it is in its infancy. Once it fully established, it will become much more difficult to root out, and other ruthless corporations, organizations, and even governments will want to emulate the success of Microsoft's campaign.

This is the nightmare vision of the end of the social internet as we know it.

Re:Best for headshots

[cheekyjohnson]

"When you pay $100 for a pair of blue jeans, you're also being scammed"

But... all the cool kids have those! Anyone without them is a loser!

Re:Best for headshots

[TheKidWho]

For starters, those fans don't make economic sense, but that was never the point. The point is to get the best performance possible.

The fact is that the replacement of a fan/heatsink with a item that isn't supplied by the systems manufacturer makes no sense and doesn't, actually, get you any worthwhile gain for the money you spend.

This is just a sign of your ignorance, an aftermarket heatsink will reduce the CPU temperature by tens of degrees. It's the difference between getting a 3.5ghz overclock on air versus a 4.2ghz overclock on air. It's a pretty big difference performance wise, on the order of 20%. As an anecdote from my own testing, my Core i7 920 hits around 90C at 3.6ghz with the stock fan/heatsink while on my Scythe Mugen 2($50 heatsink/fan) it hits 68C at 3.8ghz. For some people that extra 20% of performance is important, for most people it isn't.

(I hate overclockers - I challenge any overclockers to ring up a significant number of CPU cycles gained by overclocking when you take into account their overall average computer usage. Yeah, you might get a handful of FPS better but the games aren't designed for that anyway so all you gain is stretched textures on a unnecessarily high resolution moving slightly faster than your perception would ever allow you to detect anyway).

Cool man. Did you ever think that some people do it because they enjoy the tweaking? Some games also require you to run overclocked if you run the game at high resolution(2560x1600) to max out the game. On the i7s with multiple GPUs the issue is very apparent. A lot of overclockers also contribute their spare CPU power to groups such as Folding@Home doing Protein research.

Expensive fans, heatsinks, etc. do not make economic sense for 99.999% of users.

Sure, so what? See above.

Overclocking is the go-faster stripes of the PC world.

Not true at all, the main reason people overclock nowadays is to turn their $200 CPU into the equivalent of a $1000 CPU. I purchased my i7 920 for only $200 last year and I have it performing faster than the i7 960 which cost $999 at the time by overclocking it. A lot of people do it for price/performance reasons more than anything else. The money saved from purchasing an "inferior" CPU goes into the purchase of better cooling equipment to manage the heat.

If I want a cooler PC, or faster graphics, or whatever, it tends to be that after a certain point (usually the mid-to-top-end equipment), the returns are so minuscule as to pale in comparison to *ANYTHING* else I might do. I've seen processor overclocking gains that are wiped out (by orders or magnitude) by installing a better antivirus (yes, they were running an AV on their overclocked, behind-the-firewall, games-only machine that was sucking up about 25% of their CPU time but thought that running their systems out of specification by a handful of percent was "worthwhile"), or taking a couple of items out of my startup, or putting the PC so it's not blowing its exhaust air at a solid wall.

This is true to a point. I recently built a comptuer for my friend to play Starcraft II maxed out, total cost was only $550, miniscule compared to the $2k I spent on my rig and it plays the game nearly as well. However that's not why I spent $2k on my computer. Regardless, why do you assume overclockers are idiots? You'd think most of us would be smart enough to recognize where slowdowns are occuring in our computers since most of us are seeking the best possible performance for the money. Just because your friend is an idiot doesn't mean we all are.

And when you factor in the extra cost, you really are better off NOT touching a bog-standard off-the-shelf system but instead saving all that money and time you would normally spend tweaking with a) playing the damn game and b) putting the money sav

Re:Best for headshots

[TheKidWho]

If you buy those expensive digital cables for stereo, you are NOT getting what you paid for -- you've been scammed.

It depends, similiar to your Alieve/naproxin sodium example, you can buy high quality cables for reasonable prices as well and there IS a difference between high quality cables and trash cables.

When you pay $100 for a pair of blue jeans, you're also being scammed.

Not true, it just means you didn't buy a piece of crap pair of jeans from a sweatshop in the third world. Have you ever worn an expensive pair of jeans? I'll admit they are overpriced to a point, but they tend to be much more durable and comfortable than regular jeans.

When you pay three times for a bottle of Alieve what you would pay for a bottle of generic naproxin sodium, you're getting scammed.

To a point, you're also paying for the fact that Alieve has a known high level of quality control for their product. To some people that piece of mind is worth the price. See melamine contaminated food imported from China for an example.

Re:Best for headshots

[Captain Segfault]

And if those fans / heatsinks really made economic sense, then every server room in the world (from little offices up to large datacenters) would be using them.

They don't fit. The giant heatsinks and fans popular among overclockers would not fit in a 1U case, whereas noise in server rooms tends not to be a big deal so cases full of small loud fans are the norm.

Re:Best for headshots

[GrumpySteen]

> You do, however, usually (but not always) pay for what you get.

Not according to the RIAA.

Re:Best for headshots

[kiddygrinder]

it works fine, just at 50% of the frame rate.

Re:Best for headshots

[cheater512]

There was that test on a large group of audiophiles who couldn't tell the difference between stupidly expensive cables and coat hangers.......

(Btw pretty much all jeans are made in the sweatshops. Some are better quality but dont kid yourself on their origin)

Re:Best for headshots

[bassmadrigal]

I think you are forgetting about the love people can have for their interests. Car lovers souping up their engines even if they can only drive 65mph legally (unless you live in Germany like me :D ). "Let's make something faster that will give us worse fuel economy and have no practical benefits unless we take it to a track."

How about architects? They design these grandiose buildings that due to all the windows require more energy to keep the temperature maintained. Are archways really necessary? Why put a chandelier when a cheaper light fixture could work?

You can see this in every aspect of our lives. Scrapbooking... do we really need all these fancy designs for our pictures, rather than just labeling who is in it and what they are doing (or where they are)?

The fact is that people have their interests. Some peoples interests include overclocking. I have done it before with great results. I didn't even do it for the gaming, but for video encoding. 20% faster on a 2 hour long video that you are doing a 2-pass encode on saves you quite a bit of time. Especially when you have batch operations with 30 some-odd videos. And my PC lasted 5 years until the USPS broke it while in transit when I tried to ship it over to Germany.

I have considered overclocking my current machine (AMD X2 6400, 4GB), because occasionally I run into minor stuttering when it is transcoding high-def video that is streaming to my xbox. Could I buy a new quad-core processor that would do it without a problem? Absolutely. Could I shut down a bunch of stuff running on my machine that is taking processor time from the transcoding job? Absolutely. Could I pipe lower quality videos to my xbox that don't require transcoding? Absolutely. But I could also overclock my processor and not have to do any of those other things. It allows me to squeeze extra life out of this processor before I finally decide to bite the bullet and upgrade.

And as for your using stock heatsinks/fans, there are reasons to upgrade even if you aren't overclocking. Upgraded HSF (Heatsinks and fans) are usually better performing. Keeping the heat down will also help with power consumption. They are also quieter. My stock AMD HSF can get quite noisy when my computer is going full bore. Plus you can get some cool looking ones, which goes back to the love people have for their computers. My computer has a side window and the majority of my fans are blue lit. I get comments from people as it is with the stock heatsink. If I got one that added blue leds to the fan or just something a bit flashier, well it adds to the overall appearance.

I think you should just be more tolerable of what people like to do in their own time. If they want to spend their time overclocking, or replacing their car's exhaust, or spending time cutting out all these "cute" things to go into a scrapbook... let them (and preferable without berating their interests at the same time).

Not just useless, but actually toxic.

[QuoteMstr]

Trades that happen this fast are only good for further enriching large investment firms that can afford to spend millions on clever algorithms for shuffling numbers around. This speedup lets these companies make even more money without creating one damned thing that's useful to any living person.

Limit trades to one per second per institution, and while you're at it, add that tiny per-trade tax. Finance should be boring. Let's encourage people to focus on the real economy that operates in the world inhabited by you and me.

Re:Not just useless, but actually toxic.

[buchner.johannes]

Can someone explain to me what the system does in these 126 microsecond? Is it sending packets through the world, doing some complicated calculations, solving locking resources? It seems an awefully long time to add to a table and update some stats.

Re:Not just useless, but actually toxic.

[Ethanol-fueled]

The idiots in charge of the stock markets believe that, since Linux doesn't crash, their stock markets won't either.

Re:Not just useless, but actually toxic.

[mpoulton]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of. These high-speed arbitrage and "quant" investors may make a profit without creating a product (though collectively their track record is not very good financially), but their profit margins are vanishingly small and they serve a critical role in equalizing prices between markets and keeping prices up to date as market conditions change. In short, your complaint about these trading practices smacks of jealousy and sour grapes, and it ignores the valuable role they serve in the markets.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

It is the time measured from when a bid/ask order is sent from the customer's network port, until it has been processed/stored and possibly matched at the Exchange, and back again.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

The idiots in charge of the stock markets believe that, since Linux doesn't crash, their stock markets won't either.

And if Linux crashes, Linus will give you a full refund.

Re:Not just useless, but actually toxic.

[ikkonoishi]

Lots of data going in lots of data going out. Millions of people make trades on this thing constantly.

Re:Not just useless, but actually toxic.

[dakameleon]

I respect your optimism and idealism when it comes to these things, but Index Arb desks are making some of the most effective, near-risk-free profits for the big banks, and it's little wonder the LSE wants to be at the forefront of this market. You don't have to pay a computer a bonus, and the programmers behind this hardly see the same kind of money as the big swinging dick traders who try to spot the macro inefficiencies. Furthermore, the same strategies and speed advantages are used for algo traders to allow big blocks of trades to go through as best possible without shifting the market, making more cash when the trades are billed to the client at a weighted average instead of the true cost.

But then you don't see these numbers in the breakdown of the Goldmans profit numbers, and you never will. In the casino of the share market, the dealer is helping the sharks fleece the sheep.

Re:Not just useless, but actually toxic.

[NoSig]

I can't tell if you are are someone defending your own work to assuage your conscience or a troll pretending to do so, but anyway: There is no value to equalizing prices in less than a second, especially when the "equalizing" you are talking about is really just pocketing the difference. The only value there would be in equalizing prices in less than a second would precisely be to remove the threat that these people pose. Since sub-second trades are necessarily automated trades, they also cannot be doing anything sensible to keep prices up to date as real world conditions change, as that requires understanding the real world which an automated system cannot. They serve only the function of increasing profits for their investors - how little or how much damage they cause in the course of that is what is debatable.

Re:Not just useless, but actually toxic.

[1s44c]

Much as I'm sure that Linux will perform faster and with far more stability than Windows in all given applications the parent is spot on here.

More speed isn't the answer here, less speed is needed. If anyone wants to buy and sell little bits of companies at those kinds of speeds they clearly have no interest whatsoever in the companies they are trading. They can only be gaming the system for unearned gain.

Investment bankers ( and I don't mean real investors ) are parasites on the capitalist system and anything that enriches them impoverishes someone who works for a living.

Re:Not just useless, but actually toxic.

[nacturation]

Can someone explain to me what the system does in these 126 microsecond? Is it sending packets through the world, doing some complicated calculations, solving locking resources? It seems an awefully long time to add to a table and update some stats.

One eighth of a millisecond is an awfully long time?

Re:Not just useless, but actually toxic.

[atomic brainslide]

while in theory your idea is correct, the harsh reality is that in practice, the large investment firms increase their profits drastically because there are actually two markets. this isn't strictly legal, but it's there. the large firms have dedicated connections to the exchanges with guaranteed SLAs and lower latencies than any other regular participant in the market. this allows them to stuff the buy/sell queues and rapidly cancel orders before they go through. the purpose of this is to deduce other bidders' price points and gain an edge. there are a number of such hedge funds (and even a major bank whose name escapes me), for example, that have had perfect trading days for over a year. statistically impossible outcomes like this only come from gaming the system in the above mentioned manner. as usual, the regulators are asleep at the wheel and the markets become more volatile week to week with increasing flash-crashes exactly because of these schemes. more efficient markets these are not.

Re:Not just useless, but actually toxic.

[nacturation]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of.

The faster the trades, the greater the disparity between how fast a normal Joe investor can trade vs. how fast an investment firm plugged directly into the exchange can trade. Let's say I'm down the block and my ping times are an awesome 6 milliseconds. A large investment firm plugged into the exchange might have a ping time of 1 millisecond (I'm being quite conservative here), even 1% increase in benefit I see the investment firm is going to see a 6% benefit.

Yes, they play a role in equalizing market prices (more power to them) but they can also profit from the fact that ordinary investors are acting on information that is outdated because they are unable to receive the information as fast as the large institutions can. The greater trading speed only increases this disparity.

Re:Not just useless, but actually toxic.

[Mitsoid]

This sounds interesting, can you please provide a citation for those who wish to research?

Re:Not just useless, but actually toxic.

[davester666]

It increases the time delta between when your broker trades the stock up a little, then puts through your trade by selling you the stock he just bought, but for a little more than what he bought it for. Every time.

Re:Not just useless, but actually toxic.

[harlows_monkeys]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value

I doubt this is true for real markets. Value ultimately comes down to human reactions to events in the real world, and how that affects our demand for goods. Since human perception speed and decision time is limited, I expect that there is some maximum trade rate, beyond which the market ceases to be optimal (if we measure optimality in terms of producing efficient allocation of real-world resources) and starts reacting to itself. I suspect that this is one of those areas where to get an optimal result, you need to inject some noise into the system (add random latency to trades).

Re:Not just useless, but actually toxic.

[MyFirstNameIsPaul]

The regulators are not 'asleep at the wheel'. They are playing their historic role of do what the boss tells you, and if the boss doesn't tell you to bust one of the huge firms, you don't. If the regulators are incapable of protecting the small investors, then get rid of the regulators, but don't blame the technology. Your assumption that the market is more volatile today than ever before is weak. Take a look at the Dow in the 30s.

Re:Not just useless, but actually toxic.

[Tom]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value.

It might, if the trading were actually being done at the exchange, for market prices. It isn't. Most trading today is only registered at the exchange, but the actual deal is brokered elsewhere.

Also, the amount of liquidity a market needs is subject to discussion. Do you really need to have a counterpart available this second for a market to work? That is nonsense, the market (the real market, not the speculative one) wouldn't burn and die if you had to wait a second or two or even *gasp* five for a deal to go through.

Providing liquidity is a valuable role. However, you are ignoring the fact that everyone is in it for the money. If the cost of liquidity, i.e. the amount of profit the high-speed traders extract from the market, becomes too high, the market also suffers. Somewhere, there is an optimal point between the positive (more liquidity) and the negative (the cost of this liquidity).

Re:Not just useless, but actually toxic.

[LingNoi]

It might crash but at least it won't have an 8 hour down time like their previous windows based system had. Oh, and you can forget about the imaginary support you'll get with that windows license as the London stock exchange found out the hard way.

Re:Not just useless, but actually toxic.

[hughbar]

There's a huge problem with the word 'value' in the above.

To declare interest, I'm ex-investment banking and not too proud of it. The 'values' and 'derivatives' exchanged are often not mapped to anything happening in the 'real' world [think manufacturing, [useful] services even], however they do have a negative impact on it [factory closures, bank bailouts paid by the taxpayer, for example].

Try a thought experiment, does anything useful change in 126 microseconds? Bread get baked? Pizza cooked? House built? Seed planted, if you want to get rural and idyllic?

Incidentally, I'm not against simple futures, for example, they smooth the farmer's year and have a purpose. I am pretty much against most exotic financial derivatives and against short-term [126 microseconds, for example!] 'investment' to use ironic quotes...

Re:Not just useless, but actually toxic.

[Anonymous Coward]

Just from some quick Googling:

100% perfect trading streak

61-day perfect run

There are some in-depth analyses of the "Flash Crash" recently, for instance at:

Flash Crash Analysis

Re:Not just useless, but actually toxic.

[Pinky's Brain]

The HFT algorithms aren't value based, how could they cause the stocks to accurately reflect value? HFT is a pure arbitrage play, gaming the restricted strategies of long term investors (in some cases legally restricted). Forcing volatility beyond normal variance to try to benefit from stop loss orders is a perfectly valid HFT strategy ... does nothing to make stocks reflect value though.

HFT has one benefit, increased liquidity, and one downfall, increased volatility. For the moment humans are still the only ones capable of judging value ... the HFT algorithms add nothing there.

Re:Not just useless, but actually toxic.

[wvmarle]

This is not about investors. They are barely if at all affected by these fast trades. Investors are people that buy stocks to hold for long-term gains (either dividends or value increase).

This is about speculators. The people that try to ride the natural fluctuations to make a quick buck.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

no button, it's all CLI, dweeb.

Re:Not just useless, but actually toxic.

[fraktalek]

It's also not true that it is about *large* investment companies, resp. large is relative. For example RSJ Invest is one of the biggest and it has only about 5 employees, AFAIK. So it is in fact a small company which became financially pretty big over some 15 years of careful development of statistical models and algorithms.

Re:Not just useless, but actually toxic.

[Pinky's Brain]

Themis Trading has a white papers on the kind of information HFT algorithms get to play with (for a fee of course, the exchanges want their cut).

http://blog.themistrading.com/?p=906

PS. some of this is being addressed at the moment, but it shows where the real money in HFT is coming from ... and it sure as hell isn't being better at price discovery.

Re:Not just useless, but actually toxic.

[lena_10326]

You do realize this is a stock exchange processing large volumes of transactions requiring a high degree of availability and consistency? Not a ma-and-pop website processing 100 transactions a day. Right?

Re:Not just useless, but actually toxic.

[hairyfish]

Which specific part of Linux inherently prevents 8 hour outages? Or is this one of those 'Linuxeses R teh awes0me, indows is teh suck' type post?

Re:Not just useless, but actually toxic.

[Splab]

Yeah was thinking the same.

My database which is running pretty damned fast (general purpose) spends around 30.000 microseconds on simple add/update/inserts (timed from middle-ware calling), having a computer respond to anything in 126 microseconds is down right impressive.

Re:Not just useless, but actually toxic.

[Nursie]

the part where it's running on a mainframe?

(Yeah, I have no idea whether the stock exchange has anything to do with mainframes, but I know you can put linux on 'em)

Re:Not just useless, but actually toxic.

[buchner.johannes]

Anonymous Coward writes:

It is the time measured from when a bid/ask order is sent from the customer's network port, until it has been processed/stored and possibly matched at the Exchange, and back again.

ikkonoishi writes:

Lots of data going in lots of data going out. Millions of people make trades on this thing constantly.

nacturation writes:

One eighth of a millisecond is an awfully long time?

davester666 writes:

It increases the time delta between when your broker trades the stock up a little, then puts through your trade by selling you the stock he just bought, but for a little more than what he bought it for. Every time.

lena_10326 writes:

You do realize this is a stock exchange processing large volumes of transactions requiring a high degree of availability and consistency? Not a ma-and-pop website processing 100 transactions a day. Right?

Johannes concludes: no one knows what it does. Yet we are stunned of the millions of transactions flowing around.

The limiting factor can't be throughput/number of transactions, given that the data flow is continuous.
The closest to an answer is that it is network latency. But why is it relevant to use Linux then? Because the TCP/IP stack is so great? Wouldn't the specs of the network (fibre?) be more interesting?

Re:Not just useless, but actually toxic.

[LingNoi]

No, i'm talking about the hours of downtime that the previous system experienced (my bad 7 hours not 8). It was a simple fact (get the facts haha).

When one of the numerous Linux exchanges goes down for that long then call me.

Re:Not just useless, but actually toxic.

[janwedekind]

I couldn't agree more. There was an interesting PBS documentary about the financial crisis. This guys are extreme. They take everything to the edge. They make competitive risk estimates by grouping financial debt obligations with seemingly uncorrelated risk. And then they "realized" that the risk goes down with the amount of trading you can do in a second.

The system rewards players who figure out how to accumulate ownership of resources. There seems to be little reward for long-term developments which require prolonged periods of financial trust.

Well, why am I even writing this? No reward in that either ;)

Re:Not just useless, but actually toxic.

[MadnessASAP]

Well that's actually true, but instead we'll be getting panics on the market.

Re:Not just useless, but actually toxic.

[1s44c]

I wonder if you'd be saying the same thing if they were replacing Linux with a Microsoft solution. Probably not.

Of course I would. The problem here is not the OS.

Re:Not just useless, but actually toxic.

[kgrey38]

30,000.
http://en.wikipedia.org/wiki/Decimal_mark#Countries_using_Arabic_numerals_with_decimal_comma

Re:Not just useless, but actually toxic.

[AlecC]

Entirely agreed. The socially constructive purpose of the stock exchange is for people (investors, fund managers) to make medium term investments in a market in which the can make and liquidate investments easily. As such, their decision times will be many second, or even minutes. In order to maintain market liquidity and to be responsive, it is useful to have speculators willing to trade slightly faster than the real investors, making tiny profits by keeping the market fluid. But they only need to be, say, ten times faster than the "real" investors. A trade every second or two is plenty to allow the market to do its job. And a nano-tax on trades would provide useful damping on a system with natural positive feedback.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

It's not only updating tables...

The exchange gets a cut of every sale, so at the very least, the more sales you have the more you can profit.

It's more insiduous than that though. Between the price a buyer wants to pay and a seller wants to receive, there's a certain spread. Faster trades allow the exchange to take advantage of that spread. Sometimes it lasts a second or so, but trading volume means a second can make thousands of dollars.

Faster exchanges also allow 'tasting'. The average investor doesn't get to take advantage of it, but the large houses do. They can float a price out there and see how many people are willing to buy at that price. Then they can test a higher price... Then higher.. At some point they reach a price that maximizes their profit.

It also means that certain brokerage houses can get their trades in faster. So that means a popular and rising stock goes to those houses that pay for the privilege of first dibs. These houses can then set the price on the stock.

There are dozens of other ways that faster trades help.... Of course, none of it helps us, the average stockbroker.

Re:Not just useless, but actually toxic.

[martin-boundary]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value.

This is fundamentally wrong. The only way to bring the market mathematically closer to the ideal is to speed up time for everybody in the market. But that's certainly not what's going on.

Only some market participants get to trade at a faster rate in reality, so what this leads to is a bias in the prices which weighs the preferences of those privileged participants more than those of the other, unprivileged, participants.

Put another way, the asymptotic optimal values computed in a two tiered market where some players can act much more frequently than others are not the asymptotic optimal values computed in a single tiered market where all players can act at the same frequency. It's a fundamental misunderstanding of market dynamics to think otherwise.

Re:Not just useless, but actually toxic.

[kiddygrinder]

and if windows crashes, i hear steve gives you a sexy massage.

Re:Not just useless, but actually toxic.

[crowne]

Well ... in the investment banking world, BSD has a very different meaning to what we're used to over here in technology.

Re:Not just useless, but actually toxic.

[rtb61]

So the London stock exchange has updated it's operating system to accelerate it trades, just as it seems likely that laws will be implemented to decelerate trades. It makes logical sense that trades be slowed to a 24 hour cycle to stabilise the global share market a prevent algorithmic stock price collapse or major firms distorting the market electronically with rapid internal trades.

Re:Not just useless, but actually toxic.

[networkconsultant]

Utilizing FIX or some other Exachange based protocol, the server looks for your order, finds the cheapest holding house and then requestes that order, once posted it is then filled by whomever had the lowest bid out of about 200 institutions world wide. A typical trade involves about 20 to 80 database queries.

Re:Not just useless, but actually toxic.

[TaoPhoenix]

Freddie and Fannie were sitting in a Feed
T-R-A-D-I-N-G....

Re:Not just useless, but actually toxic.

[TheLink]

Actually what is most disgusting is:

When those algo/HFT systems have bugs or lose big
a) the stock market rolls back the trades[1]
b) the small timers beating those algorithms get sued.[2]

But they don't do that when the small timers make mistakes or the algo/HFT systems beat the small timers.

Even though many of the HFT bunch are doing dubious stuff:
http://www.nytimes.com/2009/07/24/business/24trading.html
http://www.nytimes.com/imagepages/2009/07/24/business/0724-webBIZ-trading.ready.html

[1] http://www.reuters.com/article/idUSTRE6456QB20100507
[2] http://www.computerworlduk.com/news/security/3244186/norwegian-traders-convicted-for-outsmarting-us-stock-broker-algorithm/

Re:Not just useless, but actually toxic.

[Nutria]

it ignores the valuable role they serve in the markets.

What valuable role?

The ability to buy a tiny fraction of 1% a company, decades after it's IPO?

Or the gambling which is what 99.999999% of stock trading is.

Re:Not just useless, but actually toxic.

[TheLink]

It's just front-running and other normally illegal stuff by a different name.

Effectively a tax on everyone else.

Re:Not just useless, but actually toxic.

[transami]

Tobin Taxes Now!

Re:Not just useless, but actually toxic.

[gundersd]

It may be true that nothing useful in the real-world changes in 126 microseconds... however...

The way I look at it is that 'the market' is like a big complicated electronic system which contains a lot of complex feedback loops (some of them more stable than others). Imagine tweaking a random knob on such an electronic circuit and watching the effects of that tweak ripple through the circuit until it (hopefully) reaches a steady state again.

Increasing the latency causes changes to ripple through the system in a way that a steady state may take a long time to occur (or may never occur) as market participants end up making decisions based on old data. Sure, it may be easier for a human to observe what's happening but it doesn't necessarily mean things will be any more stable.

Lowering the latency to trade is equivalent to increasing the bandwidth of the components in the system, helping the steady state to be reached sooner. Yes, in the worst case, this might allow the feedback loops to veer outside the stable region within the blink of an eye, but that's why there are things like safety cut-offs.

I agree that at first glance it seems that 126 microseconds should be fast enough for anyone, but when you consider the sheer volume of market participants, the mind-boggling number of trades that are executed, and the complex network of relationships between different stocks, I think that having a market that can reach a steady state as quickly as possible under various "tweaks" of input parameters is probably, on the balance, a good thing.

Re:Not just useless, but actually toxic.

[Rockoon]

Try a thought experiment, does anything useful change in 126 microseconds? Bread get baked? Pizza cooked? House built? Seed planted, if you want to get rural and idyllic?

But isnt that the point?

I would think that an exchange where useful things CANT happen between the decision to buy and the actual fulfillment of the buy would be a desirable one.

Re:Not just useless, but actually toxic.

[aliquis]

I assume it's while handling all the other trades to. Just not one trade.

Re:Not just useless, but actually toxic.

[aliquis]

And if Chernobyl crashes nuclear power get you!?

Re:Not just useless, but actually toxic.

[aliquis]

Funny looking at that map.

The British and their empire, always doing things different :)

Old emperors of Brittan would had been so proud of Steve :)

Re:Not just useless, but actually toxic.

[aliquis]

[citation needed]

Since there's normally an order depth and many of us would be very small traders in comparison to the whole system?

(Eventually you can set prices in 1 "small units" instead of 5 where people normally put 0/5/.../95, not sure.)

Re:Not just useless, but actually toxic.

[zevans]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of. These high-speed arbitrage and "quant" investors may make a profit without creating a product (though collectively their track record is not very good financially), but their profit margins are vanishingly small and they serve a critical role in equalizing prices between markets and keeping prices up to date as market conditions change. In short, your complaint about these trading practices smacks of jealousy and sour grapes, and it ignores the valuable role they serve in the markets.

Good try, but if there's one thing we've learned since 1986 it's that stock markets are no longer a measure of any meaningful long-term company value.

The "value" you're oh-so-efficiently measuring there means nothing useful to anyone except another trader.

Re:Not just useless, but actually toxic.

[Hognoxious]

What about Henry Ford? He paid his workers very well by the standards of the time and he made a useful product at affordable prices.

Re:Not just useless, but actually toxic.

[williamhb]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of. These high-speed arbitrage and "quant" investors may make a profit without creating a product (though collectively their track record is not very good financially), but their profit margins are vanishingly small and they serve a critical role in equalizing prices between markets and keeping prices up to date as market conditions change. In short, your complaint about these trading practices smacks of jealousy and sour grapes, and it ignores the valuable role they serve in the markets.

Except that we're long since at the levels where high frequency trading is the only customer for its own increased efficiency. The only kind of purchase that can notice a difference from a 400 microsecond latency down to a 126 microsecond latency is a fully automated one, because it takes a human orders of magnitude longer just to twitch his finger towards the "go" button, let alone actually press it. To any non-algorithmic trade, latencies were already vanishingly small.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

I actually had something to do with their trading setup. They're using "state of the art" technology, which, frankly isn't ready, and that was the real cause of their downtime. Just looking at the shear size of their network forwarding tables immediately clarifies why it crashed, and why it will crash again.

(IP) Multicast just doesn't work (reliably) in a network of that size, and no OS is going to change that unfortunately.

Apparently doing anything else is somehow "unfair".

Re:Not just useless, but actually toxic.

[Anonymous Coward]

Having worked for the Exotics desk in a Derivatives environment I can tell you that the complex instruments they invent are complex in order to dazzle and deceive the customers - the traders simply take advantage of their superior knowledge in that domain to make money out of the clients by charging the client more for something that they don't really need, a bit like some car mechanics convince clients to get new starter engines when what they really need is new spark plugs.

Most clients could have their needs satisfied with cheap derivatives like futures and options which can be gotten directly from an Exchange, but then the banks wouldn't do quite as much money and the traders wouldn't get millions as bonus.

More in general, banks make money from "investment" not because they're any better at it than others but because they have a number of structural advantages that others don't:
- They're big enough to have custom systems done to aggregate information
- They can borrow cheap money thanks to their government guarantee ("too big to fail") which they then use to "boost the returns" of their own capital by leveraging.
- They have tiny trading costs since they are registered directly with the exchange and need not pay brokerage costs.
- They can have their trading systems connected directly to the exchanges.
- They have their own trading platforms and sell to customers directly Over The Counter.

In fact I still work for the industry (they pay well and I'm a brain whore) and the amount of wastage I see around me would not possible in an environment with real competition where companies have to fight every day to be and remain profitable.

Re:Not just useless, but actually toxic.

[forkazoo]

The regulators are not 'asleep at the wheel'. They are playing their historic role of do what the boss tells you, and if the boss doesn't tell you to bust one of the huge firms, you don't. If the regulators are incapable of protecting the small investors, then get rid of the regulators, but don't blame the technology. Your assumption that the market is more volatile today than ever before is weak. Take a look at the Dow in the 30s.

I've researched the Dow in the 30s. Yes, today's market really is more volatile in many ways. Nobody in the 1930's would have cared about shaving tens of microseconds off of trading time because the system operated on human-comprehensible time scales at that point. To pull off the sort of schemes modern HFT uses by exploiting a few microseconds, you would have needed hours in the market of the 1930's. Sure, it was full of speculators, but the market at that point was at least dominated by actual traders. Not just robots skimming off the top.

Re:Not just useless, but actually toxic.

[Americano]

There is no value to equalizing prices in less than a second,

You are wrong, of course. They are providing liquidity in the market - which has tremendous value, if you understand how the markets actually work, and don't just assume they're there to funnel money from investors into scammers' pockets.

Should they not be compensated for providing that liquidity? They make a (small) profit on these trades, in return for providing a ready flow of cash to support other people trying to buy and sell.

Re:Not just useless, but actually toxic.

[khallow]

Trades that happen this fast are only good for further enriching large investment firms that can afford to spend millions on clever algorithms for shuffling numbers around. This speedup lets these companies make even more money without creating one damned thing that's useful to any living person.

Except, of course, for the liquidity that these "large firms" provide to everyone else who trades on the market. Enabling betting trade is itself useful to living persons.

Re:Not just useless, but actually toxic.

[gagol]

Black Sheet of Deficits? Blantantly Sunken Devise?

Re:Not just useless, but actually toxic.

[e065c8515d206cb0e190]

It's a trading system. I want to say it matches Buy orders with Sell orders.

Re:Not just useless, but actually toxic.

[khallow]

The limiting factor can't be throughput/number of transactions, given that the data flow is continuous.

The primary limiting factor is how much liability the stock market can afford to pick up. For example, if enough traders discover a way to legitimately siphon a fraction of a pence from the London Stock Exchange with each trade, then the stock exchange would probably go bankrupt.

A secondary consideration is the reputation of the stock market. Trades that don't go through when they should are going to piss traders off (to the point that they move to other markets or even try to get their money back from the market itself).

So whatever the LSE's system does, it needs to be reliable and not have any ways for the LSE to consistently lose money. That means the underlying system has to be reliable and predictable. While I doubt linux versus Windows will have a thing to do with money losing trades, that choice has a lot to do with the overall reliability of the system. Linux simply can be made far more reliable.

Re:Not just useless, but actually toxic.

[e065c8515d206cb0e190]

A lot of people thought like you when they wanted to ban the trading of futures in the late nineteenth century.

And since this is Slashdot: remember when Congress wanted to ban the telephone? It's pretty much the same. Technology advances, whether you like it or not.

Re:Not just useless, but actually toxic.

[khallow]

I respect your optimism and idealism when it comes to these things, but Index Arb desks are making some of the most effective, near-risk-free profits for the big banks, and it's little wonder the LSE wants to be at the forefront of this market.

And your point is? We shouldn't allow arbitrage? We need to feel emo about big banks? Keep in mind that the profit isn't risk free for two reasons: 1) the big bank has to invest in trading infrastructure and personnel for a chance at arbitrage profit, and 2) somebody else can get it first.

But then you don't see these numbers in the breakdown of the Goldmans profit numbers, and you never will. In the casino of the share market, the dealer is helping the sharks fleece the sheep.

Hard to believe that you speak of arbitrage and casinos in the same post. I suppose it's because you don't understand how arbitrage helps reduce the "casino" atmosphere of markets. An arbitrage opportunity is a mispricing of two or more disparate markets (this is a far from perfect summary of arbitrage, but it does cover even arbitrage in stock option trading which is the current market versus the same market in the future).

If nobody could make that trade, then the various markets involved would trade at different prices. Someone would be getting a worse deal than they'd otherwise get and someone would be getting a better deal than they should. Arbitrage couples these markets together to deliver better, more accurate prices for everyone.

Sure, only traders with significant resources and, most importantly, low transaction costs (ie, people like "big banks") can trade arbitrage effectively on modern markets. But how is it a bad thing that an asset traded in one market is closer priced to the same asset traded on another market? Or assets in current markets (along with their standard call/put options) trade consistently with the asset at a future time?

To claim that arbitrage is wolves fleecing sheep is ignorant. The sheep pay even worse prices without arbitrage.

Re:Not just useless, but actually toxic.

[rwa2]

Word... HFT is not much different than the plot for Office Space. "Making" money from money by taking other people's money from their transactions just by essentially skimming what would otherwise be roundoff errors off the top.

On the plus side, at least it's a plug for Linux performance?
Obligatory: http://xkcd.org/808/

Re:Not just useless, but actually toxic.

[mcgrew]

Or perhaps the part where the admins are competent? I took a programming course once, taught by the guy in charge of the Illinois Secretary of State mainframe. He took us on a tour of it, very impressive.

But most impressive to me was that they have an average of 1/2 hour of outage per year.

Re:Not just useless, but actually toxic.

[TheTurtlesMoves]

Liquidity measured in microseconds no less. You know I just don't know what I would do, if after talking to my broker (yea old school), or just selling online, if it would take 1 sec rather than 130ms.

I mean sheesh, think of the interest I could earn in 870ms.

In fact why do we have 90day bank bills. What about some 90msec ones?

Re:Not just useless, but actually toxic.

[TheTurtlesMoves]

With a chair.

Re:Not just useless, but actually toxic.

[TheTurtlesMoves]

But if the wrong person provides that "liquidity", they get sued.

Its a sham plain and simple. So BS about liquidity would mean the rules would the same for everyone. But the rules clearly are not, as this post also points out.

Re:Not just useless, but actually toxic.

[sockman]

Companies spend millions getting space physically close to the exchange data center, and low latency connections to it. I think he's pretty sure.

Re:Not just useless, but actually toxic.

[TheTurtlesMoves]

More than a million times? Really?

Re:Not just useless, but actually toxic.

[rcp]

There's a series of decent (but marketing-focused) articles explaining low-latency messaging around exchanges, and high-frequency trading at Solace Systems, makers of low-latency messaging appliances:
http://www.solacesystems.com/tag/low-latency
From that feed, here's a Reuters story about HFT that's more than just fear and frothing at the mouth:
http://uk.reuters.com/article/idUKN173583920091202?pageNumber=1

- Richard

Re:Not just useless, but actually toxic.

[Lobachevsky]

Liquidity isn't just about there being _someone_ willing to buy or sell; it's about the spread. Do you want to go back to 25 cent spreads from the early '90s? Most spreads today are 1 cent. If you're happy paying 25 cent spreads to get rid of automated traders, I'd say that's a bit like chewing off your arm to swat a fly. Most automated traders make anyways from 0.1 cents to 0.5 cents per share. Comparatively, retailers like E*Trade charge customers $9.99 for trades that average around 400 shares, which is 2.5 cents per share. Mutual funds like Fidelity often charge 1 to 2% management fees on investment, which is 30 to 60 cents per share on a $30 stock.

Trying to bend the rules of the market to wipe out a segment that makes 0.1 to 0.5 cents per share is silly when there's zero effort concerning E*Trade making 2.5 cents per share, or Fidelity making 30 to 60 cents per share. Professional services like Lime (a high-end version of E*Trade) charge 0.1 cents per share. No one is angry that E*Trade charges 2.5 cents per share while Lime charges 0.1 cents per share? Oh, that's right, because it's "only $9.99 !!"

That's the irony of all of this. The average person writes of $9.99 to E*Trade but the media tries to get them concerned about "costs" that are effectively 1/20th of that. I put quotes around "costs" because the spreads have come down from 25 cents per share in the early '90s to 1 cent nowadays. So the average person benefited 24 cents on the spread, and is angry that liquidity providers make 0.1 cents per share? Where was the anger in the early '90s when specialists (a cartel of liquidity providers) were raking it in, making 10+ cents per share? Where is the anger now at Fidelity _losing_ our money in 401ks _and_ charging management fees of 30 to 60 cents per share, annually?

The biggest to benefit from automated traders going away are the Larry Ellisons and other market manipulators who want to buy up companies without "moving the price up." It's all misdirection, trying to convince you and me that we're being hurt. It's the "death tax" all over again. The average person was never affected by estate tax, yet the media convinced us we should be against it, just so some rich jackasses can save money. The same deal is happening now. Rich folks want to buy out the public companies you and I are invested in, cheaply, and stealthily. They don't like automated traders sniffing their actions with pattern matching heuristics and raising the prices (benefiting long-term owners like us).

Re:Not just useless, but actually toxic.

[khallow]

Try a thought experiment, does anything useful change in 126 microseconds? Bread get baked? Pizza cooked? House built? Seed planted, if you want to get rural and idyllic?

Yes. All of the things you mention, for starters. I'm being a bit pedantic, but you do any of those activities in a long string of periods of 126 microseconds.

It surprises me how many people do a job for years and fail to understand what it is they do. You should know what a baker, pizza maker, home builder, and farmer have in common. They all trade. People like derivatives traders take that to the purest level, trading fairly abstract things, but still things with an attachment, however tenuous, to the real world. Nothing wrong with that. The trades themselves don't close factories or cause bank bailouts.

The real problem is leverage. For example, when you were a banker, how much collateral did you have to put up for your borrowed money? 1 unit per 10 of debt? More? In the real estate securities market, there were apparently people who could borrow 50 units for every unit of assets they had. I can't comprehend that level of foolishness, though I'm sure it make good bank for a time for the people who could get away with it.

Bottom line, what do you think would happen if pizza makers could get 50 to 1 leverage? I could use the pizza store I owned to borrow enough to build 50 more. I could use my car as collateral for one or more pizza stores (depending how nice it is). In a few months, we'd be up to our eyeballs in new pizza restaurant construction. There'd be incredibly specialized stores catering to the gay, vegan, Hispanic boardgamer.

And after the bust, when people realize that they didn't like pizza all that much? Pizza would be vilified, an epithet for people who need something to hate. Pizza makers would be scoundrels of the Earth who don't make anything connected to the real world, unlike bread bakers, home builders, farmers, or even the investment banker. Much would be made of their negative impact, such as factory closures and bank bailouts (paid by the taxpayer), for example. Even pizza makers would be self-flagellating themselves over their worthlessness.

The above is an interesting sociological phenomena, but the lesson boils down to high leverage causes massive fuckups every time no matter the industry, no matter how "real" the product is, and is the number one cause of market crashes and trigger for recessions.

Re:Not just useless, but actually toxic.

[peterbye]

Should Microsoft be responsible to help troubleshoot my website when it crashes? Or maybe I just wrote a shitty website.

MS provided a lot of in-house development resource for the Tradelect system, that was a major factor in winning them the contract.

Re:Not just useless, but actually toxic.

[Anonymous Coward]

Funny looking at that map.

Not half as funny as this one

Re:Not just useless, but actually toxic.

[Compaqt]

That's a great set of links. Thanks for sharing. I've bookmarked them.

It's one thing to have a neutral set of rules, and let the chips fall where they may. But it seems the new rule is Goldman Sachs must always win, and if it doesn't, you go to jail.

For the libertarians out there, this does not concern ownership of the means of production. This is about free and open markets.

Re:Not just useless, but actually toxic.

[coldtone]

- Matching buy and sell orders in an in memory book. Normally each symbol will have its own book.
- the database is written to after the transaction, multiple machines track the transaction, and log files are written too.

Re:Not just useless, but actually toxic.

[Splab]

other than your flaming, 30ms is not microseconds.

And as someone else pointed out 30.000 is due to locality the same as you yanks would write: 30,000.

Re:Not just useless, but actually toxic.

[hoggoth]

You cannot come up with a procedure or technology that will defeat greed.
One trade per second per institution? They will register 1,000 institutions and have their trades auto-distributed throughout their sub-companies.

Re:Not just useless, but actually toxic.

[spiffmastercow]

You're serious? This is slashdot.. Probably half of us are developers that have built or worked on one system or another that's made a difference in society. It's not always a big difference, but it's more than the financial market pricks have done for society.

Re:Not just useless, but actually toxic.

[Lobachevsky]

Higher latency translates into higher spreads. People quickly forget that stocks used to have 25 cent spreads, and now mostly only have 1 cent spreads. Say Microsoft on Nasdaq is the "primary" (leading indicator) of the true stock price. Microsoft in Chicago Stock Exchange is effectively a copy. When MSFT on Nasdaq goes up, for whatever reason, the folks with active offers on CSX will lose money unless they can cancel. If the latency is too high and those with sell limit orders on CSX are guaranteed to lose money in this scenario, they'll quote at a larger spread to "make up the difference". Think of it like sales tax, the company "pays" it, but it's really passed on to you.

Why is there a spread at all? Why are there bids at $18.01 and offers at $18.02? Why not just have "one price" for a stock, and everyone buys and sells at this agreed upon price? For the same reason options charge a premium. You're not going to be able to buy cheaply a 1-year call option of Microsoft with a strike price a couple cents out of the money. Such an option is expensive because Microsoft could easily be worth more by a couple cents in a year's time. Similarly, a put option is also expensive, because Microsoft could easily be worth less by a couple cents in a year's time. Both are true. Similarly, with "high latency" exchanges, the stock could either move up or down. You might be thinking "but wait a minute, what's the chance the stock actually moves up or down in 120 microseconds?" But you're forgetting about conditional-probabilities. If no one executes the lingering bid or ask limit order, whether the stock moved or not is immaterial. The only scenarios that matter are those that are subsequent to the bid or ask limit order being executed. The thing is, limit orders are rarely executed until there is information that the stock is about to move. When a stock is about to move up, buyers whack the ask limit orders. When the stock is about to move down, sellers whack the bid limit orders. So the probability of the stock moving, _conditional_ on getting your limit order filled, is quite high. Liquidity providers need to quickly realize a stock is about to move up and cancel their ask limit orders before losing money. If high latency or regulations prevent them from cancelling their ask limit orders effectively, they'll no longer place them 1 cent away from the bid. They'll go back to the early '90s of bid-ask spreads that closer to 25 cents.

Re:Not just useless, but actually toxic.

[tibit]

Thank you, thank you, thank you! This deserves +5.

Re:Not just useless, but actually toxic.

[aaarrrgggh]

While I am against HFT, I do trade options which have significantly less liquidity than your average stock. My scale as an individual investor is that I want to get a trade through within 5 minutes, and would really prefer to have it done in one minute.

For an investor with much more at stake, it is easy enough to see how a one-second interval could be make-or-break.

I think the exchanges should be forced to add random delays into each trade in order to create some risk to the HFT folks. 100-500us should be enough for anyone...

Re:Not just useless, but actually toxic.

[khallow]

You're welcome. I'm just fighting the good fight.

Re:Not just useless, but actually toxic.

[Dishevel]

Perception of value can change VERY fast. And perception my friend is everything.

Re:Not just useless, but actually toxic.

[Dishevel]

STFU.

We all know the rich people are evil. Banks, Big Business, Wall Street, and Big Oil are all out to kill babies. Our only hope is a benevolent World Government run efficiently and compassionately by the UN. With the assurance that we will all be made completely equal in all respects.

Do not try to defend them. You will be put down for it. Right so as well.

Re:Not just useless, but actually toxic.

[Archangel Michael]

There are a couple of simple fixes to stuffing and canceling problem. Simply add a "fee" (tax) for every order and cancellation, which is waved if the order is accepted. Stuffing and canceling would simply slowdown, because it would become immediately less profitable.

The other option would be to have time limits on orders, meaning they couldn't be canceled for a set period, say one or five minutes (whatever the magic number is). This would keep the liquidity, and make sure that people making offers are legitimate ones.

If you did a combination, where an order that has exceeded its order time (1 or 5 min), automatically waves the fee (along with accepted offers). You don't want to punish legitimate offers, only the speculative ones fishing for perfect information.

Re:Not just useless, but actually toxic.

[Dare nMc]

What the GP says still makes sense. This type of trading has been around forever, by making the trading faster, it means the arbitrage traders have to get faster and faster systems as well, for the same benefit. It didn't eliminate the game, but it made the game more difficult. I think your point is the fewer and fewer who have the resources can still take advantage; doesn't necessarily mean it got worse. I don't have any idea how one could figure out if the overall cost to the market as a whole went up or down, but the fact that it still exists isn't proof if it is better or not.

Re:Not just useless, but actually toxic.

[Americano]

You should really read your own links:

The two men managed to work out how the computerised system would react to certain trading patterns. This allowed them to influence the price of low-volume stocks for their own gain.

Manipulating stock prices is what got them in trouble.

Re:Not just useless, but actually toxic.

[UnknowingFool]

Indeed, speed in a transaction matters for large volume traders and not average investors. Suppose a public company makes an press announcement about their major product. This will affect their stock price. If the stock was trading at $25.20, it might rise up or down by a certain amount. Suppose it was good news that causes the price to go up ultimately to $25.30 at the end of the day. If a large trader was able to place an order for 100,000 shares at price of $25.25. Then they turn around and sell at $25.30 this turns into a $5,000 profit. For a small investor it isn't a large profit, but a large volume trader might make thousands of these trades a day leading to much more profit.

Re:Not just useless, but actually toxic.

[NoSig]

OK, please tell me how that works. It seems to me that since high-frequency traders typically hold a stock for less than a second, the only service they are offering are to make a good sell at most one second sooner than it otherwise would. Unless they are buying from someone else who are also a high frequency trader, selling a second sooner or a second later serves no function to improve the market. Indeed, if the high frequency traders usually make money, they will have traded this worthless service for real money, which seems to me to be a misallocation of funds to an unproductive activity - a case of the market rewarding worse-than-useless behavior. So what am I missing here?

Re:Not just useless, but actually toxic.

[TheLink]

Describing this as getting sued for outsmarting an algo is pretty misleading. The traders in question did find some flaws in the algo, but rather than exploiting them directly to profit from the algo machine, they used the algo to manipulate the market as a whole, so they could profit from that. They understood what would happen when they started manipulating the algo, and they should have understood that market manipulation of this kind illegal.

Misleading? Illegal?

Go see one of the links I posted: http://www.nytimes.com/2009/07/24/business/24trading.html?_r=1

Quote: "High-frequency traders often confound other investors by issuing and then canceling orders almost simultaneously. Loopholes in market rules give high-speed investors an early glance at how others are trading. And their computers can essentially bully slower investors into giving up profits -- and then disappear before anyone even knows they were there"

Sure looks like one rule for the HFTs and another for the rest.

If you don't think the HFTs do all that and worse, you can google for evidence yourself.

Re:Not just useless, but actually toxic.

[marcosdumay]

But it is a lot less cycles of the memory clock of that same machine. And it isn't enough for any desktop to complete a disk writting. Their database probably uses a battery powered RAM cache for the disks.

Re:Not just useless, but actually toxic.

[s73v3r]

Especially at those instants when someone presses the 'B' key instead of the 'M' key.

HFT does nothing more than make overpaid finance guys richer without actually having to do or produce anything. Except it has the added side effect of making the market much more volatile for everyone else.

Re:Not just useless, but actually toxic.

[sjames]

Actually, no. The whole reason they're worried about sub-second trading is that they know well that they have less than a second to take advantage before the market equalizes without them. Their margins are small but their trades are prolific. They siphon billions annually directly out of the market and into their own pockets.

We would all be better off if they would take up a more honest profession like used car salesman.

If I had to pick a more appropriate time quantum for trades out of the air, I'd have to say not less than 5 MINUTES. Setting a minimum hold time on a stock would also be a worthwhile consideration.

Re:Not just useless, but actually toxic.

[marcosdumay]

And the Arabic peninsule is the only mapped place where people don't use arabic numerals... That is funny, but there is something wrong with the map, shouldn't the extreme east be all red?

Re:Not just useless, but actually toxic.

[khallow]

There is no value to equalizing prices in less than a second, especially when the "equalizing" you are talking about is really just pocketing the difference.

And you claim this because? I tire of people who don't have a clue. Just because subsecond trading has no value to you doesn't mean it has no value to someone else. And obviously, very obviously it does have considerable value to someone else.

Re:Not just useless, but actually toxic.

[Americano]

So what am I missing here?

The entire broad context of the market itself.

High frequency traders in the market tend to lower the bid-ask spread for equities, meaning that the markets generally become more liquid (and thus less volatile) because you can take a position or sell a position more easily.

If everybody in the market is holding a position for long-term investment, think about what that does for equity prices: If there's a million shares of a company available, and nobody wants to buy? If there's a million buyers, and nobody's willing to sell because everybody in the market is a 'long term' trader?

Somebody who buys a stock with no intention of holding it for long term is essentially providing a 'loan' to the person they buy from, and relying on the fact that somebody else will buy the security they've just purchased for a slightly higher price than they bought it at to turn a profit on that 'loan'.

They also foster competition among brokerage houses, which reduces per-trade costs, which benefits everybody making a trade, including 'mom and pop' investors.

You're thinking in terms of "why does it matter if a single trade is delayed a few seconds," but the markets operates on hundreds of millions of trades from hundreds of millions of actors. Delay them all by a few seconds, or a minute, or hours, congratulations, you've just introduced HUGE inefficiency into the market, guaranteeing that the 'insiders' are the only ones who will ever make a profit, because they'll be the only ones with sufficient information to buy and sell well.

Re:Not just useless, but actually toxic.

[EdgeyEdgey]

How exactly does HFT provide liquidity?
I believe your post is correct if you are talking about market makers who have to accept all offers below a certain market size, but HFTs can withdraw their offer faster than an investor can accept.

I am more than willing to be wrong. Please correct me.

Re:Not just useless, but actually toxic.

[TheTurtlesMoves]

And that is *exactly* what the all the other traders are doing. Running computerized systems doing things like stuffing and canceling etc for their own gain. But why is that they are not sued? They are the sanctioned ones.

You can't have this much perfect trading without "working how the computerized systems would react to certain trading patterns" for their own gain.

But between this, and other exclusive rights (ie naked short selling etc) to "special" status traders that makes this and other parts of the financial sector a farce.

As for liquidity and spreed. 1 second trades would achieve both just as good as the current system as far as the over all market is concerned. Perhaps not for the 100us traders thou.

Re:Not just useless, but actually toxic.

[aliquis]

Sure they don't use them? I just got the impression they just used slightly different separators (and eventually "the right ones"?)

Re:Not just useless, but actually toxic.

[Americano]

I decide I want to sell 100 shares of MSFT today - right now - at a price of $25.27, and I intend to take my $2527 and invest it into ~8 shares of AAPL, trading at $310.25.

I put my shares out for sale... and... huh. I can't sell. Nobody's interested in buying right now, or I won't know if anybody is for a few hours because of some artificial latency introduced into the market. SO I wait.

And the offer comes back, somebody willing to buy my 100 shares for... $24 per share. Do I want to accept? Well, it's the only offer I've had to buy, so I guess I better take it.

So I take my $2400, and offer to buy shares of AAPL at 310.24. And I wait a few hours... and somebody's offered to sell me 7.5 shares at $320 per share.

In an 'efficient, liquid' market, I would have been able to make the sale of MSFT & AAPL immediately, and I would have lost less money in the process. HFTs are the ones buying my share of MSFT *right now*, expecting to turn around and sell those in a batch to someone else in about 20 minutes, for pennies (or fractions of pennies) of profit per share. I lose less money, they get a small amount of money per trade, and make it up in volume.

So, because there wasn't liquidity in the market, my $2527 worth of 100 MSFT shares turned into 7.5 shares of AAPL worth $2400, all because I couldn't find somebody willing to accept a trade quickly, so I could take advantage of the *current* market prices. Remember, bid/ask spreads aren't guaranteed to be "a penny or two."

Re:Not just useless, but actually toxic.

[Americano]

Running computerized systems doing things like stuffing and canceling etc for their own gain. But why is that they are not sued? They are the sanctioned ones.

Why don't you ask the Norwegian government, which is who went after the two traders you're up in arms over. The charges in Norway say that they didn't just "use the information to turn a profit in the market," it says they *manipulated stock prices* for their own gain.

As far as "perfect trading," one link provided is to a 'stock analyst' newsletter, and the other link shows that 'perfect quarters' don't seem to be as rare as you think they should be:

It was the first time Bank of America had a perfect quarter since acquiring Merrill Lynch in early 2009.

A JPMorgan spokesman said the last time the bank had a perfect run was the first quarter of 2003.

Goldman Sachs — which is fighting an S.E.C. suit claiming the bank defrauded customers on a complex mortgage investment — posted its first perfect quarter ever.

A Morgan Stanley spokesman said the firm’s last perfect run was the second quarter of 2007.

So 2 of them have had perfect runs in the last 12 quarters; a third in the last 7 years. Is it any wonder that they'd be very careful about their trading activities and oversight given the current regulatory climate and public opinion, and that that might result in them having a 'perfect quarter' at the beginning of 2010 in the midst of an economic recovery?

As for liquidity and spreed. 1 second trades would achieve both just as good as the current system as far as the over all market is concerned. Perhaps not for the 100us traders thou.

As for liquidity and spread, if you introduce an arbitrary 1 second latency for every one of the tens or hundreds of millions of orders crossing an exchange every day... what will that do for volume, volatility, and efficiency? Hint: it won't have a positive effect. And all it will do is lead to a pulse, every second, in which the big guys get their orders first in the execution queue, and mine are last - why? Because they'll still have the fastest connections, and the largest orders. Your proposal to add some arbitrary amount of latency solves nothing, and will not ever solve the problem you think it will.

Re:Not just useless, but actually toxic.

[EdgeyEdgey]

First, thanks for the reply.

I decide I want to sell 100 shares of MSFT today - right now - at a price of $25.27

A price is the point at which someone willing to buy. Lets say that normal market size (http://www.investopedia.com/terms/n/normalmarketsize.asp) is 10 shares, so we receive $252.70 and still have 90 shares. The market maker has to offer a price and has to accept that price. Repeat the process 10 times and you have sold all the shares, but the price gained for the remaining 90 shares is different to the initial quoted price. But this is just restating your post.

You describe how market makers create liquidity, but not how HFTs create liquidity.

In an 'efficient, liquid' market, I would have been able to make the sale of MSFT & AAPL immediately, and I would have lost less money in the process. HFTs are the ones buying my share of MSFT *right now*, expecting to turn around and sell those in a batch to someone else in about 20 minutes, for pennies (or fractions of pennies) of profit per share.

HFTs already know that they can sell on those shares. There is not waiting and they take no risk. They do not enter into trades that they cannot instantly get out of. Therefore they provide no liquidity.

Is this argument incorrect?

Re:Not just useless, but actually toxic.

[jbengt]

So, it sounds from the article like it had nothing to do with Windows and everything to do with a flimsy system structure.

It may have had nothing to do with MS Windows. But it did have something to do with a flimsy structure that MS helped build, with lots of .Net and C# goodness and integration with MS Ofiice. MS advertised it loud and clear at the time as an example of how MS Windows was chosen over Linux, how MS technology was well worth the small extra costs, and how MS could deliver robustness and speed that Linux couldn't.

Re:Not just useless, but actually toxic.

[jbengt]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of.

Since, for a lot of trading, the definition of value is what the market will pay, having "prices at any instant accurately relfect value" is the limit of a feedback loop gone mad, and that can easily lead to disaster. One of the tried and tested ways to prevent feedbackl loops from getting out of control is to slow down the rate of feedback until the damping effects can prevail.
Also, a lack of latency does not equal liquidity, at least not in the real world.

Re:Not just useless, but actually toxic.

[Americano]

You describe how market makers create liquidity, but not how HFT's create liquidity.

This presumes that market makers and HFTs are two mutually exclusive things, when in fact, they are not (or at least, not *necessarily*). Many HFTs are (officially or unofficially - depending on the exchange & their relationship to it), market makers as well. Even if they're not designated market makers, their presence helps reduce the spread that market makers are offering (they will buy and sell to take advantage of the spread, exerting pressure on the bid/ask prices to get closer and closer together), which makes investment more efficient for 'regular' traders.

As far as 'no risk' for HFTs, of course they have risk. They could lose money if a stock tanks just like anybody else, and they have to have a buyer or a seller in order to unload their positions. They are not (necessarily) required by the stock exchanges to stay in the market in bad conditions (designated market makers would be, unofficial ones wouldn't), such as during the "Flash Crash" back in May, but many of them stayed in anyway.

A report by the CME Group reported that high frequency traders had no discernible impact on the flash crash back in May: the report.

The SEC also found that they perhaps magnified the impact of the crash, but that many of them also stayed in the market... providing liquidity... throughout.

There *are* issues where HFT programs could (and should) be subject to more regulation - flash functionality (essentially, a 30ms-faster peek on large pending orders for people who pay a fee for it) is a big one, and has mostly been disallowed already by the exchanges I'm aware of. Use of dark pools is another one that could probably benefit from increased oversight.

Essentially, the trading volumes created by HFT (estimates I've seen range from 50-70% of market activity) help to smooth out the big swings which can be caused by large trades - they act as a damping mechanism on the volatility of the market, and several studies (here's one from the Journal of Finance) have found that algorithmic / HF trading has a net-beneficial effect on the market.

Re:Not just useless, but actually toxic.

[Tom]

For an investor with much more at stake, it is easy enough to see how a one-second interval could be make-or-break.

Seriously? If a one-second price difference can make or break an investor, don't you think that is ample proof that the markets are way, way too volatile? There is nothing left of real price-finding if these differences can happen in those times. The real world doesn't change that fast, and last I checked investments are about real-world things (companies, goods, etc.)

Re:Not just useless, but actually toxic.

[Tom]

Good points there, except for a few details:

True, the high-speed traders only make fractions of a cent per share - but they make it ten, twenty or a hundred times a day, and they are often high-volume traders, too (an algorithmic trading company a friend of mine works at once held 10% of a major industrial corporation - for a few minutes).

Two, I'm not invested in any public companies. I used to work at the stock exchange, I know better than to put my future into what is basically a casino. If you want to get rich at the casino, you have to be the casino.

Re:Not just useless, but actually toxic.

[Americano]

After the big guys have made their money, are 150ms trades really any benefit to the rest of us compared to 250ms trades, 400ms trades, 5s trades?

Numerous analyses have indicated that yes, there is really a benefit to the rest of us, in reduced trade prices, smaller bid/ask spreads, lower market volatility, increased liquidity, and so on. These are economic analyses of the actual market data, not just knee-jerk "THEY'RE RICH FROM THE STOCK MARKET, therefore they must have done something evil" pronouncements on slashdot.

This is not to say that high-frequency trading requires no additional or no different oversight, or that we should stop enforcing our existing regulations - but 'high frequency trading' is not the devil people are making it out to be.

Everyone else claims that that's a smokescreen for some exclusive protected profit;

Some lines of business require a larger initial investment to get started in. Does the company that contracts to build a skyscraper do so at the expense of the contractor who builds a single-family house? Or do they both benefit from each others' activities in the market?

The existence of construction firms specializing in large projects - skyscrapers, dams, power plants, etc., does not do so as part of a nefarious scheme to keep the single-family ranch house contractor poor. The large company is better capitalized, has the necessary staffing, equipment, funding, and education to take on these tasks, and as such, reaps a profit on the work it is qualified for. Why should the stock market be any different?

High frequency traders are often well-funded with lots of capital, which gets circulated in the market creating benefits for everybody else, and creating a profit for the person injecting that capital. Legitimate barriers to entry (the need for a large amount of seed capital to get started) is not an intrinsic evil of any industry. If the government started saying "ONLY 4 large banks can engage in high-frequency trading," I'd agree with your pronouncement that they are getting some sort of 'protected' market.

But that's not the case. If you have a 20 million dollars or so to invest in developing high-frequency trade algorithms and technology, you could go ahead and enter that market, too, and you'd probably make a good profit at it.

Steep barriers to entry are not 'evil' - they are merely too high for someone without significant capital to invest, which means that only some people are able to take advantage of the market opportunity. If everybody in the building trades was restricted from building skyscrapers because not everybody could afford to start Modern Continental, that would not be a 'victory' for the small-scale contractor any more so than eliminating high-frequency trading would be a 'victory' for the small investor.

Re:Not just useless, but actually toxic.

[socceroos]

Aaah, right. It all makes sense now. There was a network outage so they decided to dump Windows and .NET. Yeah, network outages - dump Windows.

Drinking the marketing spin kool-aid?

Re:Not just useless, but actually toxic.

[EdgeyEdgey]

- flash functionality (essentially, a 30ms-faster peek on large pending orders for people who pay a fee for it) is a big one, and has mostly been disallowed already by the exchanges I'm aware of.

Yep, fortune-telling is my main bugbear, as well as the ability to instantly remove an offer. Confidence in the market is damaged if all investors do not have the same access.

I admit that HFT algos are not all bad. They do help institutions make large trades whist minimising market movements, so aiding this sort of liquidity.

Thanks for the detailed reply and all the links. The forthcoming paper will be an interesting read.

Re:Not just useless, but actually toxic.

[NoSig]

If HQTers only buy goods to sell in less than a second, then there better (for them) be someone to sell it to a second later - and even at a higher price. So if I'm a human making trades I'll hardly even be able to notice the difference in speed of finding a buyer. All that will have happened is that I'll be getting a lower price. As I understand what you wrote, you are saying that if all trades are sped up by less than a second, then this provides information to all market participants that would otherwise only be available to "insiders" - and that this information will never be available on a timely basis without HFT. Unless you are referring to insider trading, I don't know who "insiders" refer to, but more importantly I don't see how HFT provides information to anyone, since HFT is necessarily automated trading and so this trading isn't based on human understanding of the world and it isn't based on pertinent information about the goods being traded that is withheld from everyone except HFTers. What am I missing?

Re:Not just useless, but actually toxic.

[NoSig]

Yes, it very obviously does have considerable value to the people making money from engaging in it. Other than that using the word "obvious" doesn't stand in the place of an actual argument.

Re:Not just useless, but actually toxic.

[Chowderbags]

It's people making lots of money in ways that are completely inaccessible to the rest of us. Even the people making the algorithms to make the trades aren't really paid most of the money that's made. I really doubt that there's a significant gain in liquidity by allowing sub-second trades (at least, liquidity in any sense that matters for the average investor), and flash trading is siphoning off money while doing no work. But maybe I happen to think that investing actually means holding onto something for more than a fraction of a second and that the value of a company cannot swing noticeable amounts from one second to the next for no apparent reason.

Re:Not just useless, but actually toxic.

[Americano]

It's people making lots of money in ways that are completely inaccessible to the rest of us.

As I asked in an earlier follow-up in this thread: should nobody be allowed to build skyscrapers, because the tools, techniques, and capital required are 'inaccessible to the rest of us'? Numerous mathematical studies of the market (I linked one of them earlier as well) show that HFTs have a net-positive effect on the market, including (but not necessarily limited to) increasing liquidity, damping volatility, and reducing per-trade costs by fostering competition among brokerages.

You can doubt that's the case, but until you can point me to your analyses published in a reputable financial journal, I'm inclined to believe the guys with the economics degrees and publications.

But maybe I happen to think that investing actually means holding onto something for more than a fraction of a second and that the value of a company cannot swing noticeable amounts from one second to the next for no apparent reason.

What's interesting is, again, that several studies have concluded that HFTs don't have any appreciable long-term impact on a stock's 'actual' price for the long-term investors, and that the existence of HFTs in a security acts as a damping mechanism to reduce volatility: when an HF algorithm notices a change, it's likely that it'll be moving quickly to take advantage of that change, which tends to drive the price back towards the 'real' value - think of it as constant minor course corrections, rather than a couple giant swerves & recoveries.

As I've also said elsewhere in this thread - none of this says that there is NO room for sensible regulation & oversight of high-frequency trading; however, the knee-jerk view of it here seems to be that it's "money for nothing, and so somebody is taking advantage of someone else and cheating the system," and that's simply not the case. HFT is not the unmitigated evil that most people here seem to think it is, and if you look at studies done on actual market performance and market data, they generally conclude that these activities have a net-positive (or, at worst, net-neutral) effect on other (long-term) investors in the market.

Re:Not just useless, but actually toxic.

[hughbar]

People like derivatives traders take that to the purest level, trading fairly abstract things, but still things with an attachment, however tenuous, to the real world.

That's exactly one of my major points, 'abstract things' and [I would say, because I'm arguing on that side of this] no connection to changes in value in the real world, within the 126 microsecond slice.

I agree completely about leverage and hence about fractional reserve and the major of money being issued as debt. This whole thing is due for an major re-think, it's part of what's destroying us.

Re:Not just useless, but actually toxic.

[khallow]

Yes, it very obviously does have considerable value to the people making money from engaging in it. Other than that using the word "obvious" doesn't stand in the place of an actual argument.

Since you agree with me, it doesn't matter if my argument was "actual" or not, though that in itself would be strong evidence that it was an actual argument. And my emphasis on the obviousness of the argument is because it really should have been obvious to you.

Re:Not just useless, but actually toxic.

[khallow]

That's exactly one of my major points, 'abstract things' and [I would say, because I'm arguing on that side of this] no connection to changes in value in the real world, within the 126 microsecond slice.

I suppose I have different interests in the matter. For example, I've worked for somewhere around 10 years on problems far more abstract than derivatives. My view is that something isn't useless just because it is abstract. You do have to have a way to connect it to the real world, but once you do, abstract tools can often be powerful, allowing you to do things that you couldn't do before. In the case of derivatives, they have the power to hedge against arbitrarily defined sorts of risk.

A common alternative to such things is hope. That is, you hope the bad thing doesn't happen, because you have no way of preparing for it, if it does.

I agree completely about leverage and hence about fractional reserve and the major of money being issued as debt. This whole thing is due for an major re-think, it's part of what's destroying us.

I can't disagree, but I think a lot of the problem is in degree rather than kind. The US debt wouldn't be a serious issue if it were a factor of ten smaller. Same goes for private debt. And of course, the same goes for leverage. Anything is crazy at 50 to 1 leverage. Some things aren't crazy at 5 to 1 leverage (though a lot of stuff remains so).

Re:Not just useless, but actually toxic.

[NoSig]

I don't think anyone has ever claimed that HFT is bad because the high frequency traders themselves don't make money. That's not the issue.

Re:Not just useless, but actually toxic.

[glodime]

The study you linked to does not seem to be as supportive of the net benefit of Algorithmic or High Frequency Trading as you implied. The paper merely confirms that such trading increases liquidity in rising markets over a short period (less than 5 years). The paper does not indicate the value of the additional liquidity provided and does not compare that value to the costs of providing the liquidity. The paper does not consider in depth what those costs may be.

On page 34 the authors indicate a limitation of there study and offer a cursory glance at one negative effect of algorithmic trading:

...it remains an open question whether algorithmic trading and algorithmic liquidity supply are equally beneficial in more turbulent or declining markets.

...the decline in depth has hampered the ability to trade large amounts without substantial costs. ...innovations can offset some of these effects. For instance, some "dark pools" such as LiquidNet and Pipeline represent a modern version of an upstairs market, allowing traders with large orders to electronically search for counter-
parties without revealing their trading interest.

Re:Not just useless, but actually toxic.

[Americano]

Yes, that single paper does say that their analysis is a fairly short period of rising prices. But it then goes on to say:

Like Nasdaq market
makers refusing to answer their phones during the 1987 stock market crash, algorithmic
liquidity suppliers may simply turn off their machines when markets spike downward.

However if you read the other article I linked about the May 2010 "Flash Crash," you'll see that the SEC also concluded that some of them remained quite active in trading throughout the decline - lending some support to the notion that they have positive effects in the "more turbulent or declining" markets discussed by the Journal of Finance article.

The takeaway here is, once again, not that HFT is "unambiguously good and needs no regulation or oversight." Neither is it true that it is unambiguously evil, and in need of complete banning, which is more or less what is being suggested by many people throughout this thread.

Re:Not just useless, but actually toxic.

[afidel]

Actually, I think the modern stock market does a VERY poor job of establishing the accurate value of a company, instead it accurately reflects the instantaneous value of the equity security. While at first blush the two are the same thing, they are in fact very different. It is the absolute focus on the security price that has ruined so many once great companies. CEO's and other executives are no longer incentivized or in many cases even allowed to look beyond the current quarter to build long term stockholder value, because of the focus of the large money holders on the instantaneous return from things like HFT's the focus on the core value of the company gets disconnected from the equity price.

Re:Not just useless, but actually toxic.

[khallow]

I don't think anyone has ever claimed that HFT is bad because the high frequency traders themselves don't make money. That's not the issue.

My point here is that someone else thinks HFT has considerable value. The label is pretty broad and includes some stuff that can even be consider DoS attacks on fellow traders. But even if you get past the gaming of the system, you still have value in HFT just from providing liquidity on the time scale of milliseconds and faster.

Re:Not just useless, but actually toxic.

[NoSig]

I'm willing to consider that. It seems to me that the extra liquidity on the timescale of milliseconds can be useful with high probability only to someone who is himself doing HFT. So no service is rendered to the rest of the market participants. Is that wrong?

Re:Not just useless, but actually toxic.

[Gothmolly]

Why not one trade per two seconds? Since we're already in your arbitrary fantasy land, why not one trade per day? Would that make you feel better?

Re:Not just useless, but actually toxic.

[khallow]

I'm willing to consider that. It seems to me that the extra liquidity on the timescale of milliseconds can be useful with high probability only to someone who is himself doing HFT. So no service is rendered to the rest of the market participants. Is that wrong?

It's pretty accurate. I would point out that there is coupling between nearby timescales. Someone who makes liquidity on the order of milliseconds is helpful to someone who makes transactions on the time scale of seconds. They in turn are useful to people who make transactions on longer time scales. If I make decisions on the time scale of months, it's very unlikely that HFT will have any direct use to me. But HFT would help the fauna that exists at faster time scales than me. That means a bit more economic activity and wealth overall, something that would help me.

Re:Not just useless, but actually toxic.

[NoSig]

That means a bit more economic activity and wealth overall, something that would help me.

Well that's interesting. If you buy to hold for a few minutes or more, HFTers won't be of any help to you, it seems to me. Economic activity does not necessarily imply growth. I see HFT as a whole to be extracting wealth from the economy without growing the economy or otherwise contributing something substantially useful. In the mean time funds and human capital are being allocated to carry out HFT, thus making those resources unavailable for useful pursuits. If that is true then it seems useful to change the rules of the game to make HFT uneconomical. Do you believe that would be a bad idea?

Re:Not just useless, but actually toxic.

[khallow]

That means a bit more economic activity and wealth overall, something that would help me.

Well that's interesting. If you buy to hold for a few minutes or more, HFTers won't be of any help to you, it seems to me. Economic activity does not necessarily imply growth

The other claim, more wealth does imply growth. The point about higher economic activity is simply that it provides more opportunities for anyone who trades to make trades. Even if you're not bringing in more income to your particular activity, a higher frequency does tend to smooth out the bumps and make things more predictable.

I see HFT as a whole to be extracting wealth from the economy without growing the economy or otherwise contributing something substantially useful. In the mean time funds and human capital are being allocated to carry out HFT, thus making those resources unavailable for useful pursuits. If that is true then it seems useful to change the rules of the game to make HFT uneconomical. Do you believe that would be a bad idea?

As I note, currently the label HFT includes some relatively parasitic and destructive trading strategies such as creating a pile of phantom bids to either probe other traders' software triggers or even to fill the market space with noise, effectively acting as a DoS attack on other HFT traders.

These can be ruthlessly dealt with by making bid placement cost money (obviously this is a "change the rules" idea but one with a bit of nuance). For example, if placing a bid cost a penny, then placing thousands of bids a second (or much more!) costs at least tens of dollars per second. The strategy might still work, but now every $10 per second of cost means almost $300,000 lost per day. If someone tried to play that game over the 5,000 stocks of the Wilshire 5,000 (virtually all capitalization of publicly traded stocks), then that'd be at least hundreds of millions of dollars lost per day, that they'd have to make up somehow.

It's also worth noting that the primary targets of such opportunistic HFT are other HFT traders. But stop orders are a fairly predictable HFT phenomena. That's one reason I never use them.

There are other forms of HFT parasitism, particularly fronting. If I know you're going through with an order that can trade at a price further from the market value than current bids or asks, then I can quickly buy out small orders ahead of your trade and list them somewhat greater to complete your trade, making effectively risk free, profitable transactions. Large market orders (sell directly to current market price) are particularly vulnerable to this trick. That's one of the reasons I never make market orders and would be reluctant to place large orders.

The point here is that there are negative features to HFT, but these are usually associated with exploitable, bad trading practices, such as flawed computer trade programs, or using mechanisms that give you less control over your market actions such as market orders and stops.

So what are the advantages of HFT? It's pretty much already been discussed. It adds liquidity and market responsiveness to fast time scales and the income from such activities adds economic activity and wealth to the overall economy.

And recall that every trade on the market is between parties that willingly entered into the trade (or willingly took on obligations that forced them to make the trade). This happens even at the HFT level. Trade by its nature increases wealth overall and enables the traders to do things they couldn't otherwise do.

Given it's relative isolation from the realm of ordinary investors, I really don't see a big need to regulate out the teeth, the aggressive and ruthless nature of this particular market space. Indeed, I think that nature would help reduce problems at this level by quickly taking wealth from incompetent HFT players.

Re:Not just useless, but actually toxic.

[BillAtHRST]

Actually, it's not so much about the spread as it is about the size. The decimalization of the markets in 2001 had the effect of fragmenting liquidity, by allowing arbitragers to "ping" the market with orders with very little risk. (Google "SOES bandit" for more info).

Also, 25-cent spreads (a "quarter") had all but disappeared by that time, except for very thinly-traded issues. The average spread was a "steenth" (6.25 cents) back in those days, and in retrospect that wasn't so bad. It left enough room for both buying and selling broker to make a little money while giving the customer "price improvement" -- i.e., a better-than-quoted price -- esp. for large trades.

The SEC's rataionale for moving to decimal pricing was apparently to help the "little guy", but it missed the point that most "little guys" have their money invested with big mutual funds, etc. Decimalization actually hurt them (and their little-guy customers) by fragmenting liquidity to the point that it became very difficult to execute large orders without "price impact" (which is when arbitragers find out that a big fund is buying or selling and try to jump ahead).

At the time I was consulting with the 2nd largest "wholesale" broker on Nasdaq, and they shut down the business because they could not make money at sub-penny spreads. Fewer market-makers means less liquidity which leads to situations like the May "flash crash" where so-called "market-makers" run for cover by posting "stub quotes", rather than "maintaining an orderly market" as specialists on traditional exchanges are required to do.

Yes, specialists and market-makers made good money back in the old days, but for the most part, they earned it by risking their own capital. (And by having a vested interest in the overall health of the markets). When all is considered, it seems that "we" (as in society) may be paying a much higher price today.

Re:Not just useless, but actually toxic.

[NoSig]

And recall that every trade on the market is between parties that willingly entered into the trade (or willingly took on obligations that forced them to make the trade). This happens even at the HFT level. Trade by its nature increases wealth overall and enables the traders to do things they couldn't otherwise do.

Trade increases wealth by allocating services, goods and other resources in a more beneficial way than if there were no trade. Not all trade accomplishes that. If the two of us swap our possessions twice, we haven't really accomplished anything - worse, we've wasted our time and so actually decreased total wealth. I can only see that HFT decreases wealth, by misallocating funds to the HFT activity without there being any substantial global benefit to that and, as you point out, there certainly are downsides even ignoring the misallocation of funds.

The main point of my problem with HFT is that there is no sense in the guy with the slightly faster internet connection or computer algorithm having an advantage. It produces competition for faster speeds that has no advantage, unlike e.g. the competition to allocate funds to the best investments. On top of that, this useless speed competition extracts money from the rest of the market, e.g. in the ways you point out.

Insider trading is illegal, yet it too is between parties that willingly enter into the trade.

Re:Not just useless, but actually toxic.

[khallow]

If the two of us swap our possessions twice, we haven't really accomplished anything - worse, we've wasted our time and so actually decreased total wealth.

Why do you say that? You've never changed your mind and returned something? Or buy something that didn't work out?

I can only see that HFT decreases wealth, by misallocating funds to the HFT activity without there being any substantial global benefit to that and, as you point out, there certainly are downsides even ignoring the misallocation of funds.

All I can say is that I see things differently and I've explained my point of view. And the fact that HFT is a growing business indicates to me that there is substantial global benefit to it whether you see it or not.

Re:Not just useless, but actually toxic.

[khallow]

The main point of my problem with HFT is that there is no sense in the guy with the slightly faster internet connection or computer algorithm having an advantage. It produces competition for faster speeds that has no advantage, unlike e.g. the competition to allocate funds to the best investments. On top of that, this useless speed competition extracts money from the rest of the market, e.g. in the ways you point out.

Insider trading is illegal, yet it too is between parties that willingly enter into the trade.

Sorry, I just had to reply once again. This speed is useless to you. That's all. It's obviously not useless to some other people. I doubt you're interested in pokemon trading cards or sculptures made of varnished moose poop either. But people buy them, meaning they have value to someone.

And I'll add, we may well find that complex, automated decision making at the microsecond level has value far beyond the HFT marketplace.

I remain in favor of HFT because I think it had considerable net benefit to society despite the flaws and because I'm interested in where the arms race ends up taking us.

Re:Not just useless, but actually toxic.

[NoSig]

And the fact that HFT is a growing business indicates to me that there is substantial global benefit to it whether you see it or not.

I don't understand, wouldn't that just mean that it is profitable? Not that HFT is the same as stealing, but if stealing is a growing activity, I don't think we can infer from that that stealing has a global benefit, though it may have a local benefit to the thieves.

Re:Not just useless, but actually toxic.

[NoSig]

It's not that it is useless to me. My point would remain even if I was personally profiting from HFT, as indeed I may be indirectly doing if my bank or pension fund is engaging in it. My complaint is that a market that makes HFT economical is not as useful to society as one that does not - just as for insider trading. The difference is that insider trading has to be illegal to make it a bad prospect, because it is hard to detect, while it is not necessary to make HFT illegal to make it uneconomical. Your suggestion of a tiny tax per transaction is a good start. There is no relation to whether or not HFT is useful to me personally.

Re:Not just useless, but actually toxic.

[tukang]

One of the few levelheaded responses I've read/heard about the economic meltdowns in the past few years.

Software Only?

[ticklish2day]

Is this improvement purely because of the change in software technology or were there simultaneous infrastructure and process modifications? The article doesn't really say.

Re:Software Only?

[Anpheus]

It's my understanding that it's also all new hardware.

Re:Software Only?

[Anpheus]

I don't know if it solved all of their problems, perhaps they were also unhappy with the .NET system in other ways?

Re:Software Only?

[KZigurs]

Itanium was being phased out and isn't really well supported by latest centOS kernels*.

*Joking, joking. Regardless, whatever they had couple of years ago probably was overdue to be replaced anyway. And these kinds of latencies suggest that they are using something a bit more interesting than Client For Microsoft Networks over poorly terminated thinwire ethernet in their datacenter.

Anonymous Coward.

[Anonymous Coward]

Your post was fast... are you using Linux?

Linux: 1, MS: -1

[ThePromenader]

Not so sure that the networking speed is due only to the Linux OS switchover - The LSE obviously updated its hardware too.

I'd be really be interested in the makeup of the LSE network support - do they rely on their own developers/deployers, or do they have a support deal with Linux?

Re:Linux: 1, MS: -1

[RightSaidFred99]

Yeah, probably to people who don't know a lot about software you've summarized it.

For the rest of us, we know that architecture matters more than platform, and that barring extremes of performance requirements it also matters more than native code versus non-native code.

Since this is an extreme range of performance (in terms of latency), .NET was a poor choice - Java running on Linux with the exact same architecture would have performed as badly or worse.

Fanboys will pretend this means something in the Linux versus Microsoft debate, when in fact it means absolutely nothing other than that whoever implemented their last version was retarded and delivered a horrible product.

Re:Linux: 1, MS: -1

[LingNoi]

I found this further into some other links about it.

At the time the LSE maintained that TradElect was not responsible for the outage, but has since, nevertheless, made aggressive steps to replace the platform by acquiring trading firm MillenniumIT, the supplier of its new system.

I'd post more info but slashdot has broken copy and paste.

Re:Linux: 1, MS: -1

[Anonymous Coward]

It was costing them so much to maintain their systems due to support and modification contracts that they just out and out bought an ENTIRE company whose sole product was...trading systems (For about 50 million'ish pounds IIRC).

In essence they bought a development department lock stock and barrel and it was STILL cheaper than their existing setup.

Re:Linux: 1, MS: -1

[Billly Gates]

I also wonder what platform as well. Java vs Php and the database software could make it or break it regardless of OS.

Re:Linux: 1, MS: -1

[Sean Hederman]

The software was written by Accenture with assistance from Microsoft, so that would tell you all you need to know.

Re:Linux: 1, MS: -1

[JohnFluxx]

If you call Microsoft fanboys, then sure. Microsoft did a huge ad campaign about how the LSE switched from Linux to Windows, proving that Microsoft was ready for the big time.

Re:Linux: 1, MS: -1

[RightSaidFred99]

That tells me nothing.

Nobody clearly pointed out what the problem was with the original solution. It crashed, so what. Generally software crashing is because the application developer created a bug or there was an unforeseen event. This has nothing to do with Microsoft Versus Linux.

Now this new system, on new hardware, probably running natively compiled code and highly performance optimized runs faster than a .NET solution. Wow! Quick, call the press!

You're jumbling together a bunch of different dimensions of a solution and lumping them all into "Durr, Linux > Microsoft, this proves it!!".

Anybody with a lick of sense realizes how retarded your crowing over this is.

Re:Linux: 1, MS: -1

[Sean Hederman]

Okay, well to anyone who's ever had to work with Accenture code, it would tell them a lot.

I agree wholeheartedly that it has nothing to do with MS vs Linux, I think it has to do with another shoddy Accenture implementation. Even the .NET decision has nothing to do with it IMO, I'm a firm believer that algorithms and design have far more impact than OS or language choice.

Oh, and calm the hell down. It's a discussion, not a flamewar.

Re:Linux: 1, MS: -1

[Anonymous Coward]

Not true the original software being replaced was, the new millennium software originates from malasia

Re:Linux: 1, MS: -1

[flyingfsck]

Well, obviously the LSE wanted a real-time system and Accenture and Microsoft used .NET, which was a total failure on their part. You cannot do real-time with .NET - Idjits...

Then on top of being dog slow, it fell over, costing the LSE a ton of money. So they probably implemented it with an Access DB and Exchange mail server as well.

So, MS touted this as a major win and then fell on their asses.
1. Euphoria:
http://web.archive.org/web/20080303191622/www.microsoft.com/casestudies/casestudy.aspx?casestudyid=51828

2. Reality:
http://blogs.computerworld.com/london_stock_exchange_suffers_net_crash

3. Tux to the rescue:
http://www.computerworlduk.com/news/networking/3244936/london-stock-exchange-smashes-world-record-trade-speed-with-linux%22%22

4. The dead cat bounce?
http://moneycentral.msn.com/investor/charts/chartdl.aspx?symbol=MSFT&CP=0&PT=11

Re:Linux: 1, MS: -1

[Blakey Rat]

Knowing Accenture, I'm guessing the stupid .net decision was theirs, and Microsoft's involvement was more along the lines of, "they want to use .net for THIS? Fuck... well let's see what we can do with it!"

Has anybody been involved with a *successful* Accenture project? How are those guys still in business?

Re:Linux: 1, MS: -1

[KnownIssues]

http://www.computerworlduk.com/news/it-business/3242509/london-stock-exchange-hit-by-glitch-after-linux-launch/

Granted, it's only one hour downtime, not eight. I'm not going to defend .NET here, it's obviously much hated by the /. community, but you're already seeing that Linux isn't going to make this a rock solid platform simply by virtue of being Linux and/or open source. Linux, .NET, Java are all just platforms and they are all flexible enough to do whatever you're trying to do and all complicated enough to screw up when even very intelligent minds are at work.

Re:Linux: 1, MS: -1

[jsailor]

While I can't speak to any specific exchange directly, I can say that their designs are fairly similar and fairly simple.
Fast links provide low serialization delay (the time required to put bits on the wire). A 512 byte packet requires 4.256 microsec at 1 Gbps and .4256 microsecond at 10 Gbps (40 or 100 Gbps are not linear jumps)
Fast switches forward and sometimes filter traffic. the faster equipment requires between .7 and 6 microseconds to begin spitting out packets. think about systems that start forwarding traffic prior to the entire packet being received. These are Ethernet values. Infiniband switch can be 2+ times faster.
The network design is very simple. Not your traditional vendor-derived three tier architectures, but 1, 1.5, or 2 tier architectures. The largest variations are in the handling on security and market data distribution. Firewalls are slow and typically deprecated. Market data is a religious decision marred by questionable technology implementations
Utilization is typically below 1% for any reasonable measurement period. Bursts may push an interface to line rate fractions of a second. Keep in mind that 10 GigE can carry 14.8 Mpps - so these bursts stress switch buffers and the receiving systems.
The total network latency within an exchange's network is well below 100 microseconds with some offering delays below 50 microseconds and a 1 or 2 that are below 30 microseconds.
The largest contributor to delay is, and will be for some time, the matching engines and other hosts that process the trades.

Re:Linux: 1, MS: -1

[h4rm0ny]

That tells me nothing.

Actually, though I understand and agree with your general point, I suspect you've never had any major dealings with Accenture. Microsoft can be good or bad (though with something as lucrative and high-profile as this, they're generally pretty good), but Accenture... Print out every Dilbert comic you've ever read, mush it into a great big soggy pulp and squeeze the distillated management greed and stupidity out into a barrel. Add a shot of highly toxic mercury for the Hell of it. This is what Accenture board members have in their veins instead of blood.

I think you saw that mention of Microsoft and thought: "another Linux cheerleader". Yep, fair enough. But to those of us familiar with Accenture, his post makes a lot of sense. No company in the world does bloat and overrun like Accenture.

Well, maybe EDS.

I love this "Ad" by Microsoft:

[ClarkMills]

http://www.youtube.com/watch?v=BwSM55bsCrM

I could watch it over & over... It puts a smile on my face... :)

http://news.cnet.com/8301-13846_3-10036286-62.html

Cheers... Clark

Communist Linux

[Anonymous Coward]

Communist Linux - bringing speed and efficiency to the dregs of capitalism.

Re:Communist Linux

[Anonymous Coward]

As long as they don't run any large file transfers in the background, har har.

Re:Communist Linux

[shutdown -p now]

It all goes as planned, comrade. Remember: the faster the capitalists trade, the quicker they will sell us the rope with which we will hang them! ~

Re:Communist Linux

[definate]

LOL It's hardly communist. Produced by a community, and provided for free, does not define ANYTHING as communist. It's as communist as much as Obama providing socialized medicine is communist. (Just for clarity, it isn't)

Re:Communist Linux

[clang_jangle]

Ever had to copy large files with Windows Explorer? Linux may have an IO issue compared to FreeBSD (for instance), but compared to Windows it totally rocks.

Re:Communist Linux

[KnownIssues]

The beauty of capatilism is you sell yourself the rope with which to hang yourself.

Re:Communist Linux

[haruchai]

Are you talking about copying from one local disk to another or to or from a network share?
Anyway, there are a few Windows utilities that speed up file copying - I've tried a few and the improvement was remarkable.
I don't know why M$ can't fix this for explorer.exe

Here's a good free one (FastCopy): http://ipmsg.org/tools/fastcopy.html.en
It's not very pretty but it sure gets the job done.

Re:Communist Linux

[KZigurs]

Ah, but do you demand payment upfront or extend credit?

Gee

[TubeSteak]

The record breaking times were measured on the LSE's Turquoise smaller dark pool trading venue, where trades are conducted anonymously.

Dark pools are part of the problem.
Transparency is critical for a functional marketplace.
Dark pools only require trades to be listed after the fact...
Which isn't as useful as it sounds, even if brokerages weren't completing the trades in/across-house, where disclosure is not required.

Anonymity and secrecy are anathema to a functional market

Re:Gee

[Anonymous Coward]

Well not quite. Dark pools still generate a price after execution. Most non dark order books are anonymous anyway. The main difference is that you don't see the order depth. You put it what you see as a fair price, if it excutes great. The real motivation for Dark pools was to add more stability to execution when large positions needed to be cleared. A large position could skew price. Dark Pools allow you to reduce your exposure risk to a certain extent. However clever alogols these days can sniff this out. And thats when the trouble starts.

PR Rubbish

[abigsmurf]

It may be that a Linux network is able to conduct trades faster than a windows one. I'd be willing to be though that most of this dramatic speed increase is down to new hardware. The windows network the LSE was on was what... 4+ years old?

Re:PR Rubbish

[Sean Hederman]

I think it has little to do with the Linux vs Windows, and more to do with badly designed and implemented software.

Re:PR Rubbish

[Robert Zenz]

...and more to do with badly designed and implemented software.

So we are back to Linux vs. Windows, huh? ;)

Re:PR Rubbish

[Anonymous Coward]

The hardware the LSE used has been updated continuously because of the problems with the windows implementation. They've been adding hardware for years before admitting the windows software was just not adequate (IIRC, the same day they announced they where switching to Linux, they also announced a new hardware upgrade to try to make the windows system suck less in the period between the announce and the switch).

In fact, the record is probably due to the fact the hardware platform was grossly over-dimentionned during the windows era, and is now more than adequate for the Linux system.

Re:PR Rubbish

[X.25]

It may be that a Linux network is able to conduct trades faster than a windows one. I'd be willing to be though that most of this dramatic speed increase is down to new hardware. The windows network the LSE was on was what... 4+ years old?

It is obvious that you've been following this, and that you are aware of the history of the project.

Ah, you have no idea what you're talking about? Never mind, don't let that stop you from making yourself look like a retard.

(Hint: Microsoft/Accenture implementation was rolled out on the newest hardware at the time, and was slower than pretty much everyone else, when project went live)

Re:PR Rubbish

[abigsmurf]

Nice of you to result to personal insults without even supplying so much as a single citation.

Whether or not the MS implementation was slow or fast at the time is utterly irrelevant. The article is comparing benchmarks for a 4 year old network to a brand new one which almost certainly has new hardware.

It's the equivalent of saying "Microsoft make the best consoles because the Xbox 360 outperforms the PS2!".

Hip hip hooray for Wine

[SpaghettiPattern]

Hip hip hooray for Wine!

I'm sure they took the complete system as it used to run on Windows and then simply ran it on Linux using Wine, thus allowing for great comparison.

Or maybe they completely redesigned the system, avoided known bottle necks and halved the response time.

I'm a UNIX developer since many, many moons but I find these stories lacking detail highly disturbing. Sure, nice Linux PR but on /. I'd hope to get a bit more context information.

Re:Hip hip hooray for Wine

[antifoidulus]

Seriously? You seriously think they would use Wine for a real time system? Are you out of your mind? Much more likely what they did is they ported over a lot of their code to run on a heavily modified real-time Linux(whose scheduler is a hell of a lot different than the one used by desktop distros). If you know what you are doing writing portable C isn't all that hard.

Re:Hip hip hooray for Wine

[SpaghettiPattern]

Seriously? You seriously think they would use Wine for a real time system? Are you out of your mind?

You should read past the first line. Of course I don't.

If you know what you are doing writing portable C isn't all that hard.

It isn't that hard but not straight forward. Study widely distributed software written in C and you'll see. The Perl make process heavily scrutinises your system before compiling anything. Not particularly for the faint-hearted.

Have you ever developed for a financial institution? Decisions take ages, the result must be there pronto and you will never ever be allowed to support more platforms than used. Want to write a fancy configure.sh which intermediates programmers understand and newbies not? Use stuff that has been around since the mid-80s? Forget it.

My take is they engineered the system and that if they would have done a sane design it surely would have run even under Windows. Note that I say this against my personal preference and interests.

It wouldn't surprise me if the promoting force was the utterly unreliable system they had before and that hence budget was freed.

Re:Hip hip hooray for Wine

[antifoidulus]

It isn't that hard but not straight forward. Study widely distributed software written in C and you'll see. The Perl make process heavily scrutinises your system before compiling anything. Not particularly for the faint-hearted.

Actually I have, used the EXACT same code on an iPhone, a mac and Linux and it worked fine. While writing portable C is easier, it's obviously even easier to write non-portable C. I kind of liken it to the old arcade game Dragons Lair. When you first start you are going to die a lot because you don't know the way forward. However with a little practice(and a lot of quarters) it becomes almost second nature and you can usually breeze through (though there is still the chance for an occasional slip up)

Re:Hip hip hooray for Wine

[KZigurs]

LSE is NOT a real time system (in the definition of guaranteed execution time for given volume) therefore there is nothing that would prevent it from being run from wine indeed ;) Even the record is the best case scenario, they were quoting approx 2x-3x that as a median.

(obviously that doesn't mean that the systems haven't been writen with care avoiding allocations in critical loops, probably using rolling buffers liberaly and running it on a modified kernel with BKL finally removed and yummy predictable O(1) scheduler developed in-house.)

trading time

[MSG]

trading speeds of several hundred microseconds

Several hundred? All of the stories I've seen in the past about TradElect indicated that trades were measured in the milliseconds... The new system seems to be about twenty times faster.

Re:trading time

[MSG]

But no one in the real world talks about hundreds of hundreds. Sixteen thousand microseconds would be the common verbal representation of the number.

The year of Linux desktop...

[mathfeel]

1. Create a super fast OS that take over the world's stock trading
2. Ruin the economy so that no one can afford to buy proprietary product any more
3. ...
4. Linux on every desktop!!!

Re:The year of Linux desktop...

[networkconsultant]

damn all i have is underpants.....

Re:The year of Linux desktop...

[fotoguzzi]

5. Come up with a consistent way to cut and paste text.

Light

[pgn674]

Distance light travels in 126 microseconds: 23 miles, or 38 kilometers. That's fast.

TFA isn't detailed on what exactly takes 126 microseconds, calling it 'trading time,' 'trading speed,' 'latency,' and maybe how long it takes to process orders. I'm wondering, where are the computers that are doing the trading? Do they need to be positioned within a few miles of the stock exchange servers, with dedicated fiber optic lines between them? Because it sounds like that'd be necessary to make this kind of speed up in the stock exchange servers meaningful.

Re:Light

[Anpheus]

If it's anything like the NYSE is planning, they let the investment banks build their high speed trading server racks right next to the stock exchange racks. Literally.

Supposedly there's even ways to get your server physically in the same rack as one of the NYSE's exchange servers, but I think that might be taking it a bit too far, or maybe an overzealous reporting on the matter. But I don't think it's even disputed anymore that they're letting banks colocate with the exchange.

Re:Light

[Anonymous Coward]

That is exactly what they are, the land prices around the servers in New Jersey has sky rocketed as trading corporations are buying land as close to the servers as they can possibly do in order to gain an advantage over their competitors. This whole millisecond trading is ludicrous and I am sure it will result in a next big bubble as the price is not based on any real evaluation of the corporation traded.

Re:Light

[dakameleon]

Yep, and major trading firms will do anything to get closer to the exchange servers. I worked for a Major Bank which had a major data centre well outside London which hosted all the "slow" apps, and a small (but well cooled) server room in the City a few blocks from the LSE building. Each and every app in the central data room had to justify its need, and every so often you would hear about acquisitions of real estate closer still.

This is of course pre-2008; I'm no longer so intimate with the details of server rooms at major banks. C'est la vie.

Re:Light

[Nursie]

The world of high finance has become just a bunch of racks frantically swapping bits around.

Yet when it screws up, the shocks are felt everywhere, for some reason.

Something is sick on this planet, when automated behaviour of electronic systems decide who eats, who can buy a new mansion, who gets a miserably low pension, whose house is going to be taken away and who's going to pocket a billion dollars in profit.

I'm more and more coming over to the side of those that say the whole finance sector is parasitic in nature and needs to be destroyed.

Re:Light

[Krneki]

If you don't like the developed world, you can go and live in Africa*, they don't have any stock market there to ruining their life. *troll disclaimer I mean the most underdeveloped African countries,

Re:Light

[dintech]

Co-location. A lot of exchanges will let you put your servers in their building. Then the arguments start about who's network cable is shortest and which router it's on. I'm not joking.

Re:Light

[Nursie]

I'm not a leftie. Neither am I anti-capitalist. I just think the level of abstraction from doing anything in the least bit worthwhile here is... nuts.

Re:Light

[khallow]

Yet when it screws up, the shocks are felt everywhere, for some reason.

Leverage.

Re:Light

[L3370]

Actually even in Somalia, one of africa's most volatile regions, there is a thriving stock market that is ruining (enriching?) there lives. http://www.npr.org/blogs/money/2009/12/a_pirate_stock_exchange.html Oddly enough, Somalia would be a pure Libertarians dreamland. No taxes, no real government to fuss with, and everyone is free to set up any enterprising business.

Bonus points if guns are used to make that business even more profitable.

Re:Light

[TubeSteak]

Co-location. A lot of exchanges will let you put your servers in their building. Then the arguments start about who's network cable is shortest and which router it's on. I'm not joking.

Ultimately, the solution to all this is to give the exchange a heartbeat and to only execute trades on those beats.
Randomize the queued orders that accumulate between the beats and we can finally kill off high frequency trading.

Equal access is one of the prerequisites for a perfect market

Re:Light

[Krneki]

Free market at last.

trade speed

[Eivind]

Trade-speed is irrelevant to investors.

It's relevant to highly speculative robot-trader algorithms that try to make a profit by arbitraging sub-second timing-issues. But this is a zero-sum game: one trader can only gain $X by taking advantage of timing if other traders lose PRECISELY $X, so to the sum of traders, this is irrelevant.

Stock-exchanges, make a living trough fees. The fees are coupled with volume, i.e. a broker that has a larger volume of orders, will pay higher fees.

So lower latency is good for the stock-exchange, neutral for traders on equal grounds and negative for those suckers who play at daytrading. It -does- tilt the table towards those with machinery though, but the effect is irrelevant for traders who aren't extremely short-term.

In short: yet another reason to invest rather than speculate.

If you buy and sell 20 times today, each time with the table tilted a tenth of a promille against you, you'll on the average lose 2 promille, plus the fees. This doesn't sound like much, but a trader that does this 200 days a year, will have lost 20% of his profit to the tilted table. (if his flat-table profits where less than 20%, he'd thus run a minus)

Meanwhile, the investor, who holds stock on the average 5 years, will also lose a tenth of a promille in every transaction, but since he's got 2 transactions in 5 years, that works out to 0.4 transactions/year -- thus his loss relative to the flat table is 0.4 * 0.01% = 0.004% pro year, which is irrelevant.

Re:trade speed

[Eivind]

No idea. You tell me, in your third language.

Re:trade speed

[Eivind]

Not quite.

Translation: If the system is set up to favour those with 20ms response-times over those who are slower, it is dumb to play against that system when your response-time is a lot longer.

It's not really stealing, it's just favouring a certain type of trader (namely one who trades a lot, and is very fast)

It's logical that the stock-exchange would favour this sort of trader, afterall that's where they make their money. (never forget that the stock-exchange itself, is a business)

Re:trade speed

[Eivind]

Every time one trader buys, another sells. Like I said: it's a zero-sum game. The only way one trader can come out ahead, i.e. buy at a *better* than average time, and sell at a *better* than average time, is if another trader comes out behind, exactly equally much.

Thus you'd expect that 50% of traders come out ahead, and 50% come out behind. But it gets worse: there's fees associated with every transaction, so the real odds are more like 30% come out ahead, 70% come out behind. And you're more likely to come up behind the more often you trade. (since more frequent trading translates to bigger fees)

A *good*, as in genuinely upper-quartile trader, will still come out ahead, sure. But because of stuff like this, tilting the table in advantage of machines, odds that you, or any other individual small-time investor will be in that quartile, sinks all the time.

It's one of those games where the best way to win, is not to play at all. (notice that the game of speculation is distinct from the activity of INVESTING)

1 per second? Not even that

[Anonymous Coward]

Stock trading should be at least limited to speeds that humans operate at, which is more than one second. Hell, I think that large, industrial companies' stocks could well be traded once a month or so (especially they shouldn't be traded immediatelly after quarterly reports, to prevent idiotic panic reactions). New software companies... Perhaps once a day? Stock trading is supposed to be long term investment based on how the companies are doing, not automized microsecond manipulation.

That said... Who am I to say that people shouldn't be allowed to throw their money to that stupid system if they so want to do? Rather, we should make the society less dependant on that... So that the next time the market crashes (which it periodically does), it doesn't take the rest of the society down with it.

Re:1 per second? Not even that

[NoSig]

That's interesting. Whenever vital information about a company is known to come out, the stock of that company should not be tradeable until sufficient time has passed for a human to reasonably process the information.

Re:1 per second? Not even that

[The Mighty Buzzard]

The thing is, if people don't have the freedom to be both panicky idiots and greedy fuckers who're happy to take advantage of that, they're not free at all. Freedom is the government leaving alone unless you're screwing with someone else's freedom; only letting you do what's morally right by their standards is the basis for most oppression.

So, yeah, let them make money off people who aren't as good at stock trading as fast as they like. Doing otherwise would basically be telling them they're not allowed to make any more money because you find the amount they already have distasteful. Pretty much 100% sour grapes.

Re:1 per second? Not even that

[skegg]

For those who may not know, these are known as trading halts.

What makes me furious is looking over historic share price graphs and observing share prices rise a day or so *before* a company makes a positive announcement (e.g. a huge mineral find, scientific breakthrough, profit increase, etc).

Re:1 per second? Not even that

[The Master Control P]

The idea of government only preventing someone from directly interfering with the freedom of others sounds great until you realize that (1) everything everyone does affects everyone else, so the only way to actually satisfy this constraint is for everyone to do nothing, (2) not only is it possible to harm people through second-order and higher effects, that's the overwhelmingly dominant means by which people in industrialized countries come to harm today, so relaxing the constraint to not directly harming others is effectively useless, and (3) the relationships between cause and effect are, a large majority of the time, of such high order that accurately and objectively assigning blame/responsibility for harm is effectively impossible. The world is vastly too complex to be effectively managed by an idea so simplistic or black & white.

Absent the availability of a superhuman-class intellect that's both able and willing to solve the optimization problem, we settle for global stability constraints. Stamping out actions whose only tangible effect is to crash whole stock markets so hard the operators hit the big red "Shut. Down. Everything." button sounds like a damn fine constraint to me. Or, "your freedom to be a greedy dick or a panicking moron ends where the viability of the world's economy begins."

Re:1 per second? Not even that

[shutdown -p now]

So, yeah, let them make money off people who aren't as good at stock trading as fast as they like. Doing otherwise would basically be telling them they're not allowed to make any more money because you find the amount they already have distasteful. Pretty much 100% sour grapes.

If the only outcome of people trading on the stock market and failing was less $$$ in their pockets, I couldn't possibly care less - just as I don't care about casinos and lotteries.

The problem is that, when stock markets fuck-up big time (again), the ripple effect is severe enough that the only way to avoid it is to stock up on supplies and bug out to the woods. We've seen this in practice more than once already. Since, in the end, I somehow find my paycheck been affected, I feel perfectly entitled to advocate for stock market regulation.

Your freedom ends where my nose begins - but, in a working society, we all have our noses stuck into each other's business, so in practice the point is moot.

Re:1 per second? Not even that

[definate]

Well, not quite. Freedom is loosely defined. Though, given freedom means liberty, then there are two concepts of liberty.

Negative Liberty which is the laissez-faire approach, which you're talking about.

Positive Liberty which is the socialist approach, which you're not talking about.

As it stands, which ever one you choose, is a subjective choice. All attempts to make it objective, are usually absurd, and based on the maximization of "utility" or similar, and are open to extremely wide open to error/critique.

For the record, I'm a negative liberty proponent. I don't find people telling me to be free, particularly helpful in these complex systems, with unintended consequences, that we live in.

Re:1 per second? Not even that

[The Mighty Buzzard]

Considering secondary and higher order removal of effects is just a way to talk yourself into taking away the freedoms of others for their own good. That, my tyrannical friend, is just as bad as taking them away for personal gain or spite without the benefit of at least being honest about it.

If an exchange wants to take prudent steps to keep their market from inadvertently getting hosed, more power to them. It's their sandbox they're inviting others to play in. The government butting its nose in is another thing entirely. If for no other reason, then because they have an abysmal track record of not screwing things up worse than before.

Re:1 per second? Not even that

[The Mighty Buzzard]

You have a very odd idea of what people want out of society but that's off-topic enough I'll let it go.

As for others actions indirectly, adversely affecting you? That's life. Get used to it because it is never going to change. We're not entitled or promised a safety net and anyone trying to give us one wants our freedom as the price. A bit here and another bit there; eventually it's gone.

Re:1 per second? Not even that

[Anonymous Coward]

As for others actions indirectly, adversely affecting you? That's life. Get used to it because it is never going to change. We're not entitled or promised a safety net and anyone trying to give us one wants our freedom as the price. A bit here and another bit there; eventually it's gone.

Yeah, it always starts with regulating traffic and ends with concentration camps. There is absolutely nothing in between but one giant slippery slope.

Re:1 per second? Not even that

[The Mighty Buzzard]

Ah, see there, just because something gets called a type of liberty doesn't make it liberty much in the way that mountain oysters are most assuredly not oysters.

Participation in government is not liberty of any sort. It is abdication of easily defined individual liberty in exchange for comfort, security, and someone to blame if things don't go your way. That is the fundamental nature of all forms of government.

I suggest that liberty can only be measured in a bottom up manner (individuals vs. society, provinces vs. their nation, nations vs. the world). Given non-vague enumeration of an individual's rights, his liberty can be objectively observed and measured; not so a society. Society's rights are the collective will of the people which you can attempt to measure but what you receive as a result is not a boolean answer as to whether a right has been infringed or not but a statistic. Given that both lies and damned lies can scoff at statistics from a moral high ground, that makes them completely unacceptable data when dealing with something as fundamentally important as liberty. It also means politicians will instantly fall in love with them.

All that being said, I'm not in favor of either way if taken to the extreme. Most anything taken to its logical end ends up looking absurd except to the foolish or the mad. I simply favor giving up as little liberty as absolutely necessary to maintain the stability of the nation. Everything else I will happily take care of on my own.

Re:1 per second? Not even that

[The Mighty Buzzard]

What you say in jest, I say in seriousness. Fortunately it usually happens over many lifetimes.

Re:1 per second? Not even that

[khallow]

The problem is that, when stock markets fuck-up big time (again), the ripple effect is severe enough that the only way to avoid it is to stock up on supplies and bug out to the woods. We've seen this in practice more than once already. Since, in the end, I somehow find my paycheck been affected, I feel perfectly entitled to advocate for stock market regulation.

It's worth noting that the stock market itself, hasn't fucked up once. Leverage is the mystical beast you should be worrying about. Leverage turns a bad trade from affecting just the trader to affecting the trader and the people who loaned him money.

Re:1 per second? Not even that

[Citizen of Earth]

when stock markets fuck-up big time (again), the ripple effect is severe enough that the only way to avoid it is to stock up on supplies and bug out to the woods.

The thing about HFT is that when it does fuck-up the markets, the markets naturally fuck-down again within minutes. Use limit orders and you won't get boned. I'd like to see more flash crashes/explosions because they create enormous opportunities for more sensible investors.

Yeah, but

[toygeek]

Does it run LINUX? Oh wait, it does. Cool!

Re:Yeah, but

[1s44c]

Does it run LINUX? Oh wait, it does. Cool!

If I built a robot that stole cash out of your bank account and it ran Linux would that still be cool?

Some things are not cool no matter what OS they run.

Yes it's a really bad analogy but still.

Re:Yeah, but

[oji-sama]

Bender runs Linux? Cool.

Re:Yeah, but

[aiht]

If I built a robot that stole cash out of your bank account

...

Yes it's a really bad analogy but still.

Wait, that was an analogy? I thought you were just talking about TFA.

Wait up zealots.

[metrix007]

This speed is not specific to Linux, it is because it is a new system which has been optimized.

Had they switched to BSD, Solaris, OS X or even a a recent version of Windows and optimized the application, then it would be just as fast.

Sorry kids, thems the breaks.

Why oh why do people have a need to spread reverse FUD and give "linux" props it does not deserve?

Re:Wait up zealots.

[RyuuzakiTetsuya]

No, but they hopped ship from Windows anyway.

I'm guessing there's serious TCO and maintainability issues being addressed.

I had a funny conversation with a guy in another department, he's bitching about 32bit vs 64bit ODBC for some long overdue infrastructure changes, I looked at him and said, "I did sudo yum install mysql" and just ported my packages over.

He was a little pissed...

Re:Wait up zealots.

[Sean Hederman]

It had nothing to do with TCO, maintainability or anything like that. It has to do with the fact that the software they're switching to is the fastest in the world. The fact that it runs on Linux is actually a disadvantage, since most of their skills are MS-based. But they don't care because the software is so fast and customisable.

Re:Wait up zealots.

[_Sprocket_]

Why oh why do people have a need to spread reverse FUD and give "linux" props it does not deserve?

Because Microsoft highlighted the TradeElect system in their "Get the Facts" campaign. Oddly enough, if you read the press kit Microsoft put together, you'd get the impression that a recent version of Windows and an optimised application was in place and doing very well. Maybe Linux does get some small slice of kudos after all.

http://web.archive.org/web/20070706203521/http://www.microsoft.com/windowsserver/facts/casestudies/lse.mspx

Re:Wait up zealots.

[LingNoi]

Actually you're completely wrong. They bought an entire company who specialises in exchange systems on linux, so none of their skills are in Microsoft based systems. Buying that company was cheaper then their previous MS based system which suffered a complete loss of days trading and was much slower.

There's really no disadvantage here. They've saved money, got a more stable system and have someone to complain at when things go bad unlike their previous MS based solution where the company in charge of it absolved themselves from responsibility simply blaming Microsoft.

Re:Wait up zealots.

[Sean Hederman]

The LSE's existing staff are MS-based. My point was that since the software they were looking at was Linux-based, that was actually a point against it when they were evaluating. They went with it anyway because the speed advantages were too good to resist, and it was proven technology. I promise you that the LSE care nothing for MS vs Linux. If there had been an MS-based system with the same performance and reliability, they would have been happy going with that. There wasn't, so they didn't.

As for "the company in charge of it", the current system is owned and developed by the LSE. It was handed over to them by Accenture some years ago. Since they now own the new system as well, I'm not sure exactly who you think they'll "complain at when things go bad" that they don't have now.

So, in summary, the reason they bought this software was that it would have been too expensive and risky to get their existing systems to the performance characteristics required, which is no surprise to anyone who has ever had to deal with Accenture-developed code.

Re:Wait up zealots.

[udippel]

http://download.microsoft.com/download/1/c/0/1c0aa1b6-bb2b-4371-a2a0-80c8c8497b6d/LSE_WinServ03.doc

Héhéhé! As of now it is still there; in the anonymous stomach of Microsoft.
Now I'm waiting for the fanboi to give Stevie B. a call. And then he'll start swirling chairs, yelling "I'll fxxxing kill that fxxxing pussy that didn't remove that fxxxing doc!!"

Re:Wait up zealots.

[LingNoi]

No you're changing your original point. Here it what I responded to before you changed it.

It had nothing to do with TCO, maintainability or anything like that. It has to do with the fact that the software they're switching to is the fastest in the world. The fact that it runs on Linux is actually a disadvantage, since most of their skills are MS-based.

It is based on TCO. Their existing system cost more money then it cost them to completely replace it.

It is based on maintainability. Their existing system suffered a 7 hour downtime on one of the busiest Mondays around the time of the US financial collapse.

It has very little to do with being fast. The LSE isn't one of the busiest trading exchanges. In fact it's only around 9th in the world.

Re:Wait up zealots.

[Sean Hederman]

Hmmm, I'm not sure how you think I'm changing my original point, but okay.

The #1 reason for switching has to do with performance. The system they've chosen is very stripped down in comparison with TradElect, allowing for greater speed. It also is more flexible, which is useful since the LSE operates other exchanges trading engines as well. Reliability is a major factor as well, agreed.

Whilst the LSE is 5th by market cap, they also run trading for Borsa Italiana, small parts of the Tokyo Stock Exchange, and the Johannesburg Stock Exchange. In total managed market cap that puts them at #2 behind the NYSE. No idea where you got that 9th figure.

I also don't know where you're getting this idea that perf was unimportant, but you can forget it. I can assure you that in the stock exchange business, speed is king. They would happily suck up somewhat increased operational costs if they could speed up the trading, since their profits would soar. The problem with TradElect is that it doesn't scale too well, so their costs rise faster than profits as they boost it's performance. With the MIT acquisition they're getting an even better deal: increased speed plus hopefully lowered costs.

Increased speed encourages companies and brokers to work with you, it encourages smaller exchanges to run on your systems, it increases the money you can make from co-location. Not only that, it obviously also allows you to greatly expand your listings base should you choose to do so, and go after massively increased volumes. From their own press release: "commitment to strive for performance enhancements and efficiency in the markets we operate, ultimately delivering value to our clients". It doesn't mention TCO or costs anywhere there.

Re:Wait up zealots.

[LingNoi]

Your link shows capital traded, not actual trades done. LSE is small when looking at the amount of trading done through it compared to other exchanges. That is, the amount of trades done, not the amount of those specific trades.

Sure performance is an issue but that has nothing to do with what you said.

You keep trying to hand wave away the fact that TCO wasn't an issue here at all but it is a factor.

From looking at your linkedin I know you've invested a lot into only Microsoft technologies which is why you're being their resident apologist but the simple (real) facts are that they've saved money by switching away from Microsoft only technologies.

Re:Wait up zealots.

[codepunk]

Yea sure because they are running a off the shelf ip stack on those boxes right? Come on don't talk silly the reason these guys can achieve that speed is because they had access to the source and had the ability to modify and tweak the system to their needs. Look when speed means the difference between winning and loosing I would certainly pick a OS that I can modify to my needs.

Re:Wait up zealots.

[Sean Hederman]

Look at my blog and whilst I do indeed use Microsoft technologies, I think it's pretty unfair to label me a Microsoft apologist. I also find it interesting that you use this "Microsoft apologist" label to hand wave away the evidence I presented, a clear cut case of attacking the person, not the argument.

I have not disputed that their costs have been lowered. I have focused on performance issues because I know for a fact that performance is the primary reason they switched. You keep focusing on other areas, which whilst important, were not of primary importance in their selection criteria. I thought you were genuinely interested in getting information, which is why I tried to explain to you the main reasons they switched.

If you looked at my LinkedIn profile, then you know where I'm contracting. How the hell do you figure you know the inside details better than me?

Microsoft, get these facts!

[Idaho]

Remember that this very stock exchange moving to a purely Microsoft/.NET based solution was widely touted in Microsoft's so-called 'Get the "Facts"' campaign. Microsoft was involved (with Accenture) in the implementation of the project, not just in selling some Windows licenses. So this screwup should really be a PR disaster for them. If Microsoft themselves cannot even get a .NET project to work in places where their Linux-using competitors have no trouble at all (Chi-X is also Linux-based), then that sure looks like a platform in trouble to me.

Remember that the entire thing crashed down for an entire trading day, something that you can imagine didn't go over well, and together with the high latencies and other numerous problems, was the reason they dumped it for Linux.

Re:Microsoft, get these facts!

[wrook]

As much as I want to agree with you, it's really pure speculation. Projects fail for a huge number of reasons of which computing platform is just one. It might be a PR disaster, but not for any legitimate reason. I daresay it is not impossible to build a decent trading system on Windows/.NET. I have implemented systems in .NET before and while it's not my platform of choice I don't really recall anything about it that should be a show stopper here.

But while we're speculating, it would be interesting to imagine a project failing precisely because of the hype about .NET. Most good programmers understand that a problem can only get as simple as it can get. After that no tool is going to help you make it simpler. But MS (and virtually all other proprietary tool vendors) promise that their tools can make projects magically simple. The only ones that get fooled into believing this are managers and bad programmers. These are the people who tend to flock to such projects.

So, it seems possible that a project could self-select itself for failure based on unrealistic expectations of the computer platform they are using. Well, when I say "it seems possible", what I mean is I've seen it happen time and time again ;-) Whether it was the case here, I doubt I'll ever know. Bad programmers usually have no clue why their projects fail (or serendipidously succeed). That's why they are bad programmers.

Re:Microsoft, get these facts!

[dbIII]

I suppose if the point of the project becomes to be a showcase for a very narrow platform (dotnet) instead of actually getting the job done with a lot of choices on a wide platform (anything at all that runs on any version on *nix) then you have immediately reduced the chance of success.
In hindsight it once again looks like the only good outcome from using Accenture/Andersons in a project is for the lawyers. I wonder how long it will be before they have to change their name again?

Re:Microsoft, get these facts!

[UnknowingFool]

Well there hasn't been much detail about the LSE crash other than it was hinted that it was an upgrade that went bad. And it was blamed on MS. What has been curious is that MS has not responded to the this. If it were a project that I had done a lot of PR work (and MS has), I would try everything to prevent the LSE from naming my product publicly even if it were my fault.

As much as I want to agree with you, it's really pure speculation. Projects fail for a huge number of reasons of which computing platform is just one. It might be a PR disaster, but not for any legitimate reason. I daresay it is not impossible to build a decent trading system on Windows/.NET. I have implemented systems in .NET before and while it's not my platform of choice I don't really recall anything about it that should be a show stopper here.

The criticism before this project for .NET is that it doesn't scale very well. With .NET you can develop desktop applications fine but it has not been used in many large scale developments like this before. There were probably many things that were unknown about optimization and performance that MS didn't know and didn't have experience dealing with before.

Take for instance, supercomputing. Many of the top 500 supercomputers run Linux; however, it isn't Ubuntu or commercial Red Hat. The supercomputing versions of Linux have been customized for high speed, highly parallel computing. MS finally has one or two entries in the list but again those were specifically customized by MS. Now with other stock exchanges that use Unix or Linux, there simply has been more experience creating one for exchanges.

But while we're speculating, it would be interesting to imagine a project failing precisely because of the hype about .NET. Most good programmers understand that a problem can only get as simple as it can get. After that no tool is going to help you make it simpler. But MS (and virtually all other proprietary tool vendors) promise that their tools can make projects magically simple. The only ones that get fooled into believing this are managers and bad programmers. These are the people who tend to flock to such project.

Good points. MS has touted that .NET is the solution for everything from mobile to servers to desktop. In my experience a general tool is not often good at specialized functions unless it can be highly customizable. In the case of .NET (unlike more open systems like FOSS), it is controlled by one vendor. Unless that one vendor is highly agile and constantly willing to change their product based on customer expectations, it is not likely to work out in the long term. With Linux the customer could change it themselves if they were willing.

So, it seems possible that a project could self-select itself for failure based on unrealistic expectations of the computer platform they are using. Well, when I say "it seems possible", what I mean is I've seen it happen time and time again ;-) Whether it was the case here, I doubt I'll ever know. Bad programmers usually have no clue why their projects fail (or serendipidously succeed). That's why they are bad programmers.

Also remember that there was a disconnect here. LSE hired Accenture to manage it who hired an outside software company. The programmers at the software company may have only did what was necessary to complete the basic objectives. They themselves may not have had a vested interest other basic functionality.

How scientific is this?

[bertok]

I'm willing to bet that this has little to do with Linux.

It's a great server OS, sure, but lets look at this realistically:

- the Windows / .NET trading system was based on Windows 2003 and SQL 2000, and was deployed in 2005.
- the Linux-based system is under development now, to be deployed next year.

Just based on that, you'd expect substantial performance differences from just using newer hardware. Chances are that the original kit was certified as a part of the solution, and hasn't been replaced since. With all-new gear, I'd expect about 2-3x the CPU performance, and if they're using 10GbE (likely), then 10x the network performance.

Even ignoring the hardware and the OS, one would expect 90% of the performance to be determined by the application, not the OS. Decisions like writing the software in .NET versus C or Java, or using a special-purpose Java runtime would make a huge difference, irrespective of the OS.

On top of this, the software stack is completely different, and developed by a different team. Just about every design decision, small and large, will be different.

To make a fair comparison, you'd have to run the new software on both OS platforms, on the same hardware.

Re:How scientific is this?

[jimicus]

Exactly - I've seen perfectly stable Windows systems.

I have a strong suspicion that a large number of the "problems" associated with Windows have got less to do with the platform and more to do with the sheer number of incompetent morons it seems to attract like moths to a lightbulb. Put similarly skilled people to work on a Linux platform, and I'm sure you'd see similar results.

Re:How scientific is this?

[bertok]

I'm not so sure. for certain workloads, In production, postgresql queries I've written have been 4 times faster on a linux server, compared to a windows server.
Now I'm willing to accept arguments regarding postgresql's potentially differing levels of optimisations for the two platoforms, but the fact remains, particularly for IO intensive workloads, that the underlying platform has a measurable effect

Of course, but you did a like-for-like comparison, the same application across two different platforms. The article is about two dissimilar systems, on different hardware. There's no way to draw any conclusions from that.

Benchmarking is complex -- subtle things can completely ruin results, especially when multiple operating systems are used. For example, on Linux, starting a process is cheap, but on Windows, starting a new thread is cheap. Software written using the "Linux way" runs terribly on Windows, so a lot of Linux developers just assume Windows is "slow". I've seen this with some PHP-type web frameworks that fork a new process for every query, they can barely do 20-50 requests per second on IIS. Meanwhile, ASP.NET can do thousands on the same box, because it uses a thread pool.

Re:How scientific is this?

[bertok]

I'm willing to bet that this has little to do with Linux.

I'm betting it has a LOT to do with Linux.

Remember, it wasn't just trading speeds that doomed the .Net solution. It was the unreliability.

Sure, it COULD be a statistical fluke that the Linux-developed system proved more reliable and performant. On the other hand, this isn't an isolated incident, and so far the statistics seem to be falling in favor of Linux where the rubber meets the road. The LSE bid was made after looking at similar systems already in existence.

Nice try, though, Mr.. Ballmer.

Why would .NET be unreliable? Are you expecting it to occasionally answer '2+2' with '5'?

The solution was most likely at fault, not the managed runtime! It certainly wouldn't be more or less reliable than, say, Java.

I mean seriously, .NET is a terrible choice for a latency sensitive application, as is SQL Server. Neither provides any kind of time guarantee. Java and Oracle would be just as bad.

If microseconds matter, you code in C or C++, and rely on specialist databases, or roll your own for the purpose. That's how mainframe applications are written, and surprise, they work. All of this can be done on Windows. If you turn off everything that's not strictly required, Windows is as stable and reliable as any commodity server OS out there. It used to be unstable, true, but it's rare to see a crash that isn't driver-related these days. I've seen Windows terminal servers with 2 year up-times.

The LSE project had buzzword-compliant architecture bullshit written all over it. It was simply a badly designed application written by consultants who tried to satisfy Microsoft's marketing needs instead of making pragmatic decisions to solve problems.

Don't blame the tools, blame the tool users.

Re:How scientific is this?

[xaxa]

With the salaries paid to developers in the financial sector in London (150-200% of normal from the start, plus a big bonus) they shouldn't have a problem recruiting decent developers for either platform.

Re:How scientific is this?

[jimicus]

The sort of person you see applying for that and getting it is the sort of person who has testicles so big they cart them around in a wheelbarrow. The sort who you see on The Apprentice and want to punch in the face because they can't stop going on about how wonderful they are.

Their actual ability may not be even remotely correlated with this.

Re:How scientific is this?

[gbjbaanb]

I tell you what, that's the 3rd comment I've seen that's made the excuse that it must have been better hardware performing better, nothing to do with .NET v Linux at all. Are the Microsoft marketing department working overtime today? You'd think you guys would have a selection of excuses to use instead of just the 1.

Frankly, I'm surprised you havn't banged on about how it was written in .NET 1.1, and that .NET 4 is soo much better now. Still, the facts are that Linux is an established player in corporate, enterprise, mission critical systems.

Re:How scientific is this?

[xaxa]

I'll have to check with some "friends" (i.e. any of the 30-50% of people from my CS class at university who took a job in the finance industry, and who I can no longer afford to have after-work drinks with). Most of them are probably capable -- that's why the banks etc tried hard to recruit from the university -- but most of the really clever guys took jobs elsewhere (Google, academia, small start-up, etc).

Re:How scientific is this?

[codepunk]

To see the light you must understand that when it comes to high speed trading speed and latency counts. In fact speed and latency are the only thing that counts. The first thing you have to remember that these guys are not running off the shelf shit everything in the system is modified to run at those speeds. Which system am I going to use when microseconds can mean 10's of 1000's of dollars? That is correct I am going to pick the system that I can tweak, tune and modify just to gain a couple of microseconds of advantage on you.

Re:How scientific is this?

[guruevi]

I'm pretty sure that 'replacing hardware' was included in 'how much does it cost to get this to do stuff faster'. I'm also pretty sure that Microsoft got included at first when the LSE came and said that it needed to go faster.

The problem was that Windows/.NET/SQL Server couldn't perform under this high load with predictable latency and either Microsoft gave up on them or they gave up on Microsoft. As any web hoster will tell you, SQL Server just craps out if there is too much unpredictable (random) load. .NET has very unpredictable load responses especially when many applications share the host and is not close enough to the bare metal like C or C++ is to get better performance out of it (it's like Java, but not cross-platform). Windows has no low-latency or real-time anything especially not in combination with .NET and the TCP/IP stack is really bad as well.

Apparently it was cheaper to just build a new system than to upgrade their hardware, software and/or developers. Management people know how to get most of their stuff done for cheap even if they don't understand the technical issues behind it.

Re:How scientific is this?

[Builder]

As an aside, some of your friends are assholes. Our general rule is that those of us who made it into the higher paying roles, or have fewer expenses (no kids, etc.) pay for more of the drinks than the others. That way we all still get to hang out, be we plumbers, geeks or bankers.

As a result, we're a more diverse group than many other circles of friends that I've interacted with and we have some great times.

Re:How scientific is this?

[xaxa]

If you work in the City I think you end up simply not seeing "normal" stuff, and you forget what "normal" is (in the same way I forget how much cheaper some things are in North England). Many of the bars (and shops, and restaurants) in the City have outrageous prices, which match your outrageous salary. The last time I met up with the "friends" the cheapest drink was £12, and they ended the night by buying all the "novelty" drinks -- things costing £100-200 and served in a fake pirate's chest etc.

With real friends we meet somewhere else, perhaps later in the evening or at weekends. British culture and custom requires everyone to pay their share in a pub ("I'll get this round" "no, no, it's my turn"), so if we're somewhere with reasonable prices no one is priced out. It's sort-of OK to let the rich guy buy more rounds, once everyone's bought one round each.

Re:How scientific is this?

[Builder]

Holy christ - Ok, we're just not posh enough. I could probably afford the occasional night out with 12 quid drinks, but I have a sneaking suspicion that I wouldn't enjoy the kind of place that charges that kind of money :D

There are still plenty of civilised bars in the city where drinks don't cost the earth, but at the same time you're not spending the evening with the yobs :)

Plus, most of my mates are foreign imports, so we're not precious about people 'having to stand their round'. I'd rather my brother in law save the cash for my sister than buying drinks :D

Fast crash

[Zoxed]

Great news: now the economy can crash even faster !!

NASDAQ under 100

[harlows_monkeys]

NASDAQ is under 100 usec.

clearly not: prices are chaotic, fractal, etc.

[r00t]

Prices are discrete, quantized, non-differentiable, etc.

Prices are chaotic and somewhat fractal.

Going faster does not solve this. Think of sign(sin(1/x)) as x approaches zero; it changes rapidly but this doesn't make it smooth.

Hourly trades would be reasonable. You get a few minutes to submit secret bids, the exchange gets nearly a half hour to match them up, the exchange gets a few minutes to publish results, and you get nearly a half hour to decide on your next bid.

There is no reason that the finances of normal corporations and normal investors should be subjected to the abuse of today's stock market.

Re:clearly not: prices are chaotic, fractal, etc.

[Pinky's Brain]

This is far too sensible to ever happen.

Re:clearly not: prices are chaotic, fractal, etc.

[dintech]

You get a few minutes to submit secret bids, the exchange gets nearly a half hour to match them up

This is similar to what the open and close auctions are on most exchanges except there are indications as to what the mid price is. It all gets matched at the end of the auction. If you think you're being fleeced, why not just trade the auctions instead of intraday continuous trading?

Re:clearly not: prices are chaotic, fractal, etc.

[Kjella]

That's going way in the direction of overkill and would create a shadow market that operates independently of the official one, and even if it didn't it would take too long time for the market to settle after big stock news. I would say the point is to remove the dependence on your connection so that it's possible for everyone to submit orders without being ripped off by microsecond traders and "fake" orders that are immediately canceled.

It's well known that the human reaction time is around 300 ms if you're prepared for it, like watching a stock ticker and waiting with the finger on the submit key. If we say twice that and add in a generous 300 ms of round trip time (I get 150 ms Norway - California) then I think a one second pulse with 0.1s matching is fine. Any plain consumer DSL/cable line should do.

That way there's alway a risk that your order is matched and small traders are part of the same matching as everyone else. I suppose they could still play the sub-second high speed trading game between themselves, but I doubt it would be that interesting with only themselves playing. Of course computers can still try seeing macro trends in all the stocks and speculate in that, but that I consider "fair".

Re:clearly not: prices are chaotic, fractal, etc.

[khallow]

There is no reason that the finances of normal corporations and normal investors should be subjected to the abuse of today's stock market.

And they aren't. I think that's the thing people don't get about high frequency trading. It doesn't affect people who aren't also high frequency traders (or employ market strategies like stop orders, that operate on the near instantaneous time frame).

Re:clearly not: prices are chaotic, fractal, etc.

[Lobachevsky]

Not to mention it makes takeover bids cheap. Why punish shareholders of a company and benefit a billionaire like Larry Ellison in his appetite to buy companies?

The ability of the market to move up when it sniffs billionaires trying to buy up the stock cheaply is critical to giving the mom & pop (shareholders) the value they deserve.

All the brouhaha over automated trading is silly when considering a simple, fundamental question -- do you think the price of the stock is wrong? If you think it's too low, buy it, if you think it's too high, short it. If you think the price of the stock is perfectly fine, then why on earth is anyone complaining?

speed is counterproductive

[Tom]

One shouldn't forget that these high speed tradings are not in the interest of investors, companies being traded or other market participants with a real-world interest. They serve only the high-speed/high-volume speculative traders, specifically algorithmic or automated traders.

Even Wall Street is slowly waking up to the problem:
http://online.barrons.com/article/SB50001424052970203952604575552190237324972.html?mod=BOL_twm_mw

This was a bomb about a week ago when it was published, the guy making those statements is a Wall Street billionaire, not a hippie communist.

Re:Why?

[__aatirs3925]

corporations don't need to have public options in their stocks. most of the traditional corporations out-there do this but a lot of the newer corporations have privatized it so there's less risks and they can do whatever they feel like doing without worrying about shareholders. Personally I hate the stock exchange system.

Planet Money

[VValdo]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value. Latency contributes to the very inefficiencies that you blame these "large investment firms" from profiting off of.

Let's just relax and listen to this episode of Planet Money on high-frequency trading.

I know. It's NPR. They fired Juan Williams. Whatever. (FWIW: on the Diane Rehm show from Friday, they defend Williams, say there was a "rush to judgement" and say that NPR went "off the handle". You'd NEVER see most news employees do that on-air.)

W

Re:Planet Money

[khallow]

Let's just relax and listen to this episode of Planet Money on high-frequency trading [npr.org].

You want to waste 21 minutes each of our collective time? How about you summarize what you meant to say and save me 21 minutes. I know your time is valuable, but mine is more valuable to me.

Faster than "ping localhost"

[zarlino]

ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=0.053 ms

Re:Faster than "ping localhost"

[Anonymous Coward]

Umm, 0.053 ms equals 53 microseconds, and last I checked, that is a shorter time than 126 microseconds.

Re:Faster than "ping localhost"

[Prof.Phreak]

I'd imagine they calculate latency from the time a market order hits their system, to the time it gets executed. An order comes in (UDP message), they do timestamp1, walk the book (heap or balanced tree) to find a match. if match found, `execute' if no match, add order to book. either way, generate an output message and add it to "out" queue [that happens at almost identical time as the original order arrival time]. The output process picks up thing from queue, and does timestamp2 (either order executed, or is now sitting on the book). No network or disk I/O is involved in such latency numbers. The output process sends outgoing UDP message and forward message to another thread to persist the event in database.

If they do things very carefully, they may manage to do input (read message, walk book, and place structure in output queue), and have output process pick it up on the *next* time slice... having the order turn around time in their system be very low. OS is probably irrelevant, but I'd imagine them having a lot more control over such detailed scheduling on Linux... (e.g. having OS interrupt you in middle of walking your book can double or quadruple your turn around time).

Offtopic - for gods sake why?!@

[AbRASiON]

Can someone please tell me how to get back to the old comment system? I'm checking slashdot, daily as usual - looking forward to reading insightful comments and lo and behold each day I forget this abomination is here, it's awful.

I've tried emailing the feedback address, it bounced surprisingly, I'm about to give up on the site. What's the solution here guys.
Sorry to be offtopic but I can't be the only one.

Re:Offtopic - for gods sake why?!@

[shutdown -p now]

Can someone please tell me how to get back to the old comment system? ... I've tried emailing the feedback address, it bounced surprisingly, I'm about to give up on the site. What's the solution here guys.

What new comment system?

* posted from my ELinks

Re:Offtopic - for gods sake why?!@

[Alioth]

Go into your preferences and turn Discussion 2 off.
There is a bug in Slash that means (or at least a week ago, meant) that the settings don't always save first time so you may have to try a couple of times.

I agree, Discussion 2 sucks (it uses masses of CPU time and any discussion with more than a couple of hundred comments requires an 8 core system to merely run at a snails pace). I wasn't all that pleased they turned it on for me when I've had it switched off, I've got rid of it now.

Re:Offtopic - for gods sake why?!@

[AbRASiON]

Thank you so much - how the hell did the new one get turned back on? I remember them rolling out a shitty system a year or two ago which I disabled, for some reason it just re-enabled itself.
Slashdot is usable again.

Re:Offtopic - for gods sake why?!@

[xaxa]

Can someone please tell me how to get back to the old comment system?

Has something changed? I've seen several comments like this in the last week or so, but as far as I can tell nothing significant has changed for at least a year.

Re:Offtopic - for gods sake why?!@

[Blakey Rat]

There's a bug in their bugtracker (which they haven't read). There's also a bug in there about the feedback email bouncing. (Which also hasn't been read.) I'm guessing all their email addresses are broken, and the Slashdot staff is just sitting around with a big grin going, "wow our website's great, NO BUGS!" Or their incompetent nitwits, one or the other.

The solution is to go to Help and Preferences, and under Discussion: Viewing uncheck the top-most item. (It's called something like "Dynamic Discussion System D2" IIRC.) That should give you the old comments back. There's a *different* setting for the index pages, which you may have to toggle as well.

Re:Offtopic - for gods sake why?!@

[Sir_Lewk]

That's actually the issue. I usually use the 'classic index' so that slashdot will render nicely in browsers like elinks, but lately it's been defaulting back to slashdot 2.0 anyway.

Valuable role they serve? Oh please.

[Viol8]

The only role they serve is to line their own pockets at the expense of ordinaryinvestors who then lose out as the prices are pushed up and down at the whim of various trading algorithms. This might come as a shock to you but stock exchanges don't exist for the sole benefit of investment banks and hedge funds. They're supposed to be for everyone and should be one the best ways of democratising the financial system. But no, the big boys have to screw over everyone else as usual.

THIS

[Joce640k]

If you want to stabilize the global economy put a tax on all stock trades. Stocks and shares should be long term planning, not microsecond.

Re:THIS

[asc99c]

Yet oddly, in the UK at least, there is a tax on actually buying shares, so my long-term investment account has various charges for stamp duty as my money goes in there. The tax is only applied to buying actual shares. It isn't applied when messing around with futures and complex derivatives etc that have caused the problems.

I think this needs to be exactly the opposite way around. I don't think there should be a tax for dealing in shares, but instead a tax on dealing with various futures products. Not too severe a tax but enough to dissuade this sort of high speed speculation.

Re:THIS

[khallow]

If you want to stabilize the global economy put a tax on all stock trades. Stocks and shares should be long term planning, not microsecond.

And why do you think stability is a worthy goal? Let me put it this way. Suppose I'm the CEO of some publicly traded business. And in the news today, we find out that I just absconded with every stealable asset in the company leaving the company with a massive debt to asset ratio and making the stock completely worthless. Stability means that the market continues to trade the stock as a considerable price even though everyone knows it's gone. Unstable markets would react quickly to reflect what happened.

That's the tradeoff. Stability versus more accurate pricing of the securities traded.

Re:THIS

[fotoguzzi]

...I just absconded with every stealable asset in the company leaving the company with a massive debt to asset ratio and making the stock completely worthless.

Rather than use up 26 precious words to describe a hypothetical business disaster, couldn't there be a shorthand using a standard keyword. Let's try it:

Suppose I'm the CEO of some publicly traded business. And in the news today, we find out that I Zuned. Stability means that the market continues to trade the stock a[t] a considerable price even though everyone knows it's gone. Unstable markets would react quickly to reflect what happened.

Re:THIS

[Joce640k]

The point is that when that happens nobody should be able to sell their (now worthless) shares on to other people who haven't heard the news yet.

Re:THIS

[khallow]

The point is that when that happens nobody should be able to sell their (now worthless) shares on to other people who haven't heard the news yet.

Why not? Why shouldn't the first bringers of news get rewarded? And there's a lot of news. What's the criteria for deciding which news is good or bad enough that you shouldn't be able to trade on it?

Re:THIS

[Joce640k]

You really don't see a problem...?

If you know your stocks are worthless and the other guy doesn't then it's fraud, simple as that.

Re:THIS

[khallow]

You really don't see a problem...?

If you know your stocks are worthless and the other guy doesn't then it's fraud, simple as that.

And why's that? I'm not misrepresenting the stock (which would be the sort of thing fraud is) or anything else fraudulent. I simply know something they don't know and trade on it. The problem here is that there are always knowledge differentials, routinely very significant ones. When should those knowledge differentials result in suspension of trading or other action (such as prosecuting someone for insider trading)?

Currently, trade is suspended either when a big drop occurs or when someone in a decision-making position feels like it. I simply don't see the reason or need for it.

Re:THIS

[Joce640k]

I suppose you believe that painting lead bricks with gold paint and selling them as gold ingots is Ok, too?

It's all just "knowledge differential"...

Re:THIS

[khallow]

I suppose you believe that painting lead bricks with gold paint and selling them as gold ingots is Ok, too?

The difference is that you are knowingly misrepresenting something for financial gain. That's the sort of thing fraud is. When I sell a stock on the market, I routinely don't know who's on the other end and I make no claims about the stock.

Re:THIS

[Joce640k]

But you know your stocks are worthless and he's paying 100 times too much for them, right?

Re:THIS

[khallow]

But you know your stocks are worthless and he's paying 100 times too much for them, right?

You didn't make any promise to him that the stocks would be worth anything. He put up the bid knowing that this sort of thing could happen and accepted the risk. Hence, it is not fraud.

What prog language was used is just as important

[Viol8]

While its always nice to see Linux succeed I can't help thinking that if the code had been written in C/C++ on Windows (as I assume it has been on this linux install) instead of using the hopeless managed slug known as .NET , then on the same hardware I suspect it would have been a much closer run thing. Why MS wants to deliberately hobble itself in this sort of scenario instead of using the fastest possible runtime system is anyones guess. Are they trying to promote Windows or .NET? They need to decide. If they'd have used C++ people might have wondered whether .NET was a bust but Windows for still up for the job - now they have a situation where people KNOW .NET is a bust and Windows has been dragged into the shit along with it.

Open source widens the gap between rich and poor

[captainpanic]

This stupid system makes it possible to earn a massive amount of money by buying and selling shares at the right time. The really stupid thing is that at some point, the money earned with those transactions can be exchanged for real goods, produced by real people.

People who don't add anything tangible to this world get free money, and can buy goods for it. It's madness. If I hadn't grown up in this world, I doubt that I would ever understand and accept it...

Re:So what are they using?

[jabjoe]

Yes, I believe so. I thought they had bought some C++ stock exchange development software house, but I can't find anything about it now. In which case it's not so much Linux faster as C++ faster, which we already knew. Not to take anything away from Linux, but I'm not sure it's what is making the big speed difference.

*sigh* This again

[Anonymous Coward]

Arguably the most important idea of communism was that workers own the tools of their trade. In traditional sense, this could mean (Marx didn't really go into the details of the implementation) that workers of a factory democratically vote for all the important issues (wages, etc.) and as such the workers benefit from their work (as opposed to one guy at the top pocketing the money) if they do it well... and have to tighten the belt or begin doing something else if they do the job is unnecessary or done poorly. There was more about the utopia that this would lead to, but that was the primary concept.

It is indeed true that socializing medicine has nothing to do with communism. Socialism means that government owns the production facilities, Communism means that workers own them, Capitalism (In original meaning of the word, not as a synonym for "free trade") means that some other private entity benefits from the people who work for him. The three are mutually exclusive concepts and communism is at least as far (perhaps further) away from socialism than capitalism is. After all, communism and capitalism both rely on question and demand while socialism includes the idea that some things aren't economically feasible but should be provided for the people anyways. Communism is effectively capitalism where workers of a company own all its stock and each worker owns a fair portition (Not necessarily "equal" as people who have worked there longer could own more because of that and it would still be canon).

Now, Linux is - to some extent - communism. It obviously isn't socialism (no government owns it) and it isn't capitalism (nobody at the top owns it and benefits from the people who work for him) but rather the community (the people who work to develop it) own it, own the tools to develop it, make decisions about it and benefit from their work for it. So, while it is a project, not a economic concept (So you can't say "Linux is communism". Communism is communism. Linux is Linux.), it certainly is based on a lot of the ideas that make up the foundation of communism.

Markets

[captain_dope_pants]

The stock market used to be full of bears & bulls. Now it has penguins too :)

Re:What prog language was used is just as importan

[jabjoe]

Have to say I've been wondering the same thing. Microsoft is pushing towards .NET for the whole platform while the Linux world stays C,C++ and only something else, normally python, where speed doesn't matter. Java and Mono aren't very often used for popular apps and tend to be replaced for smaller, faster, C/C++ alternatives. This move is only going to make the Windows platform seam even slower compared with Linux (or Mac and everything else for that matter).

I see many comments...

[OneSmartFellow]

... indicating that this sub-second trading is somehow 'gaming' the system.

The simple fact is that it makes no difference how fast or slow trading occurs, each trade (or order) is the result of someone thinking they can predict the future price. Whether they think that from insider information, careful review of the companies products/balance-sheet/ management/patents/whatever, or some stupid algorithm trying to outperform someone elses stupid algorithm is immaterial. 'Better' information is everything. To some people the better part simply means they can make a decision quicker than someone else.

I knew it!

[Overzeetop]

You're the guy who tried to figure out if he could have all the fractional cents from interest bearing bank accounts deposited to an account with your name.

great!

[alien9]

growing finance bubbles at twice the previous speed

Constant speed?

[gmuslera]

2 years ago the markets more than speed, needed acceleration, falling at 10 ms/s^2

Totally scientific!

[FranTaylor]

As they say "the proof is in the pudding"

You can theorize all you want about who could have done what, but in the end what matters is the implementation.

The simple and basic fact is that Microsoft with all their corporate might and power cannot fix the poor latency of their implementation.

It's the ordinary investors that benefit

[Ed Avis]

In fact it's about cutting down the slice taken by 'large investment firms' and preserving more value for the individual investor. More liquidity means that you pay a smaller bid-ask spread when you buy or when you sell. Arbitraging the short-term market movements means that there's no longer such an advantage held by sophisticated traders (who would, for example, try to buy a large block of shares in bits and pieces, to keep the price low) over more straightforward buying and selling.

Liquidity and lower spreads, oh my...

[Tuan121]

Liquidity Liquidity Liquidity.

High frequency players try to make lots of tiny little profits and in turn provides LIQUIDITY to the other investor base that does not trade nearly as often as they do.

And because there is so much liquidity from high frequency players, spreads are kept very very small.. Which is a good thing for everyone (except the old school people that made money screwing people on high spreads, thus they are becoming extinct for the most part).

So let's summarize what these evil traders are doing:

Providing High Liqudity ---> Lower Spreads ---> Means that your average person / shortish/medium/long term hedge funds/investors/whatever is able to easily trade in financial markets without having to worry about being screwed by high spreads or not being able to trade when they want to.

Yes, these are just evil, evil people... they allow you to sit at your computer when you should be working and click buy/sell whenever you want to and be able to complete the transaction. You people need to stop blaming the "big banks" and all that crap for everything that is wrong in your lives.

*Disclaimer: I work at a medium term hedge fund. We have no interest in doing ultra high frequency, it's a pure technology game. We instead enjoy the research aspect of applying physics/other type of models to the markets; high frequency is only interesting for the technology side. But it's great there are so many people that do because it's easier for us to trade.

You need to read more before commenting

[schon]

It's a great server OS, sure, but lets look at this realistically:

- the Windows / .NET trading system was based on Windows 2003 and SQL 2000, and was deployed in 2005.
- the Linux-based system is under development now, to be deployed next year.

You missed

- the Windows / .NET trading hardware has been upgraded continuously because it was unable to cope with the load.

Just based on that, you'd expect substantial performance differences from just using newer hardware.

Sure, except for the part that the both are running on new hardware.

Chances are that the original kit was certified as a part of the solution, and hasn't been replaced since.

"Chances are" - except that is 100% wrong. They had problems since day 1, which were blamed on the hardware, so they've been constantly upgrading it trying to fix the problem.

Even ignoring the hardware and the OS, one would expect 90% of the performance to be determined by the application, not the OS. Decisions like writing the software in .NET versus C or Java, or using a special-purpose Java runtime would make a huge difference, irrespective of the OS.

The old system was written with the help of MS. They were the ones that said that .NET was the best way to implement it, and they even touted this in their press releases.

On top of this, the software stack is completely different, and developed by a different team. Just about every design decision, small and large, will be different.

Of course it's completely different - that's the entire fscking point.

Nice success story

[apexwm]

Great to see others finally dump Microsoft's over-bloated OS and software, and use something more scalable and efficient, not to mention more AFFORDABLE. Now it's time for Dell to dump ASP.NET on their own website, it's horribly SLOOOOOOW.

The proof isn't in the pudding!

[iserlohn]

The proof of the pudding is *in the eating*!

Re:NASDAQ official trade data dissemination uses M

[UnknowingFool]

You're most likely correct... why? NASDAQ is proof, in & of itself is why, because NASDAQ uses Windows Server 2003 & SQLServer 2005 + IIS to power a 24x7 "bulletproof & bugfree" uptime in a MAJOR stock exchange for years to a decade++ now in fact (w/ MS stuff acting as the "official trade data dissemination system" as it was shown to be at LSE here -> http://www.youtube.com/watch?v=BwSM55bsCrM doing the same role for LSE as it does for NASDAQ). It appears to be a fault of either Accenture, or the staff @ LSE why MS stuff didn't scale as well or as reliably there as it has at NASDAQ for years now running stable & strong 24x7. Oh, by the by: I also agree with your looking at this from a hardware-based perspective also, as that has a bearing on performance also, and strongly so

No, you're slightly misinformed. The NASDAQ exchange systems themselves runs on Linux. NASDAQ runs their Exchange and messaging servers on Windows which is a completely different system. MS also has a case study showing that SQL Server is used by NASDAQ not in the real-time transactions but in reporting after the trade.

Rubbish

[toby]

Trading this fast brings the market closer to optimal economic efficiency, where prices at any instant accurately reflect value

This has been refuted many times. It's precisely wrong, at least when exploited by HFT methods, which completely defeat meaningful pricing.

Re:Pffft

[codepunk]

Actually Linux has everything to do with being able to achieve that sort of speed. Your standard off the shelf operating system cannot achieve that sort of speed without the
ability to tweak the OS to your needs.

Re:What prog language was used is just as importan

[codepunk]

Yes perhaps it would have been quicker but would it be quicker than what can be achieved on Linux. These systems use a heavily modified ip stack it is not your general off the shelf architecture. To obtain the speeds and volume we are talking about here full access to the OS source and the ability to modify it in any means possible is a must.

Re:Proofs of how NASDAQ uses MS OS & servers..

[UnknowingFool]

You are completely missing the point and trying to spin it like a shill. NASDAQ does not use Windows for the actual trading. That is the system the LSE migrated from Windows to Linux. Their email and reporting systems use Windows which doesn't have the same performance requirements. Often this is misreported as the email system of Windows (Exchange) is confused with the function of NASDAQ "exchange".

In this context, the trading system is the mission critical system. From your posts "10,000 queries a day" or "100,000 queries a day" is several orders of magnitude from being acceptable in trading. NASDAQ reports that they can make 64,000 transactions per second. An email server up 24x7. So what? A reporting server up 24x7. So what? The most important thing to NASDAQ is that their trading system is up and running.

Garbage collection pauses...

[bored]

Every time I hear about some big or realtime .net application getting kicked out. I wonder if its was the customer Patric Dussud is talking about when he describes a very large system that gets paused for an extended period of time while the garbage collector runs (http://channel9.msdn.com/Shows/Going+Deep/Patrick-Dussud-Garbage-Collection-Past-Present-and-Future). Long enough that the cluster heartbeat decides the node is down and shoots it.

I, for one, welcome our

[couchslug]

... faster flash-crashing (no, not Flash) stockmarket overlords!

Re:Java?

[EvilRyry]

No, actually the system is C++.

Re:You're putting words in my mouth I didn't say

[UnknowingFool]

In the context of this whole article and thread, we have been talking about the trading system of LSE. The LSE has decided that Windows and .NET were not suitable and replaced them with a Linux system. You keep bringing up the irrelevant facts that the NASDAQ uses Windows for other systems "24x7". So what?

Re:What's the difference?

[s73v3r]

Long term investing in a company is just that: Investing in a company, giving them capital to help them accomplish a goal. Short term speculation, however, is simply playing the numbers, and hoping to outguess someone to make money. No different than poker, really. Long term investing is also much more stable, and much less disruptive to the market than short term, especially high frequency trading.

Re:You can't back up your own mistake, period

[UnknowingFool]

Again, this whole thread is about "trading". You keep bringing up reporting and email.

Show us reading here where I said that Microsoft Windows Server & SQLServer 2005 were being used as anything other than the official trade data dissemination system, ok?

How about below where you are confusing two different functions.

By myself simply showing how/when/where/why Microsoft Windows Server + SQLServer 2005 do well in a high tpm high volume environs in a stock exchange called NASDAQ (out of Chicago Illinois USA iirc), as the official trade data dissemination system @ NASDAQ & it does well, 24x7 in failover clusters... & it used to be used in that capacity @ LSE as well before but didn't do well.

You keep implying that NASDAQ has done it but the LSE hasn't therefore it was a failure on the LSE. But you keep comparing a reporting system that NASDAQ runs on Windows with the trading system that LSE ran on Windows but ditched for Linux. You keep equating reporting and trading here. They are not the same thing. I keep pointing out that NASDAQ runs trading on Linux so that makes your comparison rather irrelevant. Also to undermine your points, I think the LSE still uses Windows for reporting.

The biggest clue that the two systems are completely different is what you describe as "high tpm" Because in your MS systems it is about 2 or 3 orders of magnitude from being acceptable in trading. 100,000 queries a day? That's 2 seconds of trades on NASDAQ (the same freaking company). That's fine for an external reporting system; it's not so good for trading.

See the comparison? It shows that 1 team @ NASDAQ does do it well, and the LSE team did not, period, for the SAME TASK @ hand! One team @ NASDAQ does it well, & LSE's team does not, using the same products (MS ware).

Again reporting and trading are not the same task. That is the flaw in your logic. The comparison is not suitable.

(This tends to agree with my conclusions in my post that the TEAMS around these wares matter as well, plus, the hardware too which has advanced since LSE used MS stuff, yielding more speed in THAT alone, which is NOT comparing "apples to apples" as the poster I replied to stated & I agreed with also)...apk

All I said is it's not an apples to apples comparison but you are trying to make it out to be one. See all your bolded texts above.

But if we were to discuss the problem in detail, the problem is that a reporting system is not a real-time system that a trading system is. And Windows and .NET are not well suited to this task. Even after upgrades to the system, even after MS themselves were involved with the deployment, the LSE decided Windows wasn't up to the task and went with Linux. The LSE publicly blamed Windows for the crash and even touted that they would save money by moving to Linux.

Now if I were MS and I had a ton of PR as well as case studies on the LSE trading system, I would do everything I could to (1) not to be publicly blamed even if it were my fault, (2) try to convince the LSE to stay on Windows, and (3) at least get the LSE to downplay the migration. MS could not do any of these.

126 microseconds what it cant do

[tempest69]

126 microseconds is the time light can travel about 38 kilometers. So NO, packets are not traveling far in that time. it's near the latency of standard Gig-E, so traders use Infiniband. And get closer to the exchange, as every 30 meters is a microsecond of advantage.

Re:Answer 2 questions Unknowing Fool, ok? apk

[UnknowingFool]

Question #1: What is/was LSE's "INFOLECT" and what was it built with and what is/was it for?

Question #2: What is NASDAQ's "MDDS" and what was it built with, and what is it for??

Answer to BOTH = BOTH systems were built using Windows Server 2003 + SQLServer 2005 and were for the purposes of trade data dissemination & booking.

*Sigh* The problem you haven't quite grasped the details so let me be as clear as possible here. Infolect/TradElect was LSE's TRADING system in that it processes transactions of purchases and sales. NASDAQ's MDDS system is used for REPORTING in that it messages to externals systems about the purchases and sales that have been made. The overhead screens that you seen on the NASDAQ floor and scrolling bars are examples of reporting systems. When it comes times to actually book an order, NASDAQ uses Linux (The terminals that traders use). However you cannot use the overhead TV screen to purchase or sell orders; you have to use a terminal to do so. If you don't understand the difference between reporting and trading then there is no hope you'll ever understand any other points.

This is no world's record

[jtara]

Hate to burst their bubble, but we were executing trades with ISLD in better time than this circa 2002 or so.

Pretty sure we saw some confirmations in the 80uSec range, and regularly in the 100 uSec range. Have to pull out the logs to be sure. We had a small high-frequency trading operation, which was probably one of the first. I got the idea to co-locate in the same building as ISLD at 50 Broad St. (We started out trading from St. Louis, LOL - moved to co-location at our broker 30 miles up the Hudson, and then into 50 Broad.) We initially co-located at their Internet provider at the time, located in the building, in order to get the closest Internet connection to receive quotes from ISLD via their ITCH interface. Later, we were able to obtain the use of a fibre in the elevator shaft and bypass the Internet part, and eventually were able to start entering our trades directly through OUCH, bypassing our broker's equipment, and sharing the same fibre connection (most firms had to use dedicated T1's at the time). (We leased T1s to some of the other ECNs, and got some data from our broker's system.) When ISLD moved to NJ, we moved our equipment to a co-location facility that they provided. (They never had an official one at 50 Broad.)

Not sure if ISLD had transitioned to Linux for the OUCH servers at that point, or were still using MS-DOS. Our own equipment used Windows 2000.

Josh Levine at ISLD was a fanatic about low latency. Most firms couldn't care less, so he was eager to work with anyone who showed an interest. We had a lot of discussions on how to decrease latencies, and he introduced me to TCP_NOWAIT, which disables the Nagle Algorithm in TCP stacks.

Other ECNs were resistant to change. I'm not sure if they knew just how far ahead of them ISLD was. At that time, based on our logs (we logged the timing of every trade with microsecond resolution) ISLD was the no-contest hands-down winner, with REDI a not-so-close second, followed by Instinet, and crap. ARCA was hampered in the extreme by their Chicago server location. Of these, REDI was the most willing to listen, and did cooperate in making changes to reduce latency. INCA and NASDAQ obviously eventually "got it", since they bought ISLD essentially for it's technology. There really wasn't much to the technology - it consisted of nothing more than an approach of insisting on knowing just where every microsecond goes, and eliminating unnecessary delays.

Current NASDAQ stats are better

[jtara]

BTW, here's the current published latency for NASDAQ's OUCH interface. 98uSec.

http://www.nasdaqtrader.com/trader.aspx?id=inet

I should point out that the times that I logged circa-2002 included a round-trip through our server. That is, they were measurements from the time our software decided to initiate a trade against ISLD to the time we received and processing a confirmation message indicating that we had bought the shares.

The NASDAQ figures quoted above are from the customer's network port, throgh NASDAQ's equipment, and back to the customer's network port. (I assume measured at the NASDAQ facility.) So, it's a significantly shorter path.

So what do we take from this?

- Linux can do ~100 uSec trades today.

- Linux and MS-DOS could do ~100 uSec trades several years ago.

- The additional latency of a customer-side trading system of Windows-2000/MFC several years ago was probably much less than 100 uSec, since the total round trip was in that range.

- .net might suck. Or maybe just un-talented programmers.

The fact that we aren't see 50uSec or 10uSec trades today is not surprising, given the exponential growth in volume...

I suspect that OUCH is still the undisputed king of low-latency execution.

Furthermore, NASDAQ's volume is very significantly greater than the LSE's, and OUCH-entered orders represent a significantly greater proportion of NASDAQ's total volume than LSE's "dark pool".

Wow, faster than an I/O interrupt

[lsatenstein]

Somewhere I believe that the microseconds are really 1000 times to fast. Should not the measurement be milliseconds or just over 1/10th second.

Re:NASDAQ mdds system does do trade booking

[UnknowingFool]

MS case study contradicts you: "MDDS receives direct feeds from NASDAQ's trade reporting system, and collects the data, storing it in SQL Server 2005. It is then available in real time for queries by market participants, including those using the NASDAQ Workstation, a Web-based tool that connects to NASDAQ trading systems."

"NASDAQ Deploys SQL Server 2005 to Support Real-Time Trade Booking and Queries"

"support"!="process" Samba "supports" Windows networks. Samba != Windows. MDDS helps traders in that they can gather information about trades, companies, prices, histories. In that way they support booking, but MDDS itself is not for booking. If you read the case study, the biggest clue is that MDDS could only handle 100,000 queries a day. The NASDAQ prides itself that it can handle 64,000 transactions per second. A point you have never addressed. If MDDS was used for booking, it is painfully inadequate. All you've shown is that you don't understand details.

Re:UnknowingFool, MDDS does trade booking too &

[UnknowingFool]

Where Mr. David Lester, CIO of LSE, plus Mr. Robin Paine their CTO backed him. Both even said INFOLECT is LSE's Trade Data Dissemination system then... he said it himself there, take a listen/look in fact!

Trade Dissemination == Reporting. Trade processing == Booking. Again you don't seem to grasp definitions very well. Read the MS case study again. It talks all about reporting and how MDDS would "help" booking. It doesn't mention that it would process the trade at all.