Wall St. Trading Servers To Power Off-Hour Clouds?

miller60 writes "As cloud computing gains traction, some Wall Street firms running armadas of servers to power high-frequency trading operations are contemplating leasing out their excess computing capacity after the trading day ends at 4 p.m. 'Once 4:30 rolls around, we don't need those machines,' said one CTO of a market data firm. 'There may be an opportunity there.' A similar revelation led to the creation of the cloud computing operation at Amazon.com, which built its infrastructure to handle peak Christmas-season loads that lasted just a few weeks each year."

Comments

Is This Secure?

[WrongSizeGlass]

I wonder if this poses any security concerns or problems? Is letting 'cloud users' access the servers that run out financial markets really a good idea?

Re:Is This Secure?

[DJoffe]

Is letting 'cloud users' access the servers that run out financial markets really a good idea?

No.

Re:Is This Secure?

[LostCluster]

We've already got Virutal Machine tech and several other ways of saying "Don't cross this line!" so unless it's implemented by an idiot I'd say we'll be okay.

Re:Is This Secure?

[Anonymous Coward]

I wonder if this poses any security concerns or problems? Is letting 'cloud users' access the servers that run out financial markets really a good idea?

You pose a good question. The answer is: It's all in the implementation.

a. What do these wall-street computers do? Seriously. Do these servers store data about stocks? Are they merely high throughput, low latency information distribution machines? In order to figure out if it's a security threat, we need these answers

If no data is being stored on these machines, then I see no reason why they can't wipe the VM and let it load up some default image. From there, it can do tons of cloud computing stuff.

But in the same regard, there have been articles on Slashdot about compromising a VM. I'm sure that it's low risk, but it would have to be a very targeted attack against a firm.

We need more data.

Re:Is This Secure?

[sopssa]

But Virtual Machine's are only as good as they're designed. Even the most known and biggest vendor VMWare has had serious bug and exploits in their software. For one example see this, which let an exe running in the guest OS exploit a vulnerability in the VM code to get code run in the host OS. A serious security risk, especially when were talking about Wall Street. Even getting an access to their internal network opens new possibilities.

Just because of this I think it's a stupid idea. Even more so because the gain is not really that much, but it can be really destructive. Someone will find a way to exploit it.

Re:Is This Secure?

[Huh?]

I could see this being useful as some sort of "private" cloud during off hours (i.e. selling time to banks for batch and accrual operations...etc), but allowing the unwashed masses access to the underlying infrastructure of something as important as financial trading just seems like a recipe for a security disaster.

Re:Is This Secure?

[Anonymous Coward]

We need more data.

I'm sure that we'll get plenty of data once someone hacks one of the servers.

Re:Is This Secure?

[toastar]

Is letting 'cloud users' access the servers that run out financial markets really a good idea?

No.

Citation needed.

The NAS and compute cores just need to be separate systems

Re:Is This Secure?

[c0d3g33k]

Is letting 'cloud users' access the servers that run out financial markets really a good idea?

No.

Citation needed.

http://www.answers.com/topic/common-sense

Re:Is This Secure?

[temolate]

they could keep all data on a SAN during the day and power that off (and/or physically disconnect it) and then keep all their client data on a separate SAN (and have that off/disconnected during the day). and it's not like that would be an *extra* SAN cost because they're going to have to keep the client's data somewhere either way.

Re:Is This Secure?

[icebraining]

IANASA*, but wouldn't putting the data servers (DB servers?) in separate machines and physically disconnecting the power before allowing access to the other machines be enough?

* I Am Not A Sys Admin

Re:Is This Secure?

[Hooya]

You could have the exploit install something into the host OS and have it run when the regular stuff is back on and connected.

Re:Is This Secure?

[cr_nucleus]

IANASA*, but wouldn't putting the data servers (DB servers?) in separate machines and physically disconnecting the power before allowing access to the other machines be enough?

It wouldn't as a compromised machine would just need to wait until the db gets back online...

Re:Is This Secure?

[fuzzyfuzzyfungus]

As secure as not doing so? Probably not quite? Pretty secure? If implemented correctly.

If this were just a matter of "Hey, let's give some random punters shell accounts on our major servers so they can run jobs at night!", it would be a transparently terrible idea. Bugs that allow an authenticated user with shell access to do unpleasant stuff to other users and/or the system aren't all that uncommon. However, particularly in modern Big Serious Setups, there can be significant fluidity and decoupling of "a given server"(in terms of its software configuration, data, users, access controls, etc.) and the actual boxes or blades with the CPUs and memory in them.

If, for instance, you have a whole bunch of compute servers that are booting via PXE or hardware iSCSI initiators, the only local state would be in the firmware embedded in various chips. With a change to your switches' VLAN rules, you could make your daytime trading storage, systems, and state completely inaccessable, and switch over to booting the compute nodes from the night storage side. If you were really paranoid, you could probably even arrange things so that there would be connections you could physically disconnect/power down so that even a super sneaky remote reconfiguration attack on your switches wouldn't help.

This would be riskier than doing nothing with them at night, of course, one can think of any number of possible fuckups, or really subtle persistent firmware reflash attacks against specific buggy devices, or what have you; but for good enough money, the risk would probably be worth it.

Re:Is This Secure?

[fuzzyfuzzyfungus]

With PXE boot, or iSCSI, or just an automated imaging job, there is no reason for the "host OS" to be constant.

In the first two cases, the 'host OS', and any 'local' programs would just be a blog of data loaded on boot by your compute machines from a networked storage system that would, presumably, be powered down or VLANed off during the night. Like pulling the physical HDDs; but a whole lot easier to automate.

An automated re-image cycle at beginning and end of day wouldn't be as elegant; but would have the same effect.

If you are running on any serious scale, and the vast majority of your machines couldn't be blown away and rebuilt swiftly and automatically, you already have a problem.

Re:Is This Secure?

[Michael Kristopeit]

With just an automated imaging job, there is no reason for the "host OS" to be constant.

the imaged host OS could be exploited to exploit the local network to alter the automated imaging jobs...

Re:Is This Secure?

[icebraining]

PXE is part of the BIOS.

While I'm not a server sysadmin, there are motherboard which have jumper to physically write-protect the BIOS chip. How could they alter the PXE booter using software only?

Re:Is This Secure?

[GraZZ]

Having worked in an electronic trading operation (brokerage side) I can already say that security is of very little concern in this kind of operation. Of the three corners of business operations, these systems need to be fast and cheap; quality concerns are secondary as long as you're not completely breaking the clients' trades.

Arguably security isn't very important in high frequency trading. You're trying to get an order to market before it can react to something. Once the order is there, people already know (a lot) about it.

Re:Is This Secure?

[DJoffe]

http://www.google.com/search?q=virtual+machine+vulnerabilities Sigh ... when will people ever learn.

Re:Is This Secure?

[ottothecow]

These are not the computers that make up the underlying infrastructure of financial trading.

These are the computers that other people use to tap into those computers for data and then make lots and lots of calculations on that data. The worst thing that could happen if you (Mr. Trading Firm) open up your after hours cpu cycles to joe-coder (or jack-hacker) is that they might be able to hack in and steal or front-run your model.

Sure, it sucks for your company, but you were the one who decided to do it and you were the one who screwed up the security. I am sure people get their models stolen or abused all of the time through other means (employees changing hands, industrial espionage, whatever) so the security risk is nothing new.

Finally, I would not be surprised to find that the firms who want to do this basically already have their own compute-farm clouds. Models are stored and worked from less powerful machines that then look to your local compute farm when they need to do the heavy lifting. If I can farm out my SAS computations to a system with far more CPU cycles (and faster, larger storage devices)...why can't some prop trading do the same with their models?

Re:Is This Secure?

[nospam007]

"Arguably security isn't very important in high frequency trading. You're trying to get an order to market before it can react to something. Once the order is there, people already know (a lot) about it."

So making other people's order go through slower while letting your own orders get through fast is the way to make money out of it?

Re:Is This Secure?

[davester666]

You are making the argument that they should go ahead with this because it is possible they COULD implement [and maintain] some kind of cloud setup while keeping their core financial setup completely secure and functional, ready to go each day at 8:30 am or whenever.

Yes, they COULD do it. It is theoretically possible.

Experience, history, bone-crushingly stupid decisions by the financial industry [hello, welcome to the recession, would you mind giving us $100 billion just to tide us over for the next year or so] say no, they don't have permission to do this. And it seems we need someone more powerful than any of the 3 branches of the US gov't to make sure they don't do it.

And by "bone-crushingly stupid decisions", I mean making decisions that result in the entire world going into financial difficulty. Of course, these same decisions made a whole lot of individuals incredibly wealthy, with evidently no significant downside for them.

Re:Is This Secure?

[dyingtolive]

Virtual Machines aren't the magic solution people typically regard them to be, no matter how amazing the tech is. The problem with VM and market data is we're talking literally millions of messages a second. VMs can't support production level processing of that level of traffic. I know for a fact that RMDS (the Reuters infrastructure software) isn't supported on any 'virtualization' technology other than Solaris Zones. I can also tell you that it has almost zero consideration put into it when it comes to security. Generally the philosophy is "Padlock the front door, leave all the others wide open."

Also. These market data people are the same ones who can't track down a god damned duplex mismatch between two boxes WHEN I TELL THEM WHERE IT IS. If places start actually doing stuff like this, I'm going to be doing a whole lot of flinching in the short future.

Disclaimer: I support the stuff.

Re:Is This Secure?

[VolciMaster]

...so unless it's implemented by an idiot I'd say we'll be okay.

I've worked for one of those "financial institutions"... the IT departments aren't always run by the brightest folks on the planet. And the staff, while generally good, doesn't know how to do anything that isn't rigidly defined in their guidebooks :-|

Re:Is This Secure?

[marcosdumay]

That wouldn't be enough. You'd want to get rid of all information stored by the uncontrolled processes, that means, you want to also change the boot disk, BIOS (or watever boots your system) firmware, and firmware of network devices at least (HBAs mainly) at least. And you'd want no local disks. Also, of course, you'd want to physicaly turn off everything that holds your trusted data.

One could make a server that makes those thasks easy, but current servers don't.

Re:Is This Secure?

[Michael Kristopeit]

if a single imaged host OS is exploited, couldn't they man in the middle the network stream from PXE to the imaging servers to continually protect other servers and have them protect itself?

i'm not saying it's easy, but this whole discussion steps from the worry of VMWare exploits that no one has proof exist.

like it or not, some of this financial trading data is surely already transmitted over the internet in some encrypted form... renting out the idle number crunching machines shouldn't be a problem as long as sensitive data is physically removed from the system before the customers are let in.

Re:Is This Secure?

[tehcyder]

unless it's implemented by an idiot I'd say we'll be okay

Well then it's lucky there are absolutely no idiots working in either IT or financial services, isn't it?

Security nightmare

[UndyingShadow]

This seems like a security nightmare waiting to happen. I understand everything would likely be virtualized, but it just makes me nervous that you would be able to rent time on servers that interact with the stock market, especially considering how panicky the market can be, and how badly everyone suffers when it does panic.

Re:Security nightmare

[Tanktalus]

There is no such thing as perfectly-secure. They could already be hacked today, this isn't changing that. It merely introduces a new attack vector: the VM sandbox.

If the ROI is there, this can be mitigated. If they create a cloud using their existing hardware, and move their own apps into a priority cloud on that hardware, and sell the excess CPU time, then not only does an attacker need to figure out what VM they are in, and what, if any, vulnerabilities there are in that VM that they can exploit, they have to cause the parent virtual machine (let's face it, there's no reason why a virtual cloud needs to be sitting on physical hardware directly - and, for this purpose, AIX, Sun, and the mainframe are already on virtual machines anyway) to run arbitrary code that would then go and find other virtual machines, find the one of interest, and then cause THAT virtual machine to give up information.

Breaking IN to a virtual machine might prove more difficult than breaking OUT of a virtual machine. And you may need to break in to all of them just to find the one you need.

Security by obscurity isn't the same as no security. It's not perfect, but it does reduce the exposure.

And, besides, maybe they only sell CPU time to other corporations where they can better track who has access to what, with passworded VPNs that only go directly to the cloud that the password given grants access to.

It's all about ROI, and whether they can make it work while improving their overall financial picture. I bet they can. I'm not betting whether they do or not.

Re:Security nightmare

[PingPongBoy]

  If they create a cloud using their existing hardware, and move their own apps into a priority cloud on that hardware, and sell the excess CPU time, then not only does an attacker need to figure out what VM they are in, and what, if any, vulnerabilities there are in that VM that they can exploit, they have to cause the parent virtual machine (let's face it, there's no reason why a virtual cloud needs to be sitting on physical hardware directly - and, for this purpose, AIX, Sun, and the mainframe are already on virtual machines anyway) to run arbitrary code that would then go and find other virtual machines, find the one of interest, and then cause THAT virtual machine to give up information

The reality that you see might be virtual - this may be more true than it sounds. So it was in the storyline of The Matrix.

The other side of the coin: why all the fuss about the security of the stock market? It's the monkeys that run this zoo. Who was chairman of Nasdaq? Bernie Madoff. All this talk about Dow 11,000 might be just talk - can YOU verify the prices and volumes of trading? Can YOU hack into the electronics and trace all the transactions? Madoff was able to operate for years because no one was able to trace his ass. The entire market can be more insidious. If an individual wants to sell out, but the money isn't there, that can be explained away by a sudden stock drop - good luck verifying it.

The hackers have already positioned themselves on the inside!

Not a new idea...

[LostCluster]

Compuserve was founded by a life insurance company to make a consumer service run on their business systems with the expectation that the consumers would use it during non-working hours.

When it comes down to it, nobody needs their clock cycles 24/7 at even load, even though that's what computers are designed to do. Shared services for the win!

Re:Not a new idea...

[geekoid]

" nobody needs their clock cycles 24/7"
What about computers the service cloud computing?

Re:Not a new idea...

[Anonymous Coward]

What about computers the service cloud computing?

What about sentences the make sense?

Re:Not a new idea...

[Ironchew]

When it comes down to it, nobody needs their clock cycles 24/7 at even load, even though that's what computers are designed to do.

Except for the halt instruction, found in most CPUs, that drastically reduces power consumption while the CPU isn't doing anything. Running a computer at 100% load sucks up more energy, directly and indirectly (e.g. cooling). That's why I don't run x@home distributed computing at 100%; it wears down the computer faster.

Re:Not a new idea...

[smchris]

No kidding. It was my understanding they all were. I was more of a GEnie man myself because it was cheaper -- which was GE's philosophy -- heavy on cheap because they were going for the off-hour gravy and hardly advertised. That said, the management did a great job with what they had into the 90s.

Really, just interesting it took that long for the light bulb to go off on Wall Street.

Re:Not a new idea...

[Anonymous Coward]

Getting value out of unused resources is cool. You may recall that when AOL bought the Compuserve service, a company called Worldcom (later MCI Worldcom) bought the Compuserve network infrastructure. What is less well known is that Worldcom was originally known as LDDS (Long Distance Discount Service), a telecom company that was spun off from a gas company.

The idea was that this regional gas company (I don't recall the name) had overbuilt their urban pipeline infrastructure, and so they had all these empty tubes, and someone had the bright idea of filling them with *something*... Fiber optic lines!

Well, the board of directors and stockholders in the gas company were conservative and didn't want to get into that business, so a team from the company spun off and started their own telecom business, and bought the pipes and right-of-way from the gas company. The parent saw steady growth as most utility companies do, and of course LDDS eventually fell under good old Bernie Ebbers' control (as MCI Worldcom), and he, along with his buddies at Enron, practically destroyed the stock market around the turn of the century!

Re:Not a new idea...

[tehcyder]

Getting value out of unused resources is cool.

And for your example you choose MCI Worldcom/Enron, which is a bizarre definition of cool.

heh...yeah.

[Pojut]

$20 says this idea was cooked up by someone who heard about "cloud computing" on the radio while in his cushy office, signing official looking papers and making a big fuss about "revenue".

Wow

[dachshund]

I sure hope Wall Street is utterly confident in the security of their operating systems, VMs, low-level peripheral firmware, etc. Because if they're not absolutely confident, they should treat all of those machines as potentially untrusted from the moment they open them up to the world. This holds even if they constantly re-image.

When you're talking about the kind of money Wall Street stands to lose from a clever security breach, no amount of paranoia is too ridiculous.

Re:Wow

[amirulbahr]

Re-image? Yeah, I'm sure the NYSE "re-images" their clusters once a month because you know Windows always feels faster after a fresh re-image.

Get rid of them entirely

[TubeSteak]

Not to derail the conversation, but high frequency trading doesn't contribute much to the stock market's ability to set optimal prices.

What actually happens is that high frequency traders squeeze in while prices are moving and they siphon off money. Neither the original seller, nor the original buyer gets the best price, and the high frequency traders make a mint.

Re:Get rid of them entirely

[wjousts]

Exactly this. High frequency trading really perverts what the markets are for. Make a law that you have to hold a stock for at least a day (maybe even just an hour would work) and you might restore some sanity to the market as well as giving those traders who don't have server farms running advanced trading algorithms a fighting chance.

Re:Get rid of them entirely

[LostCluster]

Yep. Market regulations are all about a "level playing field"... no using information others can't get yet for trades. Just ask Martha Stewart.

I really think a rate limit that makes high-frequency trading impossible is a good idea. Just like the TV ad goes, why do you want to sell something you just bought in an auction? Everybody in the room already said they wouldn't pay what you paid.

Re:Get rid of them entirely

[Anonymous Coward]

Not really. Someone has to quote a bid and an ask all day long, quickly so you get your shares when you want. And with competition between high frequency players trying to extract a spread or rebate, you get some pretty tight spreads giving you a good execution price without you having to pay the hidden cost of the spread. Most heavy volume stocks are a penny wide.

Second, no one can "siphon off" money without taking a risk. Prices moving don't imply "free money" for the taking and there is substantial risk involved. No one in the market gets compensated for zero risk. It's like any other business where people take risks with their capital to provide you with a service. In this case, that service is providing you with liquidity.

Re:Get rid of them entirely

[Herkum01]

I had thought about this for a few years and had two ideas on it. The first was like yours, the person has to hold onto the stock for a set period of time.

The second was don't display any information about trades during the day, just accumulate orders. At the end of the day match up the buy and sell orders.

They need to stop rewarding this stock exchanges on the number of trades and instead focus on providing stability for the fair pricing of stock for companies.

Re:Get rid of them entirely

[TubeSteak]

Exactly this. High frequency trading really perverts what the markets are for. Make a law that you have to hold a stock for at least a day (maybe even just an hour would work) and you might restore some sanity to the market as well as giving those traders who don't have server farms running advanced trading algorithms a fighting chance.

You'd only have to slow them down by a second (literally 1 second) to completely bring their business model to a halt.

HFTs buy zero second data feeds from an exchange and some exchanges actually sell pre-zero-sec access.
Meaning that the HFTs can see trades a few hundred milliseconds before they post, allowing the trader to sneak in and buy/sell.
The SEC is looking to ban the practice, but hasn't gotten around to doing so yet.

Re:Get rid of them entirely

[jackchance]

Make a law that you have to hold a stock for at least a day

That's an interesting idea. But it's obviously flawed. Let's say i buy some pharma stock at 9am. At 12 pm the FDA announces that they have found that pharma's main product to be dangerous. Should I not be allowed to sell?

I would argue that derivative speculation and complicated leveraging packages are what make the market insane.
Other people have made similar comments.

Re:Get rid of them entirely

[aamcf]

They might not get the best price, but they get the price they are willing to sell (or buy) at. If you don't want to sell your stock at $10, then don't. Nobody is forcing you to.

It might seem odd to us, but the reason that high-frequency trading works is because to someone, somewhere thinks that it is better to buy XYZ at $10 now rather than $9.99 at some unknown time in the future.

Re:Get rid of them entirely

[aamcf]

How would this work for the market makers et al? Sounds like your suggestion would make market making more difficult, thus reducing liquidity, and thus reducing prices.

Re:Get rid of them entirely

[aamcf]

If a company doesn't like the way a particular stock market works they can just go and get listed on another one. If an investor doesn't like the way a particular stock market works they can just invest elsewhere.

Re:Get rid of them entirely

[aamcf]

Why should the practice be banned? There has always been an advantage to getting certain information first. If your trading strategy depends on you being first to have some information, then either pay to be among the first to get it, or change strategy.

Re:Get rid of them entirely

[AcidPenguin9873]

I have never understood this desire for a continuous 1 cent spread. I think HFT use that argument to justify their existence. The market does not require it.

Market makers set bid and ask prices, and when we trade stock, the market maker makes the spread. If HFT weren't there, market makers would probably make more money due to a wider spread, but I'd be okay with that since they are the ones REALLY providing liquidity.

Second, no one can "siphon off" money without taking a risk.

Bzzt. The whole point of HFT is to exploit microsecond-long arbitrage opportunities, which are by definition zero-risk.

Re:Get rid of them entirely

[aamcf]

But why eliminate it at all? Why ban a practice that increases liquidity?

Re:Get rid of them entirely

[aamcf]

The information published on market data feeds is public information; it is not insider trading. Everyone is on a level playing field. It is just some people can run faster.

Re:Get rid of them entirely

[maxume]

If they aren't contributing much information, then, by definition, they aren't taking much of the money either.

If it is really a huge problem, buyers and sellers will move to trading systems with higher per-trade fees, or with promises about the nature of the other party in each trade.

Re:Get rid of them entirely

[metlin]

Seriously? You link to two posts - one by a blogger and another by someone from the American Trucking Association - on financial derivatives?

Even people who've decried them (think Buffett or Taleb) have used them themselves. Leverage is a useful tool, especially when companies are trying to stretch the dime when they are low on reserves.

Derivatives, speculation, and derivative speculation are merely natural progressions of an increasingly complex financial eco-system.

What made the market insane are government and tax payer subsidized credit and incentives for financial institutions to keep being destructive, and getting rewarded at the end of the day. The things you mention are merely proverbial straws, and have existed in one form or another long before this crisis.

Re:Get rid of them entirely

[afidel]

Because the entire idea of the SEC is that all information is to be known by all parties at roughly the same time, it's why insider trading is illegal for instance.

Re:Get rid of them entirely

[jackchance]

Seriously? You link to two posts - one by a blogger and another by someone from the American Trucking Association - on financial derivatives?

You are totally right to call me out. I had read a much better article , i think by Krugman, in the NYTimes but i couldn't find it in the 10 seconds that i tried.

This is a bit better

Re:Get rid of them entirely

[dintech]

some exchanges actually sell pre-zero-sec access

I work in Europe doing exchange connectivity (using Wombat/direct connections) for a major investment bank and I've never come accross anything better than servers co-located at the exchange. I always imagined that anything "pre-zero" or different data would be illegal in the civilised world. Which exchange are you talking about? I'll talk to my traders and get them a connection...

Re:Get rid of them entirely

[dintech]

why do you want to sell something you just bought in an auction? Everybody in the room already said they wouldn't pay what you paid.

Because the information available on stocks (market data, economic data, news, etc....) changes very quickly. All those thousands of financial intruments and economic indicators are all interrelated.

Re:Get rid of them entirely

[wjousts]

Make a law that you have to hold a stock for at least a day

That's an interesting idea. But it's obviously flawed. Let's say i buy some pharma stock at 9am. At 12 pm the FDA announces that they have found that pharma's main product to be dangerous. Should I not be allowed to sell?

That's the risk you take when you invest. To turn your argument on it's head, you buy 1000 stocks in Prescott Pharmaceuticals at 9am. At 12 pm the FDA annoucnes that their main product causes "testicular cranberrying", you read that over lunch and rush to sell your stock at 12:05 pm. Meanwhile, Large Investment Bank Co. buys 10,000 stocks in Prescott at 11:59 am, the news breaks at 12 pm and their high speed trading network dumps all their stock at 12.00:01 PM. By the time you get to sell the price has already tanked and by 1 PM, Large Investment Bank Co. is buying back the stocks they shorted at 12.00:02 PM

You get screwed because unless you are Large Investment Bank Co. with access to high performance servers running right at the exchange, you can't possibly win.

Re:Get rid of them entirely

[wjousts]

I like that idea.

Re:Get rid of them entirely

[EvanED]

If an investor doesn't like the way a particular stock market works they can just invest elsewhere.

Ooo, is this another one of those non-choice "choices" I have that makes capitalism so great?

"Once 4:30 rolls around..."

[mcmonkey]

Said the CTO who is now looking for a job.

NYSE closes at 4:30. But there are other markets. And the data flows 24/7.

There is no reason for these systems to have spare cycles.

Re:"Once 4:30 rolls around..."

[MobileTatsu-NJG]

Are you one of the guys that maintains those machines?

Re:"Once 4:30 rolls around..."

[LostCluster]

The servers that run the global markets are, well, global. There's no need for a trading firm to have resources in NYC after business hours, because the people and the computers that do things on the markets elsewhere are, well, elsewhere.

Re:"Once 4:30 rolls around..."

[keithpreston]

Location is the reason these system have spare cycles. They build data centers across the street from the market so that they have the lowest latency access to the market (it matters for high frequency trading). A data center on Wall Street is useless for other markets.

Re:"Once 4:30 rolls around..."

[wjousts]

I had a very similar thought. The obvious way to make use of the servers during hours when the markets are closed is to use them in other markets that are open. For example, trade the NYSE during the day and trade the Asian stock markets at night. Synergy!

Re:"Once 4:30 rolls around..."

[LWATCDR]

That has got to be the most expensive data center for m^2 on the planet.
They don't care about power or heat it all would comedown to foot print.
I bet IBM sells a lot of POWER cpus there.

Re:"Once 4:30 rolls around..."

[eastlight_jim]

Believe it or not, trading by computer has now become so quick that companies vie for server space closer and closer to the place where trades are taking place. This article" even goes as far as suggesting that a 1 millisecond advantage in trading applications can be worth $100 million a year to a major brokerage firm with such low-latency trading becoming more common.

I think the ping to Asia may be a bit more than that!

Re:"Once 4:30 rolls around..."

[mystik]

These High-Frequency trading systems require sub-second responses from the market servers. They're usually collocated in the same structure as the marketing servers, and firms pay handsomely for that kind of space and network access.

When that market closes, you don't want these machines trading overseas, the latency to reach those servers would negate the entire purpose of these machines.

Re:"Once 4:30 rolls around..."

[NeumannCons]

It's my understanding that the high frequency traders need machines that are physically near to the market they're trading stocks on to minimize hops, lag, etc. and to chronologically beat everyone else who's trying to do the same. Everything is built to make transactions that can be executed almost immediately to take advantage of stocks going up or down before everyone else does thereby altering prices.

I'm going to guess that trying to play that game for servers overseas where lag can be measured in seconds won't work when your competition has servers located in the same building the market is in.

Re:"Once 4:30 rolls around..."

[toastar]

Said the CTO who is now looking for a job.

NYSE closes at 4:30. But there are other markets. And the data flows 24/7.

There is no reason for these systems to have spare cycles.

So your going to try to make high frequency trades that require ultra low latency in Japan with your servers that are in new york?

Re:"Once 4:30 rolls around..."

[Anonymous Coward]

So of course, the logical answer is to put these servers into a containerized data center, launched into low earth orbit from a giant railgun, and splashing down in the harbor of Hong Kong or some other financial center. Just do this twice a day, back and forth to New York, and they'll always be co-located during market hours.

Then: profit!

Re:"Once 4:30 rolls around..."

[thoughtsatthemoment]

So they were trying to get 1 millisecond by moving closer to the source. Since it's a software system, I guess the code is already optimized to nanosecond level already.

Re:"Once 4:30 rolls around..."

[afidel]

Don't be so sure, the NYSE runs largely on HP DL585's running Linux. Their next generation systems appear to be running the Xeon 5500, so probably DL380 G6's.

Re:"Once 4:30 rolls around..."

[LWATCDR]

Kind of shocking to be honest but then just how do you get the most performance per M^2 today? Or maybe they already have monster machine rooms left over from the Mainframe days.

Re:"Once 4:30 rolls around..."

[afidel]

They just built a 400k sq ft datacenter over in NJ, mostly hoping to attract these sorts of trading systems. As to what kind of square footage they have on Wall St, I have no idea.

Re:"Once 4:30 rolls around..."

[LWATCDR]

Well over in NJ the cost of real estate is probably a lot less than on Wall Street. Not as cheap as say NC or eastern WA but a heck of a lot cheaper than Wall St. There X86+Linux probably gives you the best bang for the buck.

Cost to Society

[Oxford_Comma_Lover]

Yes. If Amazon went down tomorrow and never came back, society would be fine. If the stock exchange were taken over by malicious but hidden computer software for months, and then finally was taken down, the damage to society would be MUCH more severe. It's not just a way of exchanging everything, it's a way of establishing who owns what. If suddenly nobody knows who really owns every stock that's traded in the last six months, we've got a major frikkin problem. We shouldn't, maybe, but we do because money is an illusion.

Re:Cost to Society

[LostCluster]

A stock-trading unauthorized program is the nightmare of financial IT so there's frequent checks to make sure that doesn't happen. If a financial company doesn't know what it owns, it doesn't know much at all.

If something is being artificially inflated or deflated there will be people asking "Why?". A human rogue trader, trading with money that isn't his and doing something other than what the money's owner has authorized him to do is an international story when one happens. A software rogue trader wouldn't last very long.

Re:Cost to Society

[BitZtream]

the damage to society would be MUCH more severe

Only if someone told society that it was gone. The stock market effects a few select people drastically, but really has little influence on our daily lives in and of itself.

The panic and fear generated as a result of a market failure as people start hording for no reason other than CNN or FOX said the world was coming to an end are what causes problems.

If it simply ceased to exist the world would change very little. Stocks are based on what someone thinks a stock is worth, its precieved value.

Lets face it, I own more than a few shares of Apple, but until I sell them they are barely useful for starting a fire. If the stock market ceased to exist, their actual value would be identical to what it is now.

Re:Cost to Society

[interkin3tic]

We shouldn't, maybe, but we do because money is an illusion.

You mean how you can fold bills just right and tilt it and it looks like the president is smiling or frowning???

Re:Cost to Society

[Jenming]

The servers in question are not running the stock exchange. Rather they are owned by some bank to operate one of their trading strategies. That bank possibly could lose some money and if f it was a big enough amount of money it could effect the rest of the stock market, however its not like compromising the stock exchange as a whole.

Re:Cost to Society

[DigiShaman]

The stock market effects a few select people drastically, but really has little influence on our daily lives in and of itself.

Your wrong. The stock market has a profound impact on everyone's lives throughout the world. While it doesn't impact us directly, it does through many other financial abstract layers. Everything from your IRA, savings, to corporate reinvesting which including hiring of new employees to expand a business.

Have none of you learned from the Great Depression. Take away the stock market and watch what happens!

Re:Cost to Society

[metlin]

You know, I almost half agreed with your comment, until I came upon that last sentence. Seriously?

A dollar bill can directly translate into a can of coke; the $10 in your wallet directly translates into a DVD; the $300,000 in a bank account into a house.

A representation of worth, perhaps. A convenient exchange mechanism/medium with other social, political, and economic constructs (e.g. fear, value etc) projected on to it, perhaps. But an illusion? Only a fool would think so.

Re:Cost to Society

[Oxford_Comma_Lover]

My Last sentence was a reflection of having dinner on the stove and only having a second to write it. =)

I wonder if he bothered to

[geekoid]

consult with his technical people.

What am I thinking, Of course not.

Re:I wonder if he bothered to

[sunderland56]

Especially since these are not particularly fast computers. They are optimized to do financial transactions - something which requires incredibly fast I/O, but generally only requires a single addition per transaction.

So, if your cloud requires blindingly fast I/O, great (but then the NYSE would need to buy a bunch more disks). If your cloud requires some compute power, these may not be ideal machines.

Re:So what they are saying is

[Midnight Thunder]

I still don't understand why the Stock Market can't run 24/7.

Trading is still done by people, who have a horrible tendency to want to do things like eat, sleep and such forth.

Re:So what they are saying is

[ottothecow]

It can run 24/7...we just don't want it to.

In fact, with after hours trading and foreign markets, it really does run 24/7, just not on the floor of the NYSE (and not as liquid).

I would posit that a good reason for this might be that the stock market is already panicky enough and being closed for most of the 24 hour day gives people a little more time to thing about what is going on. A perfect example is companies that release earnings and other important news after the closing bell. It gives people time to process the information rather than giving the fastest guy a chance to make a quick profit. For the *real* purpose of the stock exchange, it does not need to be open. If your goal is to raise capital for a business (or invest in one) rather than speculate and day-trade, the current market hours are just fine.

Veeeeery bad idea

[Angst Badger]

I'm sure that there are lots of people who would like to run jobs on machines that normally process billions of dollars in transactions.

Of course, a certain percentage of them are going to be probing for security holes so they can steal financial data, but no worries -- the government will bail them out, right?

destroy them

[roman_mir]

I dream that this entire high frequency scam is declared illegal and all involved are placed where they belong with all of their property confiscated.

Here is what happens when HFT is done: 2 parties agree on a price, the HFT meddles with in a way, that takes out money from the transaction, so the buyer sells lower and the seller buys higher. That little bit of difference is stolen by HFT.

These are thieves, we are discussing here, understand that. So they found a way to make some profit on their infrastructure? Well, great for them. 4,000,000 transactions per second they are talking about for one shop. That's 4,000,000 thefts per second.

Re:destroy them

[roman_mir]

so the buyer sells lower and the seller buys higher.

- a typo obviously. Buyer buys higher and seller sells lower than anticipated. The HFT transaction takes the edges. There is no value for economy other than the bank account of the HFT transaction owner becomes bigger. They didn't care about what was bought, what was sold, they have no idea what is happening with what is bought/sold, they are taking money from participants who may, in theory, have done something productive with it. Of-course that's not what happens. Just understand, that many people who own mutual funds and other investments are the suckers in this game of stealing the penny from the 'tray for everyone', to put in Office Space terms, so that /. would understand.

Re:destroy them

[blahplusplus]

I would argue that the stock market itself is unproductive to how a healthy business runs, the constant desire for growth is actually _unhealthy_ it inspires the constant pump and dump mentality of a business's employee's. You see this especially in the game's industry where entire teams are laid off and their skills go unused.

Re:destroy them

[BitZtream]

Funny, our ability to use tools puts a big gap between our species and the rest.

Yet ... here you are ... pointing out how its unfair that someone uses a tool that you don't have, yet I'm sure you'd be okay with using the tools you have at your disposal for making decisions.

If you have a problem with their tools then either get their tools or make it fair on everyone by not allowing any tools.

Of course ... if you don't allow any tools then the only way to play is to actually stand on the trading floor, which is once again going to cut you out.

Lifes not fair, stop pretending it is or that it should be and you'll get a lot further in life.

The stock market is one massive money laundering scam anyway, you're an idiot if you think of it as anything more than gambling, its far too disconnected from reality to be anything else. Due to the panicky nature of people and the ability for CNN to control that panic, I'd wager there is more entropy in the stock market when its 'stable' than there is in most lotto machines.

Re:destroy them

[roman_mir]

If you have a $100 and you are holding it in paper at home for 10 years, then it means that in this time you have lost at least half of the purchasing value of your paper, so your purchase power has gone down from $100 to $50. That is quite a fair estimate of inflation in normal conditions. Question is: are you OK with it?

Due to the fact that there is always someone who is willing to do extra work to live better than the rest, some new money will be created from the value of the work that is done by these individuals. Of-course what happens now is that the brightest and best connected/educated individuals have figured out that the most money can be made by stealing, not by providing services that people want, and that's the problem with trading on stock market. It started as a good idea: stock released by a company to collect capital to create something of value that will pay back in dividends, instead the actual stock (the paper) itself became the product that is sold and bought, the dividends are not what is being sought. Of-course there are still long term investors, who invest into solid companies with solid products (it's Asia mostly now) and they still do get paid dividends. It really used to be the primary objective: collect money from investors now, use the money to grow business and investors get paid.

Stock market today has little to do with this idea, it is no longer relevant what the business does, how the company is doing, it is about buying and selling the paper (electronically). It is all about manipulation and shorting at this point by these market makers who have enormous power because they see all the bets.

The stock market is completely rigged right now, just like a bad casino, where the house knows all the cards in everybody's hands but it does not bet against people directly, it just gets in between a bet that is made by two parties and it covertly makes 2 other bets with these same people, but it's done so fast that these parties do not even know about it, it's like a magic trick.

In reality it should have never been allowed for private interests to control the information about everyone's bets this way without any oversight (and nobody really checks anything anymore, it is obvious that the government officials are in on it and also do not understand the game.)

It's all screwed up. The best bet at this point is to buy real things: coins, stamps, furniture, energy, canned food, whatever and to avoid wasting your money in these market conditions. This market will flop soon, it will implode because it is hollow and heavy at the same time.

Re:destroy them

[roman_mir]

Straight from TFA, 4MILLION HFT transactions on that hardware per second. Whether all of them end up in a real purchase or a sale or whether they are done TO PUSH the prices up or down and immediately canceled, every transaction they do there is another theft.

Re:destroy them

[L3370]

coins...ok that sounds good. They become collectibles and precious metals typically outpace inflation. But do note, if you are buying gold now you ARE buying at a peak price. That is inherently risky.
stamps...maybe
Furniture????...I never knew love seats to be an appreciating asset.
Energy and canned food? ok now you sound paranoid. We've been in economic slumps before, and I think we are strong enough to get back out. In the mean time, others will continue to make money off the atmosphere of fear.

Re:destroy them

[roman_mir]

Furniture that is collectible obviously. Stamps also can be collectible or not, same with everything.

Whatever you are buying, my point is valid: this market is rigged and it will collapse, since it is based on free money that is printed by the Fed and that is loaned to financial firms at no cost, who turn around and play with it in various ways. The next 'bubble' is in Treasury bills (but it is all the same bubble, all the way back, the housing, the tech bubble in the nineties.) The point is that all of this is the bubble of the free money that is being given by the government to the preferred people, corporations. They are sucking out the value of whatever used to be produced and saved, they are almost done now. It will collapse and then the prices will skyrocket because the money itself will be worth no more than the silly paper it is printed on.

Food, energy, nonperishable goods, collectibles, weapons probably, any of that is better than playing in this rigged game unless you are playing with free money, unless you are the preferred financial institution and are getting free money, unless you own these HFT machines.

Re:destroy them

[roman_mir]

certainly the stocks themselves have become the products that are bought and sold and they are dipping and raising suddenly due only to the manipulation that mostly has nothing to do with the actual company behind them (unless the company is doing the manipulation of-course).

No question that the era of investors putting money into a business and getting dividends with publicly traded companies is gone, in the West at least. It is all manipulation, but HFTs are so blatant and nothing is done about it, where are the regulators? Well, they don't understand the game and the top brass is in on it.

Re:destroy them

[roman_mir]

Just because you are OK with this theft does not make it any less of theft, just shows what you are.

Re:destroy them

[Anonymous Coward]

As a former developer from a very successful HF trading firm, I can tell you that's not how it works. You are forgetting one crucial item, which is that there is a spread in the market. At any time, the price at which people are willing to sell, and which other people are willing to buy, is different. I'm willing to sell a share for $3, but you only want to pay $2 to buy it. (This spread is unrealistically wide to make it easier to illustrate the point). Unless one of us changes our mind and is willing to sell for less or buy for more, there's no transactions going on.

There are many ways of doing high frequency trading, but here's one common way. The high frequency trader, call him T, steps in and says that he willing to sell it for $2.75 and buy it for $2.25. I think, hmm, I would really like to get $3 for my share, but unless I can sell it, I'm not making any money. $2.25 is better than $2, so I'll sell to T for $2.25 instead of to you for $2. Two things have happened here: Now I'm able to sell my share, and I wasn't able to before because $2 was just too low. Also, I got a better price than if I bought from you.

T now owns 1 share, and offers to sell it for $2.75. You decide that you really need to buy a share. At least $2.75 is a better price than $3. It is not as good as the $2 price you really wanted, but you can't always get the price you want, so you buy the share for $2.75. You got a better price from T than you would have gotten from me.

T gets $0.50 for his trouble. It is not risk free, but I'll get to that later.

One way of looking at this is that T is a parasite. After all, you could have been willing to buy for $2.25 and made the transaction. But the point is that you weren't willing to do it. You were free to offer 2.25, but you didn't do it. I was free to sell for $2.75, but I didn't do it. Without T's intervention, you and I could have stared at each other for days without buying or selling.

Thus, the other way of looking at it is that T made the market more liquid, and decreased the spread between bid and ask. I made more than I would have without T, and you paid less than you would have without T.

T's transaction is not without risk. For instance, the market may move the wrong way, leaving T to take a loss. I may sell my share to T for $2.25, but then the market drops, and no one wants to buy it for $2.75. T drops his price to $2.65. Still no takers. He drops it to $2.55. Still can't sell. At this point, he can either chase the price down bit by bit, which is dangerous because it can lead to a huge loss, or give up now and sell at whatever price buyers name, which is probably more like $1.80 at this point. Instead of a $0.50 gain, he takes a $0.45 loss. There are other ways to loose money, too.

Also, T is not alone in the market. Other people are doing this, too. Someone will see T's juicy profits, and offer to buy for $2.30 and sell for $2.70, taking less profit, but getting the business instead of T. But someone else will offer to buy for $2.40 and sell for $2.60. The high frequency traders pile on, and the spread shrinks until it is so small that only the most efficient traders can make any money, and they usually make only fractions of a cent per share. As a retail investor, this is great. With more competition, I can sell for more and buy for less than without these guys.

Re:destroy them

[L3370]

This mentality has been seen before. The Great Depression brought on a generation of hoarders. They'd save their cash buried in the back yard, and have a basement full of canned meat, tin foil and a mountain of "collectibles" sitting in there garage. Items they can't bare to part with...because they might need it one day when the next Great Depression rolls by. Fast forward a few decades. Their cash wasn't sealed properly and half of it has rotted beyond recognition. Inflation took care of the rest. The canned meat is questionable, tin has been replaced with cheaper abundant aluminum, and their mountain of collectible items are worth less than the purchase price.

Either way you invest you inherit risk. Your suggestion of food and guns seems like a good idea if you think the world market drops to zero and everything fails without the hope of recovery. But in that situation. Guns and canned food aren't investment grade commodities, their survival tools. That isnt returning profit. Its breaking even at best.

We've seen plenty of economic downturns but historically it has ALWAYS risen afterwards. This is where you must ask yourself...Should I bet on the system that has a 100% success rate in returning profit over time, or should I prepare for the end of the world?
Or you can be smart and invest in the market system, and use a small portion of your profits and income to prepare for a couple years of tough times.

Re:destroy them

[roman_mir]

What we are observing right now has not happened in human history before. The West sees its jobs disappear and move to the East while it is no longer the nation that saves and lends, it is in deep debt. This is not going to be like before because the setup is different. What is coming is a collapse of the dollar.

Re:destroy them

[martin-boundary]

This sounds good, but you're making a number of economic mistakes:

1) The trader T is virtually alone in the pre 1 second market, when compared with the size of the original market. Thus the original market has been split into two markets, one market for a small number of privileged players and one market for everybody (privileged and unprivileged players). That always leads to suboptimal prices. To get optimal prices in the market, it's necessary to coalesce these two markets.

2) The idea that making an extra sale is always better than not making a sale is wrong. In the market, the nonexistence of a transaction is just as important as the existence of a transaction, so by facilitating a deal the trader T is biasing the natural outcome, which again results in suboptimal price movements.

3) You are implicitly treating T as a regular market player, when you describe T's profit and risk strategies. Regular players are good for the market, but T is not a regular player, instead T is a player in two markets, which cannot interact until we find out how to do time travel. The interaction is only one-way. The actions of T in the 1 second market lead to certain advantages in the initial portfolio of T when viewed in the later market. These advantages would not exist if T did not have the 1 second privilege, due to all the other (privileged and unprivileged) players in the market. But these other players cannot act, as they are not part of the first market. T therefore plays the role of a middle man which cannot be routed around.

4) The total costs of joining the high frequency trading markets are nontrivial, and as such represent an inefficiency in the market that results in suboptimal outcomes for the market as a whole (eg pre and post 1 second). Since it is physically impossible for everybody's servers to be collocated, the only way to fix this inefficiency is to allow nobody to perform high frequency trading.

5) The privileged (1 second) market is necessarily automated. This leads to a market in which all players are algorithms. From the economic perspective, this is again inefficient, as the algorithms are not the players, the companies are. An algorithm (especially one that needs to be fast) is only a limited representation of an economic preference function.

Re:destroy them

[indigodotnu]

You misunderstand: In a transaction you have a buyer and a sellers, and a facilitating party such as an exchange like NYSE. High Frequency firms participate on both the buy or sell, but never act as a middleman to the overall transaction.

Re:destroy them

[ottothecow]

Please explain who they are stealing from.

What is happening is this: you are selling your car and you put out an ad in the paper saying "anybody want to buy my car" and Mr. HFT says "ok how about $10,000". If you accept his offer, it means you were perfectly willing to accept $10,000--if Bob was willing to offer $10,001 but waited too long to call you, it does not mean that Mr. HFT stole from you. If Mr. HFT turns around and sells the car to Bob for $10,001 (at a whopping $1 profit), it doesn't mean he has stolen from Bob--Sure bob could have gotten the car earlier for $10,000 but he has demonstrated that he is perfectly willing to pay $10,001 for the car.

If you don't like the price offered, you do not have to enter into the transaction. No sane person who actually understands the issues would say that the high-frequency traders are stealing. The questionable part of the practice that the SEC would like to ban is the way in which they get the information--they are the first guy on the paper route (or sometimes they work at the printing press) while bob is a few houses down. This gives them an unfair advantage that allows them to hop on these deals first--remember, the seller is still perfectly willing to sell for their price and and the buyer is perfectly willing to buy and all parties walk away satisfied. The only problem is the unfair advantage which does not make it stealing.

Re:destroy them

[sgt101]

Be careful with your statement "ALWAYS risen". It can be a long time (20 years after the last depression) before the rise happens, and when it happens it happens according to a stock market index; the assets in that index can be and are completely different from the assets that started off in it. How long do companies last? 80 years is very unusual.

So - by the time the "market" rises; the stock you bought is worthless, a world war has happened and all your assets are now priced in rembi and... oh - you are dead from effects of the market driven healthcare system that you insisted on.

Wait... wait...
 

Re:destroy them

[sgt101]

Holy heaven above ! Where has this AC been for the last 3 years?

Various participants are out of money.

The market dried up.

We were all robbed to restart it.

There is more robbing going on.

The next round will *not* be pretty.

Don't think about the 1930's, think about 1789.

Re:destroy them

[sgt101]

I have to enter into the transaction. Because my taxes are used to bail out the victims (companies) of these trades, and I work for one of the victims as well.

These trades are not made on the basis of the shareholder value delivered, they are based on scams; insider information, market manipulation, rumours, momentum trading, chartists.

This is done to move money about; to move it from the old lady down the street to the bastard in a shiny suit. This is not a moving about done on the basis of high intelligence and hard work - it is a moving about done with trickery and force majeure, her money is in the bank or a pension fund; until it isn't and she has to live off foodstamps and cat food. Great reward for a life of hard work.

This is robbery.

Re:destroy them

[roman_mir]

I do not misunderstand. They first sell to one and then buy from the other, the transaction does not happen between the real buyer and seller, it happens between HFT thief->buyer, seller->HFT thief.

Re:destroy them

[roman_mir]

you know, it is VERY telling that most of the replies to my post are from Anonymous Cowards.

Do I believe that people have a 'RIGHT' to make profit? No.

Do I believe that HFT provide anything but harm to the market? Yes.

Re:destroy them

[roman_mir]

The OTHER thing that the HFT provides the market with is DATA on who is the criminal asshole.

Re:destroy them

[ottothecow]

I really think you are confusing high frequency trading with something else. Having somebody trading quickly in the middle is not going to screw over little old ladies, it is just going to narrow the spread. There are a whole host of other things (like rampant speculation on questionable derivatives) that could screw stuff up but HFT is not one of them.

It doesn't even really mess with the value of the stock--the end shareholders are still paying what they think it is worth (the HFT people certainly are not holding these stocks...so there has to be somebody who wants to buy them). The thing that people don't like is that its a bunch of guys making a lot of money simply because they can afford to get at the market data faster than anyone else (which is why the SEC doesn't like it...it is not an even playing field).

Re:destroy them

[therealkevinkretz]

Your repeating that these freely-entered-into transactions are "theft" twice as often doesn't make it twice as true, or half as false.

Re:destroy them

[therealkevinkretz]

If your assertion is that taxation is theft, then argue that. Because the rest of what you've said doesn't amount to what you claim.

Re:destroy them

[roman_mir]

You saying that those transactions are 'freely-entered-into' makes no sense and is false. The transactions between parties A and B do not intend to include intermediary transactions between A and C and then C and B.

I can repeat it another time: this is clearly theft.

Re:destroy them

[therealkevinkretz]

They are freely entered into. If A purchases at $X and B sells at $X, it's irrelevant and transparent to A and B what goes on in between. If you purchase something from a retailer and (unbeknownst to you at time of purchase) someone else delivers it, or if they never actually held the inventory but really just sold it for some other company without a storefront, it's still a freely-entered-into transaction. You've shown nowhere how any of this is either th "theft" or not "freely entered into". Maybe you'd be less murky and meaningless if you shared an explicit example instead of your abstractions of whatever it is you mean.

Re:destroy them

[roman_mir]

Creating and canceling orders all within milliseconds so as to check the willingness of the purchase or sale request is what is happening.

If you want to buy 10000 shares for $10 each, it means you are willing to accept up to 10000 shares at any price up to $10. Say the market is at $9.80. The flash orders are created in say batches of 100 shares, say at $9.85, $9.90, $9.95, $10, $10.05. They noticed you were buying up to and including at $10, but did not bite at $10.05. Now they do another 1 or 2 flashes and figure out that you are willing to buy at $10, so they offer to you $9.99. You are buying from them while they are buying away from someone at 9.80 tops.

They just made 19 cents on your ass, but not because the market was willing to do it, they figured out your hidden information: what is your price limit. This is the piece of information that they figured out that allowed them to steal from you, the market did not need you to buy at your maximum and you did not need to buy every share at that maximum either.

You were willing to buy at 10, sure, but that means you were willing to buy at MAXIMUM 10.

There was no real supply at 9.99, they created the 'supply' for your specifically and immediately.

Re:destroy them

[therealkevinkretz]

"You were willing to buy at 10, sure, but that means you were willing to buy at MAXIMUM 10. There was no real supply at 9.99, they created the 'supply' for your specifically and immediately." So? If I'm trying to sell a car for $10000 and someone places an ad seeking the same car for $11000, am I doing anything wrong by "creating the supply" for the car at $11000? No. In your example, I was willing to pay $10/share, and that's what I paid. If someone found shares for $9.90 and sold them to me for $10 - so what? I don't have to offer $10. I could have offered $9 and increased my offer incrementally. But, in your example, I didn't. You've never shown "theft", apparently don't know what "theft" is, and haven't shown me a transaction that either participant wasn't willing to enter into.

Re:destroy them

[roman_mir]

Theft - when someone takes something that does not belong to him and gets away with it.

Re:destroy them

[therealkevinkretz]

Super. If (your example) you offer to buy something at $10, and someone sells something to you at $10, not matter whether they purchased it at $9.99, or $9, or $2, nothing's been "taken" from you.

Re:destroy them

[roman_mir]

No. The actual owner sells it at minimum of 9.80. The buyer will pay a maximum of 10.

Then there is a 'magician', who gets into their heads for a millisecond to read their thoughts and decides to make 19 or 20 cents on the transaction that would have been done just fine without him and tricks the real participants without them even noticing.

Taking your thoughts and picking pockets quickly so that nobody notices to move something minus 20 cents from pocket of one to pocket of another = theft.

you can now come up with how this is not theft, irrelevant to me. I would shoot this 'magician' in the eye with a 30 millimeter cannon.

Re:destroy them

[therealkevinkretz]

You go ahead and keep repeating your story of how the person who bought at what he offered to buy at, and the other person who sold at what he offered to sell at, were the victims of theft. It doesn't make it so. Your "magician" put in place a trading system that A and B chose to use. The rules by which your "magician" operates are readily available and A abd B both had other options if they didn't like those rules. You're a broken record who doesn't understand what he's so loudly complaining about.

Re:destroy them

[roman_mir]

The rules by which your "magician" operates are readily available and A abd B both had other options if they didn't like those rules

- lies, lies, lies. The system was changed very recently and A and B are not even aware of the change. The system became something else from what it was intended to be.

Companies were traded on merit. People were investing or shorting based on merit. Then came the HFT systems. There was no merit, there was no intention of investing. Everything HFTs are doing is stealing. They use their super fast systems, located within a microsecond computer time distance away from the trading floor, UNLIKE A and B. HFTs use their systems to 'read news' and decide the market movement based on this NOT for investing, only to figure out the market general direction.

None of this is investing, it is all about theft, it will destroy the market.

Re:So what they are saying is

[ragethehotey]

I still don't understand why the Stock Market can't run 24/7.

because then, computerized trading systems would have a distinct advantage over humans.

Re:So what they are saying is

[LostCluster]

All trading instructions, no matter how technologically implemented, come from people. Somebody has to write and fund the program that says "Buy XXXX when its price reaches $Y" even if they're not attending it at the time it happens.

Just look at what a mess after-hours markets are. Sometimes they're offering tomorrow's price today, but sometimes they get bent out of shape. Don't you dare buy a stock Jim Cramer promotes on Mad Money in after hours... somebody who owns that stock would love to jack up the price on you. Worse yet, somebody can short that stock and likely have a profit by 9:30am the next business morning.

If you want 24/7 markets, try currency trading. The US Dollar is up for trades at nearly all hours of the day.

What goes around comes around...call it GEnie.

[Quarters]

I guess if an idea is 20+ years old the statute of limitations has run out and someone can use it again as NEW and EXCITING.

Re:What goes around comes around...call it GEnie.

[BitZtream]

GEnie is still around, owned by another company under some other name doing EDI for some companies that refuse to join the Internet and just use SMTP to accomplish the same thing. Its still around never the less.

EC2 is because of christmas servers?

[euxneks]

Can someone post a source for the claim that Amazon did EC2 because of Christmas time servers that are no longer in use during the year?

Re:EC2 is because of christmas servers?

[Anonymous Coward]

Can someone post a source for the claim that Amazon did EC2 because of Christmas time servers that are no longer in use during the year?

You'll probably get sources, as Amazon said so initially. Of course that's best forgotten now as it has become an inconvenient fact.
Notice how every Christmas their servers go down due to mysterious "hackers" poisoning their DNS server or what not. Every year - a different excuse, and every time - around Christmas.

Basically, don't use Amazon if you have mission critical processing around Christmas, or even worse, if you expect a peak around Christmas.

Awesome ... not really

[BitZtream]

Since most computing power is needed during the business day, during the same time that they need their processing power ... it would seem they have little value in what they have to offer.

What do you do during the peak time when they don't offer you processing time?

This pretty much limits the data that can be handled on these servers to batch processing of data that doesn't have constraints on how long it can wait.

Basically its useful for low budget scientific research. Thats not saying its not very useful in that sense. Just recouping wasted CPU cycles could be great for science in general. I just don't see there being enough of a market for a stock exchange to justify the expense of running the business.

Re:Awesome ... not really

[BradleyUffner]

Since most computing power is needed during the business day, during the same time that they need their processing power ... it would seem they have little value in what they have to offer.

unless you are on the other side of the planet, then your peek time is their down time.

Re:So what they are saying is

[Lunix Nutcase]

You act as if they don't already.

Who is this Wall Street guy?

[alexmin]

I've been in trading business for a while but never met him. Still "press" keep talking about him. Especially when it comes to getting someone to pay for something guvmint wants. Enyone met him yet?

It Depends...

[FrozenGeek]

It could be secure, given the following constraints:
1. The computers contain NO mass storage at all.
2. The mass storage is external to the computer and is disconnected from the computer during cloud computing.
3. The computer is rebooted from CD (or DVD) before and after cloud computing.
Of course, the odds on those constraints being met are pretty low if non-technical types are involved.

This won't happen

[L3370]

These servers make a *ridiculous* amount of money for their owners/shareholders. Revenue from hosted services would be a drop in the bucket in comparison....no a single water molecule in a bucket. Leasing out their servers in the off hours would barely pay for the brokers cocaine-and-hookers slush fund. Hardly enough to warrant the effort and associated risk.

It seems like a novel idea, but in the end I bet it remains just that.

Wikileaks

[Snarf You]

This might explain how Wikileaks did it.

I, for one, would like to rent one

[gman003]

It would be great for batch rendering. Sure, the hardware might not be optimized for it, but it would probably be cheaper than renting one that is. And running it during off-hours is actually a plus: set up a scene during the day, render overnight, then see the results in the morning.

Re:So what they are saying is

[RichardJenkins]

being closed for most of the 24 hour day gives people a little more time to thing about what is going on.

You mean like: "ARRRGH! What the hell are they doing with our servers while we're not there!!!

All you need to know...

[frank_adrian314159]

...said one CTO of a market data firm...

Note that it wasn't the CTOs of the actual trading firms speaking. In reality, almost every cycle that's not being used during the trading day is being used either in trading on foreign markets when yours are closed or running stats to drive the next day's domestic trading. A "market data" firm? Yeah, I can believe that. Real trading companies? Not so much.

Microsoft Office Clouds

[Doc Ruby]

Is the Microsoft Azure cloud software available to install and run on these Wall St server farms during the off hours? I could see running the two separate installs on two separate fiberchannel SANs, and physically switching between them (plugging cables) and rebooting/reflashing to ensure none of either was left available to the other during the alternate operating cycles. A lot of Wall St server farms are optimized to run Windows, because otherwise the farm is too slow. If a local install of an Azure cloud could be quickly deployed and removed, that could be another huge new Microsoft product. Or, if these farms let MS operate it, a huge MS cloud service for which it doesn't have to pay for the hardware.

Re:Microsoft Office Clouds

[Anonymous Coward]

I would love to tell where I work or identify myself to lend credence to what I'm going to say. But I value my job too much... I work as a trade support specialist and systems engineer at an electronic trading firm dealing with many exchanges in a few countries.

We are finishing the process of scrapping Windows from over 100 machines simply because they are too slow and cannot be optimized enough at a workable cost. When speaking to some of the exchanges I have a good laugh with them about Windows machines and we definitely see poorer performance from the dark pools (exchanges with no market data) which are built on Windows. Large exchanges typically do not have Windows servers anywhere near the order entry or market data systems as a whole.

As to the notion of using the off-hours cycles, we have given a passing thought to using them for SETI or the like, but always came to the conclusion that the power bill from dozens of colocations worth of servers being run at full tilt squashes all real intent at doing this. These are some really powerful machines! Even if we charged just for electricity and cooling it would be hard to make a profit from it. Seriously, one colocation has 5x the service rating than does most people's relatively large homes. And we certainly don't run at a low load!

Re:Microsoft Office Clouds

[codepunk]

I work on trading systems and I can assure you that nothing we run is on Windows.

Re:Microsoft Office Clouds

[Doc Ruby]

Well I write trading systems and all kinds of other line of business apps for some of the biggest hedge funds, banks and brokers, prime and otherwise, in Midtown Manhattan. And I can assure you that they're running lots and lots of that software on the Windows platform.

Just because you don't do it doesn't mean they don't.

Re:Microsoft Office Clouds

[Doc Ruby]

Well, I'm in the business too, and I'm not going to break my NDAs and specify whose MS software I service. But I do write and maintain apps for some pretty big traders (and directly related financial businesses). There's lots of MS platforms in their core business ops. Lots of Windows server farms, particularly running SQL Server and business objects. Tremendous horsepower, both in-house, and colo at telco hotels for low latency to exchanges - and at leasable datacenters. And starting to move some services to clouds. They're interested in Azure, and waiting to see what it's like when it's ready for prime time.

But their biggest obstacle is letting their data and algorithms, or anything in their critical path, live at Microsoft. If they had an Azure cloud distributed among the locations they control themselves, they might be closer to moving their apps to that model.

And if they could make money off their sunk hardware costs while it's "sleeping" (except for some hefty datamining and OLAP procs), they would. They'd sell their grandmother if it had a chance at a profit. And computing services rarely sell for less than the power and other ops costs to run them.

So there's probably a future in this. I wonder who else has seen more of it than I have yet. So far, only people who haven't have chimed in here.

Re:Microsoft Office Clouds

[lewiscr]

So there's probably a future in this.

Then they can start trading futures on the CPU cycles they're not using.

But what happens to the cloud at xmas?

[Anonymous Coward]

But so far as I know Amazon doesn't shutdown the cloud at Christmas? Meaning they have to have enough capacity now to handle both the cloud AND their peak Christmas load. Well I guess they could sell the spare capacity for cloud computing the rest of the year... Recursion, anyone?

Oh look!

[bmo]

It's the original CompuServ business model all over again!

We really have come full circle.

--
BMO

Re:So what they are saying is

[_merlin]

A perfect example is companies that release earnings and other important news after the closing bell.

That doesn't happen much - for the most part, the exchange just puts a trading halt on the stock for the announcement period, and then lifts it after people have had time to digest the information. This would work just as well with 24-hour trading.

Tut-tut!

[ThatsNotPudding]

These Wall Street chaps know what they're doing! Their immaculate track record speaks for itself!

Nope.

[therealkevinkretz]

Nobody would bet their farm (pun intended) running someone else's software on their critically important server farm. Nor would any admin worth this salt invite that amount of someone else's traffic onto the network they use for trading. Or limit the already-often-restrictive windows of service time. The biggest flaw in this supposition is that most firms who have this much infrastructure are big enough that their trading isn't limited to US markets, and so would be trading over many time zones, limiting the useful downtims.

Problematic

[hesaigo999ca]

They would have to devise not only a proper time sheet of when to shut them off in advance to also let most of their servers
become secluded, but also then to regain proper connectivity to their own network and be able to handle the massive load of prestocks in the morning.... also time tables should add a sort of maximum possible allowance no matter what in case the peak season (as christmas is for amazon) would sort of super load their connections and avoid having to shut it all down in order to regain order for their own machines.

[Off-topic] I tried the link in your sig

[ElMiguel]

You should mention that all of the questions in Frustration Trivia are so USA-centric that people from other countries need not bother.

Re:[Off-topic] I tried the link in your sig

[LostCluster]

It's hard to write a game that's Internationally playable. I once wrote a 30 question chat room game where the Canadian players had completely different correct answers than the American players. (IE. Who is the host of The Weakest Link?) as a farewell to the Canadian audience on Prodigy.

There's a reason why I serve "America's Most Uo-to-Date Trivia Game" at the bottom of every page.

Re:[Off-topic] I tried the link in your sig

[ElMiguel]

There's an international Western culture. You could ask about the Moon race, Shakespare's plays, the Roman Empire and so on. But if you don't want to do that, you should at least say "American trivia" instead of just "trivia".