Slashdot Mirror
Undiscovered Country of HFT: FPGA JIT Ethernet Packet Assembly
michaelmalak writes "In a technique that reminds me of the just-in-time torpedo engineering of Star Trek VI: The Undiscovered Country, a company called Argon Design has "developed a high performance trading system" that puts an FPGA — and FPGA-based trading algorithms — right in the Ethernet switch. And it isn't just to cut down on switch/computer latency — they actually start assembling and sending out the start of an Ethernet packet simultaneously with receiving and decoding incoming price quotation Ethernet packets, and decide on the fly what to put in the outgoing buy/sell Ethernet packet. They call these techniques 'inline parsing' and 'pre-emption.'"
This is the madness of high-speed trading...
No sig. Move along - nothing to see here.
These people could be used to develop more efficient hardware for everyone to use, or fix medical conditions, rather than make rich traders even richer at the expense of another economic collapse. It seems wrong that our economy prioritizes high frequency trading so much.
So that you can send your order before even the price quotation comes. Oh , wait, those guy already can do that and send packet back in time : http://news.slashdot.org/story/13/09/25/1955220/somebody-stole-7-milliseconds-from-the-federal-reserve /sarcasm
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
Things like high frequency trading make me want to vom. Essentially, all they're doing is shuffling money around, taking advantage of an outdated system, and increasing risk for the entire world.
It'd be great to see this kind of innovation in something that actually is useful and valuable - not for creating an incremental improvement on a corrupt system.
This is investing?
Fucking parasites (and their toolmakers).
Makes no sense.
I have a friend who works in the defense industry but interviewed with HFT firms around 3 years ago. My friend also had this idea, and discussed it with me since I am close to the industry.
If an industry outsider like my friend had this idea within a week or two of merely interviewing for jobs, it is a good bet many others had already conceived it and even gotten it working before that.
Someone's gotta leave a penny...
Ok I can see it having implications in high speed control applications now that I think of it. Something like CERN, NIF or any space agency could use it. But we wont have any money to build anything because its more profitable to build shit like this to skim parts of pennies out of the jar than to actually build cool shit that needs FPGA controlled switches.
No matter what Wall St apologists say, this is skimming. There's no polite way to put it. The argument that HFT creates liquidity is a complete lie. All it does is shuffle money from 90% of the people trading into the pocket of 1%.
Just think of the expenses that could be involved if one of these programs screws up. Instead of sell! sell! sell! One could buy! buy! buy! the wrong stock in very large quantities. Instead of a billion dollars in buggy whips one might end up with a billion dollars in donkey whips. What a bummer.
Ya know, as a Morlock, I'm starting to look at them less as gated communities and more as veal pens.
Set a minimum x hours before a stock can be resold.
What is with the vitriol here? Why should buyers and sellers not be able to come together to make a transaction at any time that they like?
In Chicago, traders are working faster than light.
You have to be a rich bankster to achieve faster than light trading, though. If anybody else does it it's cheating.
Actual full summary ending: "They call these techniques 'inline parsing' and 'pre-emption' and 'greedy unfair asshole companies making billions for their owners through cheap, cheating tactics like this to undercut smaller startups and push them out of the market so their fat owners can buy another personal jet instead of handing off the profits to their investors/customers'"
In The Undiscovered Country they modified a torpedo to home in on gas emissions from a Klingon Bird of Prey. This is a story about building trading algorithms into an ethernet switch.
Apart from "needing to do something quickly," I really, really can't see the connection.
systemd is Roko's Basilisk.
Making lots of money by pushing network packets around faster, to no real net benefit to anyone. Other than the ones pushing the packets.
a minimum processing interval for trades, such that trading would only take place every X number of seconds/minutes/hours so as to eliminate the need for dedicated and rapid uplinks to the stock market in order to be competitive?
Other than perhaps eliminating HFT, I can't see a whole lot of major changes to the market and with longer intervals auditing trades would become much easier to do since there would be less data to sift through for any given period of time.
But maybe that's a simpleton's view of the situation.
They call these techniques 'inline parsing' and 'pre-emption.'"
And everyone else has been calling this "pipelining" for decades.
Ezekiel 23:20
An interesting technique/technology, created for an utterly worthless and counterproductive "industry". I know they say that HFT (High Frequency Trading) is supposed to "increase liquidity" but it produces nothing and removes billions from the industries that actually do something (I would even include entertainment & service(s) industries in that). If I had to come up with an analogy to match my perception of HFT it would be an individual who spends tens of thousands of dollars to develop a device that they can bury at a stop light and siphon a little gas from each vehicle that stops over it. It provides nothing to the roadways (save for a little bit of new pavement), doesn't create any really useful technology (at least vs its cost) and basically steals from everyone who is unfortunate enough to stop over it that are likely going to work, transporting goods, or going to buy goods.
I think we just found our culprit.
Doctors destroy health, lawyers destroy justice, universities destroy knowledge, religion destroys spirituality
tax every transaction.
Also tax rolled back oops, my bot "ran amok" transactions. Also track these and send the history to our regulatory watchdogs who, supposedly, will take an interest in the chicanery of anyone abusing this "feature" of the markets. Or just don't allow them to be undone in the first place. caveat emptor, bitches.
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
We should be surprised? Porn has driven more technological advances than most will ever care to admit or acknowledge.
not nearly as much as warfare.
regarding the news, if merely bypassing some local communication api layers yields such a signifficant performance boost, one has to wonder what the decision-making algorithms actually do. obligatory: what could possibly go wrong?
Having a pick set either declares you as an intruder, or someone who loses their keys more often than the pick set...
HFT is morally thievery, ethically abusive, and deserves to be regulated as such.
OR, alternatively, brokers, market makers, and exchanges need to fully and repeatedly disclose to investors the nature and impact of HFT. Those of us trying to time the market are wasting our time, the HFT guys have this down to milliseconds. Trying to find arbitrage is impossible.
Of course, when the Fed stops blowing up this bubble, then the market will collapse to a lower level, and there will be other investment vehicles that can attract capital. Until then, your savings account and CD are paying zilch, so you buy funds and houses. Since banks really don't want to go back into the savings business where they are accountable to their true owners, the depositors, we are facing a Fed intrusion that has no end in sight. Until that ends, individual capital is almost superfluous. And the stock market is the only game in town for many of us.
My rental property looks real good right now.
deleting the extra space after periods so i can stay relevant, yeah.
The impact of this technology will be actually very small.
The reason is that there is no much liquidity at milliseconds level.
Another argument: if N HFT trading firm are chasing a single order only 1 can get it and N-1 are out of the game.
It would be much more interesting if the post would be:
the trader go to minutes interval. And about HFT - it topped on about 2008 and goes down
since then because of a competition.
I have so many FPGAs in my network gear that my pings are NEGATIVE.
Have you got your own gaming network gadg to give you the edge in gaming?!
IF YOU BUY NOW YOU GET SOME GAMER FUEL FREE.
TERMS AND CONDITIONS APPLY.
This still doesn't enable FTL communications. The through-planet neutrino network suggested in that article is more likely.
"When information is power, privacy is freedom" - Jah-Wren Ryel
We need to stop rewarding folks for high-speed trading. It basically steals money from the folks genuinely invested in the companies whose stock is traded and adds no value to the system at all.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
But is it secure?
The fix for High Frequency Trading is 5 minute rolling averages for all buy/sell orders. This would allow the free flow of capital with some level of price uncertainty that would clear out all the volume based on miniscule profits
This would be done by placing all trades into a five minute escrow account and once the five minutes is up, the trade is completed on the average price during those five minutes. Five minutes is huge in computer time, but not so big to humans. This would only affect non-human based trading.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
Some things in life are simple:
Almost everyone agrees that HFT is evil.
Nothing is being done about it.
How can both of these things be true at the same time without revealing a serious, dangerous flaw in our political and economic system?
Assorted stuff I do sometimes: Lemuria.org
Someone please fact check before publishing summaries...Ethernet "packets" don't exist. Learn your networking 101 please.
(the segment of data is a FRAME, and may contain packet data)
Education for the win... http://en.wikipedia.org/wiki/Ethernet_frame
Trackball users will be first against the wall.
As usual when an "HFT" article appears on slashdot, there is little or no commentary on the actual technical issue at hand, in this case putting custom code into FPGAs on Arista switches. Nor is there any discussion or curiosity about what actually happens in such code as far as HFT is concerned. Instead, a band of emotionally challenged, bitter trolls descends to squelch any useful conversation. Most of the negative DDOSsing comes from people who don't know what HFT is or does; they just "know" it is another way "rich" people are taking money from "poor" people, and comment accordingly with all of the fervor and intelligence of a typical 18 year old member of the local college chapter of Democratic Socialists of America. They have very little or no idea who participates in HFT, they have given very little thought to the complex issues behind why HFT has arisen in the first place, and most of their proposed and regurgitated "solutions" to HFT would laughably do nothing more than to put more power in the hands of price fixers or further entrench too-big-to-fail Wall Street institutions. Reading these posts is pointless, and the all-around antipathy for innovation is surprising in a forum renowned for technology.
The news you refer to was notable precisely because it is physically impossible for an electronic signal to get from DC to Chicago in the specified timeframe. I think your culprit is a leaker and accomplices guilty of insider trading, not HFT.
I thought that's what FPGA stands for.
Future Packet Generation Algorithm
Here's the original article. There's also a video. http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/
I can imagine this technology can be used for other practices, like a massively parallel world system used in games or simulations.
Cuts down on latency and routing issues with multi-node services by allowing switches to be temporary shared-data proxies.
http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/
Taxing every transaction would be too drastic. However, if you trade stock soon after you buy it, you are either gambling (taxable) or have prior knowledge (illegal). Assuming people that do that will be at least gambling, you should tax profit as gambling profits and not make losses tax deductible. Adding a 40% gambling tax on all gross profits made on each transaction that has a sale of the share within a week and putting the burden of bookkeeping of this on the gambler^Winvestor, will make it a lot less profitable to do HFT, but it wont hurt actual investors. Not only that, but it would give the government a nice income they can spend on improving the infrastructure of the USA and making government work better. In the same line, all derivative trades are basically gambling on a value going up or down, so should be taxed the same, unrelated to how long they run or when sold/purchased.
I was promised a flying car. Where is my flying car?
I think you missed my point.
The point with the dual listed stocks is that you have the same stock on 2 different exchanges – a quote based and a order based exchange. Going head to head the quote based system beat the order book system.
By the way, there are a many tricks that one can use the game a order book system. So you are not so much eliminating a problem as switching from one set of problems to another.
If cheaper and deeper markets (using empirical evidence) does not convince you what do you need?
That innovative product will probably be used for something else at some point.
Here we see finance that benefits the real economy, by developing a product instead of providing capital. How weird.
Due to the extreme time constraints it's very likely to be as simple as possible and have to ignore many of the edge cases and error checking normal networking hardware has to worry about. That doesn't leave a lot to be interested about especially since we have no detail at all to talk about and it's boring to discuss what has been left and what the consequences of doing so are. Do you really want to see a lot of posts along the lines of "only an idiot would cut out ..." for something cut out because time is far more important than accuracy?
So we can blow up the economy even faster ..
Rent seeking is when regulatory mechanisms are put in place to extract value.
You are not obligated to buy or sell stock.
Almost everyone has investments for retirement, most of them would have a non trivial amount in stocks.
Even if I wasnt obliged to I'd still like the option to buy stocks without the rent seeking leeches taking thier cut.
Add a 0.0001% tax to all financial transactions. Has minimal effect on anyone buying shares to hold but makes HFT unviable.
Actually it had to do with the inability of the authors (and most commenters) to do simple math. The article said: "[...] were placed on Chicago exchanges 2-3 milliseconds after 2 pm." The speed-of-light-delay between the dc and chicago is 3.2 ms. So, no, not "physically impossible".
HFT provides liquidity, and liquidity is of the utmost importance to traders.
But why should investors suffer and be forced to pay a subsidy to HFT's to keep other traders happy?
HFT is very misunderstood by people who don't participate in or understand trading. HFT in fact adds tremendous value to all market participants by dramatically increasing market efficiency with the significant proportion of trading volume it's responsible for.
Investing is very misunderstood by people who dont participate in or understand investing (like traders for example)
Just so I understand this correctly.
Dedicated hardware is now being produced to allow bottom-dwelling scum to steal more of our money?
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
Hft make money off of retail, uninformed traders. Why don't schwab, etrade, fidelity give retail options other than market or limit orders both of which end up feeding hft and are so profitable they pay the brokers for them? Institutions have access to algos, why not retail? It used to be mysterious and incredibly expensive but now neither one of those is true. Someone could implement the getco zero plus algo and house it at Nasdaq for $20k per month and $300k up front. Who's with me? Let's starve the hft. Most retail has one big advantage, we don't need immediate execution, we can cancel/replace all day long until we get a good price.
According to the article posted elsewhere on slashdot about this:
Somebody placed massive orders for gold futures contracts betting on exactly that outcome within a millisecond or two of 2 p.m. that day -- before the seven milliseconds had passed that would allow the transmission of the information from the Fed's "lock-up" of media organizations who get an early look at the data and the arrival of that information at Chicago's futures markets
"within a millisecond or two" is still physically impossible given the 3.2 millisecond bound implied by the speed of light in a vacuum, but even the 2-3 millisecond figure quoted in your other article is much less than the actual fiber latency of about 6.5 milliseconds between Chicago and NYC (presumably similar between Chicago and DC), and is still somewhat less than the fastest microwave links available between NYC and Chicago, which come in at about 4.1 milliseconds. (I'm not sure there are any microwave links operational between Chicago and DC as of yet.) These figures are somewhat worse than physical speed of light bounds because they include actual geographical routes and hops, and an index of refraction in the fiber case.
Which is why there should be an investigation - it certainly looks like someone was sitting on the news beforehand and traded on it as soon as 2PM rolled around. But it could also be that a local news organization broke the news embargo ahead of Washington DC. If I understand the article, it seems like news organizations have access to such announcements beforehand so they can prepare ahead of time release their canned articles at the same time. That would be interesting to find out too - I can see local newsmen tipping off their trading buddies where/what to look at precisely 2PM, or just inaccurate network time synchronization.
So if you don't do those, which exploits are you running on the market to drive up prices and take a cut?
HFT is the cause of "flash crashes" already. Combine this with markets already pushing 90% cross correlation between internal market and intermarket trades and then the general lack of fundamentals in the bubble business that are flying now, and you've basically created a supercritical system just waiting for a crash worse than all the market crashes of the last 150 years combined.
Seriously, again with this nonsense.
Your wilful ignorance here is truly staggering.
http://www.nytimes.com/imagepages/2009/07/24/business/0724-webBIZ-trading.ready.html
where do I set the "anti-evil" bit in my transaction?
The tech was there from the very beginning.
Ethernet encoding allows for pause in the data and also allows for canceling transmission of the packet by utilizing Link error code (risking that routers will complain about too many link errors).
Very strange, I'd say.
Prove me wrong then.
Describe how preventing the buyer and seller from communicating with each other and getting in first to screw both over is not an exploit - and don't give me that shit about doing such a thing not being a form of manipulating prices - that is a very blatant lie and you know it.
Tell me how it benefits society in some way and is not mere parasitism and a total waste of talent of the people involved.
I'm waiting, but all you've supplied was pre-emptive insults and no actual justification for what you do. I'd like to hear how you benefit society. Also don't insult us with the liquidity crap since you vultures pounce on trades that other people are going to do anyway.
s/do not with to/do not wish to/
So when Pete buys from Paul and you get a cut from their transaction how is that helping anyone? They could just as easily sell to each other if HFT didn't exist and one of those two would be getting the benefit you are getting.
Why do you skirt around such a simple issue and call people "idiots" just for bringing it up?
You don't actually know how HFT makes money - that's part of the reason you assume it's bad.
I start a company, XYZ we are a superHFT, we pay money to the exchange and get the data 1 second before HFT do. Any trade you try to make I profit from and you lose.
My SHFT company takes away 99.999% of your profits as I can trade faster than you. All HFT stop trading and you have no job anymore, are you happy about the situation?
Now you know how they make money, they leech it all from the slower people. You only think its fair because its you doing the leeching.