Slashdot Mirror

← Back to Stories (view on slashdot.org)

Australian Company Promises Switching Hardware With Sub-130ns Latency

Posted by timothy on from the time-to-start-straightening-the-undersea-bits dept.

snowdon writes "The race for low-latency in finance and HPC has taken a major turn. A bunch of engineers from Australia have 'thrown away the air conditioning' in a traditional switch, to get a 10G fibre-to-fibre latency of less than 130ns! Way faster than more traditional offerings. This lady (video) would tell you that it's equivalent to just 26m of optical fibre. Does that mean we just lose money faster?"

14 of 77 comments (clear)

  1. Low latency post by Anonymous Coward · · Score: 5, Funny

    The trick was not reading TFA.

    1. Re:Low latency post by jamesh · · Score: 2

      Remember what happened last time someone copied an Australian invention without paying the licensing fee! We own low-latency now.

  2. Goldman Sachs will be pleased by CanHasDIY · · Score: 2

    Now they can suck our economy dry with even more speed and efficiency!

    --
    An enigma, wrapped in a riddle, shrouded in bacon and cheese
  3. HFT Should be illegal by the+eric+conspiracy · · Score: 3, Insightful

    This is not good news; it promotes a trend in a technological approach to making money in the stock market that should be flat out illegal.

    1. Re:HFT Should be illegal by vlm · · Score: 5, Insightful

      This is not good news; it promotes a trend in a technological approach to making money in the stock market that should be flat out illegal.

      Why? I know what it is, how it works, how it makes money, and I'm not overly concerned. So they make a market, and they make it really fast. Eh.

      All I ever hear as a rationalization for dislike of it is "workers of the world unite" and "1% OWS" and "something bad happened to the markets, I don't like witches, so lets hang the witch because she has property I like that we can take from her" and "I hate the rich" and all that sort of stuff. I never hear a good moral or ethical explanation of why doing what you normally do, but really fast, is so wrong.

      Another good question, is assuming you "make it illegal" how in the world would you enforce that? What would the reg look like? Like... your connection to the market must be this far away blah blah even if it means making a 1000 KM coil of fiber on the data center floor, or maybe all traders must connect to "the market" over geosync satellite links so the average latency is both long and more or less equal?

      Please don't confuse conventional HFT with "Flash trading" which is basically window dressed up insider trading, corrupt as all heck. If you do insider trading and trade your own book etc thats just illegal, but if you do it really fast people like to pretend its just "flash trading" and not wrong.

      The easiest way to "get rid of" HFT is to fix decimalization. Back when we traded on eights thats too wide to make money on a HFT strategy, and if we allowed millionth of a penny trading quotes that would narrow bid/ask resulting in too little money generated by HFT. Decimalization seems to be almost the perfect pricing system to result in massive HFT strategies, so if you really hate HFT (and WHY?) then just fix the pricing structure so its not profitable anymore. Rather than playing games with legal enforcement of favored or disfavored behavior, simply make disfavored behavior unprofitable. I suppose the opposite solution of going big would work just as well, fine we'll only trade stocks on dollar values now no fractions of a dollar.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    2. Re:HFT Should be illegal by CanHasDIY · · Score: 2

      This is not good news; it promotes a trend in a technological approach to making money in the stock market that should be flat out illegal.

      I always thought that if those losers in Anonymous (or anyone else sick of Wall St. and their economic stranglehold) wanted to actually do something meaningful, they would create and release an open source HFT program anyone can use, and help the populace beat the banksters at their own game...


      nudge nudge...

      --
      An enigma, wrapped in a riddle, shrouded in bacon and cheese
    3. Re:HFT Should be illegal by Nethead · · Score: 2

      I never hear a good moral or ethical explanation of why doing what you normally do, but really fast, is so wrong.

      The nuns back in grade school gave us moral reasons not to wash "down there" really fast. Although through much "scientific discovery" I concluded they were wrong (or just trying to keep a good thing for themselves.)

      --
      -- I have a private email server in my basement.
    4. Re:HFT Should be illegal by vlm · · Score: 2

      The problem with HFT is the "fake" orders placed. Computer trades typically will places tons and tons of buy AND sell orders for prices they really have no intention of executing on just so they can be first in the queue.

      Yeah, but why is that a problem? IF they get executed on they lose their shirt, if not who cares? Using "problem" as in ethical or moral dilemma that we should legislate out of existence, not "problem" as in "I don't personally like it, just like I don't like strawberry icecream, so men with guns should forcibly stop people from doing what I don't like". I need something more than its bad, because by definition its bad. A real world example with numbers.

      AC you and I both think one internets (or karma point, or whatever) is worth about $10 and you have one internets for sale asking $10 and coincidentally I'd like to buy one internets and I toss in a at the market order for one internets.

      A flaming squadron of HFT dumps a bunch of sell orders from 9.95 to 10.05 right before my order hits. So I end up with $9.95 for one internets. Whats your problem? You believe the value is $10 per internets so the HFT just got screwed out of five cents and I made a profit of 5 cents because I paid $9.95 for what we both think is $10 worth of internets. The latest price for one internets is now $9.95 but its worth $10 so you throw in a order to buy too, after all, whenever the price is less than the value, if you got cash you buy. Dumb HFT sells you something worth $10 for $9.95 so you just made money too. Dumb HFT wants to sell even more at 9.95 but it ran outta internets... well you'll sell the HFT internets at $10 a piece, after all you only paid 9.95 a piece. Ha Ha dumb HFTs they suck. Its very hard to make money off a HFT scheme. Its hardly a license to print money. I'm failing to see how its "wrong".

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    5. Re:HFT Should be illegal by vlm · · Score: 2

      The problem with HFT is that it undermines the whole point of central markets and business investment into human endeavour

      Please advise me as to the purpose of "central markets" and "business investment" and then explain why increasing the bid ask spread and decreasing liquidity by the destruction of HFTs somehow helps achieve those goals. Or alternately, why lowering the bid ask spread and increasing liquidity somehow magically prevents the achievement of those goals. I strongly suspect you can't.

      The problem with a tax is, from the retail non-HFT perspective, a HFT looks to them like a microscopic tiny transaction tax in the sub thousandth of a percent range. The solution proposed is always to smack everyone with a tax perhaps a thousand times higher, but more than a thousand times higher for HFTs in order to destroy them.

      So lets say for the sake of experiment that on long term average when I sell a block of 100 shares of electric company stock, HFTs increase the sales price by a buck by improved liquidity and narrower bid/ask although they reduce the amount I take home by about a penny. So they gave me a buck but stole a penny from me. I disagree, but whatever, we'll roll with this "they stole my penny" thing. Anyway find me a tax proposal that won't simply result in the destruction of the HFT industry, and lower liquidity means I'll get an even lower sales price, costing me maybe $10, and the new tax means I'll pay the fedgov maybe $10 as a tax. So we've saved a penny of HFT loss at a cost of only $20 to my bottom line.

      So with HFTs I make an extra 99 cents, would have been a buck but they stole a penny from me.
      Without HFTs I'd have the baseline
      With taxes designed to destroy HFTs I'd lose $20, but at least those nasty HFTs would be dead so I can dance on their grave, even in my poverty.

      As a non-HFT investor, whats in it for me?

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
  4. Meanwhile... by bertok · · Score: 5, Interesting

    Even if a lot of research has been put into reducing the latencies of switching technology, the vast majority of real-world deployments are nowhere near where they should be. The result is corporations spending millions upgrading their core switching, and then the result is the same or worse than what they had with ordinary gigabit technology.

    I've been to more than half a dozen sites recently with new installations of 10 GbE, but with terrible network performance. All too often, I'm seeing latencies as high as 1.3 milliseconds even for servers on the same switch. Read up on bandwidth-delay product too get an idea of why this would severely nerf throughput. The odd thing is that at a number of these sites, older servers with 1 GbE connected to the same switching infrastructure get 100% of wire speed without issues.

    I don't know exactly what the root cause is, but I'm starting to suspect that the extra latency is coming from somewhere that network engineers don't usually test, like CPU power management taking an excessively long time to wake up a core to respond to a packet. What I think happens is something like this:
    - The fast network delivers a burst of data very quickly.
    - The receiver CPU starts to slowly wake up from sleep mode, while the sender is waiting for an ACK packet, because it has finished sending an entire TCP window.
    - The sender CPU goes to sleep, because it still has nothing to do.
    - The receiver CPU finally gets around to the packet, which it processes quickly and efficiently, sending the ACK back.
    - The receiver OS sees that the CPU is "0.1%" busy, has nothing to do now, so it sends it to back to sleep.
    - The sender CPU starts to slowly wake up, while the receiver is also asleep, waiting patiently for more data.
    - The cycle repeats, with more waiting at every step.

    With slightly slower networks or CPUs, the CPUs never get a chance to be idle long enough to enter a sleep state, so everything is always ready for more data. I've seen 3x improvements in iPerf TCP throughput by simply running a busy-loop in a background process!

    1. Re:Meanwhile... by Cassini2 · · Score: 3, Interesting

      The number of interrupts per second that a modern processor can handle has stayed relatively fixed for a large number of years, and network response time is a strong function of interrupt performance.

      It has been a while since I benchmarked interrupt response on a variety of processors with the same block of code. However, when I last did it, a 32 MHz 8-bit PIC18 series microcontroller (MCU) was capable of processing a real-time interrupt at 10,000 times per second. A much faster 3 GHz x86 CPU could manage 70,000 interrupts per second under Real-Time Linux. This wasn't much of an improvement, considering the x86 CPU is theoretically 100 times faster, was capable of executing multiple instructions per clock, and the little 8-bit PIC MCU has a hard limit of around 8 MIPs.

      Why does this happen?
      For a variety of factors including:
      1. The amount of data a modern CPU must flush on an interrupt is huge. Most of the cache needs to be flushed and replaced, and this means a great deal of memory bus bandwidth.
      2. Main memory latency in the modern PC is huge. Modern compiler designers assume it to be on the order of 100's of clocks.
      3. Modern operating systems have to reload the page tables for every user/kernel mode transition. This takes time, and possibly more interrupts.
      4. The I/O response time of the PCI bus is terrible relative to the cycle time of the processor. This means that most of the network traffic is transferred via DMA. However, this optimization only helps in terms of bandwidth, not latency.
      5. Multiple cores don't really help. Synchronization overhead is a killer. Tricks like software transactional memory can help, however typical I/O devices lack the ability to support software transactional memory. Even simple techniques, like giving each core its own I/O and thus eliminating synchronization overhead, are often not implemented. Linux is still trying to eliminate the "Big Kernel Lock".

      In short, the interrupt response latency in modern processors has really not improved much. This was a big discussion in some of the Ethernet working groups, because some of the simulations pointed out that 10Gb Ethernet performance might be limited by interrupt response time on the processors.

    2. Re:Meanwhile... by Bengie · · Score: 2

      There is hope around the corner, Netmap claims to share no locks among established TCP connections. It also claims to scale extremely well with many cores. A 950mhz single core CPU can saturate a 10Gb interface. It can also attach a given connection a to a given core and have the NIC forward the data to the L2 cache so it's ready to be processed. It is also capable of handling send/receive events on different cores for the same connection.

      Should make for some fast routers. It is being ported to Linux and is in FreeBSD 10 Head.

      Modern OS's tend to use DPC(deferred procedure calls, or something) in place of many interrupts. This keeps the CPU from cache-thrashing. Not great for firewalls, but great for user-mode programs.

    3. Re:Meanwhile... by joib · · Score: 2

      Some additional points:

      - FWIW, Linux finally got rid of the BKL in the 3.0 release or thereabouts.

      - Many (most?) 10Gb NIC's are multiqueue, meaning that the interrupt and packet processing load can be spread over multiple cores.

      - Linux and presumably other OS'es have mechanisms to switch to polling mode when processing large numbers of incoming network packets.

      That being said, your basic points about interrupt latency being an issue still stand, of course.

    4. Re:Meanwhile... by bertok · · Score: 3, Interesting

      That's similar to what I'm seeing.

      I found a "DPC ping" tool which queues a simple Kernel task from User mode, and measures the response time as if it was an actual network ping.

      I found it interesting that it was well correlated with both ping times and tested network throughput, and that the DPC ping time wasn't consistent. Some fast CPUs had terrible DPC pings, and some older slower models were much faster. The operating system also contributes significantly, I found some combinations where the average was OK, but there were regular spikes of high latency.

      Now that I've done more benchmarks across more sites, I've been recommending jumbo frames much more often. Previously I though it was a nice-to-have, but these days I'm starting to be of the opinion that without it the money spent on "10 GbE to the server" is just wasted.