Slashdot Mirror


Robust Timing Over the Internet

ChelleChelle writes "The NTP (Network Time Protocol) system for synchronizing computer clocks has been around for decades and has worked well for most general-purpose timing uses. However, new developments, such as the increasingly precise timing demands of the finance industry, are driving the need for a more precise and reliable network timing system. Julien Ridoux and Darryl Veitch from the University of Melbourne are working on such a system as part of the Radclock Project. In this article they share some of their expertise on synchronizing network clocks. The authors tackle the key challenge — taming delay variability — and provide useful guidelines for designing robust network timing algorithms."

12 of 178 comments (clear)

  1. make all wall street traders own stock for 1 day by Anonymous Coward · · Score: 4, Insightful

    let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.

  2. can't trust self if microsoft by at10u8 · · Score: 4, Interesting

    Microsoft refuses to consider the notion of clocks accurate to within 2 seconds.

    1. Re:can't trust self if microsoft by Rockoon · · Score: 4, Informative

      He was modded troll, but hes right.. sort of.

      The problem isnt "windows" .. the problem is "x86 computers."

      Accurate timing on the modern PC is ridiculously difficult due to the hardware situation. The original IBM PC and its clones came with Intel's 8253 or 8254 Programmable Interval Timer (PIT) chip and back then it was pretty good. This timer chip had an internal frequency matching the CPU's of the time of 4.77mhz, and ticks were evenly distributed between 4 internal counters, so each counter had 1193181.81818181.... ticks per second. Pretty damn good.

      Many of the old-school people might be familiar with the 18.2 ticks per second of the DOS clock. This rate was a direct consequence of configuring one of those counters to emit an interrupt every 65536 ticks, the longest possible interrupt interval due to these chips using 16-bit words.

      Anyways, over time many manufacturers stopped using Intel's PIT's and started emulating them with other hardware (such as PMT.) None of these emulations can be configured to tick at exactly the same rate as the Intel PIT, and even among these emulated timers there wasn't any consistency.

      To counter the serious problem of accurate timing on the PC, Microsoft and a few other large companies moved to define a new standard named HPET. Many new motherboards will have an HPET timer, but not all of them, and there are some problems with some HPET implementations as well (such as appearing to tick backwards sometimes.)

      The issue of accurate timing on PC's has considerable effect on multi-player game developers.

      A reference to some of the issues, and some of the attempts to fix them (source code)

      --
      "His name was James Damore."
  3. PTPd? by Jeremi · · Score: 5, Interesting

    I'm surprised the article didn't mention PTPd, which is an implementation of the IEEE 1588 precision time-synchronization standard. I was under the impression that was the standard way to solve this sort of problem when NTP wasn't enough.

    --


    I don't care if it's 90,000 hectares. That lake was not my doing.
    1. Re:PTPd? by apharov · · Score: 5, Informative

      (disclaimer: just finished my Master's thesis on a related subject) PTPd is ok, but not in itself up-to-date at the moment. It doesn't implement the most recent IEEE 1588-2008 standard, which has significant improvements compared to the 1588-2002. About the 1588 in general: its main selling point is the ability to do hardware timestamping (when using hardware with support!) of the two-way timing messages between master and slave. This eliminates the very significant timing jitter that happens in the software stack before the messages are timestamped. For reference, commercially available master-slave implementations using IEEE 1588 achieve synchronisation within tens of nanoseconds within LAN, and microseconds to tens of microseconds within WAN, depending on network conditions. So overall I think that while RADclock might be ok as an alternative between NTP and IEEE 1588, it doesn't really bring anything new to the table. Some of the stuff in the Rideaux/Veitch paper has also been used with IEEE 1588 for quite some time, for instance the filtering for fast timing packets is a necessity for accurate synchronisation with IEEE 1588.

  4. Re:Ridiculous by Anonymous Coward · · Score: 5, Informative

    This sort of thing probably has something to do with it:

    http://www.nytimes.com/2009/07/24/business/24trading.html

  5. Re:Trust yourself... by ls671 · · Score: 4, Interesting

    Network issues. If say, on average, once every 25 polls, ntpd doesn't get realistic data from higher stratum servers because of the network, don't let ntpd crank up the frequency to ridiculous values like it does when this occurs.

    There are realistic values for the frequency on every machine with a good clock. It is ridiculous to set the frequency below or above these values.

    Last time I checked, there is no way in ntpd to configure these values. The typical ntp guru reply will be: "Get a decent network connection". The author in TFA noted that such a "decent connection" is virtually impossible to achieve because of "variable delay". He also noted that it is sometime better to trust yourself, which is kind of what that script does.
     

    --
    Everything I write is lies, read between the lines.
  6. Re:Speed of light and relativity by 1s44c · · Score: 4, Insightful

    I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.

    Like most smart people you are forgetting how dumb most people are.

    Wall street traders don't understand much about time. They only understand how to obfuscate financial transactions like a high speed shell game. To them more speed means more chances to skim off a percentage and the laws of physics don't even get considered.

  7. Would make sense by Sycraft-fu · · Score: 4, Informative

    GPS is accurate to about 50 nanoseconds. All kinds of devices that need precision time get it from GPS. You don't need much more than a standard receiver, just one that is designed to place a high priority on time updates. The GPS system itself keeps very accurate time since each satellite has an atomic clock, and they all sync to the master clock.

    To me that would seem the best way, if accuracy is really important and the systems are high end. Have each device have its own receiver and just have them sync to that. Don't sync them to each other, since you aren't going to get anything more accurate.

  8. Mod parent down. by Anonymous Coward · · Score: 5, Insightful

    Actually this kind of trading (especially arbitrage) narrows the spread making it cheaper for ordinary people to enter the market.
    So if you ban it you go back to only the big boys being able to afford to execute trades.

    However, using any system, high frequency or otherwise, to deliberately manipulate the market is a different thing.

    I'm sorry I'm posting as AC but it could affect my job.

    1. Re:Mod parent down. by the+eric+conspiracy · · Score: 4, Insightful

      Mod parent up. The efficiency and liquidity of world wide stock markets increases their usefulness to all involved. My ability as an individual investor to buy and sell at will is a huge factor in my willingness to participate.

      My father used to pay $100 to get a simple trade executed, and it would take hours to get it done. I can do the same trade for $5.00 and get it executed in a second or less.

      So what if some financial institution is making a profit on this. It is a win-win situation.

      The problems in the system are elsewhere - where people in the system are taking advantage of their position to trade based on information not available to everyone. Fortunately most of that is illegal.

  9. Re:make all wall street traders own stock for 1 da by brianerst · · Score: 4, Insightful

    I think people outside of the industry don't really understand the need for day traders.

    I write trading software for the futures and options industries (I was the lead architect for the CBOT's Order Routing System, and have written trading systems for other exchanges and banks). The type of day trading that you and the GP have issues with is an absolute requirement for liquid markets and the efficient working of those markets.

    Let's say you are a firm that needs to buy a large number of shares or contracts (in the futures world, you buy contracts for the future delivery of product). In the type of buy and hold market that seems "fair", that large order is going to go into a market of other buy and hold traders that are only going to trade out of their positions if the trade makes sense in their long-term strategy. Huge effective tax rates are going to make that trade unprofitable unless you've held the position a very long time (increasing your risk of market volatility) or you charge a premium (increase the price) in order to offset the tax hit.

    This type of "illiquid" market raises prices dramatically. You have to factor in the time value of money, the risk exposure of holding shares for a long time (increasing the likelihood of a bad market event) and tax overhead. All that means that buying shares/contracts is riskier and selling them is more expensive.

    Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).

    Now, when that original firm wants to buy that large number of shares, they are doing that in a market where there is almost always a large number of shares being offered at competitive prices. The day traders will sell off shares and pick up shares all day, just trying to end the day flat (holding nothing) to reduce after market risk.

    Day trading does extract a price (they are making money) on the market, but the efficiency gains that they bring outweighs that price in almost all markets. They distribute a lot of the risk involved in trading among a bunch of smaller firms, rather than concentrating all the risk in big funds. This is generally a good thing - the government and the market in general are better off when a small firm goes bust than when one of the "too big to fail" firms starts having problems.

    Now, the rise of algorithmic trading is causing some stress in this pattern - I can see some real issues coming in the next few years as these algorithms become more and more precise. I'm less concerned about a rogue algorithm dragging a market down (or up) than of all the algos converging to a point where the market gets stuck because the algos are just too damn efficient. But that's a post for another day...