Slashdot Mirror
Robust Timing Over the Internet
ChelleChelle writes "The NTP (Network Time Protocol) system for synchronizing computer clocks has been around for decades and has worked well for most general-purpose timing uses. However, new developments, such as the increasingly precise timing demands of the finance industry, are driving the need for a more precise and reliable network timing system. Julien Ridoux and Darryl Veitch from the University of Melbourne are working on such a system as part of the Radclock Project. In this article they share some of their expertise on synchronizing network clocks. The authors tackle the key challenge — taming delay variability — and provide useful guidelines for designing robust network timing algorithms."
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
How about GPS receiver + exact known location?
I would think just eliminating an unknown latency and replacing it with a known one (radio waves generally travel at a consistent rate) would get you a pretty accurate time. If you add one more known variable which is the exact location of the device, then you should be able to get an even more accurate time.
Microsoft refuses to consider the notion of clocks accurate to within 2 seconds.
I'm surprised the article didn't mention PTPd, which is an implementation of the IEEE 1588 precision time-synchronization standard. I was under the impression that was the standard way to solve this sort of problem when NTP wasn't enough.
I don't care if it's 90,000 hectares. That lake was not my doing.
This sort of thing probably has something to do with it:
http://www.nytimes.com/2009/07/24/business/24trading.html
Network issues. If say, on average, once every 25 polls, ntpd doesn't get realistic data from higher stratum servers because of the network, don't let ntpd crank up the frequency to ridiculous values like it does when this occurs.
There are realistic values for the frequency on every machine with a good clock. It is ridiculous to set the frequency below or above these values.
Last time I checked, there is no way in ntpd to configure these values. The typical ntp guru reply will be: "Get a decent network connection". The author in TFA noted that such a "decent connection" is virtually impossible to achieve because of "variable delay". He also noted that it is sometime better to trust yourself, which is kind of what that script does.
Everything I write is lies, read between the lines.
I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.
Like most smart people you are forgetting how dumb most people are.
Wall street traders don't understand much about time. They only understand how to obfuscate financial transactions like a high speed shell game. To them more speed means more chances to skim off a percentage and the laws of physics don't even get considered.
GPS is accurate to about 50 nanoseconds. All kinds of devices that need precision time get it from GPS. You don't need much more than a standard receiver, just one that is designed to place a high priority on time updates. The GPS system itself keeps very accurate time since each satellite has an atomic clock, and they all sync to the master clock.
To me that would seem the best way, if accuracy is really important and the systems are high end. Have each device have its own receiver and just have them sync to that. Don't sync them to each other, since you aren't going to get anything more accurate.
While I think that's a bit extreme, I think you and the GP are both on to the right track. I think stocks should have a fairly short minimum hold time. A day would probably be fine, just something such that you aren't trying to gamble with them more or less, because that's why high speed day trading really is. It is gambling in every way that Vegas is.
I also thing a sliding tax would make sense. If you execute quick trades, you are subjected to a fairly high tax. 95% is a bit excessive, but something still fairly high. Then, as you say, the tax drops on a monthly basis until it hits zero. That would act to encourage investing, rather than gambling. It rewards people who put their money away for the future, and do so in a manner that allows it to help the economy grow. It would also still allow for quicker trades, but the government gets to take a cut in that case.
I really do think we need to start looking at something like this for three reasons:
1) Because there are companies that try to game the system. That's never a good thing. You game the system on a large scale, you'll break it. When you are talking about something highly important to the economy, that's not acceptable.
2) Because there are too many gamblers. There are plenty of traders who basically gamble on the market. They make day trades not based on anything about the company, but just to try and grab money from trends. This is not useful, and isn't what the stock market is for. It leads to unnecessary instability and thus should be discouraged.
3) To help combat people's fear response. A non-trivial part of the big drop in the stock market last year was people being fearful. They panicked because of the economic troubles and wanted all their money out. That means even if the fund managers thought it was a bad idea, they had no choice. You have to do what your client wants. That of course lead to more drops and more panic and so on. Perhaps tax incentives could help stave that off with people.
Actually this kind of trading (especially arbitrage) narrows the spread making it cheaper for ordinary people to enter the market.
So if you ban it you go back to only the big boys being able to afford to execute trades.
However, using any system, high frequency or otherwise, to deliberately manipulate the market is a different thing.
I'm sorry I'm posting as AC but it could affect my job.
You don't need accurate timing for that, you just need to be first.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
"Codes" is a common and long-standing jargon word in several significant sub-fields of CS, notably high performance and scientific computing. If you are unfamiliar with it, then you are simply betraying your own ignorance.
Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.
I operate a small business. I don't give a fsck about your claims that we need "liquidity" when I can't get a simple business loan for less than 8-9%, despite being well-capitalised, while my personal savings attract all of r pct, where 0.01 < r < 0.1. Quite the scam there. As a result, I use only retained earnings to pay for anything, which while a prudent thing to do given my situation, is hardly a recipe for large-scale economic growth. The financial industry does not fulfil its most basic imperative of funnelling capital to productive use, why should I care about whether some institutions can hold a stock for 1ms or 1 day?
I suppose things could be worse. Maybe in addition to being completely starved for capital, I may be subject to "protection" payments in the near future just to be able to stay in business at all! Financial innovation, 'Ndrangheta style! Wouldn't that be a boon for Wall Street and the City of London.
I think people outside of the industry don't really understand the need for day traders.
I write trading software for the futures and options industries (I was the lead architect for the CBOT's Order Routing System, and have written trading systems for other exchanges and banks). The type of day trading that you and the GP have issues with is an absolute requirement for liquid markets and the efficient working of those markets.
Let's say you are a firm that needs to buy a large number of shares or contracts (in the futures world, you buy contracts for the future delivery of product). In the type of buy and hold market that seems "fair", that large order is going to go into a market of other buy and hold traders that are only going to trade out of their positions if the trade makes sense in their long-term strategy. Huge effective tax rates are going to make that trade unprofitable unless you've held the position a very long time (increasing your risk of market volatility) or you charge a premium (increase the price) in order to offset the tax hit.
This type of "illiquid" market raises prices dramatically. You have to factor in the time value of money, the risk exposure of holding shares for a long time (increasing the likelihood of a bad market event) and tax overhead. All that means that buying shares/contracts is riskier and selling them is more expensive.
Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).
Now, when that original firm wants to buy that large number of shares, they are doing that in a market where there is almost always a large number of shares being offered at competitive prices. The day traders will sell off shares and pick up shares all day, just trying to end the day flat (holding nothing) to reduce after market risk.
Day trading does extract a price (they are making money) on the market, but the efficiency gains that they bring outweighs that price in almost all markets. They distribute a lot of the risk involved in trading among a bunch of smaller firms, rather than concentrating all the risk in big funds. This is generally a good thing - the government and the market in general are better off when a small firm goes bust than when one of the "too big to fail" firms starts having problems.
Now, the rise of algorithmic trading is causing some stress in this pattern - I can see some real issues coming in the next few years as these algorithms become more and more precise. I'm less concerned about a rogue algorithm dragging a market down (or up) than of all the algos converging to a point where the market gets stuck because the algos are just too damn efficient. But that's a post for another day...
The only really major issue with NTP is figuring out the time offset due to network latency in asymmetric network environments. The NTP protocol itself can frequency lock down to a few ppm over the internet without any problem at all. Offset errors are another matter. Over the internet 10 ms is about the best you can do. Over a LAN with a local GPS signal driving the protocol you can reduce the offset error to less than a microsecond and can probably get it down to less than 100ns without too much trouble (and without needing any hardware time stamping or netif queue prioritization), and you can frequency lock down to a few nanoseconds.
Other issues include the fact that most motherboards do not have temperature compensated timebases, and they tend to float around a few ppm (for the better MBs) to a few hundred ppm (for the stupid MBs that put crystals next to heat sources), as well as bugs in the operating systems themselves which don't get noticed until you actually try to frequency lock your timebase. This creates a multiplicative effect if you are not connected to a stratum 1 or 2 server because each server in the chain is trying to do a frequency lock against a drifting MB timebase. The result is the leaf nodes in the chain can get frequency locks but the locks are to a drifted frequency instead of to the correct frequency because the server itself can't instantly correct for its drifting timebase. This is why you see the frequency lock jump around by +/- 50ppm in a seemingly uncontrollable fashion once you get past stratum 2.
NTPD was famous for not being able to frequency lock while doing an offset correction at the same time. The NTPD program has also had many issues over years, such as not using a proper dual-staggered linear regression to control offset and frequency corrections, which is why I wrote dntpd for DragonFly. I wonder how many of these old issues with NTPD have been fixed over the years, maybe they have.
Network jitter is largely irrelevant. It isn't a problem if you use a proper linear regression over a long enough period of time. The linear regression can be used to calculate what the jitter is and thus form a good knowledge of the baseline accuracy of the protocol. A medium sized run over a few minutes will get you down to 10-15 ppm on your frequency lock and you can get down to 1-5 ppm within about 10 minutes (assuming the motherboard has an accurate timebase of its own).
The sad thing about all of this is that it takes just a single resistor to temperature compensate a crystal, a resistor most MB manufacturers don't put in. Some don't even use crystals anymore for their PLL input (and the crystal on the RTC doesn't help much because the RTC doesn't have a fine enough grain timer to poll easily). Sigh.
-Matt