Slashdot Mirror
Google's New Public NTP Servers Provide Smeared Time (googleblog.com)
Google says it has built support for the leap second into the time servers that regulate all Google services. An anonymous reader shares a blogpost by Google:No commonly used operating system is able to handle a minute with 61 seconds, and trying to special-case the leap second has caused many problems in the past. Instead of adding a single extra second to the end of the day, we'll run the clocks 0.0014% slower across the ten hours before and ten hours after the leap second, and "smear" the extra second across these twenty hours. For timekeeping purposes, December 31 will seem like any other day. All Google services, including all APIs, will be synchronized on smeared time, as described above. You'll also get smeared time for virtual machines on Compute Engine if you follow our recommended settings. You can use non-Google NTP servers if you don't want your instances to use the leap smear, but don't mix smearing and non-smearing time servers.
Does this mean that my DPS will be slightly higher on December 31st?
My eyes reflect the stars and a smile lights up my face.
Google has been smearing leap seconds over NTP since 2011. This is a public reminder that Google NTP will be serving smear seconds because there is one coming.
"don't mix smearing and non-smearing time servers"
Best Slashdot Co
At 4.37 it'll report that 4.38 molests goats. 4.38 will retaliate by claiming 4.37 killed a man and lied about it. 4.39 will accuse 4.38 of secretly having two wives who know nothing about one another. 4.40 will claim 4.38 and 4.37 are having a secret affair and are making up allegations about one another to hide the fact. 4.41 will claim 4.40 is a multiple felon. 4.42 will accuse 4.41 of cheating on his taxes...
You are not alone. This is not normal. None of this is normal.
Jesus Christ, this sounds like an April Fool's joke.
"Only Trump made that extra second for Americans possible!"
or
"A typical smear campaign by Trump supporters!"
Pick your side.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Instead of adding a single extra second to the end of the day, we'll run the clocks 0.0014% slower across the ten hours before and ten hours after the leap second, and "smear" the extra second across these twenty hours.
I wonder why they're "smearing" the time over 20 hours rather than 24, which would seem the more obvious solution.
I had a spare second to waste.
But was it smeared or unsmeared? Enquiring minds want to know!
We used to call this fuzzy logic back in the 1980's.
https://en.wikipedia.org/wiki/Fuzzy_logic
Let's just slow that down. Hope that wasn't important. What is 0.0014% between friends?
Friends that do low-latency high stake stock trading may have issues if some of competing parties run on a skewed clock (or syncs with something that syncs with something that runs on a skewed clock) and others don't. That affects not only the latency, which is important enough, but as the skew increases until the end when the leap seconds kicks in, and because the leap second is added at midnight UTC while bourses around the world are offset from that, it could mean that some might get a large part of a second more than others at cutoff time, which is an eternity in this context.
*pushes glasses up* *straightens pocket protector*
Actually, they would need to run the clocks 0.0013889% slower. If ran the clocks 0.0014% percent slower, they would gain 1.008 seconds instead of just 1 second even.
Hire me Google. I'll save the internets for you.
Can we just move to TAI and convert to UTC only when interfacing the meatspace?
NTP shouldn't have to smear it. All that will happen is that one second your computer will think it's on time, and a couple seconds later your computer will think that it's a second behind and correct itself against the NTP server. Computer clocks are inaccurate enough that it's expected they will drift by a few seconds in a day anyway and NTP corrects that quite handily already.
Instead of making NTP servers smear a second over twenty hours, here's an idea, how about just implementing support for it at the OS level? It's not like this phenomena is new. It's been around since 1972, longer than many countries have been implementing daylight savings, yet that little gem happens twice a year without a hitch. Despite this, proposals for abolishing leap seconds continue to abound, and claims that the next one will make the sky fall continue to waste people's attention. There have been 27 leap seconds since 1972, and not a single one of them has broken the world yet.
I suspect Google spent more time adjusting their NTP servers to wingle around it than it would have taken for them to, say, update Android to support it.
Why? Where do you see an advantage in not smearing leap seconds?
I can't ever see a story like this without thinking "O cursÃd spite".
Bruce Perens.
If your work depends so precisely on timing, then you should be handling your own timekeeping.
Unix time - (also known as POSIX time or Epoch time) is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, [note 1] not counting leap seconds.
This is fine for many systems but what about the systems that do count leap seconds. In January they will be 1 second ahead of POSIX time. I'm thinking of industrial equipment and electrical infrastructure. I'm also wondering about how log files will be matched up when different systems are assumed to be synced to under 1 second.
Oh wait, we will just see what breaks and kludge it in January.
From the blog:
That is the problem; it's not well determined in advance, so simply distributing static tables of data is just not going to work all the time. Moreover, those tables are highly sensitive to political nonsense in any given locale. Thus, the Google folks are basically saying "All right. We declare ourselves to be the Great Overseers of time; at any given moment, just set your clocks to what we tell you!"
I suspect very few here will any any sort of sympathy for people who do low latency stock trading. Some even suggested to add time skew or random jitter to stock tickets precisely to prevent this sort of trading.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
Smeared seconds come after sloppy seconds.
Really? You think getting every program on the planet that that interacts with Google's time servers to reliably handle a minute with 61 seconds without any ill effects is simple?
Meanwhile, very little software, even time-sensitive software, has has any concept of real time, only clock-time, so making the clock run very slightly slower accomplishes the same thing while making everything look like (very slightly faster) business as usual, with the only potential conflicts coming from slight ( 1 second) inconsistencies between different time servers. And I would venture a guess that very few applications rely on different time servers being perfectly synchronized to begin with.
And as they mention, Google is actually attempting to coordinate a smearing standard with other time-server operators as well.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
To be even more clear, every clock on the planet is constantly providing inaccurate time, since we've established a very precise standard unit of time (the second) that bears only an approximate relationship to the non-constant physical phenomena that originally defined it (1/86,400th of a mean solar day)
--- Most topics have many sides worth arguing, allow me to take one opposite you.
by Stanislaw Lem comes to my mind reading this.
"Well, the Monday me on Monday night became, Tuesday morning, the Tuesday me, and so on."
http://webcache.googleusercont...
NTP Pool is there for a reason.
If your work depends so precisely on timing, then you should be handling your own timekeeping.
No, the concern isn't whether your time is correct, but whether it is the same as the time others use.
That is not solved by adding an atomic clock. In fact, it might make it worse.
It's actually closer to the correct implementation - leap seconds are the technical solution to try and get us back closer to the correct time. Making seconds marginally longer is actually more accurate.
I suspect very few here will any any sort of sympathy for people who do low latency stock trading.
That's short-sighted thinking. Low latency traders exist, whether we like them or not[*], and it does not affect just the traders, but the stocks they trade, the companies, and other stock holders not involved in low latency trading. Their ability to do harm to others is affected when the playing field shifts, giving advantages to some. They're at war with each other, and the playing field being level helps maintain status quo, and prevent stocks from crashing or soaring when they shouldn't.
[*]: I too think they're scum. I'd like to see all stock trades have a delay of one business day.
A smeared second is stupid IMHO. People have had since 1973 to put leap seconds into their software. However, this is how NTP does it, so many computer clocks will have a smeared second even if they don't use Google.
UTC with leap seconds was set up to support celestial navigation. You can still take out your sextant and determine your position to a km or so using standard clock time. There is still a feeling that that is a useful attribute.
My personal feeling is that the Internet should just adopt TAI, but I have never gotten anywhere with that proposal.
Instead, this will go on until some plane crashes or rocket explodes or there is a massive exploit* due to a leap second being incorrectly handled, and then this will be fixed.
* There are some security protocols that make implicit assumptions about the time being roughly coordinated. On leap second day, those assumptions may be false,
If the system is time sensitive. Say like some sort of GPS system, where if the time is a little slower then your calculations may be off.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Or you could use the USNO, the other official time reference for the United States.
Some sort of GPS system should probably be using the time that the satellites are providing.
Just make the stocks trade in time-coordinated batches. The batches can still be relatively fast - even every second - just so it is slow enough to stop the stupid games with the speed of communications.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
With 20 hour smearing you get that for ~10 hours (i.e. significant time interval) time on synchronized computers would be more than 1/2 sec (i.e. human-noticeable difference) from correct value. That is substandard quality for time synchronization.
Historical definition of second is irrelevant. Current time standards (TAI, UTC) are defined in terms of the current definition of second. If a clock reports time sufficienty near to appropriate time standard, then it is by definition correct.
Jesus H Christ on a flaming pogo stick!
Just when we were starting to get to the last month of this AIDS-filled dumpster fire of a year, the go and add another goddamned SECOND to it.
I usually don't say things like this, but in this case, FUCK YOU, SCIENCE!
You know what's going to happen, they're going to start dropping Dick Clarke's frozen head in Times Square, and it's never going to reach the bottom. It will be the end of time and it will always be 2016, FOREVER.
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Handling your own timekeeping doesn't necessarily mean having your own atomic clock. In this context, it means deciding on who has the canonical clock and syncing with it.
Google is explicitly saying they are not serving UTC, so if your trading partners are on UTC, you're an idiot if you sync to Google.
I don't see why this should be a problem. NASA will fix this at the NASA/USGS Rotational Tuning Facility #9?
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
This.
Every trade that occurs within a given interval (I think 1 second is too short still, maybe a minute... or 1/4 hour, since that's often how long quotes are delayed for the average trader) trades at the same price for a given transaction type. The net effect on the market would be the same, we just wouldn't have, for example, two people placing SELL orders for a million shares of something at $100/share and one of them getting $190/share while the other only gets $10/share, which does currently happen.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
But that's the problem - you can decide who you sync with, but you cannot decide who others sync with.
A time discrepancy is a problem in either case.
The most obvious solution is that the bourses should provide time synchronization, and that everyone else sync to their time, right or wrong.
... it is not the best solution. A leap second table to make it a time representation issue would probably be better.
Seems obligatory: "A man with a watch knows what time it is. A man with two watches is never sure."
How will you decide on order priority within the one-second batches? Price/time priority? Favours people with fastest connection as they can quote beyond the price they would be filled at when the batch trades and pull their quotes at the last moment if the market moves against them. Price/volume priority? Favours the bigger players who are pushing bigger orders around as well as people with faster connections. That's just two simple, obvious disadvantages to running mini auctions like this. If there was a simple solution, it would've been done already.
Some exchanges do implement technical measures to try and reduce gaming. For example KRX enforces fill ratio - there's a minimum proportion of your orders that must actually trade or you'll be penalised. This means you can't sit around placing orders and cancelling them all day with no intention of executing. But on the other hand, it means you can't readily quote, displaying an order at the price you're prepared to trade given current market picture and moving it when the market moves. This makes it harder for an investor to know what price they can actually get executed at, and leads to a silly game of trying to hit attractive orders as quickly as possible. So it still makes life harder for actual investors and favours the guy with the lowest latency price feed.
The main reason for the latency wars is just because margins are so thin. In the past, spreads were tens of cents. A market maker (back then typically one of the big brokers or investment banks) made tens of cents against the theoretical fair price on every security traded. Lower latency allows quoting tighter spreads (you can move quotes quicker if the market moves against you), so the market makers are making far less money per trade - only a few cents per security minus exchange fees and government taxes on every trade. Because the profit per trade is really low, it's a cut-throat competition to get as many profitable trades as possible.
The defacto non-solution does much the same thing to the system clock after the leap second in order to correct for suddenly being 1 second fast but maintain monotonic time. That is, adjtime (or adjtimex) gets called to slew the clock.
In many cases, the defacto non-handling is the right thing to do. Although it compresses events very slightly, it maintains causality in the system logs and such.
Interesting. I hate them both and use a Moto Droid.
Nevermore.
As I read it, the worse is 1/2 second (slowly fades to a half second behind, then leap second puts it a half second ahead and is slowly fades to match).
Since the Unix timestamp ignores leap seconds anyway, this seems like a very reasonable approach.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
That's not POSIX compatible.
“Common sense is not so common.” — Voltaire
If you're that worried, change your accepted settings.
But you're talking crap, as my NTP pool server gets PUSHED OFF the pool if it drifts anywhere near that far, or goes offline for even a short time.
Currently my jitters for uk.pool.ntp.org are:
0.071
0.079
0.050
0.190
Those numbers are IN MILLISECONDS. 0.05 of a millisecond.
And 0.478 on a stratum 1, famous, advertised, academic, public NTP server not in the pool.
How will you decide on order priority within the one-second batches?
I wouldn't do it - people who do this sort of thing for a living would. If I'm just spitballing and you won't make fun of my naivety I could try to come up with some solutions. I'm not sure why FIFO wouldn't work just fine - simply fill the buffer until the next transaction window fires. It's the same system that is in place now, with larger quantum steps.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I actually do this for a living, and I'm telling you that 1-second intervals would make the market less stable and more prone to price fluctuations. This would make the effective transaction costs for investors higher and put more money into the pockets of market makers. But sure, if that's what you want to do, pressure your market regulators for a change like that. It's more money for me in the end.
It's Google doing this. You just have do it Google's way because someone at Google arbitrarily decided it was the best thing to do for Google, regardless of existing standards, other environments or systems, or indeed the rest of the world breaking as a result.
Look at Gmail's implementation of addressing. Dots in the user portion of the address were significant in 1982 (RFC 822 / STD 11), but not for Google, who cannot differentiate Bob.Dole@ from BobDole@. Still. In twenty-freaking-sixteen.
FFS I'm not doing anything requiring microsecond accuracy if it's within one second or less then it's fine, and unless you're running some sort of mission-critical operation that requires precise alignment to the atomic clock for some reason, you should probably just take your meds and calm the hell down about it.
I don't know about apps or luddites. I do know that Trump and cows generate methane.
Slow down, cowboy! It has been 4 hours since you last posted. You must wait another few hours.
Before you go for my throat, I was simply proposing a way to put the kabash on low-latency stock trading. I don't actually have a strong aversion to it, and though I'm pretty ignorant about the whole thing tend to agree with you. But if stopping the low-latency guys is your goal, it's pretty straightforward.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
No they don't. The clock is quite unimportant for the HFT crowd. All they care and look at are the incoming orders and changes in price, the importance of the clock is only important for later record keeping by firms allowing clients to trade via their accounts (aka stock brokers) since they have to prove to the authorities later that they gave their clients the best price available at the time from the various exchanges that they trade on but that is also not a problem since you then simply aggregate the log of each exchange as you see orders coming from them (it does not matter if the clocks on the exchanges differ since what is important is the order in which you received each message, i.e that is what defines when your system could see the best price).
Exactly this, some exchanges like the Canadian Alpha Exchange uses a random 1-3ms speed bump delay on each incoming and outgoing message. Other exchanges receive orders until a specific time at which point they arrange the orders based on price and not on time and matches which combinations of bids and asks that would create the fairest price and highest trade volume, aka continous auctions.
If I were someone who actually cared about seconds and depended on high resolution timekeeping this would undoubtedly strike me as a really shit-headed idea.
Exactly. If you are part of a group making financial transactions, that group needs to decide on a canonical time source. And yes, it's relation to the rest of the world is irrelevant to the transactions.
In any event, all time sources are wrong to some degree. If you're using NTP over the internet, you will perhaps keep the error below a tenth of a second. A directly attached GPS clock will keep it closer but the error will remain non-zero.
Well, that is kind of the point of the leap seconds - to compensate for just such unpredictable changes in the Earth's rotation...
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Good point, I stand corrected.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Couldn't they have used a nicer word than "smeared", like maybe "interpolated"? (See also other aversion-causing words: "moist", "crevice", "phlegm", "penetration", etc.)
Given the behavior of the "smearing", they are NOT NTP servers you want to use with hard real time systems, especially if 1 second will prang your gear.
Sometimes, real fast is almost as good as real-time.
You don't consider the priority at all. What you do is that you calculate which price would yield the best volume and fairest price, i.e you calculate a equilibrium price. Most stock exchanges already does this during opening and closing (via the open and closing call) in order to precisely avoid price fluctuations when determining the opening and closing prices. Some exchanges like the Frankfurt Stock Exchange even run such auctions continuously during the trading day (Continuos Auctions) on some of it's markets. Another model which is done by Alternativa (http://alternativa.se/), although for highly illiquid shares, where traders enter orders for four days and on the fifth a fixing price is determined at which trades are calculated so there can be only one trading session per week (so it would of course not work for something as liquid as the Nasdaq or NYSE).
Except that a second is defined in terms of [what is emitted by] a phase transition of some particular element, or possibly in terms of the speed of light. Neither of which cares or has anything to do with Earth going around the Sun, and spinning around its own axis.
So "more accurate" is wrong. It will conflict with everything that does not both a) smear time and b) smear time in the exact same way google does.
CLI paste? paste.pr0.tips!
The question was if there could be a problem with Smeared Time.
I have an answer. It probably isn't the best solution to the problem, but it is a problem that exists.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.