CloudFlare Was Hit By Leap Second, Causing Its RRDNS Software To 'Panic' (silicon.co.uk)

Reader Mickeycaskill writes: The extra leap second added on to the end of 2016 may not have had an effect on most people, but it did catch out a few web companies who failed to factor it in. Web services and security firm CloudFlare was one such example. A small number of its servers went down at midnight UTC on New Year's Day due to an error in its RRDNS software, a domain name service (DNS) proxy that was written to help scale CloudFlare's DNS infrastructure, which limited web access for some of its customers. As CloudFlare explained, a number went negative in the software when it should have been zero, causing RRDNS to "panic" and affect the DNS resolutions to some websites. The issue was confirmed by the company's engineers at 00:34 UTC on New Year's Day and the fix -- which involved patching the clock source to ensure it normalises if time ever skips backwards -- was rolled out to the majority of the affected data centres by 02:50 UTC. Cloudflare said the outage only hit customers who use CNAME DNS records with its service. Google works around leap seconds with a so-called "smearing" technique -- running clocks slightly slower than usual on its Network Time Protocol servers.

Comments

Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

The blog post about this incident says:

RRDNS is written in Go and uses Go’s time.Now() function to get the time. Unfortunately, this function does not guarantee monotonicity. Go currently doesn’t offer a monotonic time source (see issue 12914 for discussion).

and then later it says:

When RRDNS selects an upstream to resolve a CNAME it uses a weighted selection algorithm. The code takes the upstream time values and feeds them to Go’s rand.Int63n() function. rand.Int63n promptly panics if its argument is negative. That's where the RRDNS panics were coming from.

So to me it sounds like this incident was at least partially due to limitations with the Go programming language and its libraries.

Would this incident still have happened if this software were written in the Rust programming language?

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

I don't know if you can blame the language, the devs should have added their own checks if the language didn't have a guarantee.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

Why would you even think of switching programing languages due to the simple and sadly common 'bug' of programmers not verifying parameters match a function's documented pre-conditions? My only guess is you're paid to promote Rust. Lazy programmers will write bugs in every language.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

I STILL can't tell if these Rust trolls are just doing it for the lulz or if there really is a big gang of deluded out-of-touch Rust shills out there trying to get brand recognition by any means necessary, including making themselves the biggest laughingstock since apk.

Re:Was the Go prog lang at fault? Would Rust help?

[scamper_22]

Part of the fault can go to the Go programming language for their API design.

But most of the blame goes to the developers.
I haven't coded in Go, but I googled this quickly.
https://golang.org/pkg/math/ra...
The Go documentation clearly says it panics if n = 0.

They could have
1. validated their inputs.
2. Handle the panic and assign a default value (I am assuming this is possible in Go. I have never used it)

In the end, it seems like this is just used to distribute requests. Worst case, it should log the error and then assign say the 1st upstream (default value).

But I guess then you're in the exception handling debate on whether you swallow the error and keep going or have your application crash so that you detect the weird condition.

I'm a defensive; keep the system going developer.
But others prefer to be more exact.

Re:Was the Go prog lang at fault? Would Rust help?

[Obfuscant]

The Go documentation clearly says it panics if n = 0.

And it says it panics if n is less than 0.

If you write a library function that requires positive input always, and returns positive output always, then use unsigned input and output variables. A good compiler will flag the attempt at sending such a function a signed input as a warning at least. Pedantic compilers will fail -- better than the production program failing.

And while it seems stupid, the proper action when asked for a random number between 0 and 0 is to return 0, not panic. (I believe the [ on the range means "including", but I could be wrong. If it didn't mean "including", then the documentation should be '(1,n)'.)

But then, the test cases for the DNS code should have included 0 and negative, so this should have been caught when the function was tested.

Re:Was the Go prog lang at fault? Would Rust help?

[scamper_22]

Yep.

I meant to type n (less than or equal to) 0.
Not sure if slashdot escaped it or something.

In any case, yes, the Go API was not the best taking in a signed int when negatives are invalid.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

Would this incident still have happened if this software were written in the Rust programming language?

As always, the answer is "who knows"? Programmers are creative people who can introduce a bug to a program written in any imaginable language. That said, Rust does have monotonic instants, so that would presumably help.

Re:Was the Go prog lang at fault? Would Rust help?

[WaffleMonster]

I don't know if you can blame the language, the devs should have added their own checks if the language didn't have a guarantee.

Noting math/rand is part of the standard go library and more rigorous compile time checking would have prevented this seems like a no-brainer to blame the language.

Re:Was the Go prog lang at fault? Would Rust help?

[Waffle Iron]

So to me it sounds like this incident was at least partially due to limitations with the Go programming language and its libraries.

For now, you could use a platform-specific workaround. (Just like you would have to do if you were coding in C). For Linux:

func Uptime() (int64, err) {
    var si syscall.Sysinfo_t
    err = syscall.Sysinfo(&si)
    return si.Uptime, err
}

I'm too lazy to look up whether Windows has a similar feature.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

The amount of hatred that Rust gets here at /. is absurd. It's just a programming language. It's not even a widely used one. Yet even the mere mention of the name "Rust" gets so many people here, including you, riled up and agitated, without any reason at all. It's particularly weird when we have people making utterly crazy claims like "My only guess is you're paid to promote Rust." or talking of a "big gang of deluded out-of-touch Rust shills". Seriously?!

I mean, I can sort of understand the hatred that people here had for Windows and Microsoft back in the day. At least those people likely had to use Windows and other software from Microsoft, and didn't like it. But Rust? Most people here probably haven't even seen a single line of it, never mind actually used it, or even just used any software written in it. Yet people like you go absolutely insane if somebody happens to bring it up! Rust is nothing more than a little-used niche language that likely won't go anywhere. So your overreaction to Rust is pathological.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

Exactly, it's a little-used niche language so suggesting a major system to be rewritten in it just because of potentially one API method is crazy. Why would someone suggest such a thing? Why bother suggesting any language? The original poster could have simply said Google should have put a little more thought into it's API or Google could update the API to handle it and then no one would have cared. But no, instead he claimed Go was limited and tried to imply Rust was better. When you're pushing something in a content that doesn't make sense people are going to assume you're a shill for it because bringing it up doesn't make sense.

The fact that it's Rust isn't the issue. The fact that it's potentially a hidden ad for something is the issue. Such advertising should be illegal and the products and companies which engage in it should be shunned.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

The original comment says nothing about doing a rewrite. It obviously does not "try to imply Rust is better". The only person bringing up those ideas is you. The original comment merely asks the question, "Would this incident still have happened if this software were written in the Rust programming language?", without suggesting any sort of an answer. Maybe the answer is "yes". Maybe the answer is "no". Maybe the answer is "maybe". The original comment doesn't speculate. That makes me think it's a legitimate question about Rust.

Look, I don't like Rust. I don't like its community. I don't like its code of conduct. But what you're doing is asinine. Your false accusations of "shilling" and "advertising" are nonsensical. It's even worse when you're making claims about that comment that clearly aren't true. If you don't like Rust, that's fine. But acting like a kook when somebody mentions the name "Rust" only hurts /., and worse, it puts Rust and the Rust community in a better position. They start looking like the sane ones, and people like you who jump off the rails for no reason at all start to look really pathetic.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

You might not be seeing it, but I haven't seen that much fanboyism in a very long while. They may be right, but from an outside point of view, they look completely insane.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

Part of the fault can go to the Go programming language for their API design.

Indeed - a random number function that panics. Now that is useful.

You want servers to keep running. The better approach would be to return 0. and in the interest of debugging, spam the syslog with "bad parameter -x to rand.Int63n()". Complaints puts some pressure on devs to fix things.

Re:Was the Go prog lang at fault? Would Rust help?

[Anonymous Coward]

You need to type

& l t ;

(without the spaces)

Unit test those edge cases

[Bengie]

I still wonder how these simple edge cases make their way in

Re:Unit test those edge cases

[Anonymous Coward]

There's a billion edges out there, someone somewhere missed "some minutes have 61 seconds".

Re:Unit test those edge cases

[unrtst]

Read the article then. It shows it pretty plainly: https://blog.cloudflare.com/ho...
I was going to try to guess what they were doing, but they have some actual code snippets.

AFAICT, a unit test wouldn't have caught this either (unless they planned for this sort of error, in which case the code wouldn't have been broken either). From TFA:

RRDNS doesn’t just keep a single measurement for each resolver, it takes many measurements and smoothes them. So, the single measurement wouldn’t cause RRDNS to think the resolver was working in negative time, but after a few measurements the smoothed value would eventually become negative.

So, a unit test with one negative example (which may have been difficult to mimic anyway, due to the direct usage of Time.Now()) probably wouldn't have triggered the issue on its own.

IMHO, blaming a misconception of time always going forward is just convenient here. The fix was changing this bit:
if rttMax == 0 {
      rttMax = DefaultTimeout
}

They just changed "==" to "<=". There was no reason not to have it as "<=" to begin with, even if one ignores where rttMax comes from. Any time I check if something is == to something else, and I don't have else conditions covering the other cases, I ask myself what should happen in those other else cases and ensure I'm covered. That may still have caused it to break, but it could have done:
if rttMax == 0 {
      rttMax = DefaultTimeout
} else if rttMax < 0 {
      panic("What the fuck happened to rttMax to make it negative!?!")
}
...though it probably would have been better to just log that somewhere and set it to the DefaultTimeout.

Anyway, I think it's a great example of a one character bug that only triggers on very obscure events under significant load.

Re:Unit test those edge cases

[skids]

I'm still left wondering whether the decision to put a leap second on the night tech support staff are most likely to be over halfway through a bottle of JD was A) some intentional attempt to catch edge cases where leap seconds happen during a year change or B) some tinfoil conspiracy where we'll find out billions of dollars were stolen from a system where that particular edge case could be exploited or C) just made by people so socially isolated that they don't realize just how hard it is to fix crashed boxen over a crappy 3G connection in a dive bar bathroom using a phone covered in some chick's vomit while trying to keep down that pretzel you just washed down with sparkling water.

Re: Unit test those edge cases

[Anonymous Coward]

Millennial idiot coders are too busy refactoring everything every three months to bother testing anything ever. Mature codebases are too old, dudebro, old is bad, mmkay.

Re:Unit test those edge cases

[Obfuscant]

I'm still left wondering whether the decision to put a leap second on the night tech support staff are most likely to be (conspiracy options elided) ...

Or most likely, the people who put the leap second where it was knew there was no perfect time to do it, and assumed that anyone who was writing software that was so time-critical that it cared if there was a leap second would properly handle the issue in their code.

It's not their fault that some developers using an off-beat language that has a library that panics if a parameter is invalid (and was written so that there could BE an invalid parameter, which they could have avoided) didn't bounds check their parameters to such a function.

Re:Unit test those edge cases

[Anonymous Coward]

You are thinking to specifically.

For this problem they missed an edge case that subtraction can produce a negative number.

These are the kinds of things that can be caught with static analysis, because they don't look at the big picture only at small definable outcomes.

Re: Unit test those edge cases

[Anonymous Coward]

I'm surprised all the code wasn't written in an Node.js already. Not very hip of them.

Re:Unit test those edge cases

[Aighearach]

There aren't a billion edges, there is only an edge where your code establishes a limit.

Some minutes have 60 seconds, some have other amounts. That doesn't cause an edge case. An edge case is caused by when your code assumes that the number of seconds has some specific value. So if in the code I say "if ( seconds assumes that the value will be valid, and they don't do something useful when the values are wrong. So they crash and burn. You want to either not care what the value is, in which case you don't want to even create an edge by testing it, or else make sure that you have a valid code path for all possible values. Did you establish an upper bound? You have to test what happens when the data exceeds it.

Basic stuff, which is why when there is a leap second, just one piece of junky code stopped working and nothing else had any problem. There were probably large numbers of applications that actually have leap second bugs; careful log analysis might indicate that things that happened during the leap second were recorded as having happened at the start of that minutes. So instead of crash-and-burn, all your things that would have happened at 23:59:60 would be listed as having happened at 23:59:00. That's because competent programmers do something useful when they get bad data instead of just crashing and burning.

Re:Unit test those edge cases

[Anonymous Coward]

There are two schools of thought:
* Your's don't ever crash, make sure something happens, even if it could be the wrong thing.
* Fail fast, crash immediately if you notice something happening that is not expected, don't do the wrong thing.

Both have its place depending on the environment. Sometimes you application just needs to continue, no matter what. Sometimes an application must never do the wrong thing, no matter what.

Re:Unit test those edge cases

[Anonymous Coward]

AND their Site Reliability Engineers are so green that they wouldn't test for the rare border conditions (where 80% of the crashers are) *or* design an environment where such border conditions don't exists (as Google and Microsoft do).

The kernel people got their shit straight when they last had any sort of leap-second trouble a few years ago, and regression-test the crap out of the timekeeping code, up to and including fuzzing it to smitterness. Yes, bugs were found and either fixed, or that entire section of code was reworked to be *really* resilient. All development kernels now boot with timers close to warp-around values, for example. Also, a test suite for the NTP kernel code now exists to ensure 100% code coverage (including negative leap seconds, which were never used in practice, and several leap seconds in a row, which is either an outright attack, or a userspace NTP software bug), so that codepath is guaranteed to go boom on a kernel developer if it ever grows any new bugs. The result? No O.S.-related crashes or misbehavior *at all* were reported for this recent leap second *for over hundreds of millions of machines*.

But of course that doesn't protect your userspace application from itself and from highly idiotic design errors (the Go API in question) and faulty implementation (the use of said API by RRDNS -- which is not an idiotic error, rather, it is a common error and if I'd use harsh words about it, I'd direct them to their QA and testing people/procedures/policies).

Re:Unit test those edge cases

[unrtst]

So if in the code I say "if ( seconds assumes that the value ...

Kinda amusing that your post is an example of unexpected (though well known) data causing an incorrect outcome. IE. slashdot ate part of your comment (I'm hoping that assumption is correct. Otherwise, your brain ate part of it). Sadly, we have to manually escape < (ie: &lt;) and friends here (and I have no idea what all must be escaped).

So who was fired for this mistake

[Anonymous Coward]

I want action CloudFlare or I'm suing!!!

My internet died...

[ckatko]

...at exactly midnight, while I was playing Chivalry. I kept getting laggier... and laggier... and then everyone "froze" and the client-side prediction took over. I was recording video and it was pretty funny. Everyone just kept walking forward, until they were in a wall, and kept trying to walk forwards.

It was interesting what the client prediction would let you do. You could change weapons. You could swing your weapon. You could throw axes (of which you have two) and they flew through the air, stuck in people, and even knocked helmets off. BUT, your axe counter never actually decreased. So you could just keep throwing hundreds of axes. The animation timings / speeds were unaffected. You couldn't "chant" or grunt. You obviously couldn't damage anyone.

Anyway, my internet was down until the next morning and even then, it still required a cable modem reset to fix the connection.

Re:My internet died...

[Falos]

Specifics vary, but some games have interesting latency tolerances that they're willing to resolve without a Nope shrug (rejection). Mounted WoW players often glide past "no fly zone" triggers, sometimes remaining airborne long enough to reach the unreachable. Usually inconsequential places, by design.

PoGo players (and Ingress, I'd guess) have been known to capture indoor critters by dashing at building walls in the meatspace, then huddling over their phone to block signal. The game extrapolates position, and they can trigger a now-in-range critter engagement client-side. They resume the signal, and the server resolves it accepted.

These are amusing examples, but if I was big on FPSs or fighters I'd probably be more bitchy and whining about gritty specifics. I know I didn't enjoy having to run in front of people to cast Backstab in WoW.

OT: I've heard that, on the whole, these by-hand NTP solutions actually end up working out pretty good all around. I'm too ignorant to have my own opinion.

Re:My internet died...

[ckatko]

I've actually been researching network game architecture lately and I was actually planning on doing some video-recorded analysis of various commercial-game network models when latency, jitter, out-of-order, and other errors occur. Extreme latency is a great way to "reveal" what's going on under-the-hood.

So this time, I got video and I didn't even have to set up artificial lag!

Re:My internet died...

[Anonymous Coward]

PoGo players (and Ingress, I'd guess)...

Yes, Ingress players have been doing that for years. I have seen tutorials online about it (turn on location smoothing, run fast and stop at the last second, etc).

Re:My internet died...

[antdude]

Do you have still have that video recorded to upload to share with us? ;)

Re: My internet died...

[buchanmilne]

"Anyway, my internet was down until the next morning and even then, it still required a cable modem reset to fix the connection."

Some network equipment vendors sent out field notices about 2 weeks in advance of the leap second, recommending operators to use leap-second smearing (as implemented in chronyd for example) if they had affected versions of network device firmware deployed that could crash as a result.

(We didn't have affected versions deployed, and it would have been non-trivial - at this time of year - to get all our NTP servers upgraded. It't not recommended to use non-smearing and smearing NTP sources on the same device)

Re:My internet died...

[ckatko]

Since you asked, sure.

It'll take awhile to upload the 4K video (2.75 GB for a mere 2.5 minutes) and YouTube to reprocess it. But the link, when live, will be here:

https://youtu.be/KIeM1y9S5Mo

Re:My internet died...

[antdude]

Wow, can't you encode to smaller video before uploading? Do we really need to see it in 4K? Hahaha.

the gift that keeps taking

[Thud457]

2016 says "Hi, remember me, beeotches?!!"

CNAME Flattening

[Anonymous Coward]

May have also contributed.
Would explain why only some sites affected.
Not RFC compliant.

Re:CNAME Flattening

[Anonymous Coward]

May have also contributed.

Not as such; the problem was with CNAME lookups in general. Furthermore, you can't call flattening "not RFC compliant", as the client never sees the CNAME record at the apex: Cloudflare's servers resolve it to the final address before responding.

Do we really need to compensate?

[Anonymous Coward]

We lose or gain a second here or there, who cares? The difference has been so far 27 seconds over the past 44 years or, extrapolated out, 1 MINUTE over 97 YEARS.
Are we really going to notice if the sun goes down a minute earlier every century? We already have to screw around with daylight savings & leap years why not just make February the 29th 24 hours and 1 minute long once a century and have done with it.

Echoes of time changes days gone by

[Archfeld]

I always remember time changes as busy nights in support when I worked for a large bank. The spring forward was usually a breeze, just a matter of a lot of server verifications and log checks, but the fall back was usually a messy night. Much harder to deal with and resolve issues involving duplicate timed log entries and transaction logs. I don't really miss those days...

Re:Echoes of time changes days gone by

[Anonymous Coward]

UUIDs are also fun to deal with. Especially with VM images that are copied by those who don't understand how you can have a duplicate UUID.

Re:Echoes of time changes days gone by

[caseih]

So the bank's systems didn't store transaction times in UTC or some other timezone-neutral format? How did they deal with transactions originating from other time zones?

Re:Echoes of time changes days gone by

[Anonymous Coward]

I've seen too many companies, even large multinational companies, who insist that their servers run on the HQ's local time to be surprised by this anymore.

It ain't the IT folks doing it, I'll tell you that. The smart ones quietly say "fuck that", set the system clocks to UTC and then set a TZ environment variable everywhere.

pure insanity

[Anonymous Coward]

Who the hell cares what time it is? Nice for a reference, but nothing should be so dependent upon the clock that it ceases to function properly. Geez. I know there are millions of examples of why time dependence is important (like stock market transactions), but that's the whole problem, this should not be the case. Stop building the house of cards.

Re:pure insanity

[Highdude702]

you apparently have no idea how computers and the internet and encryption type shit works do you? time is very important, if it wasn't why would you do anything right?

Re:pure insanity

[Anonymous Coward]

Most stock market transactions are not time based they are event based; somethings happens in the market, you react on it.

Time is important to measure the performance of the system and of the market. Time should also be used to detect faults and excessive latency; part of the flash crash was due to the exchange being several minutes late with reporting trades and order book status, comparing the timestamps in those messages could have made safety systems to be triggered.

Goes to show the old adage is true

[SuperKendall]

Don't use services who names are terribly ironic in times of failure.

Flare, Flame, Burn, Drop, Etc. Et.

The universe just loves to throw a wrench at such forms of un-intentional hubris just for the LOLs.

Re: Was the Go prog lang at fault? Would Rust help

[fubarrr]

>RRDNS is written in Go

Their bugs are in HR department.

Who in the world hired people who are dumb enought to use an experimental language in production?

Re: Was the Go prog lang at fault? Would Rust help

[Streetlight]

Could also be a financial consideration. The smart folks wanted too much money. Or, maybe an unpaid summer high school intern was the choice.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

" people who are dumb enought to use an experimental language"

...sort of like people who use the new experimental word "enought"?

Or not using NTP?

[Anonymous Coward]

Not sure they have nailed it down yet.

Why are they using upstream time values
and not running NTP?

https://www.google.com/amp/blog.cloudflare.com/how-and-why-the-leap-second-affected-cloudflare-dns/amp/?espv=1

The code takes the upstream time values and feeds them to Goâ(TM)s rand.Int63n() function. rand.Int63n promptly panics if its argument is negative. That's where the RRDNS panics were coming from.

Root cause of leap sec issues: Unix time

[Anonymous Coward]

The root cause of these leap second problems is the absolutely brain dead way Unix time deals with leap seconds. The clock design is broken. What Unix should've done is to keep a running count of seconds elapsed since Jan 1, 1970 and maintained a file containing historical moments when leap seconds were defined. Unix time could be converted to UTC using this file -and- the reverse conversion could be done as well. Effectively, Unix time would be monotonically increasing and equivalent to TAI or GPS time with an integer seconds offset. Simple. Exact.

Re:Root cause of leap sec issues: Unix time

[PPH]

Unix time would be monotonically increasing and equivalent to TAI or GPS time with an integer seconds offset. Simple. Exact.

Not a bad idea. But you still have to account for system clocks that drift. And need to be bumped a few seconds one way or the other periodically. If my correcting the system clock occasionally (either manually or via a cron job from an NTP server automatically) causes apps to blow chunks, these are bad apps.

Re:Root cause of leap sec issues: Unix time

[Anonymous Coward]

Even if you would use cron to schedule adjusttime. It would use drifting to get back to the right time, a UNIX clock is guarantied to never run backwards, except for exceptional situations where nothing is expected to keep working.

I think they have begun actually using TAI internally in the kernel and also include the offset in the kernel so gettimeofday() will return UTC.
I have seen that it is possible to use clock_gettime() to retrieve TAI.
And the timezone-database now includes timezones (suffixed with /right) that convert TAI to local time.

It works almost out of the box on Linux, the only thing that is required is to download the ntp-leapsecond database and install it on your system.

Re:Root cause of leap sec issues: Unix time

[PPH]

a UNIX clock is guarantied to never run backwards

By design, yes. But I've manually reset the time forwards or backwards by many minutes on a few systems (using the date command). And I can't recall breaking anything. A few apps have raised warnings about objects being in the future, but nothing that a click on 'Continue Anyway' didn't fix. I guess I just don't use shitty apps.

The Leap Seconds issue is much bigger

[GreaterNinja]

The issue with leap seconds is much bigger than just Cloudflare. I’ve found there are difference in at least 4 types of time: Google Time (their unique version of NTP time protocol), International Atomic Time(TAI), Coordinated Universal Time (UTC), and multiple NTP protocol servers. Currently there is a difference of 37 seconds between International Atomic Time (TAI) and Coordinated Universal Time (UTC). https://www.timeanddate.com/ti... When I checked the time sync of time.windows.com to time.is I noticed there is a ~33.4 second difference. Last I checked, there are hundreds of NTP severs that have out of sync times https://community.ntppool.org/ It seems a significant amount of the world is out of sync and there is no absolute consensus on what the time should be.

Re:The Leap Seconds issue is much bigger

[Anonymous Coward]

It is actually simple: You follow your Countries official source of UTC.

Anything else is a bad choice unless you Really Know What You Are Doing (i.e. you're an astronomer). And there are more choices than just TAI or UTC in that case.

Out-of-sync NTP servers are a fact of life, are caused by bad primary timesync sources (i.e. a human's fault for not taking servers with crap clocks offline, not a code bug or protocol bug), and are compensated for by the protocol *if you configured your ntp servers correctly*.

Re:The Leap Seconds issue is much bigger

[Anonymous Coward]

It seems a significant amount of the world is out of sync and there is no absolute consensus on what the time should be.

It's the 21st. century, dude.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

At what point is a programming language no longer "experimental"?

Go was first announced over 7 years ago, back in 2009, and it was being worked on well before then. It has been created by some very experienced industry veterans working at one of the most successful software and technology organizations ever to have existed. Compared to many other languages, Go and its standard library have been quite stable for years now.

Languages like Java, C++, C# and Perl were all being used for production systems well within 7 years of becoming publicly available. They've all seen significant changes since being released, far more than Go has experienced. So when did they stop becoming "experimental"?

Re:Hosts hardcodes avoid DNS issues

[Anonymous Coward]

Stop spamming your nefarious software!

You're banned from Slashdot for a reason.

You have singlehandedly made them tweak the lameness filter to the point we can't even do long insightful posts anymore, you are ruining Slashdot every time you post!

I'm on topic & IF I am 'banned'?

[Anonymous Coward]

See subject: Why can I post then you illogic logic DOLT? Guess you ran outta "downmodpoints" now spewing bs unidentifiably instead? LMAO @ U!

* "All whipslash's troll horses & ALL his sockpuppet fake account men, can't stop APK from posting, AGAIN, & again..." & HE is the punk doing the mess, not I. Blame the "trustfund baby" source, not I.

Plus, facts - /.'er LIKE & USE my work (do they yours, unidentifiable unskilled DWEEB? No):

See here https://slashdot.org/comments.pl?sid=10053471&cid=53585643/ & here https://slashdot.org/comments.pl?sid=10053471&cid=53585667/ & Malwarebytes hosts + recommends it (I could have NO higher honor).

APK

P.S.=> Powerless whimps like you aren't capable of insight (or quality work) - I am & I truly DO "get off" on watching you blow all your effete useless 'downmodpoints' in some vain effort to stop me & seeing you FAIL (lol) as always - you have NO skills... apk

Re:I'm on topic & IF I am 'banned'?

[Anonymous Coward]

Leave Slashdot already, you are banned! And you are ruining Slashdot by causing them to increase the complexity of filters, hurting discussions on Slashdot.

You are in violation of the Computer Fraud and Abuse Act, APK is a criminal .

Re:Hosts hardcodes avoid DNS issues

[Ash-Fox]

Do not trust APK's software, APK is a criminal , he is blatantly violating the Computer Fraud and Abuse Act by posting here as he is banned from Slashdot.

APK's ban evasion has lead to more restrictive filters being placed on Slashdot that hinder good discussions.

Re:Hosts hardcodes avoid DNS issues

[Ash-Fox]

Stop with your criminal spam. You are violating the Computer Fraud and Abuse Act.

All whipslash has to do is ask nicely... apk

[Anonymous Coward]

See subject: He never has & iirc he said it's ok for me to post (iirc, I have it quoted + bookmarked) - but IF he asked nicely, KNEELING DOWN vs. my UNSTOPPABLE power (lol)?

Then, I would leave as I learn zero from goofs like you, lol! It would be enough to make a tin-plated little dictator wannabe God kneel... lmao!

* This site's served MOST of its purpose to me anyhow - 1st I used those here to "make objections" I overcome w/ ease (nice thing about fellow techies is almost like 'all those eyes' on "OpenSORES" - they MIGHT spot something I overlooked (haven't yet, lol)).

2nd'ly? It's truly been FUN showing your "lord & master" + his crony sycophant sockpuppets galore/multiple identities trolls (yes, I KNOW you have them - lol, in fact, you'd be SURPRISED how much I know who is who & what-not but more importantly, HOW I know it, lol) are weak.

E.G. - When you can't prove my points validly technically wrong? You downmod. When I beat you mano-a-mano, you use undentifiable ac posts. When you run dry of modpoints (always) then you stalk me by unidentifiable ac posts (like you're doing now) - playing either (lol) "jailhouse lawyer" as you are now OR calling me 'crazy' etc. ala "Dr. Quack - 'SiDeWaLk-ShRiNk of /.' - BOTH are indicative of your DELUSIONS of GRANDEUR, lmao!

APK

P.S.=> Don't give me orders unindentifiable cowardly PUNY dolt - you are BENEATH me & totally wrong as usual too (never been asked to leave but that's what the "powers that be" DO when they're helpless, lol - look @ breitbart, Alex E. Jones, etc. as further proof thereof to THAT very effect)... apk

Re:All whipslash has to do is ask nicely... apk

[Anonymous Coward]

Stop with the criminal activities in violation of the Computer Fraud and Abuse Act. Your continued illegal activities are ruining Slashdot for the sake of your spam.

--Robert DeLong

LMAO: STFU, & to quote "Duke Nukem"?

[Anonymous Coward]

See subject: From his new book "Why I'm so great" You wish you were me & why can I run rings around your asses?

Alicia Keys says it best (it's where I am from, superior people): https://www.youtube.com/watch?feature=player_detailpage&v=z5LOE_5icNA#t=202/

"Welcome to NY: Concrete jungle where DREAMS are made of... there's NOTHING U CAN'T DO! Count on NY... yea!"

* Bottom-line: You don't own this place - all the owner here has to do, is ask nicely, & I am gone. Yes, it's THAT simple... or is it? Pride is a terrible thing isn't it.

APK

P.S.=> Unidentifiable ac, lastly? Go FUCK yourself - alright motherfucker? You're nobody & NOTHING to me (get whipslash in here to ask me to go, I am gone (he has to KNEEL proving he's helpless technically, lol!))... apk

Re:LMAO: STFU, & to quote "Duke Nukem"?

[Anonymous Coward]

Cease and desist your criminal activites immediately, you are in violation of the Computer Fraud and Abuse Act. Your continued illegal activities are ruining Slashdot for the sake of your illegally posted spam.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

Stupid programmers doing real work. It is not the language but the idiot coder.

This happens so regularly that it must be the same guy moving from job to job. Zunes not knowing the number of days in leap year.

Glad I work on real machines. Last error we had like this was how slow the carry between hardware clock and date ref. Day a 23:59:59, day a 24:00:00, day a 00:00:00, day b 00:00:00.

This is so 1980.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

HR shouldn't be making hiring decisions for anyone. They Should at most be making suggestions based on preliminary criteria set by their to-be bosses. The one to blame here is most likely the boss who gave the greenlight fro using something like Go.

What worries me, though, is that the trend of using experimental programming languages and technologies in production environments is growing fast. It's perfectly fine using new and even experimental languages to write non-business-critical parts of a system, but for god's sake, don't make your whole infrastructure rely on something as stupid as Go. Write critical parts as C modules, or anything else that has a long track record.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

Go is experimental by which definition?

Also, this is not a language bug, it's a problem with incorrect use of library code, and that can happen in any language that supports libraries in any form.

Re: Was the Go prog lang at fault? Would Rust help

[Anonymous Coward]

Blaming the Language is like blaming a toaster for shocking someone in the bath because they felt a wee bit on the hungry side.

Tools, just like features in languages, should not be made idiot proof.
A warning is fine. In this case, someone clearly never checked the spec or put in their own check just for leap seconds. Doesn't need to be in the code forever, just that event then comment-out and recompile.
Putting these checks in by default would add extra overhead. One extra check adds up even after a day. That's extra money down the line because some idiot wanted toast in a bath.

Re: Was the Go prog lang at fault? Would Rust help

[K. S. Kyosuke]

A conservative programming language in a version numbered as 1.7 hardly fits any sane person's definition of "experimental".

We don't. Accurate timepieces do.

[Anonymous Coward]

And if you want an accurate timepiece, you should accept leap seconds, and even double leap (you CAN get 61 seconds as a valid return) in your code.

Personally, I've never needed tolerances in weight of microgrammes, but that doesn't mean we can do without them. The USA doesn't need to use imperial measures, but that isn't sufficient reason to drop them. Many things aren't useful, or of noteworthy difference, to ordinary people, but that doesn't mean we can ignore the differences.

Re:We don't. Accurate timepieces do.

[hucker75]

That doesn't answer his question at all.

Re:Hosts hardcodes avoid DNS issues

[Anonymous Coward]

Stop with your criminal spam, please. You are violating the Computer Fraud and Abuse Act, as you are well aware, so not only are you doing it, you are doing it wilfully.

Re:Ash-Fox "jailhouse lawyer" wrong (not banned)

[Ash-Fox]

Stop involving us in your crimes APK. You are violating the Computer Fraud and Abuse Act.

Ash-Fox stalker = pissed I made him a fool

[Anonymous Coward]

See subject as his hosts addons fail via using admin/root privelege (dangerous security risk) https://ask.slashdot.org/comments.pl?sid=10024927&threshold=-1&commentsort=0&mode=thread&pid=53533799/ & he's "ALL OUT OF ACES" trying to play "jailhouse lawyer" now in desperation which I shot down easily too https://tech.slashdot.org/comments.pl?sid=10073651&cid=53605283/ & yes we know it's you now "AssFox" asshole stalker since you forgot to use your "registered 'luser'" account here https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601511/ but did unidentifiable anonymous posts here instead before that last link like the one I am responding to, adding "criminal impersonation" to it (Robert DeLong) https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601715/ loony bird FAILURE that you clearly are!

APK

P.S.=> Still "stinging" after the 1st link above eh? Too bad you made it just "too, Too, TOO EASY - just '2ez'" to scorch your dumb ass in HUGE error using ADMIN/ROOT PRIVELEGE ON A BROWSER (stupid security risk) ... apk

Only thing I 'violated' is your ASS, Fox

[Anonymous Coward]

See subject as as hosts addons fail via using admin/root privelege (dangerous security risk) https://ask.slashdot.org/comments.pl?sid=10024927&threshold=-1&commentsort=0&mode=thread&pid=53533799/ & he's "ALL OUT OF ACES" trying to play "jailhouse lawyer" now in desperation which I shot down easily too https://tech.slashdot.org/comments.pl?sid=10073651&cid=53605283/!

&

Yes we know it's you now "AssFox" asshole stalker since you forgot to use your "registered 'luser'" account here https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601511/ YET you did unidentifiable anonymous posts here instead before that last link like the one I am responding to, adding "criminal impersonation" to it (Robert DeLong) to your known stalking me online https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601715/ loony bird FAILURE that you clearly are!

APK

P.S.=> Still "stinging" after the 1st link above eh? Too bad you made it just "too, Too, TOO EASY - just '2ez'" to scorch your dumb ass in HUGE error using ADMIN/ROOT PRIVELEGE ON A BROWSER (stupid security risk) ... apk

Too late AssFox: /.ers disagree #1/2

[Anonymous Coward]

his hosts program is actually pretty good by xenotransplant

his hosts tool is actually useful for those cases in which one does indeed want to locally block stuff outright while consuming minimum system resources by alexgieg

I've never tried to belittle (APK's) work, I've flat out said it's good by BronsCon

take a look at the APK hosts file engine by SuperKendall

APK is kinda right. I've tried his hosts file generating software. It works by bmo

APK is totally right on this count. Adblock Plus on Firefox mobile is a dog on older, or lower end, phones. A hostfile based adblocker makes for a much better experience by chihowa

I like your host file system by Karmashock

I find your hosts file admirable by vel-ex-tech

* My code's liked/used + recommended & hosted by Malwarebytes' hpHosts - Argue w/ those folks above.

APK

P.S.=> See subject & those quoted /.'ers outnumbering you praising my work - more coming in part 2... apk

If ...

[cwsumner]

"If Engineers built buildings the way Programmers write programs, the first woodpecker that came along would destroy civilization!"

And if you did it that way because your "pointy-haired boss" said to, then it is still your fault... ;-)

Too late AssFox: /.ers disagree #2/2

[Anonymous Coward]

I support APK's stand on the hosts file by Trax3001BBS

Your premise that hostfiles are a good way to deal with advertising and malvertising is quite valid by JazzLad

No complaints from me, I like APK... Reminds me to use a host file. Also, his stuff is free by aaaaaaargh!

APK's monolithic hosts file is looking pretty good by Culture20

APK... Awesome to see he's still spreading the good word by Molochi

ABP is insufficient as a solid hosts file does everything that APK reminds us about by fast turtle

APK isn't wrong by cfalcon

APK, I know people give you a lot of shit regarding hosts, but please don't ever stop by nasredin

You need APK's hosts file by Teun

APK solution STILL relevant by Thud457

you're right about hosts files by drinkypoo

APK

P.S.=> They're in addition to https://tech.slashdot.org/comments.pl?sid=10073651&cid=53606071/ many more earlier so "EAT YOUR WORDS", AssFox... apk

Only thing I 'violated' is your dumb ASS Fox

[Anonymous Coward]

See subject as as his hosts addons fail via using admin/root privelege (dangerous security risk) https://ask.slashdot.org/comments.pl?sid=10024927&threshold=-1&commentsort=0&mode=thread&pid=53533799/ & he's "ALL OUT OF ACES" trying to play "jailhouse lawyer" now in desperation which I shot down easily too https://tech.slashdot.org/comments.pl?sid=10073651&cid=53605283/!

&

Yes we know it's you now "AssFox" asshole stalker since you forgot to use your "registered 'luser'" account here https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601511/ YET you did unidentifiable anonymous posts here instead before that last link like the one I am responding to, adding "criminal impersonation" to it (Robert DeLong) to your known stalking me online https://tech.slashdot.org/comments.pl?sid=10073651&cid=53601715/ loony bird FAILURE that you clearly are!

APK

P.S.=> Still "stinging" after the 1st link above eh? Too bad you made it just "too, Too, TOO EASY - just '2ez'" to scorch your dumb ass in HUGE error using ADMIN/ROOT PRIVELEGE ON A BROWSER (stupid security risk) ... apk

Re:Hosts hardcodes avoid DNS issues

[Anonymous Coward]

APK is a criminal, he is blatantly violating the Computer Fraud and Abuse Act by posting here as he is banned from Slashdot.

APK's ban evasion has lead to more restrictive filters being placed on Slashdot that hinder good discussions all for the sake of his spam.

Just look how many spam post he's made to this article alone!

Re:If whipslash asks me to go I'm gone

[Ash-Fox]

APK commiting criminal activity under the Computer Fraud and Abuse Act

Respectfully, stop involving me and others in your criminal activities. You're ruining Slashdot with your illegal spam posts and illegal comments.

Assfux got shafted & likes it, lol... apk

[Anonymous Coward]

See subject: How so? Easy (too easy) https://tech.slashdot.org/comments.pl?sid=10073651&cid=53606211/

APK

P.S.=> You did that to yourself imbecile... apk

Re:Assfux got shafted & likes it, lol... apk

[Ash-Fox]

You've been told to stop involving me and others in your criminal activities. You are in direct violation of the Computer Fraud and Abuse Act. Slashdot is not a platform your illegal spam and illegal comments. Your activities have only caused Slashdot to tighten filters to the point that insightful commentary is now difficult to try to deal with you.

You have spammed this article so many times, it's ridiculous!

You have previously violated on Slashdot privacy rights, promoted offers without the express written consent of Slashdot Media, your content is destructive due to what has happened with Slashdot filters and embedding advertising without the express written consent of Slashdot media. All of these are against the Slashdot's "Terms of Use" and in turn you have violated the Computer Fraud and Abuse Act.

Your criminal activities are unacceptable and your continued persistence after being advised of such means you willfully and intentionally violate the Computer Fraud and Abuse Act and Slashdot's "Terms of Use" to further propogate your spam without a care that you are responsibile for further ruining discourse on Slashdot.

You've been asked to stop, you've been told to stop, you've even been banned and you continue. Your persistance in unethical and criminal behaviour is disgusting.

Assfux blast himself up the ass, lol

[Anonymous Coward]

See subject: You keep doing this to yourself (I think you LIKE it) https://tech.slashdot.org/comments.pl?sid=10073651&cid=53606211/

APK

P.S.=> You did it to yourself, get over it... apk

AssFox quit stalking me... apk

[Anonymous Coward]

See subject: No denying you do https://tech.slashdot.org/comments.pl?sid=2024512&cid=35403488/
https://tech.slashdot.org/comments.pl?sid=3153677&cid=41509383/
https://linux.slashdot.org/comments.pl?sid=3110069&cid=41310703/
https://it.slashdot.org/comments.pl?sid=3132237&cid=41402837/
https://yro.slashdot.org/comments.pl?sid=3149609&cid=41488037/
https://slashdot.org/comments.pl?sid=3929071&threshold=-1&commentsort=0&mode=thread&pid=44175653/
https://yro.slashdot.org/comments.pl?sid=4127345&cid=44676119/
https://news.slashdot.org/comments.pl?sid=3929071&cid=44175653/
https://tech.slashdot.org/comments.pl?sid=5111247&cid=46903077/
https://slashdot.org/comments.pl?sid=5538231&cid=47694971/
https://yro.slashdot.org/comments.pl?sid=5572203&cid=47739407/
https://yro.slashdot.org/comments.pl?sid=6895585&cid=48975515/
https://ask.slashdot.org/comments.pl?sid=6999567&cid=49105879/
https://yro.slashdot.org/comments.pl?sid=7517371&cid=49865707/
https://tech.slashdot.org/comments.pl?sid=7580903&cid=49957601/
https://slashdot.org/comments.pl?sid=8667099&cid=51378337/
https://slashdot.org/comments.pl?sid=8667099&cid=51378993/
https://it.slashdot.org/comments.pl?sid=9380571&cid=52510101/
https://slashdot.org/comments.pl?sid=9986237&cid=53472979/
https://news.slashdot.org/comments.pl?sid=9962449&cid=53438541/
https://slashdot.org/comments.pl?sid=9986237&cid=53472979/
https://ask.slashdot.org/comments.pl?sid=10024927&cid=53533799/

APK

P.S.=> ... & you FAIL badly when you do https://ask.slashdot.org/comments.pl?sid=10024927&threshold=-1&commentsort=0&mode=thread&pid=53533799/ ... apk

Re:AssFox quit stalking me... apk

[Ash-Fox]

Cease and desist your criminal activities immediately.

Hosts hardcodes avoid DNS issues

[Anonymous Coward]

See NEW APK Hosts File Engine 9.0++ SR-5 32/64-bit https://www.google.com/search?...

Ads rob speed, security (malvertising) & privacy (tracking).

Hosts add speed (hardcodes/adblocks), security (bad sites/poisoned dns), reliability (dns down), & anonymity (dns requestlogs/trackers) natively.

Works vs. caps & PUSH ads.

Avg. page = big as Doom http://www.theregister.co.uk/2... & ads = 40% of it.

Hosts != ClarityRay blockable (vs. souled-out to admen inferior wasteful redundant slow usermode addons)

Less power/cpu/ram + IO use vs. DNS/routers/addons/antivirus (slows you) + less security issues/complexity.

Compliments firewalls (blocking less used IP addys vs. hosts blocking more used domains) & DNS (lightens dns load).

Gets data via 10 security sites.

APK

P.S. - Safe https://www.virustotal.com/en/... (Verified by Malwarebytes' S. Burn "seen the code & it's safe" http://forum.hosts-file.net/vi... )

Take your own advice "count stalkula"

[Anonymous Coward]

Hosts work on topic here vs. this as a solution troll - you're off topic stalking me https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ + out of aces https://ask.slashdot.org/comments.pl?sid=10024927&threshold=-1&commentsort=0&mode=thread&pid=53533417/ after you FAILED again technically vs. me.

* Apparently I've RUN YOU DRY of "downmodpoints" you issue by sockpuppets you keep (the province of FAKE NAMES for FAKE LIVES types w/ zero accomplishments of note, lol).

I'm laughing @ YOU AssFux & so is all of /. ... guaranteed!

APK

P.S.=> You don't own /. issuing threats you don't have power to enforce - get whipslashd to come and tell me publicly to stop posting here & I will (it will take him kneeling as I'm not banned)... apk

Re:Take your own advice "count stalkula"

[Ash-Fox]

Your sock puppeting is still in violation of the Computer Fraud and Abuse Act, cease and desist your unethical and criminal activities immediately.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

You got schooled by apk AGAIN count stalkula? Yes. You got your ass blown down by apk https://tech.slashdot.org/comments.pl?sid=10073651&cid=53618169/ seems like its' a habit for you. Bad one unless you are a sado-masochist, *snicker, chortle!*

Re:Take your own advice "count stalkula"

[Ash-Fox]

This sock puppeting is still in violation of the Computer Fraud and Abuse Act, cease and desist your disgusting unethical and criminal activities immediately, APK. You are knowingly violating the Computer Fraud and Abuse Act and Slashdot's "Terms of Use".

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Big burn. Burnt in your mistakes in security and 20 times stalking him. Ash-Fox is your real name on your birth certificate sockpuppet fake name for a fake life?https://images.slashdot.org/hc/51/d1e0bd526e07.jpg

Re:Take your own advice "count stalkula"

[Ash-Fox]

Your continued knowingly repettition of unethical and criminal activities show what kind of disgusting person you are, APK. Cease and desist your unethical and criminal activities immediately.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula! Back!!!" hahahahaha R o T f L m A o

Re:Take your own advice "count stalkula"

[Ash-Fox]

Cease these criminal acitivites.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ Back!" hahaha R o T f L m A o...

Re:Take your own advice "count stalkula"

[Ash-Fox]

Immediately cease these criminal activities.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ Back!" hahaha R o T f L m A o...

Re:Take your own advice "count stalkula"

[Ash-Fox]

Your persistent willful criminal activities have revealed exactly what kind of person you are. Stop involving Slashdot and others in your crimes.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ - BACK!" hahaha R o T f L m A o...

Re:Take your own advice "count stalkula"

[Ash-Fox]

Cease your criminal activities immediately, APK.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ - BACK!" hahaha R o T f L m A o...

Re:Take your own advice "count stalkula"

[Ash-Fox]

You have been told to cease your criminal acts and you still persist, APK.

Re:Take your own advice "count stalkula"

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ - BACK!" hahaha R o T f L m A o...

Re:Hosts hardcodes avoid DNS issues

[Anonymous Coward]

Drained of your ineffective down mod points https://tech.slashdot.org/comm... and proven to be "count stalkula" hahaha too https://slashdot.org/comments.... has not turned out to be a good combination for you Ash-Fox!

Re:Hosts hardcodes avoid DNS issues

[Anonymous Coward]

Sign of the cross + "Back, count stalkula: https://tech.slashdot.org/comments.pl?sid=10073651&cid=53610991/ - BACK!" hahaha R o T f L m A o...