Slashdot Mirror
D-Link Settles Danish Time Dispute
igb writes "The Register reports that DLink has settled the time server dispute described a little over a month ago here on Slashdot. They're going to stop using an NTP server they're not really authorized to chime with, and they've reached an amicable settlement over the use by existing products. The details of the settlement are, not unsurprisingly, somewhat vague, but let's hope that the good guys aren't out of pocket any more."
I currently use the Argonne national lab NTP server most of the time which is probobly government paid though it could be provided by the University of Chicago (though since my connection is on-campus, it makes the most sense).
Bottles.
Seems to me that if you run a (public) NTP server with a publicly available IP address and/or DNS resolution, that means anyone (public) can use the (public) service - no?
No.
Do you Gentoo!?
More like you can configure your own router to talk to it, but what Dlink did wasn't a public thing. As a private corporation, they turned tens to hundreds of thousands of devices at it.
If each user had done that by themselves it would be a different game, since Joe Q. Public was doing it, but D-Link hardcoded it in.
The reason for this is to avoid problems like this, where the NTP server is overloaded or the NTP client is mis-configured and overloads the server or network.
Public or not, you have to follow the rules. It is pretty well known that only 'Stratum 2' NTP servers are to use 'Stratum 1' NTP servers. This is not just a 'because we want it that way' policy. There are many good reasons for this.
http://en.wikipedia.org/wiki/NTP_vandalism
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
Yes, and yes. They are clueless, and they are cheap.
That is why pool.ntp.org was created - to provide a pool of NTP servers that these bozos can use without hammering anybody's server too badly.
www.eFax.com are spammers
These situations make no sense to me. The NTP system is very easy to use properly.
There's a great little website about how to use ntp.org servers properly.
For the quick-fix people, point your NTP capable system at pool.ntp.org.
If you live in north america, you can use the north-america.pool.ntp.org dns name instead, for only north american servers. The same applies to other continents and several country codes.
Basically, there's no excuse for hard-coding a time server in almost any situation, unless your client is completely incapable of DNS and has no access to external DNS servers.
- Michael T. Babcock (Yes, I blog)
I've told my friends (and my company) to avoid buying their stuff because it's junk (IME) We used to spec D-Link because one of our distributors already carried it and I'm fairly certain I've since swapped most all of it to Linksys or Netgear which are both more or less equal to me.
There is now a way for vendors to use the NTP pool. See http://www.pool.ntp.org/vendors.html for details.
Agreed. D-Link appears to occupy a point on the cost-quality curve that ultimately costs more in hair-pulling time than it saves in cash. Their products may be OK for lightweight use at home, but they can really give you fits in a more demanding environment.
Case in point: we recently put a bunch of DGS-1008D 8-port gigabit switches into service, and immediately started having problems with dropped Ethernet connections. Our laser printer was sucking down enough power at the onset of its fuser-warmup phase to trigger a nearby UPS momentarily. The resulting switchover transient lasted only a few milliseconds, but it was enough to reset the DGS-1008D. After a LOT of tail-chasing, it transpired that the (cheap-ass linear) wall-wart supplies that D-Link ships with the DGS-1008D lack sufficient filter capacitance to absorb even the slightest power glitch under high-load conditions (e.g., when there are several cables plugged into the switch.)
We took a few of their power supplies apart and found that the oldest ones -- which didn't have the problem -- used a 2000-uF filter capacitor at the rectifier output. At some point, they saved 10 cents by moving to a supply with only 1000 uF, rendering their product useless in many real-world office environments.
This isn't supposed to be a general "let's all bag on D-Link" thread, but hey, if the shoe fits...
Dahlmann tightly grips the knife, which he may have no idea how to use, and steps out into the plain.
In the last story the server admin stated that he couldn't change the address because it would involve far too much work. Many people rely on his services and it was costing him enough out of pocket as is.
What do they say that? - Sound like they go out of their way (advice about firewalls, etc) to let taxpayers "Set Your Computer Clock Via the Internet".
try { do() || do_not(); } catch (JediException err) { yoda(err); }
Well don't tell any of my devices, cause all of them are over 2 years old, many of them over 5 years old. Heck my "public segment", where the DSL modem (6 years old), broadband router (4 years old) and VPN device (4 years old) connect is a 15 year old 10Base-T ethernet hub. Your experience must be with Linksys, I always keep a spare D-Link broadband router on a shelf ready for when a friend or relative calls after their "Internet doesn't work" because their Linksys router fried itself. I'm continually amazed how many people think that because Linksys costs more (and now sports the Cisco logo) that it must be better.
Proper queries are only denied & not re-made if the client follows the rules.
If you check the original artical, D-Link routers do not recognize the kill request, and they re-request very quickly. So yes, he configured the NTP server correctly, AND he posted restrictions on the NTP site correctly, AND D-Link said we don't care.
It's essentially a DDOS attack on the server. There are thousands of hits with correctly formed NTP requests coming in every second - 98% of which should be directed elsewhere.
If I remember correctly from the open letter written to D-Link from the original /. posting the cost was substantial because by violating the terms of use for the NTP server D-Link caused Poul-Henning Kamp's NTP server to no longer be eligible for a special agreement he had regarding the cost of his bandwidth usage. That letter, of course, is no longer posted most likely because of the settlement agreement.
- Stratum 1 are principle time servers for a region & directly query atomic clocks.
- Stratum 2 are general use for large regions or institutions - generally they should only be contacted by Stratum 3 servers - clients only as a last resort.
- Stratum 3 are the generic NTP servers of the internet - if you're an end client you should be talking to a Stratum 3 unless none are available/unrestricted for your use.
D-Link SOHO routers do 3 things wrong.- They don't follow the NTP protocol for requests to stop using the service.
- They ignore the restrictions place on the server usage - in Denmark, for use by ISP or Stratum (2/3) requests.
- They hit a Stratum 1 NTP server as an end client.
So no, if you run a public NTP server that you have dutifully entered restrictions on, you are expecting everyone who comes to you to obey the NTP protocol. That includes following the restrictions, listening to the go away requests, and following the basic rules of who to talk to.[Analogy type=bad]
In the US there are a number of parking spaces set asside for handicapped parking in almost every parking lot. Physically you can park there if you are not handicapped, but you're not supposed to (covers both ignoring restrictions and a client talking to a Stratum 1 server). If the manager of the parking lot tells you to get your car out of the spot - you should do that(refers to the kill request in the NTP protocol). In the real world if it get's this far, the cops come & give you a ticket. On the net you get open letters calling you an arogant prick who can't be bothered to figure out the basics of the protocols you are boasting about
[/Analogy]
For the record the Danish server was not the only Stratum 1 server they hit, they appear to have taken the Stratum 1 list (almost all of which restrict usage to Stratum 2 servers) and shoved it into the routers for general use - hardly the "Good internet citizen" they claim to be.
In my experience, when starting the 'chronyd' time daemon under Linux, it will poll very often, like 15 seconds intervals. Everytime it gets an answer, it will compare it to the system clock, log the deviation and adjust the system clock speed based on the trend. After some time, the system clock will run really accurate, so the logged deviations will be small. The polling interval will then be increased in steps up to a max. limit of 4 hours. If the computer is restarted, this scenario starts over again.
Compare this to a typical Windows XP computer which seems to poll a time server once a week or so. No doubt that the ntp server will feel some clients more abusive than others.
Disclaimers:
The intervals stated above may be wrong. I haven't tinkered with optimizing my time daemons since the old pay-per-minute ISDN days so my memory is a bit rusty.
Chronyd is just an example. I have no knowledge of whether it stresses the time servers more or less than other time daemons like 'xntpd'.
Poul-Henning Kamp got 200.000 DDK (Danish kroner) which is about 33.000 US$.
;-). This information is from the danish version of computerworld online at http://www.computerworld.dk/
;-)
The settlement states that Poul-Henning Kamp must not talk about the history of problems which the D-Link routers caused. But He tells danish press that any future problemes causes by D-link equiptment will be posted around the net
His homepage is http://people.freebsd.org/~phk/
For those in america: Denmark is not the capital of sweden
Yes, but worse and out of order
Check out NTP.org. Specifically check the Rules of Engagement, The Stratum 1 list, and RFC 1305.
Now looking at everything we have a protocol that involves 2 components, an implimentation component and a social component. The actual implimentation of the protocol is laid first as "Format your request in this fasion and we will return the responce looking like this...". However, it also has things for implimenting request timing fallback and kill requests. The social implimentation of the protocol is layed out in the RoE and the Server Lists - note the regional restrictions and the authorization requests in the server lists.
From the original article which evidently doesn't have any information on the open letter anymore - D-Link took the Stratum 1 list and shoved it into some of their router NTP lookup tables. That blows off the entire social aspect of the protocol - both the permissions and the structure.
Next they implimented only the request portion of the protocol, they ignore the backoff & get lost request structures - essentially forgoing the entire error correction portion incorperated into the RFC. So up to the point of manufacture they have 3 strikes against them,
- Failure to obey the Stratum structure of the NTP system
- Failure to follow the permisions structure of the NTP system
- Failure to properly impliment the NTP connection protocol
Now there was no known issue with this until the Danish exchange turned to the Stratum 1 owner and said "You are eating a hell of a lot of bandwidth here & we can't keep giving it to you for free." At which point the problem was tracked back to a series of D-Link SOHO routers. I don't recall the exact process he used , but he started sending kill requests to anything from a D-Link router. When they ignored it & kept making requests he talked to D-LinkFrom memory the conversation then went like this:
Dane: You're routers are hammering my server & they need to stop, you don't have permission & you're violating the rules.
D-Link: How cute, have a nickle & go get yourself some candy.
Dane: WTF? The exchange is going to charge me $8K to cover your protocol violations.
D-Link: It's not our fault & if it is talk to our Lawyer.
Lawyer: I won't talk to you unless you come to CA & argue your case.
At which point it devolved to an open letter & public shaming - which by the way seems to have worked.
[note] IIRC someone calculated the estimated bandwidth from the D-Link routers using Stratum 1 NTP servers to be enough to continously flood a T1. So this isn't just an occasional knock on the door, it's pretty heavy usage for what amounts to a request packet and a responce packet from each router.
Okay, sorry to reply to myself, but I found a cache of Poul-Henning Kamp's posting about D-Link. This was at http://72.14.207.104/search?q=cache:LAdoqMDzqM0J:n et127.com/2006/04/07/open-letter-to-d-link-about-t heir-ntp-vandalism/+%22have+been+accused+of+extort ion.+I+have+been+told+that+I%22&hl=en&gl=us&ct=cln k&cd=1
(I'm not sure of any of those gibberish-looking parameters change over time.)
I think the convention is to post Anonymously to prevent karma-whoring, but the last time I tried to post anonymously, it didn't work. It seems to work when I preview this post. Okay, here's the web page:
Poul-Henning Kamp, Slagelse, Denmark, writes:
When I contacted D-Link back in November 2005 about the way D-Link products abused my NTP-server, I expected to get in touch with somebody who understood what they were talking about, I expected them to admit that D-Link had made a bad decision and I expected that D-Link would make good on the damage they were responsible for.
For the last five months I have wasted a lot of time trying to reach some kind of agreement with the Californian lawyer which D-Link put on the case. I can't quite make up my mind if D-Link's lawyer negotiates in bad faith or is merely uninformed, I tend to suspect the latter, but either way, as of this morning I decided to cut my losses.
Since no one else at D-Link has reacted to my numerous emails, I have no other means of getting in touch with D-Link other than an open letter. I realize that it will be inconvenient and embarrasing for D-Link to have this matter exposed in public this way, but I seem to have no other choice.
I will now lay out the case below in such detail that any moderately knowledgeable person should be able to understand it, and hopefully somebody, somewhere in D-Link will contact me so we can get this matter resolved.
What is NTP?
NTP is Network Time Protocol, a protocol that allows computers to transfer timestamps across the internet so that they can set their clocks to the correct time.
A number of NTP servers on the internet are connected to radio timecode receivers, GPS receivers or in some cases directly to national time laboratories primary atomic frequency standards.
How not to implement NTP in a product
A number of D-Link products, so far I have at least identified DI-604, DI-614+, DI-624, DI-754, DI-764, DI-774, DI-784, VDI604 and VDI624, contain a list of NTP servers in their firmware and using some sort of algorithm, they pick one and send packets to it.
This is about as wrong a way to do things as one can imagine. There is no way D-Link can change the list once the product is shipped, unless D-Link can persuade the customer to upgrade the firmware.
How to implement NTP in a product
The correct way, as I have pointed out to D-Link repeatedly, is to query a D-Link controlled DNS entry like "ntp.dlink.com" and populate this DNS entry with the list of NTP servers to be queried. That would allow D-Link to add or remove servers from the list by changing the DNS server files and all deployed devices would automatically see the update next time.
If D-Link had implemented the NTP feature this way, my complaint could have been handled to my full satisfaction with an emailed apology and a few minutes of D-Link's DNS administrators time.
The problem
As you can see in the table on the right side, D-Link included the NTP server "GPS.dix.dk" in the list of NTP servers to query, and they did so without asking for permission.
I have no idea how many devices D-Link has sold, but between 75% and 90% of the packets which arrive at my server come from D-Link products via this mechanism.
Why D-Link ne
If you RTFA, you'll see that the devices in question are not using DNS. They are using a hardcoded IP address, so DNS would not solve this problem.