Slashdot Mirror
Microsoft Windows Update and Network Bandwidth?
Brett Glass asks: "As we reviewed the cache statistics for our small ISP today, we noted that the traffic generated by Microsoft's Windows Update feature constituted 45% -- no, that's not a misprint -- of our total throughput. Because so many computers on the Internet run Windows, this massive resource drain occurs whenever Microsoft announces major security holes (as it did this week). The traffic could be greatly reduced, and service to users much improved, if the updates were cacheable at the ISP. But Microsoft has set up the service in such a way that the data can't be cached. (It's digitally signed, so inserting Trojans into the cache is virtually impossible; in any event, no more of an issue than intercepting the data stream.) Are others out there seeing the same pattern? How might Microsoft be convinced to make its updates cacheable, so as not to waste unthinkable amounts of bandwidth?"
it takes so long to do updates now too... we have win 98 machines we want to update and it can take overnight!
fp!!!
No no! You're supposed to buy and install and manage an internal (corporate, academic, whatever) Windows Update server and manage your internal clients yourself.... :-)
I can only speak from what I've seen in our offices, but squid (running in transparent proxy mode) very definitely caches content from Windows Update... I set it up about six month ago and remeber being really surprised (because I think I very reasonably expected it not to).
"All the bandwidth usage of Gentoo, none of the perks!"
Not being a windows user, how big are the windows updates and how often do they come?
Apple's own software updates are pretty big, although with a much smaller percentage of machines as macs they're not going to cause the same volume of problems. The last few I've seen have been around 40MB, with one topping out at 80, and most security updates (every 2 months perhaps) being 5-15MB
I visited the site linked to in the post, and it came up with a message about how it doesn't work with my browser/OS (Mozilla/Linux). Boy, that just boils my blood! Oh, wait.
compare to 95% usage last time Code Red visit. :)
The rest 5% is Netbios traffic.
lol ;-) Thought I recognised the name.
Here at Berkeley all of the Windowsupdates come from an internal server instead of externally. That way they control who gets the updates and when.
You can download the updates individually, and there is probably a way to have them downloaded to the server automatically. All you have to do is convince the users to download them from you and install them manually. Can you block traffic from the autoupdate applet? I bet that would significantly reduce traffic, at the cost of insecure customers.
What about running an internal WU server and changing the DNS entry at the local level to a local server? You'd have to keep the catalog of updates stocked and refreshed constantly, for multiple OS's, so I don't know how cost effective it might be.
-Ryan
AUWYHSTOT (Acronyms are Useless When You Have to Spell Them Out Too)
Another option is to use a systems management package (LANDesk, ZENworks, SMS, etc.) to build the packages and deploy them while only using your internal network bandwidth (once you've downloaded the hotfixes anyway).
Of course, the two options above are really meant for company networks, but even those can help reduce the bandwidth used for more important things.
I would, but if that were a real problem you could go to sourceforge and find 15 different projects devoted to solving the problem, 3 of which are usable, one one of which... well.. you'd probably already be running so there isn't a problem.
First step is to download the patches/update manually and save them elsewhere accessable to all users:
- Windows 2000 users, please visit the
Windows 2000 Downloads site.
- Windows NT 4.0 users, please visit the
Microsoft Download Center.
- Windows 98 users, please visit the
Windows 98 Downloads site.
- Windows 95 users, please visit the
Windows 95 Downloads site.
Second, we found that users would rather use windowsupdate.microsoft.com then to go to our patches/update repository, that make sense. You could forbid your users from accessing windowsupdate.microsoft.com, but it might have a problem, as some update might actually request windowsupdate.microsoft.com during installation.Therefore, we limit the priority of traffic in/out of windowsupdate.microsoft.com. Eventually we lower the prior of entire microsoft.com because that's really necessary. Users could access to windowsupdate.microsoft.com on their own as usual - if they don't mind holding up their machines for a couple of days.
This works great. Larger and bigger patches are stored locally for users, while they could still access to windowsupdate for smaller patches/fixes. Our bandwidth load lessen(to a certain degree, we still can't solve that 5-15% Netbios traffic jam
Hope this help.
~~~
Let me guess... the other 55% is porn?
Wouldn't it be nice if you could set up a caching proxy to establish a verification process with the items being cached from that server - that way the server could perform checksum verification on the file and approve the copy for distribution.
It seems that it could be an easy implementation. The proxy requests the file verification in, an XML-RPC request is returned from the server to perform the checksum, the resulting data is sent via SOAP, and approval is given or denied, causing the cache to be used or flushed.
I don't know enough about it to say how difficult it would be to have the proxy determine if the service is available, though. It needs an acronym if it's going anywhere. How about Verify Cache Request (VCR)?
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
...download the updates yourself and either push them to the users through something like SMS, or have a program check the registry in the login script. It is fairly simple.
If it's a big problem, just block off windowsupdate and redirect them to your own page. You could implement a simple scan using something like HFNetChk. It's command line and works well.
Hey, look at it this way.. at least your users are updating! That puts them above 90% of the users out there.
No one's bitching. All the person is asking is whether there's a whether he can change Microsoft's mind. Nowhere does he insult MS. In fact, he states that there is a completely justifiable reason for not letting anyone cache the updates.
Just settle down, really. Maybe you should go to bed.
are you thinking with both brain cells here?
.NET" to solve all your worries.
the red hat updates are cacheable yet individually gpg-signed. they are also freely distributable by anyone. you can set up a red hat satellite proxy server for your organization. you can download once straight from red hat's FTP server (the URLs are conveniently listed in the emails) and push them to each machine. there are probably 50 different ways you can write perl scripts to fix the problem.
seriously, this is a difference between FREE SOFTWARE and VENDOR LOCK-IN. Even Brett Glass can understand what FREE means in this context.
Microsoft probably knows EXACTLY how much of a pain this is and will happily SELL you some overpriced "Windows Update Proxy Server Professional 2000
You _have_ heard of Microsoft before, right?
The only way to convince Microsoft of anything would be to _buy_ Microsoft.
As MS's server logs flood with people using Mozilla on Linux try to slashdot windowsupdate :)
In the meantime, you should be aware that all the major service packs for Microsoft products can be downloaded as stand-alone executables. Also, the IE download page includes some critical updates. Make your own "cache" on the network, and let everybody get their updates from there.
So dare I ask what the other 55% is? Here's my guess:
No, don't check. You don't want to know.
GStreamer - The only way to stream!
You can't transparently cache, but you can set up an SUS server and point your clients at it. Software Update Services FAQ. I don't think it costs anything (beyond the cost of a Windows 2000 Server or Windows 2003 Server), and I don't see anywhere that it says you can only use it in a business... Wouldn't that work?
Time flies like an arrow. Fruit flies like a banana.
Yes, but unlike Microsoft, RH doesn't *sell* an overpriced server to let you do exactly this, and hence *does* cache just fine. Just like apt-get (actually, apt can even grok bittorrent) and yum. I would strongly suspect that urpmi and emerge can be cached as well, though I can't personally confirm it.
May we never see th
> How might Microsoft be convinced to make its updates
> cacheable, so as not to waste unthinkable amounts of bandwidth?"
Well, you could try threatening them with legal action - that usually works...
Perfectly Normal Industries
I hope that the fools at my ISP won't blame me for running apt-get every other day or so... And how about full network installations over HTTP, if updates weren't enough ?! ;)
As we reviewed the cache statistics for our small ISP today, we noted that the traffic generated by Redhat Network Update feature constituted 45%....
/var/spool/up2date a Samba or NFS mounted shared volume (or you can make it a symlink to one) on all your RedHat machines. For your first up2date execution that day/month/year/etc., use the --download flag. This will cache all the files in /var/spool/up2date. Then issue the same command without the --download flag on all machines which need updating. up2date won't needlessly download things that are cached in this directory, but your sigs will still be checked.
Actually, this is easy to combat. Just make
Maybe there's a similar solution for Windoze updates?
moto411.com
Just checked the stats for the past 24 hours (from a Squid cache). This time, *.windowsupdate.com generated 56.11% of the traffic, with a hit rate of only 2.37%. In short, Microsoft is eating (and expending!) huge amounts of bandwidth, and almost none of what is being transmitted can be cached. What a waste.
Ahem.... Red Hat updates are cacheable. But the percentage of Linux users on our network is in the single digits. Most users run Windows.
If ANY single entity was eating up 45% of my bandwidth, yes, I'd bitch.
This sig intentionally left justified.
seriously, this is a difference between FREE SOFTWARE and VENDOR LOCK-IN. Even Brett Glass can understand what FREE means in this context.
.NET" to solve all your worries.
Microsoft probably knows EXACTLY how much of a pain this is and will happily SELL you some overpriced "Windows Update Proxy Server Professional 2000
Don't be a moron. Software Update Services is free. All you need is a machine running IIS.
--Life may have no meaning, or, even worse, it may have a meaning of which you disapprove.
If your ISP provides its users with a default homepage, try adding links to cached EXEs of the updates (aka the EXEs designed for corporate users) to that page. It's convenient, probably faster, et cetera.
Karma: Excellent (fuck, even in the future moderation doesn't work!)
Why don't you subscribe to or at least take a look the ISP-Caching mailing list?
I wonder if MS is setting the proxy:nocache header in the HTTP reply, or if their client is always doing a reload rather than an if-modified-since.
Perhaps allowing Squid to be configured to ignore proxy:nocache and to convert reload into IMS based on an ACL would allow a site admin to tweak around this without breaking other sites>
www.eFax.com are spammers
How about trying something like this.
Hmmmm. Given the amount of bandwidth Windows Update takes, I wonder how much of AOL's bandwidth it takes.
.1 second. (Yes, this puts some more work on the root servers, but not much, as his name server will cache the locations of the TLD servers).
Hmmm...
On a related note: I haven't looked recently, but it used to be that Windows clients were TERRIBLE about DNS lookups - they would not cache anything, and were always making DNS requests on every little thing. I was helping a FOF set up his DSL, and his DNS lookups were taking 3-5 seconds, because his ISP's name servers (swbell) were overloaded. We finally set up his own internal name server, and set it to do the name lookups itself - time went from 5 seconds to <
www.eFax.com are spammers
There is a Control Panel setting -- automatically ask the internet for critical updates, or something.
It's better to be the foot on the boot than the face on the pavement. ~~ tkx Kadin2048
This is a nice example of how M$' our-products-are-blackboxes-policy is increasing the cost of using them.
In a world of open systems, everyone who felt like doing it could cache software updates, freeing money and bandwith for more sensible uses than trying to cure a dead horse.
Microsoft does not want you to know what they are
sending you, and this goes for the isp's as well.
This is because long before 'trusted computing'
is forced on the rest of us by hardware, they are
trying to do it by software to those gullible enough to accept this hostile force into their
computer guised in the sheeps clothing of an
innocent sounding 'update'.
... is that Microsoft send out a good number of responses with a "Cache-Control: private" header. Any public cache storing these responses is in violation of RFC2616.
This posting from the squid-users mailing list sheds some more light on the issue.
If you were wanting to break the RFCs and were using squid, then you could probably modify src/http.c to return 1 for the relevant parts of the httpCachableReply function instead of 0, but that would be a "Bad Thing"(tm) when it came to RFC compliance.
The latest update was the Java fix, and that weighed in at 5MB. If that's all it takes to spike your traffic then you're probably getting off cheap the rest of the time, with most users not doing much downloading other than mail and news.
Why don't you post some hard data instead of percentages? Saying windows update is 50% of your traffic is meaningless unless you provide background. What is your normal traffic? How close are you to capacity?
I can't say that I don't give a fuck. I've just run out of fuck to give.
http://www.microsoft.com/presspass/contactpr.asp
I am sure they could put you in touch with the right person. You could also try this newsgroup...
microsoft.public.win2000.windows_update
I've lost count of the times I've run into problems with transparent caches feeding me stale data; the last place I want to see stale data is when fetching security updates.
If you think it wastes too much bandwidth, think about the bandwidth which could be wasted by a network full of machines which were compromised due to not fetching the latest securty updates.
Tarsnap: Online backups for the truly paranoid
Automatically ask the Internet? I like that. Forget going to a vendor website or even Google. Just send a broadcast packet to all hosts with my query and the Internet will tell me!
I better check the Evil Bit on the reply packets if I ask for critical updates, so I know whether to trust them.
--JoeProgram Intellivision!
Microsoft now uses Akamai to host Windows updates. You say you are a small ISP -- contact Akamai (http://www.akamai.com/) and see about getting their servers on your network.
If you aren't familiar with it, Akamai is a hosting company for high-bandwidth sites. Instead of hosting from a main location, they give cacheing servers to ISPs for free. These servers will cache only Akamai content -- but the machine is free and they manage it.
Traffic is directed to Akamai servers via DNS, so you don't have to do any tricks to direct traffic to them. For example, if you do a DNS lookup (ie, Unix host command) on download.microsoft.com, it goes through several CNAMES, eventualy to something like a767.ms.akamai.net , which resolves to your local Akamai server, or the nearest one of your ISP doesn't use Akamai.
Off the top of my head, Yahoo and www.whitehouse.gov are other sites useing Akamai.
Invention, my dear friends, is 93% perspiration, 6% electricity, 4% evaporation, and 2% butterscotch ripple.
That's 105%!
Overrated / Underrated : Moderation
Charge Microsoft for the costs incurred for that bandwidth. Pass the savings on to your customers. Basically lets assume that MS consumes more bandwidth than any other company on the internet.
Lets bill them for taking up the bandwidth. Someone really should be paying me for the grief and aggrevation that MS causes me every time they tell me their sw is flawed, and they have figured out a way to fix that flaw and create 4 more flaws.
Regards,
Ryan Pritchard
Fun Extends All Basic Life Expectancies
I suspect that someone at Microsoft has been reading this discussion, which is good.
Most of the stuff that became cacheable, though, was for Windows XP. Windows 98 and Me updates (and we have a lot of users running 98 and Me) are still dragging the system down. I know, I know, you guys at Microsoft are trying to persuade Windows users to upgrade. But could you please not attempt to do this by making updates slower? We ISPs would appreciate it.
We noticed the same thing on our network prior to installing Software Update Services. However, it wasn't a big deal to us because we selected the "Automatically Download and Install the Updates on the Schedule I Specify..." option. This uses BITS (Background Intelligent Transfer Service) to incrementally download updates when computers are not making heavy use of bandwidth. We also made use of QoS to control the aggregate inbound flow from Windows Update.
I've got access to stats for a couple of largish connections (good mix of individual and organization traffic) and that number seeems a little high.
All the traffic I see to/from microsoft - including msn and hotmail, accounts for perhaps 30% of my traffic on a typical day. On a day when somethign like DX9 comes out, that figure goes up a bit - but still not to the 50% level.
Do you maybe have a customer who builds systems and mass-updates them? that would almost make the number reasonable....