Microsoft Windows Update and Network Bandwidth?

Brett Glass asks: "As we reviewed the cache statistics for our small ISP today, we noted that the traffic generated by Microsoft's Windows Update feature constituted 45% -- no, that's not a misprint -- of our total throughput. Because so many computers on the Internet run Windows, this massive resource drain occurs whenever Microsoft announces major security holes (as it did this week). The traffic could be greatly reduced, and service to users much improved, if the updates were cacheable at the ISP. But Microsoft has set up the service in such a way that the data can't be cached. (It's digitally signed, so inserting Trojans into the cache is virtually impossible; in any event, no more of an issue than intercepting the data stream.) Are others out there seeing the same pattern? How might Microsoft be convinced to make its updates cacheable, so as not to waste unthinkable amounts of bandwidth?"

Comments

can't be cached?

[greck]

I can only speak from what I've seen in our offices, but squid (running in transparent proxy mode) very definitely caches content from Windows Update... I set it up about six month ago and remeber being really surprised (because I think I very reasonably expected it not to).

Re:can't be cached?

[Blkdeath]

I can only speak from what I've seen in our offices, but squid (running in transparent proxy mode) very definitely caches content from Windows Update... I set it up about six month ago and remeber being really surprised (because I think I very reasonably expected it not to).

Our store Squid server caches the likes of IE 6.1, Media Player and DirectX, but the vast majority of the Critical/Security updates are not cached. Our connection is quick enough to handle it, but a PITA nonetheless due to the dozens of machines requiring updates every week.

Re:can't be cached?

[bobibleyboo]

I can also vouch for this I had a Linux Mandrake SNF Server running a transparent squid server (with a little tweaking to the max file size and the average file size) I was able to cut out about 90% of the windows update traffic at the site (The site had about 200 users) none of the transactions where cached but when it came to downloading the updates and service packs it works wonderfully.

Re:can't be cached?

[Wolfrider]

--Check your squid.conf, and look for # TAG: maximum_object_size (bytes)
# Objects larger than this size will NOT be saved on disk.

# maximum_object_size 4096 KB
maximum_object_size 32767 KB

Re:can't be cached?

[PerryMason]

My experience is that if you download a single update, such as the Media Player, IE, service packs etc, which can only be downloaded individually, then squid will cache it. If you select a few updates, such as grouping your critical updates, it wont do it.

Looking at my squid logs, it appeared that there was a problem with WindowsUpdate issuing a 0 byte sized reply to the GET request (must be somthing to do with the activex control I guess, but never really bothered to look further into it). Squid seemed to choke on the 0 byte reply and obviously didnt cache the rest of the download.

Interestingly enough, MS's caching offering ISA appears to deal with it, but I suppose that they specifally coded it with a knowledge of how their activex control works and hence it knows whats following that 0 byte reply.

Disclaimer - I checked this all out when the new WindowsUpdate first came out and havent been arsed to look at it since then. I ended up just setting up a shitty old box as a SUS server and going that route. (The only benefit to being an MSDN partner being $0 cost for licensing as I justify it as being for testing purposes ;)

there is the way that large corperations do it

[rritterson]

Here at Berkeley all of the Windowsupdates come from an internal server instead of externally. That way they control who gets the updates and when.

You can download the updates individually, and there is probably a way to have them downloaded to the server automatically. All you have to do is convince the users to download them from you and install them manually. Can you block traffic from the autoupdate applet? I bet that would significantly reduce traffic, at the cost of insecure customers.

What about running an internal WU server and changing the DNS entry at the local level to a local server? You'd have to keep the catalog of updates stocked and refreshed constantly, for multiple OS's, so I don't know how cost effective it might be.

Several options available

[questionlp]

There are a couple of options that you can choose to help reduce the amount of bandwidth used to pull down and install Windows Updates. The first one, which is available for free and runs on Windows 2000 Server, is Microsoft's Software Update Services which allows you to create a local store of the updates (for any language and all supported platforms) and point the client Windows Update to the internal server. It's not perfect but it works in a lot of cases.

Another option is to use a systems management package (LANDesk, ZENworks, SMS, etc.) to build the packages and deploy them while only using your internal network bandwidth (once you've downloaded the hotfixes anyway).

Of course, the two options above are really meant for company networks, but even those can help reduce the bandwidth used for more important things.

Out of my experience

[jsse]

Yes you can't cache it. That save Microsoft a lot of trouble and the trouble is on you. :)

First step is to download the patches/update manually and save them elsewhere accessable to all users:

• Windows 2000 users, please visit the Windows 2000 Downloads site.
• Windows NT 4.0 users, please visit the Microsoft Download Center.
• Windows 98 users, please visit the Windows 98 Downloads site.
• Windows 95 users, please visit the Windows 95 Downloads site.

Second, we found that users would rather use windowsupdate.microsoft.com then to go to our patches/update repository, that make sense. You could forbid your users from accessing windowsupdate.microsoft.com, but it might have a problem, as some update might actually request windowsupdate.microsoft.com during installation.

Therefore, we limit the priority of traffic in/out of windowsupdate.microsoft.com. Eventually we lower the prior of entire microsoft.com because that's really necessary. Users could access to windowsupdate.microsoft.com on their own as usual - if they don't mind holding up their machines for a couple of days. :)

This works great. Larger and bigger patches are stored locally for users, while they could still access to windowsupdate for smaller patches/fixes. Our bandwidth load lessen(to a certain degree, we still can't solve that 5-15% Netbios traffic jam :)

Hope this help.

The other 55%

[Electrum]

Let me guess... the other 55% is porn?

Software Update Services

[superyooser]

Microsoft used to have a corporate Windows Update site where you could download all the patches as executable files. That site was retired last year in favor of something called Software Update Services. It requires running a SUS server and appears to distribute the updates only to systems running Windows 2000 or later.

In the meantime, you should be aware that all the major service packs for Microsoft products can be downloaded as stand-alone executables. Also, the IE download page includes some critical updates. Make your own "cache" on the network, and let everybody get their updates from there.

Re:Software Update Services

[Brett Glass]

Microsoft's Software Update Services require you to modify all of the clients. Those that aren't modified still try to access Microsoft's Windows Update site.

So, since ISPs can't administer their users' systems, this really isn't an answer. Caching is a much better solution.

Re:Valid

[mbogosian]

Wouldn't it be nice if you could set up a caching proxy to establish a verification process with the items being cached from that server - that way the server could perform checksum verification on the file and approve the copy for distribution.

It seems that it could be an easy implementation. The proxy requests the file verification in, an XML-RPC request is returned from the server to perform the checksum, the resulting data is sent via SOAP, and approval is given or denied, causing the cache to be used or flushed.

Ahh, but then that would involve Reverse Engineering, which, as we know is now illegal.

Not to mention that this is approaching a P2P network, which as we know can only be used for piracy.

Sorry, we're all just going to have to live with this new "innovation" in bandwidth utilization.

Re:Stats for the past 24 hours are even worse....

[Jon Peterson]

That's terrible. I mean Microsoft releasing frequent patches for their products - and then the users are finding those patches so easy to download and install that they keep doing it!

That's so typical of Microsoft. They don't care about the little ISPs, they just want their customer base to have free, simple, access to frequent updates and fixes, without giving a damn about the impact that has on Internet traffic.

I mean, at least when slashdot directs huge amounts of traffic to some dumb site about making a spaceship out of a floppy disc or whatever, they have the courtesy to always cache the site so that it doesn't take down the whole ISP that hosts that page.

Why can't MS be more like /. ?