The Costs of Patching
prestidigital writes "vnunet has a brief but interesting article in which Craig Fiebig, general manager of Microsoft's security business unit, is quoted as saying "In dollar terms, patching is the most expensive security measures and keeping your antivirus descriptions up to date is the least." That seems like an important statement coming from a company who's patches are possibly responsible for 45% of traffic on some networks."
... to realise that it costs more to do things 2, 3, or 4 times then if they had done it right the first time...
And that is costs more to have a new programmer look at and try to modify code that wasn't written by himself/herself...
Amazing reality breakthrough!
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
This statement is also known as "an ounce of prevention is worth a pound of cure."
evil adrian
Rather than throwing away an otherwise perfectly good pair of pants, patches have allowed me to fix them and extend their life. In some cases, patches can even be fashionable. Sewing is a great skill that all geeks should learn.
Using the patch is about as expensive as smoking, but will be more benificial in the long run because after a while, you'll be done with the nicotene forevar and not need to buy patches no more.
The difficult question is whether the costs of patching outweigh the costs of NOT patching. There's a lot to be said for "if it ain't broke, don't fix it" sometimes.
However, with security patches usually you have no choice. The only decision for some security patches is how long do you wait before deploying it. Don't wanna be the first ones to put a bad patch on now, do we?
My motto is: Never give up - unless it's harder than you want it to be.
It's their damn fault. When I decide to accept the stupid auto-update "critical" patches to Windows, it's usually less then 10 days before I have to do it again. Maybe if they didn't release software a year before it's ready it wouldn't be so bad.
"Probably the toughest time in anyone's life is when you have to murder a loved one because they're the devil." -Philips
This document was part of an interesting debate over the last year and a half between MS and Novell over whose product was more buggy (measured in terms of number of patches.)
(Google cache version in html.)
Why do I h8 apple?
IMHO getting hacked is much more expensive.
responsible for 45% of traffic
But spam is responsible for, what was it Taco, 60% of traffic on networks?
I'm at 105% utilization already!
BTW, it's just as costly, if not more, to have to rebuild your linux kernel, SSL, apache webserver, or samba installation when a bug is found there.
Quit pretending that MS has some sort of monopoly on software bugs. "Bad code" is a patentless technique used ubiquitously.
I don't need no instructions to know how to rock!!!!
Sometimes I wish there was the equivalent of Windows Update for Linux. If it wasn't worth the effort I wouldn't be using it, of course, but the asymmetry between the Windows patches and Linux patches doesn't seem to matter much when all the Windows patches are applied in one go and the Linux patches require individual attention.
The software industry has known for years that the later you find a bug the more expensive and messy it is to resolve
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
If you want security on your boxen it is prevalent to install just the components you need and no more than that. For example, it is safer to have a dedicated firewall/router and a seperate desktop machine for accessing the Internet than to just connect with a 'one-size-fits-all' installation. This goes for Windows as well as for GNU/Linux and/or *BSD. Myself, I have an OpenBSD box connected to my DSL-line and patching is seldom needed (at least compared to other OS'es). This way I can fool around on my (seperate) desktop machine till my hair falls out but it won't get h4x()r3d...
!ERR: Signature not found.
People who say 'they should have patched' do not understand the stress that installing a patch however critical on a few hundred servers, then in many cases rebooting them, can put in a commercial environment.
Maybe M$ shoudl concentrate more on amking better software dso it doesn't need to be patched so much!
There's a growing sense that even if The Future comes,
most of us won't be able to afford it.
-- Lemmy
Pff.. you lamers with your fancy-pants Windows or your free Linux or *BSDs are all clueless. I haven't patched my Apple ][+'s DOS3.3 for 20 years and it still has yet to be 0wned.
Trolling is a art,
If MS wouldn't include so much "junk data" to keep their proprietary data secret in patches, they wouldn't be so large. And, if there was a way to do a patch "rollback", then faulty patches wouldn't bring down a system until a new fix-patch was released. (One of the recent MS patches was found to cause some machines to stop booting)
-----------
From Ape to Man: Evolution
After reading this post, I checked windows update and found two brand new criticals... That makes five in three weeks. If they'd get it right the first time...
rim-shot
Thank you. I'll be here all week. Tip your waitresses.
Try to enter today's date in Appleworks.
Good to see that the subscription service kept the shit from rising to the top (or first, however you want to look at it).
Just remember debian gnu/linux is a secure OS. Every package goes through extensive scrutinization and auditon by thousands of debian users. As a result, the stable distro uses slightly older packages, but they have had nearly every bug squeezed out!
If you wan't to see how secure it is, try it for yourself
Remember more secure means less patching, which means more uptime, less traffic and more profit.
"Fiebig (pictured) also acknowledged the policy of patching was rendered less effective because of administrators' dislike of network downtime."
He should say: "Since we at microsoft has a broken system where you can't upgrade stuff without a reboot, it is hard for us" - I dont think a apt-get update && apt-get upgrade in cron is that hard work.
- To understand recursion, we must first understand recursion -
Lockdown your servers, you insensitive clod!
I've applied my fair share of patches from MS, but lately I've become really nervous about doing so. I'm always thinking "what kind of DRM will they include in this one?". It's gotten to the point where I will NOT apply patches for anything but server products, and only reluctantly so. Call me paranoid if you wish, but I can't really shake that feeling. Hey MS, great way to promote security - making users reluctant to apply patches...
Black holes are where God divided by zero
..because one of the many new feature of server 2003 is the ability to update patches auotmatically.
So they will use this 'cost savings' to push the new product. At the launch event, they bagged on there older products pretty damn hard.
It's part of there latest slogan
"do more with less".
personally, I dln't know who this less guy is, or why I would want to do more with him. Ironically I prefer less to more.
The Kruger Dunning explains most post on
Yeah and? Today is Thursday, May 1 10003.
Trolling is a art,
For once, someone was actually misquoted. (Rather grievously)
As the only sys admin in a company of 50 desktops and 4 Win2k Servers I can fully support the notion that patching is expensive...but not for the company...for ME!
Guess who gets to come in the office between 8 and 10pm to apply these patches to live servers...who has to wait if someone decides to work late. Who has to cross his fingers with every patch hoping that nothing else breaks...ME! And the only thing I get out of it is to be able to leave an hour or two early that friday...woot.
Sure some things I can and do install from remote, but almost every patch requires a reboot and you just never know when a Win2k system isn't going to boot properly and require you to drive in at 1am wearing your bath robe.
Apple free since 1990!
SuSE has a utility called YaST Online Update (YOU), which has similar functionality to Windows Update. Works great for me.
I'm sure other distros have similar features. Do some looking around on Google or Distrowatch.
apt4rpm works very nicely for RH. It will not auto update kernels (I regard this as a feature) but it will pull them down if you ask specifically.
See my journal, I write things there
79% of all percentages are made up on the spot.
Oh, I can't help quoting you because everything that you said rings true
Patching should be priority, and very rarely breaks applications, yet so many companies don't patch because they feel the need to test patches first...
can someone shed some insight?
What is the patching that you speak of?
Security Professional, CISSP
Redhat network works wonders for me. It catalogues all of the software that shipped with Redhat, and lets me know which of my systems requires what errata (updated software). Third party isn't a term that most open source companies recognize. =)
Best part is, I don't have to be on my actual system to check for available updates. I just log in to the RHN and look at the list of my registered systems. This trounces Windows update IMHO.
Actually, just the act of patching may roughly equal. But UN-patching a system can be done very easily on a *nix based system. How do you UN-patch a Windows based system?
Also, when I rebuild apache, I know what I am affecting. When I install a Windows patch, I cross my fingers.
My beliefs do not require that you agree with them.
...or is this equivalent to buying a new house, and then having the builder say, after the discovery of a defective front door: "we shouldn't have to fix this. why don't you just hire a security guard with a gun to sit in your foyer 24-7?"
In other news, the sky has been discovered to be blue and the planet has been proven round....
US Democracy:The best person for the job (among These pre-selected choices...)
Same as...
Proper planning prevent piss poor performance.
Microsoft has a free product out called SUS (see subject) the SUS works in conjunction with the BSA (no, Baseline Security Analyzer) to determin patch levels of 2000/XP clients and servers it then downloads all neccessary patches in a SIS (single instance storage) at the server. In this way every patch on your network is downloaded only once. If you only have four PCs this cuts update traffic by 75%. This is nearly as effective as ISA server but it is FREE. It is not as effective as coding it write the first time LOL but it is a start.
That seems like an important statement coming from a company who's patches are possibly responsible for 45% of traffic on some networks."
Their patchesd aren't responsible - people not applying their patches are responsible. What costs more? Patching or not patching?
apt-get upgrade
That's what I do, and I'm not sure what all the fuss is about. Things get fixed, usually before I ever knew they were broken, deamons get restarted, nothing gets interrupted, life goes on ... If I took the trouble to make it a cron job, I'd never even know.
Is Mr Fiebig telling us that things don't go so smoothly if you use MS products? Or that MS can't keep up with a bunch of amatures? Do MS patches break non-MS apps? Could all this be why so many worms and viruses manage to spread across unpatched MS products? Could it be that MS patches are as bad as the bugs they fix? SAY IT AIN'T SO, CRAIG!
See what I've been reading.
Bell labs(now lucent) and various hackers have made string functions that do the same thing but are buffer safe. They are made to create more secure apps.
My question is if gcc or visualc for that matter switched to more buffer safe libraries would it make a difference? Trusted Debian is compiled with buffer safe string functions.
It may be time gnuc did this by default assuming all the apps could be recompiled without a problem.
This would seem to get rid of %90 of holes in user as well as kernel space.
http://saveie6.com/
Why is this site so US-centric? If you USians would just use the SI system you'd get 62.5 times better ratio of prevention to cure. You'd have a gram of prevention is worth a kilogram of cure.
Geesh, I seem to have an anti-european streak today...
Norris/Palin 2012
Fact: We deserve leaders who can kick your ass and field dress your carcass.
see the subject
Between the costs of patching, the two weeks of downtime per user per year, and the flaws that threaten national security, has no one yet found a good way to sue for damages??? WTF?
Healthcare article at Kuro5hin
/joeyo
2^5
Precompiled kernels work just fine and Debian's /etc/modules file makes it easy to change around hardware. Going from 2.2 to 2.4 was easy stuff. I can only imagine that they will use the same kind of upgrade policy for kernels as they do for every other package now, therefore I expect my kernels to be patched if some kind of flaw is discovered.
The poster who thinks there is less trouble to taking care of Windoze boxes than there it to Linux boxes is nuts or ignorant.
Friends don't help friends install M$ junk.
currently my favorite form of nicotine relaxation.
"You never want a serious crisis to go to waste." - Rahm Emanuel
Ha! Ha! The joke's on you! AppleWorks doesn't run on a DOS 3.3 Apple II+!
For all you clueless moderators out there, this is actually a very funny post. The guy is writing in pig latin because it is better than Microsoft's!!! Let me translate:
Hey there Slasdot dudes. I'm writing in pig latin because it's way better than the best security systems that Microsft has to offter... And no I'm not joking! One time I sent a supposedly 128-bit encrypted email to my boss (it was for his eyes only), but an Outlook bug prevented it from being encrypted. By serendipity, I had also managaed to CC one of my coworkers... No problem if the encryption would have worked right? Unfortunately, the coworker (whom the email was about) read the email and reported it to my section manager. To make a long story short, I ws put on indefinite leave for two weeks and then transferred to a different area of the company, complete with a lower salary and a shitty job. All because Microsoft can't program beyond a third grade level. (On the plus side, I ended up leaving the company three months later and started by own business. Haven't looked back since).
Phew!!! How come English-To-Pig Latin translators are easy to find, but you have to translate back by hand? Anyway, enjoy! (oh yeah and, MS sucks!!)
is that Linux doesn't require a reboot. Often, Windows doesn't either, but you're going to get the standard "reboot" messages every time.
Most people just reboot.
Man, I can attest to this... patches... especially ones that screw up systems not only cost time/money/bandwidth but they cost HAIR.. yes thats right... admins lose their hair b/c of the stress this makes them go through..... ::looks in the mirror::
arrhhggghh..
(Maybe they do and charge for it...I'm not aware of it) This can make it cachable WITHOUT the danger of trojans. Caching locally would decrease WU data transfer over the internet significantly, and all you'd have to get from MS is the md5 sum (and do you need a PGP key from the source? Not sure...I don't know anything about PGP). Simple. Bim bam boom, done. I'm pretty sure they can even use this technology for free, and it won't mess around with any of their pretty licenses...Not sure though, IANAL and I don't know much about OSS stuff.
I thought 3.0 fixed that. BTW, that's ProDOS 8, not DOS 3.3 ;)
I wonder what OS will underlie Contiki, though.
-uso.
Dreams, dreams, don't doubt dreams, dreaming children's dreaming dreams. Sailor Moon SS
Well... before the knee-jerk MS-bashing starts, let's think about it.
If you patch, you have to recompile the component, and possibly re-boot the machine or re-start the application. This is true for Linux too (unless there's a way to fast-swap kernels that I haven't heard about).
If you update, you don't need to re-start anything.
If you patch, you could have to patch just about anything on the system.
If you update, you are working through one application.
Of course, there's nothing to stop an OSS developer from writing something that just sniffs incoming data for known exploits, like a virus scanner does.
Ahhh... but that would slow the system down.
So I think you have to add "better performance" to the pro-patch argument.
But then, there is probably less effort to updating, especially if it's automated. Is there any OSS system with automated patching that people are willing to trust?
Either way, I think it's an interesting discussion. In practice, I patch.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
"In dollar terms, patching is the most expensive security measure and keeping your antivirus descriptions up to date is the least."
"If customers could do both it would eliminate the bulk of security problems."
That second sentence seems very oddly phrased. It makes is sound like they can't do both which is definitely not true. I get the feeling that this may be the beginning of some MS campaign to somehow tie Virus updates and patching into a single system. I'm not sure what the ramifications of such a system are, but my guess is that it will continue to tie people to MS products.
You have failed to close your whine tag, now you are doubly whining. Or you might be French.
(s)he said "don't have to reboot the machine every time"...
sorry to nitpick
"There are two major products that come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence."
apt-get update && apt-get dist-upgrade
:-)
y
cost: 40 keystrokes
Comment removed based on user account deletion
Comment removed based on user account deletion
Well, okay, I can't resist. "In Soviet Russia, software updates YOU!"
Well, actually you are correct here since, if you're running SuSE in Russia, and use YOU, then YOU can update itself!!!!
Today is Tuesday January 1st, 1980 ...
--
Backup not found: (A)bort, (R)etry, (S)uicide
Doh! My bad.
Actually, American.
Norris/Palin 2012
Fact: We deserve leaders who can kick your ass and field dress your carcass.
Apart from the Music dowload, uh, stuffff, at their web-store, SoftwareUpdate is the right way to do it.
The download sites are controlled by Apple (and Akamai for all I know) but Apple really serves up the content.
Also they have a better, more secure OS that's conservatively designed and carefullly implemented so viri scouring and bug fixes aren't quite so desperately required by the system owners.
M$ may be too anal-retentive for their client base's own good. The only thing they want to conserve is their cash flow.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
I think there's a big difference between AV definitions and OS patches. AV definitions can be loaded and unloaded dynamically and have minimal effect on uptime. OS patches (in Windows) tend to be all over the place. MS' System Update Server is a good idea for now - in reducing traffic due to patches. However, in most of my environments, the only things we patch regularly are IE and IIS. We typically only patch the OS pre-SP1, but after that we only apply service packs. In addition, we have IP filtering active on every Win2k server (managed via GPO Registry Settings) so we can granularly control port access. As a result, our Win2k servers have uptimes comparable to our Linux servers, with the exception of reboots related to service packs and major software installations.
If he's running DOS 3.3, he can't run Appleworks. He'd need ProDOS for that.
Secession is the right of all sentient beings.
Oh, and for that matter, if he's running DOS 3.3 on a ][+, the operating system can't take advantage of the system clock, assuming there is one.
Secession is the right of all sentient beings.
One real issue with the way MS patches there products is simply that you can't roll them back. Without large IT bugdets that support "beta" servers, the only real way to test to see if a patch works is to try it in a semi-live environment. If it doesn't work budget some time to reinstall because there is no way to take it back.
Instead of the patch/rollback thing other OSes enjoy its a shoot in the dark with MS server products. "Oops" are less costly if you can rollback. How is MS Server 2003 going to save me money if I spend just as much time or more with spooky patches?
Oh, that's cute. In other words, he's admitting that writing these patches really eats into Microsoft's bottom line and all of us need to be more vigilant about using aftermarket protection -- at our own expense -- to help them as much as possible.
That's awesome. The bullshit that sells best is the stuff they wave right in front of your face where you can get a good strong whiff of it, as Carlin would say...
My
Limekiller
you can't beat a bit of daily patching
:
I've got "cvsup ports-supfile" on a cronjob
Every day I get emailed a list of the applications that have been updated and I can choose when it's worth patching them (they might not be installed - for instance)
to upgrade my *whole* set of port installed software
#portupgrade -ra
& everything stays in regular updated form
I magically keep in step with the mozilla builds
it's great
that's why FreeBSD ain't dying
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
This "45% of network traffic" bullshit was thoroughly debunked before, yet now you've repeated it as if it was truth. Truth of convenience perhaps.
Is Mr Fiebig telling us that things don't go so smoothly if you use MS products? Or that MS can't keep up with a bunch of amatures?
Looks to me what he's doing is spin control. Claiming that Microsoft software is better than open source because it's cheaper to use antivirus configuration updates than patches - when the only way to defend open source against a viral exploit is with a patch.
Of course that completely neglects the enormous difference in vulnerability levels - so you don't have to patch your open-source stuff as often as you do Microsoftware.
His company's software is SO VULNERABLE that it has spawned a BILLION DOLLAR INDUSTRY to protect it from exploits so common that multiple new ones are released daily. And he's trying to spin this into a cost-saving feature.
What gall!
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
I spent an unpleasant Monday evening clear through to Tuesday morning this week reconstructing a G4 running Mac OS X Server.
Why did it crap out? It kernel panicked just before completing the 10.2.5 Server combo update (updating from 10.2.3)-- I mean, like a fraction of an inch of non-blue space left on the progress bar. Which is apparently the most important part of the process.
When I power cycled the G4, it would not reboot. I tried again and threw it into verbose mode, and the errors and failures just cascaded by.
To matters worse, I could not do the standard 'archive and install' thing to fix it. 10.2.0 refused to install over top of 10.2.5.
Luckily, I had good backups, and there was no loss of any of my client's data from the drive anyway. Still, I had to nuke the drive and set it up all over again, because nothing I tried to revive the existing install of the OS had worked.
Seriously, MS has had years to redesign MS Office, Outlook, and so on, to make it hard for viruses. Why is this still an issue?
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
Well... before the knee-jerk MS-bashing starts, let's think about it.
u uu uuggggggggghhhhhhhhhhhhh!!!
You've been here how long, again?
and
M$ patches on the other hand have been responsible for some of our WORST outages,
Ever tried to block ports with IPSec?
Windows 2000 Server *requires* a reboot every freaking port you close
AAAAAAAAAAAAAaaaaaaaaaaaaaaaaaaaaarrrrrrrrrrrru
14 ports and 20 something reboots later.
Have you read the moderator guidelines? Well, have you, PUNK? (and I want a Karma: Gnarly option)
Why hasn't the company that has gobbled up any novel software and made it part of the operating system never attempted to make anti-virus software part of its system? I'm not saying I have ever liked their model of embrace and embed, but why haven't they done that in an area where it really would be logical and useful?
That 45% figure is such BS. I don't see where the guy ever backed it up. Here is my reply to that article. I run Windows XP Home and keep it fully patched, at least for critical items, plus some optional items (like Movie Maker). So far this year I've installed 19 patches (Windows Update has a history feature). Let's say each update is 5MB (which is a very liberal estimate. Most are under 1MB). That's 95MB over 4 months, or roughly 20MB/month. If that is 45% of the ISP's traffic then that ISP is getting off cheap and shouldn't complain...especially if you consider that my estimate is probably 3x too high.
OpenBSD just did an audit of their code for these kind of problems. So if this is an issue for you, that is the place to start.
Lets face it-- if the bugs that cause the critical bugs even make it to beta, there is something wrong because there is a good chance they will get through even with the best testing.
The problem is not the testing or even the coders. The problem is often the application designers/architects who often are thinking "features" when they should be thinking "security."
I suspect that $1 of design is worth $10 of coding, $100 of testing, and $1000 of patching for Microsoft, let alone the poor customers.
LedgerSMB: Open source Accounting/ERP
apt-get moo
I have to do it once in a while, I always get a good laugh !!
I am no Microsoft fan, but even when if you were to write the software "right", you have to remember that,to quote Pressman, software deteriorate. Therefore even a perfect piece of software will need to be patched at some point in the future because the environment around the software will change (new OS, new hardware, unforeseen complication (can you say Y2K ;-)). Can anybody quote one OS that never needs patching ?
Once you have accepted that you will have to patch, then you do it on a regular basis, on your test box first, then you move the "patch bundle" to the prod boxes. The only problem with this method that has come up recently is the time-sensitivity of security patches, if you want to stay safe you can't really afford the slow cycle of waiting for the patch bundle to come out, let it mature, apply in dev, apply in prod. I have no answer to this one, I'd love to hear other's opinion on it.
There are strategies to reduce patches, like the one that is rarelly mentioned and that I like a lot is de Raadt's idea of code audit , once you found a bug, you know that you have made the same error somewhere else and should go through your code to find it and fix it.
A great way to handle integration testing is to participate in a group that does it together like, say Debian. I really appreciate the community's work to make sure that everything works.
1) Patching servers and Virus scanners?
2) Repairing a company's reputation after being hacked or brought down as a result of neglecting #1?
If you chose #1 you need to find a new job in a different, non-technical industry.
Dolemite
_________________________
Save the World! Use a Quote!
Wow...it took them this long to realise that it costs more to do things 2, 3, or 4 times then if they had done it right the first time..."
Well, try calculate the cost of writing PERFECT PROGRAMS that has NO BUGS, and all the features are implemented PERFECTLY.
I know about the planning process, I know about programming methodologies such as Extreme Programming but in this real world that we live in, believe it or not, NOTHING IS PERFECT and software patches become the second best thing one can have.
Muchas Gracias, Señor Edward Snowden !
Microsoft does test the various releases, and it tests the patched systems, but they have a tremendous number of users and of hackers to contend with. The mind-set at Microsoft for many years was; One computer, one user. And deliver the promise of the microprocessor. When they did the big Internet switchover they networked together systems that were "Secure enough" for single-user usage. The whole idea of hackers and hacking just wasn't properly understood by of a lot of the developers and managers. (I'm sure there were a few choice e-mails, by the smarter visionaries about network security, that they wished they'd paid more attention to).
You can look at Microsofts security problems as a fallout of the dot-com bubble, when MS and everybody else tried to very quickly adjust to the new Internet-business reality. The microprocessor revolution ran into the Internet revolution, and a certain amount of roughness resulted.
The idea that these problems are happening because of a lack of testing or of code review or of understanding how to do good software just doesn't add up. Creating good software is hard, creating good software for a large market is harder.
I am reasonably certain that Microsoft tests their products more than the Linux vendors do. Microsoft has teams of thousands of full-time testers using tens or hundreds of thousands of test machines running both automated and hands-on tests. I dont' think any Linux vendor does that. Did you think all of the 30,000 employees were coders and marketers?
I18N == Intergalacticization
Why are you using 'European' tags? Or are you unaware that it's the REST OF THE WORLD that uses Metric? Yeesh.