WGA Meltdown Blamed On Human Error

Erris writes "As commentators like Ars Technica slam WGA as deeply flawed, Microsoft is blaming human error and swears it won't happen again. 'Alex Kochis, Microsofts senior WGA product manager, wrote in a blog posting that the troubles began after preproduction code was installed on live servers. ... rollback fixed the problem on the product-activation servers within 30 minutes ... but it didnt reset the validation servers. ... "we didnt have the right monitoring in place to be sure the fixes had the intended effect"' Critics were not impressed. 'A system thats not totally reliable really should not be so punitive, said Gartner Inc. analyst Michael Silver. Michael Cherry, an analyst at Directions on Microsoft in Kirkland, Wash., said he was surprised that it was even possible to accidentally load the wrong code onto live servers ... [and asks], "what other things have they not done?' This is not the first time this has happened, either."

Comments

Have we gone backwards?

[Ckwop]

This sort of ties in with what I was saying on IRC with my friends yesterday. My central point was that all operating system have got worse over the past ten years.

I'm currently reading the Mythical Man Month (which I imagine most of you of heard of and already read) and in it he talks about the OS/360 operating system in great detail. I'm recalling this from memory so I'm sure someone will correct my mistakes but anyway, the machine had 2MB of memory and the operating system cost 400Kb of the memory. They charged something like $9.50 a month for 1Kb of system memory. That meant that every Kilobyte of memory saved was worth hundered or even thousands of dollars over the life time of the machine.

It made me realise what is in retrospect a fairly obvious statement. The cost of the operating system on your hardware is an effect that should be minizimed. The operating system exists as a framework for runs tasks and applications, not for being a self-serving execuse to munch resources.

While Moore's Law technically means something different; the adage has held true that computing power has doubled every eighteen months. This means that my machine which I bought in January should be roughly 100 times more powerful than the machine I had in 1997. Yet do I have hundred times more power to run my applications on a modern Operating System? Absolutely not.

Strictly speaking, there are no tasks I do today that I couldn't do in 1997. I can be honest that computing hasn't really got easier since then either. There's the odd innovation here and there that's nice from a usability point of view, but fundamentally nothing has really changed. For an example, Office 97 and Windows 98 are no harder to use than XP and Office 2003. The addition of an extra monitor to my compute has impacted my productivity more than the choice of software in this period.

In short, where did all these cycles go?

Now Microsoft Vista is a sort of a post-modern operating system. In every sense it is a regression. It does not allow tasks to be managed easier yet requires an enormous amount of extra resources just to operate. WGA in a sense breaks the very stability of the system. The point of the OS is to perform tasks and applications yet Microsoft can take this away from you either by malice or stupidity.

When are we going to demand more from OS vendors? When are we going to demand that future versions do the same as the previous version with less memory and less CPU overhead? Why do we pay to upgrade only to find our upgrades are wiped out by OS bloat? All of these are interesting questions, and while off-topic slightly, I'd like to see what you think!

Simon

Re:Have we gone backwards?

[Dunbal]

When are we going to demand more from OS vendors?

      I would extend this to "software" as a whole. Software seems to be in a special protected class, since companies are able to KNOWINGLY deliver a defective product and be immune from prosecution. Computer games I am looking at you. There seems to be a mentality in the industry of "ship now, patch later".

      I can't let this go without a car analogy (this is slashdot after all):

      It's like buying a new car from a dealership, only to find out it comes with 5 flat tires. But the salesman puts his arm on your shoulder and says "hey, no worries, look - there's a gas station just over there and you can get those tires fixed in no time".

      It's high time the software industry as a whole was held accountable for this sloth. And don't give me the crap about "oh but there are so many different computers and hardware and configurations". After all, ISN'T THAT WHAT WINDOWS WAS SUPPOSED TO FIX? We certainly were sold on that idea in 1995. Windows was supposed to be a common application interface that smoothed over all the hardware differences. But because it's the poorly documented, bloated, kludge that it is, programmers yet again have to rely on little tricks and cheats to get top performance out of it. Resulting in crashes/bugs on non-standard systems.

Re:Have we gone backwards?

[pandrijeczko]

The Amiga was taken down, not because there was not enough demand for it, but because it was too efficient.

Rubbish! The Amiga was a far superior machine to the IBM PC but Commodore/Escom/Gateway/Amiga Inc. did not have a single clue as to how to market it and expand it correctly. It was their total lack of incompetence that caused its death.

Amiga users (and I know because I was one of them once) were the most loyal bunch of users there could possibly be, a bunch of people who remained loyal for years despite being continually f*cked in the arse by unfulfilled promises by David Pleasance and whomever else controlled the Amiga name over the years.

Re:Have we gone backwards?

[Sancho]

From Win95 to Win98 to Win2000 to WinXP, I've seen nothing but stability and security improvements. Vista has some security improvements, too, but in my experience, it isn't any more stable than XP. What's also come with every single new release of Windows is a changed UI, more eye-candy, and features that many geeks find useless.

That doesn't mean that they're useless to everyone.

Part of the issue is that you're focusing on the operating system. Windows is really quite a bit more than that--it's an operating environment (or a desktop environment, as GNOME/KDE are described.) This means that they aren't just there to provide a framework for performing tasks--the operating environment performs tasks on your behalf, provides feedback, allows the user access to information in a subtle, yet useful way (many OS X widgets, for example, and whatever Microsoft is calling their clone of it in Vista.)

In the Unix world, we separate the operating system (kernel) from the shell (bash/ksh/whatever) from the window maanger (metacity/fluxbox/xwm) from the desktop environment (GNOME/KDE). This separation allows for immense flexibility. I can mix-and-match flavors, and even eliminate some of these layers entirely, depending upon my needs.

Windows, however, caters to the mass market. It needs consistency in order to maintain its marketshare, while simultaneously requiring each version to have a distinct look in order to differentiate itself from the earlier versions. It has to be everything to everyone in order to keep existing users and attract new ones. It makes sense to throw in as much stuff as you can, so that people will want to use their product.

Most people buying a computer will use it for the Internet (browsing, email) and maybe for creating documents and managing finances. Yes, they could do this on a 10 year old machine. The only reason to upgrade, then, is for the new UI or because their old computer broke. In either case, they aren't really losing anything. They're gaining more cycles in their new computer, and they're getting an OS that uses those cycles. If their tasks don't change, their CPU power needs (over what the OS requires) probably haven't changed, either.

In more specialized circumstances, yes, it matters. And that's part of the reason that new OS are adopted fairly slowly in the business world. Not only do we want to ensure that the change is as easy as possible, but we want to make sure that we aren't losing anything.

I think I've rambled a bit much, but the gist is, you aren't the target of Windows Vista, and Microsoft isn't just making an operating system. And that you're bringing Unix-like preconceptions into the Microsoft world.

"won't happen again"?

[haeger]

So, if it's human error that caused the problem, how can the swear that it won't happen again? Will there be no more humans working at microsoft anymore?
I don't get it?
People make mistakes and as long as people are involved in any process they will cock up from time to time.

The point about systems not being so punitive is a valid one and should be brought up more often and louder. People who've paid money for their product should not be punished for an error on microsofts end.

.haeger

Re:It's a fair point

[Anonymous Coward]

Sure, for a 24 hour window pirates would have a free-for-all in getting perfectly valid WGA results.

Actually, pirates would probably very quickly figure out how to set the WGA server failure condition in Windows to get the automatic pass without ever actually contacting the real WGA servers, which would render WGA completely worthless. Well... more so.

I don't use Windows, can't stand Microsoft, and had a hearty laugh at the news of the WGA meltdown, but the problem is not as easy to solve from a technical standpoint as you believe.

Microsoft is blaming human error

[suv4x4]

Microsoft is blaming human error and swears it won't happen again.

Self-contradictory: of all things that could happen out there, one thing will keep happening, and that's human errors.

Realistically, it's just another fail point on your OS that will blow up from time to time.

Re:Zoom

[gatzke]

Slashdot is not about journalistic integrity, it never has been. It is about nerd topics and dupes.

ACs complaining about twitter does look like astroturfing. MS has enough money to pay a few guys to beat back public opinion on well-known public tech sites. Without facts disputing the current article, it looks like you are just pro-MS ranting against a anti-MS article without any substance.

Fact- WGA broke for a while causing many people troubles.

Fact- Some people don't like having to phone MS all the time to keep a product running.

Fact- MS has paid astroturfers to anonymously post pro-MS grassroots stuff online.

Monitoring

[Dunbal]

"we didnt have the right monitoring in place to be sure the fixes had the intended effect"'

      This sounds a lot like the Bush administration's excuse... oops!

      Seriously, Microsoft is great at monitoring YOUR computer, but they can't monitor their own?

I've said it before and I'll say it again

[FoolsGold]

If the pirates are having no problems and it's the legit users who are getting fucked in the ass, why the hell does Microsoft continue to bother with WGA?

What do they gain? Was WGA suppose to convince people using illegitimate versions of Windows to turn to the light? Fuck that, they'll just download the latest cracked WGA .DLL and get on with it, while the legit users will get boned because their serial key wasn't recognized or whatever.

WGA does NOTHING to hinder piracy, at least not with any level of success that compensates for the negative affects to legit users. It's a complete joke - and yet Microsoft doesn't have the balls to admit this yet. It pisses me off to see such short-sightedness from a bunch of guys who are suppose to be experienced in business.

Re:I've said it before and I'll say it again

[AxminsterLeuven]

Microsoft has its monopoly on the desktop market because DOS and Windows have been pirated like crazy in the past. I remember when the little local computershops would throw in a little box with freshly copied diskettes with the expensive beige 386 you just bought. It seems stupid to start cracking down on piracy just when free OS'es are starting to become an attractive alternative and the fruit-logo competitor is offering a relatively cheap shiny system with more bling.

Not an acceptable answer

[Anonymous Coward]

Look, most of us here work (directly or indirectly) in software. Who hasn't had a launch fail, or a product go bad, in a way that's negatively impacted customers. Such things DO happen. Usually not out of malice, and even sometimes not from carelessness--there are things that sometimes you can't catch on a test system. So to that extent, I feel for the folks who caused this problem..

So why do I call it unacceptable? Because of the difference in standards. On Microsoft's side, they are holding the user to a high level of scrutiny, and reserve the right to cripple some OS features if Microsoft believes the install is pirated. No discussions. Go directly to "aero jail".

Which is possibly understandable if their stance is "look, we're losing billions here--we need to fight piracy." But if they're going to take such radical and punitive measures as locking down OS features based on their tool, then they have to have an absolutely rock solid fail resistant totally monitored system. Basically, they need to hold WGA to a higher standard than most business software. This needs to be the gold standard if they want people to trust the system (and TFA links to a number of other reasonably well-balanced Ars articles that suggest it is not).

Oops, we forgot to monitor the validation boxes? You can't be organic about this--add monitoring for problems as they're discovered on a system this critical not just to Microsoft, but to their customers. You have to anticipate what MIGHT happen, even if "there's no way that should ever occur." You have to think of things that should never happen, but would be problematic if they did.

The fact that they failed here, if it never happens again, might not be a huge deal. But their answer shreds confidence that this is an isolated issue. The fact that this specific failure might not happen again gives me no comfort. Because their answer indicated that they didn't get it when they designed the system, and the don't get it now.

What they SHOULD have said is "boy, this was something we never thought could happen. We have fixed the issue, and are confident we have the monitoring to prevent this specific issue going forward. And we are undertaking a comprehensive review of our validation and monitoring systems to make sure nothing even remotely close to this could ever possibly happen again." Nothing less should be acceptable.

Re:Not an acceptable answer

[morgan_greywolf]

Additionally, IT organizations need to be able to rely on their operating system software for mission critical applications. This does not instill any sort of confidence in Microsoft operating systems. And Microsoft is scratching their heads wondering why IT organizations insist on running *nix on mission critical servers...

You have a choice, people!

[pandrijeczko]

You do not need to keep yourselves tied to Microsoft's apron strings, there are alternative operating systems you can use.

If WGA or other Microsoft activities are p*ssing you off as a user, then have some strength of conviction and DO SOMETHING ABOUT IT!

Just stop with the continual whining about it...

Re:Why didn't they kill the server?

[Technician]

So why didn't MSFT just kill the server to let people's software default to "genuine" instead of leaving the server connected with faulty software?

It's an anti-piracy feature. It prevents a business from firewalling the WGA server to get "genuine" status. Remember there was an un-authorised software update site? If it works without the real MS saying it's OK, the anti-piracy feature does not work.

Unfortunately for MS is this feature does not prevent users from migrating to the alternatives. It's hard to run a monopoly when Ubuntu is legal and free for the taking. If they had a choice, the first would be that I run Windows fully paid for. Second choice is that I run a pirated copy, but they are using WGA to prevent that to encourage me into the first choice, but the result is I have gone to their worst option.. I've gone legal to the competition. MS is helping themselves break their monopoly by reducing piracy.

Re:Zoom

[iminplaya]

Well, let's be honest. Any program or OS that requires activation deserves a good bashing, and we should not support it in any fashion. And I proudly champion those who develope workarounds. Those who complain about bootleggers while benefiting from them as Microsoft and Adobe do are just as hypocritical as gay bashing republicans.

What about the future?

[Riturno]

What I really wonder about is when will these servers go down permanently? While I hate to do it, I can still install NT3.51 on an old machine if there is a critical need to pull something off an old tape. What happens in the future when WGA goes dark? Will they issue a patch to unlock the OS? At some point MS may have to limit or eliminate backward compatibility. Will virtualization be good enough? This WGA debacle leads me to more questions and concerns than comfort. To me it is not about today. Like the fun with MS formats, it is about tomorrow.

Re:It's a fair point

[Nasarius]

Oh please. You're talking as if there aren't already full cracks for every version of Windows and WGA. That horse left the barn a long time ago. It's perfectly reasonable for Microsoft to prevent "casual" piracy by people who don't know any better, but going to absurd measures to foil serious crackers has never yielded anything but a few days' delay.