Emory University SCCM Server Accidentally Reformats All Computers Campus-wide

acidradio writes: "Somehow the SCCM application and image deployment server at Emory University in Atlanta accidentally started to repartition, reformat then install a new image of Windows 7 onto all university-managed computers. By the time this was discovered the SCCM server had managed to repartition and reformat itself. This was likely an accident. But what if it weren't? Could this have shed light on a possibly huge vulnerability in large enterprise organizations that rely heavily on automated software deployment packages like SCCM?"

Comments

Sounds like IT incompetence

[areusche]

SCCM is pretty good. It makes my desktop techs jobs significantly easier to deploy assets company wide. In this case, it sounds like someone pressed some buttons without being 100% clear as to what was going on. Unfortunate someone will not be working in IT ever again.

Re:Sounds like IT incompetence

[BitZtream]

Assuming it was just a mistake and not malicious ...

Probably not. This shit happens, and that person who did it will never do something like this again. Have you ever made a massive, expensive mistake?

I have, I was 19 years old and cost my company nearly a million dollars due to a silly misconfiguration. After I discovered it, corrected the error and notified my boss, I spent most of the night throwing up. The next morning, after everyone in the company (only 15 people or so) knew what happened, and I walked through the halls on the way to the meeting with the owner and my boss, I thought I'd pass out. As I walked into the Owner's office I didn't even bother to sit down, expecting a fairly short conversation. I was asked to sit down while my boss had this very stern look on his face. So I did, cost them that much money, I can do what they ask.

The owner than proceeded to tell me the story of how, when working for a certain Germany car company doing CICS programming, he made a mistake that screwed up a production line and cost the company several million dollars. He knew exactly how I felt, and he knew that it would never happen again because I had already punished myself more than he possibly could.

If they fire the person who did this, they just wasted the whole event. The person learned their lesson and will be extremely cautious in the future. Firing them now just means someone else will get to reap the benefits of this experience, and thats pretty stupid.

People make mistakes, and in this case the software is at least partially responsible. The SCCM server should have aborted during the preflight checks when it realized it was going to take itself out in the process. The best thing this IT department can do is for the manager/director to keep the specific employe's name under wraps, stop shit from flowing down hill from above and move on. Nothing will benefit anyone if all of Emory treats the person responsible as if he deserves to pay for all the time lost in repairing the damage, he simply can't.

The hard lesson has been learned by everyone, nothing else will make anyone any better off.

Re:Sounds like IT incompetence

[bitt3n]

Have you ever made a massive, expensive mistake?

Glances woefully down at wedding ring...

The person learned their lesson and will be extremely cautious in the future.

Thinks back on previous three weddings...

An...accident..?

[geekmux]

Knowing that people have been running various kinds of centralized update services, perhaps across multiple OSes, and spanning several years now, listening to a story about an update server literally going rogue and nuking everything attached to it, and then for the coup de grace, basically committing suicide at the end by reformatting itself, does not sound like an accident.

If it truly was, I'd hate to see what the hell purposeful intent looks like.

Only a good manager could tell the difference

[Anonymous Coward]

It sounds like the commenter above was teachable - he no doubt learned his lesson.
It also sounds like the company's owner knew he could learn this lesson. That's the mark of a great manager.

Whether the Emory staffer responsible for this mistake is teachable or not, I hope his boss can tell the difference. Some folks aren't teachable, some are. If the Emory boss is worth his paycheck, he should be able to tell.

Re:Cool

[camperdave]

No, capability isn't enough. The student's personal computer still needs to be configured to PXE boot before hitting other boot sources. Even that wouldn't be enough. Something has to trigger a reboot. So, if the machine's boot order has PXE before hard drive, and has Wake on LAN configured, AND is powered off as opposed to merely sleeping or hibernating, then it *MIGHT* be affected. However Wake on LAN requires that the MAC address of the target computer be known by the issuer of the Wake on LAN command, the SCCM server in this case. The odds of all these prerequisites being in place for a student's personal computer is remote in the extreme.