Slashdot Mirror
Big Red Button Disasters?
FredDC asks: "The Daily WTF has a story about a Big Red Button disaster. What Big Red Button disasters have you experienced? Which ones have you caused? Are there any that you've heard about, or do you know of any that can happen any day now?"
I was doing I.T. support for a 400 person call center. In the server room there was a Big Red Button that was very clearly labeled "EMERGENCY POWER SHUT-OFF" near one of the sets of double-doors.
A technician from U.S. Worst had finished his work in the server room and on his way out he hit the Big Red Button thinking that would open the doors, like at a hospital.
Hilarity ensued.
Later that day I printed out several mock "Big Red Buttons" on sheets of paper to use as decoys next time the tech had to visit.
~> ftp www.workplace.domain /mis-typed/path /mis-typed/path: The system cannot find the file specified.
/index.html with something that was supposed to be a couple levels down is bad enough.
/index.html is owned by someone else entirely. Someone who now must be woken up in the middle of the night, in a different country...
Connected to www.workplace.domain.
220 Microsoft FTP Service
Name: shag
331 Password required for shag.
Password:
230 User shag logged in.
Remote system type is Windows_NT.
ftp> cd
550
ftp> put index.html
local: index.html remote: index.html
227 Entering Passive Mode.
125 Data connection already open; Transfer starting.
226 Transfer complete.
ftp>
The realization that one has just overwritten a public-facing, high-traffic
It's worse when
After I did this two or three times, I decided to stop being such a hardcore geek and got an FTP application with a GUI.
Village idiot in some extremely smart villages.
Wow, I haven't posted in forever.
Anyway, we did a big datacenter migration at my last company. I'm not going to name names, but it's a Fortune 100 company based in Austin, TX. The move was happening because we built our own building with our own datacenter.
As part of the technical staff (network engineering/security), I was given a tour of the new datacenter before it opened. My boss and assorted other folks were on the tour. My boss, by the way, was a huge...jerk.
The electrician showed us the Big Red Buttons by each of the exit doors. He also said that each of the Power Distribution Units (of which there were three) had a Big Red Button that would cut power to just the areas powered by that unit.
My boss said, not jokingly, "If you need to cut power in an emergency, see if you can figure out which PDU is involved and just cut that one, so we don't lose the whole datacenter."
I piped up: "If I'm getting 220 across my nipples, cut the whole damn room. I really don't care enough about the company to die. I can see my epitaph now: 'Here lies Dimwit. He died saving two-thirds of the datacenter.'"
Man, if looks could kill.
...but it's being eaten...by some...Linux or something...
to tell people that "Halon" is French for "Exit," so if they ever get locked in the data center, they know how to get out.
"National Security is the chief cause of national insecurity." - Celine's First Law
You know the submission queue is slow when by the time the story is posted the site has changed its name.
I used to work help-desk, and late at night there would only be two people in the quite large building - me and one of the operators. Anyone who as worked with "ops" knows they generally turn a bit strange due to them working nights with nobody around and only DAT tapes for company.
So anyway, there is this big fire alarm panel with tons of buttons that we never really thought about, until one night when it started beeping constantly. The ops guy found a key to it, and then we both stood there looking at the probably 60 buttons and flashing lights, etc. Personally, I would have chosen one of the black buttons marked "mute", but the ops guy went straight for the biggest red one on the board.
The result was more beeping, lots of red lights and about 5 fire-engines.
And as long as we're talking halon, who can forget the classic Vaxen, My Children, Just Don't Belong In Some Places.
Articles to slashdot have to be fact checked, and tested on a focus group to make sure that they don't cause emotional distress. After a two months of this, the editors will submit a form P41B with a write up, which is circulated to have it's facts, grammar and spelling checked. The legal department need to process a form P09F911029D74E35BD84156C5635688C0B for the story to make sure there are no legal implications as to publishing it, due to trade secrets, the DMCA or libel. Then it's pretty much a quiet month of tuning the write up and testing it on focus groups before publication. Seems like cramming all this activity into three months is remarkable to me.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
I took a great deal of effort to toddler-proof my study. PC and laptop with exposed buttons at desk height or above. Synth moved from wobbly stand to sturdy wall-mounted shelf. Linux server, under my desk, rehomed into a blacker-than-black case, fancy lighting rig unplugged, all buttons, optical drives and recesses safely hidden behind a plain black door. O'Reilly Wall moved from bookcase to high shelves.
I even got a "decoy" keyboard for my 11-month-old daughter to play with.
Of course, she found the UPS switch in seconds. It had a nice glowy LED above it, and was sitting on top of the Linux server just at her shoulder height.
All three PCs, the whole study, powered down, and not in a nice graceful apcupsd way, just a sudden BOINK, follwed by darkness and silence, penetrated only by a happy gurgle.
Thank heavens for Linux software RAID mirroring.
(A couple of months earlier, she managed to cause Windows to prompt "Add new hardware - Searching for drivers" [blur-o-matic cameraphone photo] by sucking the end of my iPod USB cable. Unfortunately I didn't have any Win2K drivers for a 9-month old baby. I bet Ubuntu installs them by default, even though the GNU crowd complain they're not truly free.)
Annabel is one on Sunday. Wish her happy birthday.
Andrew Oakley - www.aoakley.com
I was 6 and the manager at my local bank was in a meeting with my mother. He let me play in the next office over, and what did my young inquisitive eyes find, but a nice big red button, right there on the floor!!! I pushed it, of course as that's what I do, and next thing I know a cop is rubbing my head asking me what grade I'm in. I never admitted to pushing the button outright though.
3 weeks later my uncle approached me (remember, I'm 6): "I heard you pushed a grey button under the desk at the bank last month!"
My response: No! It was red! *busted*