University of Twente Back Online
UncleH writes "University of Twente is back online again, after the University NOC burned to ashes on wednesday. This also means that the much discussed University Campus network is also fully available again. The university already had internet access through a masquerading box in the network of their neighbours. Big hurrah for the network engineers of the University, large applause for the network engineers of SURFnet for restoring the 10Gbps Point of Presence within 36 hours after the fire."
What was the cause of the fire anyway?
The neighbours also have their site in English. They offer serverspace from an old Dutch gold storage facility (nuclear safe building deep underground). As cool as this is their quality leaves something to be desired ;-)
(the error in the last link is theirs, not mine...try it yourself by going to this page and clicking on "qualty".....really...quite funny)
I think the outpouring of emotion is because a lot of us can relate to those network engineers - but not on this scale.
It's about getting a page at 22:00 because a brownout at 17:00 fried a router and it takes you an hour to drive to the closet, it takes some time to figure out the router died, back to the office to set up a replacement, back to the closet, etc. You end up back home at 03:00. I'm not a network engineer, but I've seen some of the stuff these guys put up with.
Can you imagine the chaos those engineers and administrators faced? You have to wonder how much sleep they got over this period. And you can imagine the thoughts going through their heads as they watched their NOC burn down (of course, thoughts of "Oh shit" come after learning that nobody died or was injured - if you're worried first about the equipment instead of human lives, then yes, there's something wrong, but I can imagine the oncoming dread as you learn all your work is gone and you MUST come up with a replacement DAMNED QUICK).
Getting a page or phone call in the evening really sucks, but you can usually go home at the end of the day. These guys probably worked their asses off for days on end.
I congratulate these engineers and admins and offer a virtual beer.
Abolute, superclapping kudos to the guys that had a backup and were able to restore operations so quickly.
Can you imagine how devastating it must have been when the ENTIRE NOC WAS WIPED OUT? Can you imagine how many unemployed people there would be right now if this ever happened to a company and there was no such plan? I hope none of you ever find out, for all your jokes and goofiness.
NOW THAT'S WHAT I CALL DISASTER RECOVERY.
I think with the interesting people, their lives can't possibly be wrapped up into a nice little package.
We're not simply talking emotion here. Yes, I hate that Debian took the hit, but that's not the major issue with which many of us empathize.
Having been in systems administration for a while now (including in a university setting), I assure you that most of us read the story and then wondered, "wow, how would I have handled the situation?" Then, we looked around our offices and realized that if our business (or campus) suffered a fire in the computer area, we would be the ones expected to pull a miracle out of the hat.
I work at a business now that talks big about disaster recovery but won't ever spend the money for us to go offsite and test it. I've often half-jokingly said that our disaster recovery plan is putting a geek on a plane to our offsite location with two things: the latest backup tapes, and a blank checkbook. My fellow geek and I are going to try again soon to convince them to let us test the system. I hope we get to test the plan before we have to implement it.
That's what is so big about this story. Many of us old timers have either been there or have had to imagine being there.
JA
http://www.johnalex.org/
security.debian.org is now hosted at klecker.debian.org. However, according to Debian's security list
Can you guess which installation is going to be hit next? Thus, it's especially fitting that this server is named after the deceased Joel 'Espy' Klecker, the first (but not last) Debian developer to actually die from boredom waiting for Woody to be released.
Hope the insurance at the server farm covers fire damage....
Not for long. The Fedora project aims to bring a Debian-like community of packages and apt-get to Red Hat Linux. We are early in specification stages at the moment, but developers (NOT USERS YET!) are welcome on our mailing lists.
I personally feel that Fedora will be very influential for the Linux community in the future.
Why? Please read my 1st Fedora draft proposal .
Mailing Listsr a-announce
http://videl.ics.hawaii.edu/mailman/listinfo/fedor a-devel
http://videl.ics.hawaii.edu/mailman/listinfo/fedo
of people hard at work to get my email back online! Thank you very much!
/.-effect-building-on-fire joke here]
Pictures
[Insert own
"We live in our minds, and existance is the attempt to bring that life into physical reality" Ayn Rand
Stupid, this is NO LONGER the emergency network. Seen the date on the message? This was on wednesday. Now the emergency network ist no longer in usage, so their is no need to avoid linking to them. Only Asshole here is you, because you are too dumb.
The normal connections are back at the same speed as before the fire. And since UncleH is quite involved with the network around utwente he is perfectly able to tell whether the servers can take a slashdotting again.
The Virtual Bookcase: book reviews
I saw the following on my favorite MUD, Tsunami, thanks to Sitavan:
There was a fire today that destroyed the datacenter at the University of Twente in Enschede, The Netherlands, housing a SurfNet (the academic and research network over there) POP amongst many other things.
A friend sent me these syslog entries which I found interesting... The first one is from a router in Enschede that was destroyed, and the second one is from the router in Amsterdam that it connected to:
0.ar5.enschede1.surf.net 3613: Nov 20 7:20:50.927 UTC:
%ENV_MON-2-TEMP: Hotpoint temp sensor(slot 18) temperature has
reached WARNING level at 61(C)
lo0.cr2.amsterdam2.surf.net 1146: Nov 20 07:20:56.458 UTC:
%CLNS-5-ADJCHANGE: ISIS: Adjacency to ar5.enschede1 (POS2/0) Down,
interface deleted(non-iih)
Tsunami -- You can't bring a good wave down!
Pictures of the sudden demise of our network can be found here: ( hosting from an other uni wasn't such a bad idea after all :-) )
http://tiefighter.et.tudelft.nl/~maurits/fotosite/ index.html
see ya
Maurits