Slashdot Mirror
Tech Magazine Loses June Issue, No Backup
Gareth writes "Business 2.0, a magazine published by Time, has been warning their readers against the hazards of not taking backups of computer files. So much so that in an article published by them in 2003, they 'likened backups to flossing — everyone knows it's important, but few devote enough thought or energy to it.' Last week, Business 2.0 got caught forgetting to floss as the magazine's editorial system crashed, wiping out all the work that had been done for its June issue. The backup server failed to back up."
Then the swearing started again.
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
who needs a magazine?
Maybe not so bad as losing your entire monthly product, per se... but it does happen. I'll bet their accounting, HR, and other back office systems are probably fine. This stuff is always ugliest at the department server level in smaller operations. I'll bet they get some good Mea Culpa 2.0 editorials out of it, though.
Don't disappoint your bird dog. Go to the range.
Some stories should just come with Nelson Muntz sound files embedded.
Ha-ha!
erroneous: look me up in a dictionary
*coughs
TFA: Business 2.0 never had to rely on their backup software until that day, which is why they probably did not realize that it was either obsolete or dysfunctional.
sorry, their MAIN problem is not in any way a dysfunctional backup system. ever heard of verifying backuped data?
I imagine that they still can resemble a lot of it from other files - they should still have all the layout pieces for one, and all the authors ought to have at least rough drafts of their stories on their personal computers. The deadline's screwed, but they can probably get it out a few weeks late (or in July, depending on how often they normally publish).
HAHAHA
No June issue?
That's OK, nobody reads Business 2.0 anyway.
It seems unlikely to have crashed in such a way that a data recovery specialist would be unableto get most of the data back.
But whatever the case - there is a useful lesson here. Make sure your backups are backing something up.
Hold true, once again.
sometimes, nothing.
This reminds me of the recent uproar over a car crash involving the New Jersey governor. He was critically injured because he wasn't wearing his seatbelt, and people freaked, asking what sort of role model he could possibly be. I argued that he was an awesome role model, because sometimes people need to see a mistake end badly for someone else before they'll do what's necessary to protect themselves from making the same mistake. Seeing a high-profile magazine get hit like this can do the same for backup slackers the world over.
I don't know about you people, but after reading this (and giving it the "haha" tag) I'm going home and catching up on a couple of backups I've been slacking off on for a while.
Slashdot Burying Stories About Slashdot Media Owned
As long as those magazines that come in the smarmy black plastic covers still arrive I can't complain.
-m
There aren't a lot of ways for a machine to "crash" that loses all its data. Even a lightning-fried hard drive can have its platters removed by a data recovery lab and many files can be pulled off. A mechanical failure doesn't grind the platters into sand. As a network server it really should have a RAID too. So how exactly can "the server crash" so spectacularly that the RAID, backups, and widely available data recovery services all fail? Did the building blow up?
Tens of IT managers are getting Hundreds of IT minions to check Thousands of backup tapes and befor a senior manager walks in.
In the not too distant future, next Sunday A.D.
the problem was, as always, not the backup. I've rarely seen problems resulting from the backup process. The troublesome process is the restore. Or as a friend put it once:
Nobody wants backups, what everybody wants is a restore.
In my twenty years of IT i've seen several companies making backups like a well oiled machine. The backup process was well documented and everyone was trained to a degree, they could do it with their eyes closed. But everything fell apart in the critical moment, because all they had planned was making the backup. Nobody ever imagined or tried a restore on the grand scale. So they ended up with a big stack of tapes with unuseable data.
Backup is the mean, not the goal.
Regards, Martin
The first issue of Business 3.0.
http://www.nytimes.com/2007/05/01/business/media/0 1mag.html?_r=1&oref=slogin
Since the parent company, Time named "you" the person of the year they were simply following "you" and not doing regular backups
End of line
Huh, I guess I wasn't paying attention to when Slashdot turned into Digg, even though I read both. Here's a link to the original article, rather than what might be a splog. Especially since the article text was copied verbatim.
If they deleted the files in error, then the RAID would faithfully mirror that deletion across all physical disks...
I know that where I work we run a Disaster Recovery (DR) exercise every time we do either a hardware or software upgrade to prove that everything still works. If nothing has changed one is done every second year. We actually pull our off-site production tapes and restore to a new machine that is not in the same city where the current production machine resides. It may be over-kill for them, but, a test of that for them (or any other business) would be a fruitful exercise in that they prove that the backups are good and they can restore from a given point and carry on with a minimal loss of work.
For one of our server apps we actually have two laptops configured with all of the required software and we do restore production data from backups on a regular basis as we use that for our system testing on projects. This happens several times a year so we know that the backup and restore procedures truly work. It is also very cool walking in to the client site, plug in the laptop and show them that in an emergency they have a working machine very quickly. Not as fast as a server, but, it gets them a working machine until the replacement server arrives.
Panic now, beat the rush!
101 Dumbest Moments in Business I think they might want to revise their list. I'm sure I would like to
Wait for OS X 10.5 and "Time Machine".
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
This is my story and I bullshit you not! I work for a manufacturing company, the second largest in its field in the world. Great. However the boss really does not like spending money. We eventually got a backup system using offsite backups (with a special client) and it seems to work ok. However, when it got to 100 GBs I was told to start pruning stuff. So I did. Long and short of it, even with the most important files backed up, we still have most things not backed up. Basically I have almost half a TB of data that I am not allowed to back up because its expensive. I can only backup 5 days worth of data as they are unwilling to pay anymore money for it. The fun will come when someone wants a restore from last year. This people, is the reality sometimes. Me, well, I really dont care anymore. Im sick of having servers, important, mission critical machines sitting on single IDE disks. We sell online, great, problem is our firewall is non redundant single IDE disk. If it goes (like it has in the past) we were down for days, loosing emails, web traffic, web orders, remote ordering systems, EDI data, remote sessions, ftp, everything. DR? the solution proposed by upper management is, oh we will buy some dells and restore. Yeah thats a good idea. After waiting a week for them to arrive, what exactly are you going to restore ? This is more typical than you think, unfortunatly. Im just the guy that has to make do with what i can. No doubt when it fucks up I will be blamed.
http://www.writeitfor.us - Writing IT for the IT generation.
Jerry: I don't understand. Do you have my data?
IT: We have your backup, we just can't restore it.
Jerry: But the backup keeps the data here, that's why you have the backup!
IT: I think I know why we have backups.
Jerry: I don't think you do. You see, you know how to MAKE the backup, you just don't know how to RESTORE the backup. And that's really the most important part of the backup: the restoring. Anybody can just make them.
"I think I speak for most of us when I say .... (Score:-1, Redundant)"
Ha ha!
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
smarmy: adjective ( smarmier , smarmiest ) informal - ingratiating and wheedling in a way that is perceived as insincere or excessive : a smarmy, unctuous reply.
// This is not a sig.
If management is going to brag about cost savings, make sure that you get documentation on their comments and your warnings. That way, if/when things turn to slime, you can put it in your resume that you tried to warn them.
This may be needed for your personal recovery plan. It may also be needed if lawyers get involved and you end up facing charges.
That's the usual fly in the ointment with backups. Backups are difficult and expensive. Management always thinks they can make them easy and cheap by buying some automated solution. But the main cost of the backup is regularly testing the backups... which can only be done properly by doing a full restore... which requires available disk space equal to the size of the backup, and hours of operator time.
Story #1. Fortune 500 company. Lost some source. Big brouhaha. Edict went out: all files are to be backed up to diskettes and the diskettes sent to offsite storage which the management had contracted for with an outside firm. It took a lot of extra time, but people did it. After about two years, an important server with source code for a major product crashed. Developers tried to get the source back from offsite storage. It turns out that nobody at any point had taken any responsibility for cataloging, identifying, or indexing the diskettes. The diskettes might as well have not been labelled: the developers couldn't identify what diskettes were needed, and the offsite storage firm couldn't have retrieved them if they had.
Story #2. Medium-size scientific research organization with a Digital 11/70 running RSTS. Enlightened manager pays operator overtime pay to stay late three nights a week and do backups. Backups are performed with the "verify" option enabled. Tapes are placed in a fire-resistant tape vault every night. But no actual restores are performed. Database (Oracle, in the days when Oracle Corporation's name was still Relational Systems, Inc). is corrupted. A restore is attempted. It transpires that this version of Oracle uses the maximum record length for its files, which happens to be 65,536 bytes, and the Digital-supplied backup-restore utility... you guessed it... has a bug with records of that length. Yep. Writes 0 bytes, verifies 0 bytes.
Story #3. I worked at a place that recommended that individual developers perform individual backups using a cartridge tape system and some standard PC software. I set it up. There were two "verify" options. One used the cartridge system's read-after-write feature to read every block as it was written. The second performed the entire backup, then verified the entire backup in a second pass. Took twice as long, of course. I opted for the second method. The problem was: more than half the time, the verify would report one or two errors. And for some reason, probably efficiency of use of the tape, it didn't write file by file, it munged them into blocks. And it didn't even report the names of the files affected. Just "2 errors were encountered" or something like that. So, when that happened, I didn't see that a rational person had any alterative except to perform the whole backup again. And more than half the time, it would report a couple of errors the second time, and...
When I asked colleagues about this, it turned out that I was the only one ever to have picked the second verify option. Everyone else had picked the read-after-write-verify option, "because it was faster."
And told me not to fuss because "if it was only a couple of errors, the chances they were on files you needed to recover was too small to worry about."
"How to Do Nothing," kids activities, back in print!