Slashdot Mirror
Happy World Backup Day
An anonymous reader writes "Easter isn't the only thing some people are celebrating today. Today is also World Backup Day. What steps have you taken to be able to resurrect your data, instead of having it go to eternal oblivion?"
I've committed every one and zero to memory.
Operation Guillotine is in effect.
World Restore Day?
Wait a sec. I should think it would be "Restore" day. At least for those of the various Christian persuasions.
Faster! Faster! Faster would be better!
I always buy hard disks in Pairs and Raid them, each disk has a back up.
I've made sure the phone number for the local data recovery services is taped to the side of the server.
Build a man a fire and you warm him for a day. Set a man on fire and you warm him for the rest of his life.
Automated incremental backup of the headless servers at home, every two days (and I check the backup logs regularly). The backup disks are cycled every 4 weeks: the existing set goes to an insulated box in the garage (a separate heated building), while the previous disks come in and start with a full backup. Our 4 workstations at home all get backed up to local USB disks, but these are merely for convenience - important files are always kept on the servers, where they belong.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
Comment removed based on user account deletion
I don't like the mess, so I group the ones and zeros sequentially.
-- I ignore anonymous replies to my comments and postings.
I put all my data in a cave and sealed the entrance with a big rock -- but three days later it was all gone.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
Time machine requires about zero maintenance and will help me recover quickly if my main hard drive dies. Of course, if a fire or theft results in the simultaneous loss of the backup drive as well, I'm out of luck. So for data that's worth spending a little extra time securing, checking it in to an SVN server works for me.
I don't care if it's 90,000 hectares. That lake was not my doing.
The First Three Rules of Computing
1 - Backup
2 - Backup
3 - See Rules 1 & 2
Of course, I have no idea how to backup a world. What kind of media would you use, cosmic string recorders?
Aha, the trick is, only the ones actually carry information - that's when the bit is actually holding a voltage. So, you can compress out all the zeros and get a roughly 2:1 saving on space!
The only downside is that for decompressing, the codebook is necessarily rather large, in fact the same size as your original data. But the compression works well and it's fast!
I've committed one and zero to memory.
I'll be able to regenerate all the data using just those two numbers.
Sleep your way to a whiter smile...date a dentist!
Amazon Glacier is supposed to be pretty cool for long term archival. It's cheap per gigabyte, but the caveat is that there is a wait time to pull your data out of their archives, so it's not suitable for something that needs to be online immediately. Haven't tried it quite yet, but the idea makes sense to me. https://aws.amazon.com/glacier/
That is all.
Mission: To provide products that consume time and energy as entertainingly as permitted by the laws of thermodynamics.
I use crashplan's client and all my pcs backup to my home server. All my photos get backed up to a second disk that I keep off site. Their software works pretty flawlessly. I was going to use their cloud service but photo and video data would take months to seed.
I managed to go 16 years in the IT world, first as a sys admin and now up through an awesome mid-level management position, without any serious data management scares. (And by 'awesome', I mean I work for demoralizing leadership and I've hit a glass ceiling which will force me to go find another company to work for if I want any shot at career advancement.) I've always made sure there's many, many layers of redundancy and good processes in place.
That was until three weeks ago.
We use Microsoft DFS to sync data between two sites. Because of some other things going on, we had to turn DFS off for 3 weeks. We thought we had everyone transitioned to using the "master" file repository, the one that gets backed up every night, etc, etc. The day we turned on DFS back on, all hell broke loose.
Oh - and this is fairly important stuff: 10 years worth of CAD, design, and legal paperwork. It's a few terabytes worth. For our medium-size company, this is basically everything that we hold near and dear.
The first thing that happened is DFS completely puked and completely trashed BOTH filesystems. Fantastic, Microsoft - what a wonderful piece of shit DFS is. Fairly quickly we had to face some data integrity issues. First, we discovered apparently there was a fella at the remote site that was using the copy of files there. Great.. through a fairly manual process we were able to retrieve most of his changes to the dataset. Next, we fairly quickly gave up on trying to fix the DFS - on the advice of Microsoft it seemed to be fairly hopeless.
This is where shit gets real.
Our head sys admin had been troubleshooting an issue with a drive in a RAID'ed NAS backup device had failed. All the other backups had been shifted to other NAS devices, but that backup was so large that it apparently had just been failing. While looking for that, we also discovered the quarterly backup from December had failed (that's the point where I wanted to put on my manager hat and go rip someone a new one, but decided that probably wouldn't be the most productive thing at the moment and could save that little teachable moment asskicking until after we were out of the woods.) Now, the sys admin hadn't been completely foolish, before turning DFS back on he had run some full backups using a different NAS device.
In a f*cking brilliant stroke of disastrous luck, when we went to perform the recovery we discovered that RAID array on the backup NAS device also had corruption.
Now, how bad the corruption was and what exactly that meant remained to be seen. The backups had completed without error, it was the NAS filesystem itself that was throwing the errors. The NAS was still running and our backup software seemed to recognize the backup catalogs on it. Ok, other than what seemed to one potentially corrupt backup, it was seeming like the next best case scenario was a quarterly backup from September, and I was also staring a complete set of disks from 2010 dreading the thought of bringing them back online. Well, with nothing to do other than try a restore, we pressed the button.
That's when I went home mid-morning, chainsmoked four cigarettes on my porch and wondered what would happened if everything went south. In other words, I was contemplating my next job.
'Lo and behold, and restore worked. We had to merge all kinds of things back together to get a complete copy reassembled, then we still had to get DFS working (which took four days of syncing over the WAN.) When it was all said and done, it looked like there were just two files from one set of changes that we couldn't recover.
I think I'll go double check on the backup jobs now.
----- obSig
Amazon Glacier has really changed my backup strategy since this time last year - I now push all my own, generated content (ie: pictures, documents, things I could never get back if I lost everything) up to Glacier using the free Windows client, Fast Glacier. In February I was charged $0.13 by Amazon for storing ~8Gb of data. I tend to push new content up as and when I create it (for example, after I process holiday snaps, or get back from a day out).
Day to day file changes are now handled by Windows 8's File History feature where my changes are pushed to a small NAS (Dlink DNS-320) in my shed (technically off site?) over a Homeplug AV ethernet link. For added security I use the legacy Windows Backup application (still present in Windows 8) to create ~ monthly snapshots of the system which I store on a 320Gb external HDD. This drive is one of two which go back and forth between my parents house each time I got and visit. These disks are encrypted using Microsoft Bitlocker drive encryption.
I should get around to properly encrypting my NAS in the shed, I've been looking at encfs.
At least we've gotten off of dialup.
Faster! Faster! Faster would be better!
LTO5 Drive - 500€ (these are getting cheaper on ebay)
LTO5 tape - 35€ (shipping included, buy more / lower per unit price)
1.5TB per 35€ (23€ per TB)
4TB Drive 200€ (50€ per TB)
but to do backup with HDDs you need two so
4TB == 400€ (100€ per TB)
when hitting the I will need 8TB storage space soon, you do the math and will realize that tapes are better.
vacation photos -> it's part of your identity
No need to do anything. When disaster strikes just wait three days and it simply restores itself. Shortly afterwards the data ascends into The Cloud and becomes available forever and ever. Halleluiah!
I never trusted "cloud" backups but recently I looked into Amazon Glacier - and now my personal backups are stored with "eleven nines" reliability, encrypted, and with price roughly 10 times lower than services such as Dropbox or Google Drive. No affiliation with Amazon... but the question was "how do you do it" so this is my answer.
I've been using it for a few months, using CloudGates.net to transfer data to it (the SimpleAmazonGlacierUploader java applet had a bug in it that affects larger files - not sure if it's been fixed yet). It's pretty great - I have 136 gigabytes of data at the moment, so I get a bill for $1.36 each month. For the money, the hassle of building a server to put at a friend's place isn't worth it, and I couldn't find any other backup solutions that are as cheap. Yes, directory listings and downloads take a few hours, but... if my house burns down and I'm recovering this data, I'm not going to mind a few hours' wait.
my one's and zero's are just like my women.
I have binders full of 'em!
--
"It is now safe to switch off your computer."
I embed the most important data in Bitcoin transactions, and let the geek world mirror the blockchain.
Escher was the first MC and Giger invented the HR department.
I use rdiff-backup on each of the machines I administer (my machines and those of my wife, at home and at work, plus laptops). rdiff-backup is nice because it saves the current snapshot as a directory that looks exactly like the one being backed up, so restoring stuff is really very trivial.
The backup scripts run daily, backing up to the home directory of the user (a /home/$user/backups directory) so that casual deletion means at most a day of work lost. I rsync all those backup dirs weekly to one of three 1TB drives. They are about 60% full each.
The three of them are rotated arround. One is next to me, one in the basement, and another in a drawer at my office. They get rotated every week.
Seems pretty solid to me. A lot has to happen to leave me with a serious data loss.
RAID!=Backup.
RAID=REDUNDANCY. Clue is in the name.
Operation Guillotine is in effect.
Thanks a lot for writing up this suggestion. I had no idea Amazon Glacier was only a penny per gigabyte, and thus a realistic way for me to backup virtual machines offsite, finally, (using only my available slow home upload bandwidth). Which got me to Searching on the net...
CloudGates.net does indeed look like a useful service.
A Search engine lead me to a free Windows client called FastGlacier http://fastglacier.com/faq.aspx
This technote from 'AWS Blog' explains how to use the more standard and better documented Amazon S3 Data buckets to automatically offload data after a specified time to Amazon Glacier storage. The trick is to create a lifecycle rule. I'm inclined to try this, once I get myself better organized, although CloudGates also looks very worthy. Kudos! http://aws.typepad.com/aws/2012/11/archive-s3-to-glacier.html
Happy World Backup Day!
You can't be ahead of the curve, if you're stuck in a loop.