Slashdot Mirror
Ask Slashdot: Best Medium For Personal Archive?
An anonymous reader writes What would be the best media to store a backup of important files in a lockbox? Like a lot of people we have a lot of important information on our computers, and have a lot of files that we don't want backed up in the cloud, but want to preserve. Everything from our personally ripped media, family pictures, important documents, etc.. We are considering BluRay, HDD, and SSD but wanted to ask the Slashdot community what they would do. So, in 2015, what technology (or technologies!) would you employ to best ensure your data's long-term survival? Where would you put that lockbox?
... have always worked for me.
still the most reliable, and it's offline storage, and can be stored in an offsite safe
BluRay is hard to beat. The discs are durable and not worn by use. The drives are cheap and will almost certainly be available in 30+ years time (like you can still buy drives to read CDs), and the filesystem will be readable.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Personally, I have three external hard drives encrypted with TrueCrypt that I rotate and keep in a fire safe at an offsite building. I rotate them monthly. Cost is a little high, but it fast, easy and convenient for me. Your circumstances are likely different enough that you will need a different approach. But generally, my archive set is large (3+TB) and sensitive (taxes, bank statements, account numbers, passwords, etc) so this solution works best for me.
ASCII tastes bad dude.
Binary it is then.
Every year, I just back up my files to an external hard drive and put it in my safety deposit box in the bank. If my house burns down, I still have all my photos (long since scanned in all my old film stuff), documents, and even music. I've got the last several years in there so it would take three or so drives not working to really lose everything (after I lost everything at home). Usually I spend a little extra money to make sure I have small external hard drives that don't have wall worts to power them as they'll fit in the safety deposit box easier and I won't have to keep track of the wall worts either. In the past, I suggested my parents do the same with a flash drive and my father scoffed when I mentioned keeping on in the safety deposit box. Of course, his computer got hit with the encryption malware and they lost everything including the flash drive we back up everything several years earlier because they can't remember where it might be.
It's an open format, so its usability will penetrate deep into the future.
I've already converted my entire porn collection to Base64 encoding, and printed it out on archival paper (acid-resistant for obvious reasons); I've grown so used to it, that sometimes the alphanumeric text is enough to make me extend my coffee breaks.
I just tell people the boxes filled with reams of paper are my late grandfather's WWII anti-NAZI code-breaking attempts.
I have a decent sized file server at home, and basically have three backups:
- a complete mirror (yes, I build a second file server of equal capacity, though of much lower spec)
- a set of two external hard drives. I keep one local and one offsite and periodically swap them (every few months or so). I back up my irreplacable stuff on here
- an encrypted mirror on a VPS (using duplicity) of my home dir (small amount of data which changes frequently, I keep a month worth of daily snapshots)
I feel this provides adequate protection for my stuff:
- the file server is raid6, so I am protected against drive failure
- if that fails I've got a recently synced mirror of all my stuff
- if that fails (due to something bad that gets synced) I've got a local drive with a recent copy of my irreplaceable stuff
- if that fails (due to say, my house burning down) I've got an offsite drive with an older copy of my irreplacable stuff, plus a recent copy of my home dir on a remote server
I happen to like 2TB internal laptop hard drives (2.5").
Pros:
-High capacity
-Small form factor, will fit in most safes / lock-boxes
-Slightly more shock resistant than 3.5" drives.
-Fit my hard drive dock/drive duplicator
Cons:
-Slightly pricey because of the large capacity
Keep the anti-static bag it comes in and toss a few zip-lock bags around it for a little bit of water resistance. If the data is worth anything to you, keep a local offline archive and one at a friend's house. If anything sensitive is on it, pick your favourite encryption (truecrypt is still my goto).
Got to have 2 different media in 2 different locations away from the computer to be fool proof against all the vagaries of time. HDD is hard to beat and cheap. Blu-Ray is good and the newer denser disks coming out might work out OK, but we won't know for years. Hence, LTO tape has lots of adherents.
Your best bet is to pick more than one. You have a better handle on your needs and recovery point objectives than anyone here. Pick two (or even three) strategies that fit your needs and utilize both. Finding out you picked wrong usually happens at the very worst moment. Duplicating your efforts adds an awful lot of cushion.
Solving Unix problems since 1989...
It almost doesn't matter as long as it's more than one medium, stored in more than one place. I keep copies of everything on HDDs (and sometimes tape) here at home, but also copy the most vital stuff onto 3.5" magneto-optical disks (Fuji DynaMO -- they never caught on but they've been super reliable) and keep that in a safe deposit box at the bank. $25/year is pretty good for getting my life's work back if my house burns down. If you do choose a removable medium, make sure you keep a spare drive too. It'd be a shame to have pristine media you can't read.
I have an MDisk writer and three Blu-ray version of MDisk media. I have had them for months but embarrassingly have not broken the shrink wrap. Anyone else have any first hand with this stuff?
Um, no. Blu-ray is more scratch resistant than DVD. And the best storage for long-term backups is always off site.
Code, documents and pictures --> Printer.
Videos --> DVD
Music --> CD
Other --> USB Drive
Put the physical items in a waterproof bag.
Put waterproof bag in strong box.
Dig hole in backyard with kids.
Put box in hole.
Cover box with dirt.
Cover dirt with young tree or other large bush bought at local gardening store.
Come back twenty-five years and dig out treasure.
This may be my last time, as I am now using AWS Glacier for all our personal photos/vids and that accounts for about 100G of the total 150G in content. I also copy everything to a mechanical USB drive and store it at my office. I think AWS + USB will cover me
Our safe is in the basement, on a high shelf. Keep in mind fireproof safes need a disecant as you *will* get moisture in them. Or use zip lock baggies.
I wouldn't put anything in a lockbox. Such media will be tested very rarely, and when they do fail, it's likely you won't know until it's too late.
I'd rather use a hard drive, hooked up (NAS or mini-pc, maybe) to a network and capable of rsync. You could place it somewhere in your home, or, if available, another secure location with Internet access. Run daily or nightly automated backups.
I use M-Disk. It is a DVD disk that is made of basically indestructible material. It is certified by the DoD as a medium of over 1000 years lifespan. Here is the link. http://www.mdisc.com/
Remember that magnetic and other computer media needs a higher level of lock box protection if you are thinking of heat/fire. Believe it or not the computer media can get damaged and rendered unusable at lower temperatures than the paper will. http://en.wikipedia.org/wiki/S...
The first problem, you don't said how long you want to preserve the data without transfering it on another support!
Because, the longer you don't use the support the more you have these problems :
with HDD, the mechanical part, even when not used risk to jam. Happen for me to a HDD stored in a safe in a Bank
with SDD and other flash drive, especially when not in use, the data (electric / magnetic gate) evaporates
With opticalDisk, except some old cd made in real gold, the data will fade aways is in contact with light.
with magnetics tape, the problem will be the same as the flash drive, the magnetics elements will evaporates.
And with all these technologies, you will need the hardware to read and connect them.
You today are able to find a computer with a 8 inches floppy drive or even a 8 inches floppy drive or a computer that have the ide connector to connect the 8 or 5 inches floppy drive ?
My solution, is to backup and copy often. I transfert my backup support every year or two to a new kind of backup support. First tape and CD, then DVD, then BR, Then HDD, this year TB SSD are cheap enough to be in my near vision for the next backup. And I keep my older backup.
Ceci n'est pas une Signature !
You copy your hard drive every 3 months, but didn't recopy your optical disks for 15 years. And to you, this is 'proof' that HDD are superior?! OK, got it. I suggest reburn every 5 years on optical. You get a fresh disk and a chance to consolidate CD to DVD, DVD to BD etc.
The USB 3.0 sticks are pretty fast and 128GB sticks are getting cheaper all the time, with cheap 256GB units on the horizon. They are light, small, have good retention, and make it easy to divide your data types into separate physical units so if you only want to retrieve the family photos you don't need to pick up the tax returns and such as well.
'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
The first guy who suggested stone tablets did....
He didn't really ask for cheap. He asked for the best chances of survival. Unless the lockbox is airtight and humidity/temperature controlled, your cheap DVD media could degrade over time.
The jury is out on how long it would take for optical media to degrade to the point of data loss, as many tests seem to yield varying results depending on the quality of the media used, but depending on the kind, (writeable vs rewriteable, etc), the general consensus seems to be that the survivability of most optical media in average room temperature and humidity is several decades, (citations obviously would be handy, but I'm not up for finding them). Archival grade optical media is your best bet if you head down this road, and it's not necessarily cheap.
Flash media may be even more volatile than optical media. Without power, minute leakages over time will lead to the loss of data, possibly within months or just a few years. (Again, citation needed.... I have the flu, so research this yourself to confirm... this is just a guideline for investigative consideration).
Tape media may still be a good bet, and probably better than magnetic HDDs. Tapes are small, store lots of data, and are pretty resilient.
I wonder how long they require/expect the data to last for? Years? Decades? Generations?
There are two types of people in the world; those who believe there are two types of people, and those who don't.
I use Amazon Glacier with FastGlacier Pro for the non-confidential stuff (photos, docs, etc). http://fastglacier.com/
It costs me about $1/month for 100GB of storage, does differentials on a schedule, and I generally don't have to think about it (which is the best type of storage).
The problem with media is that data tends to degrade and technology tends to move forward. Who says you are going to have an optical disk drive in your computer in as little as 5 years? Still its an option. Maybe instead of asking what's the best, maybe you should focus on multiple options. First and foremost, make copies of everything. Take one copy to the bank and have a safety deposit box. It's old school, but unless the bank is robbed, or maybe a fire your documents are safe. Take another copy and get one of those fire lock boxes they sell at the hardware store. I don't know how good they really are in a fire, but its gotta be better than that one desk drawer they are stuffed into now. Finally upload everything to the cloud. Pick a good cloud...or multiple clouds. Encrypt your data.
Get a transporter and put it in a friend or family member's house.
http://www.filetransporter.com...
Supposedly it'll sync all your files automagically.
And you can host your transporter, too.
http://transporterhosting.com/...
don't know much about it except that it works.
For me, I have a NAS at home with mirrored drives, if one goes down, the NAS emails me to let me know. I also keep multiple copies besides the NAS, one copy on my personal laptop, one on my wife's and one on the desktop computer. About the only thing this doesn't protect against is the house burning down, but if that happens, my .mp3 collection is the least of my worries at that point.
Frank said: "Store your data on Ridulian crystal paper, tightly packed in a nullentropy capsule."
I know you said you don't want them in the cloud, but why not? Well encrypted files are quite unlikely to be in danger of decryption, and storing them on multiple cloud servers, where data loss is an existential threat to the companies maintaining them, seems an overwhelmingly successful strategy for ensuring the survival of the data.
Or something like duplicati which lets you have the benefit of encryption without the downside of having to upload your entire volume every time you want to update it. There's probably a security trade off (I don't know of any specific attacks, but I assume a single encrypted volume is probably more secure), but to me it's worth it for the convenience.
There are multiple.
1.) simple sftp reachable cloud (very cheap, no need for special cloud software)
2.) root server (you can get these with ~4tb space minus the OS) for arround 20â (you don't need a power horse) even a vserver is ok, if it has the space and you can sftp into it
traffic is mostly multiple times storage cap
3.) having offsite sftp storage is great the connection+login/pass is encrypted + save the ssh-fingerprint and check for manipulation, only filezilla needed to retreive all data
otherwise you can automate many things like offsiting your archive hashes / etc.. via script, curl can pop3s and smtps your data.
With tcls "expect" you can automate sftp backups and automatic storage.
4.) Multiple independend hosts are also a nice idea
5.) -> Test your backup system Encryption
I suggest everything you store offsite and also onsite should be encrypted.
Everything you store offsite should have at least two layers of strong encryption using different crypt-algorithms
1.) AES or twofish/serpent
2.) additional to that
That big archive should be split into 50-100mb chunks of data and encrypted with their own hash hash value as key value
you need to store the hashes
3.) encrypt the hash lists
4.) store them in cheap or free email accounts
I'm not the first and probably not the last to suggest you take a look at M-DISC. http://www.mdisc.com/. Also, with any optical disc storage you want to make sure to store them vertically. Gravity can do surprising things when given enough time.
If the documents are that sensitive, you can run the files through multiple encryption schemas with different and very hard to crack pass-phrases, before sending them to the big drive in the sky. Yes, someone coming up with perfect quantum computer and running down all encryptions to the ground, in a matter of seconds is always a possibility, but I think I can take that much risk over, losing my important documents in a house fire or earthquake or flood. Take your pick. I personally can not see myself walking to a safe deposit box every week or even month to store the latest copies of my documents, burned to a DVD or BlueRay myself.
__________
The more I know people, the more I love animals
The main thing is to keep multiple copies in multiple locations as easily as possible. I have all our important files on one drive at home. This drive is auto cloned each day to a different drive using Carbon Copy Cloner. About once a month I copy the newer files to an external hard drive. I take this drive to a different location where I copy the files to a different drive. That drive is also cloned each day using Carbon Copy Cloner to a different drive. In addition, all the contents of that offsite drive are copied to the cloud using CrashPlan. A separate copy is also copied to the cloud using another data backup site. Everything is pretty much automatic except for putting the files onto the external drive and moving them to the other location via sneakernet. I'm sure I could come up with an automated way to do that as well, but frankly I haven't felt the need to do so in all the years I have been doing this.
I have DVDs that I've burned as a teenager kept in a nice, high-quality soft "archival" binder for the last 18 years. Nearly all of them, of varying quality/expense, are unreadable due to degradation.
OTOH, I've got old 500MB harddrives that read/work just fine and are just as old. I'd expect sealed HDDs to be as good as it gets - tape is nice, but maintaining a supported/working tape drive was always difficult (used to have one). But, unlike every other type of storage, harddrives are actually capable of warning you of an impending failure. (I've been *saved* by S.M.A.R.T. at least twice, over the years.) Add some rudimentary RAID, and you're probably good. The only way I can think of to go further is to use two/three, and cycle them between your PC(often/all the time), a nearby firesafe(When you are heading in that direction), and a safety-deposit box (seasonally?).
Why can't all fpga/microcontroller manufacturers just release free optimizing compilers???
Comment removed based on user account deletion
There is no -best- medium:
Paper is always readable, but can be easily destroyed by water or fire, and stores the least amount of info per size unit than anything else.
The cloud will be present barring SHTF, but there are the security issues [1], so it needs encrypted via the endpoint.
Tape is an archival grade medium, but the drive is expensive ($3000+), it requires a fast computer to prevent shoe-shining, and either requires a program for backups/restores, or one can use LTFS to have the tape appear as a hard drive. (This route, one can use LTFS or even just tar to stash a copy of the backup program and its keys for install, then install/use the program for the rest of the tapes.) Tapes can be physically set read-only so malware can't tamper with contents. One can also buy WORM tapes that further guarentee protection against data modification.
External hard drives are cheap and easy to use... but are not an archival grade medium, can fail, and can be zapped by malware.
Optical drives can function well as WORM media, and are inexpensive... but their present capacities are minuscule (25 GB is the best bang for the buck price point, although the next gen Archival Disc format may actually make optical media viable again for backups.) If Sony and Panasonic can make AD drives and autochangers [2] at a price point well under LTO 4-6, they may just have a major untapped market. Sony does have high capacity optical disk drives... but they run in the $6000-$7000 range, so hopefully this price will drop by large amount once mass produced.
SSD is decent and fast... but it is nowhere near permanent (those electronics will bail the gates eventually), and once the data is lost, it is gone for good.
My take: I use various redundant media. Critical files get burned onto Blu-Ray media using Nero's SecureDisk or DVDisaster (for error checking/correction), stashed in an encrypted container. I also periodically buy a large external HDD, copy everything from my machines onto it, let it deduplicate, then copy all the stuff from the normal backup drives onto the volume as well. With deduplication, this doesn't use up that much space.
[1]: Never know who has access to the files, and the provider can go bankrupt at any time, allowing the next owner of the physical servers free access to the stored data without any legal ramifications whatsoever. In fact, one cloud provider even has it in their TOS saying that the next person owning their firm gets all data free and clear.
[2]: You used to be able to spend a few C-notes on a 400 disk CD changer. An optical silo holding 400 disks isn't much different, so with the 300 gigs promised this year per disk, that gives 120 terabytes of WORM media in 3-4 rack units
I don't trust any kind of cd/dvd/BR for archiving my stuff. I back up to hard drive, detached from the system when I'm not backing up, and I cycle the hardware every 1 - 2 years, because hard drives don't last forever either.
Hard drives are so cheap these days that backing up to traditional backup media just doesn't make any sense anymore.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
Get a large-capacity, multi-disk drive housing and set it up as a mirrored RAID. Over time, as each drive fails, all you have to do is swap out the failed disk and the RAID will re-mirror the data to the new disk. This is the most robust perpetual storage option. It is possible that the magnetic fields on the disks can fade over time if left in an unpowered state. The biggest downside is that the RAID is onsite, and if there is a catastrophic event such as a fire or a flood, the drive could be destroyed. A RAID is what I am currently using for my longterm, permanent storage needs. I've lost a couple of files to bit rot, nonetheless.
Optical media storage is decent for long-term storage, but there is evidence to suggest that these disks break down over twenty to thirty years and become unreadable. So, bit rot is still an issue with optical media. However, they offer the advantage of being able to store the data in a stable medium without any power required and are easily portable to safe locations. Also, newer disk technology developed in recent years is more stable than older disks.
At the moment, I don't consider SSDs as a reliable long-term storage. All it takes is one cosmic ray to flip a critical memory cell and your drive becomes unreadable. Also, when disconnected from power the charges fade over time and in as little as a few years the drives could just erase themselves. The advantage they offer is being small, lightweight and easily portable.
The important thing to remember is that technology is changing all the time, and there are newer and better alternatives on the horizon that answer the shortcomings of each of the above solutions. The biggest problem with any long term storage is bit rot, where random bits get flipped or erased over time. Storage technology companies are striving to improve all the time, so the choices available to you will also continue to improve.
Whew! This water sure is cold!
For anything that can be printed, print out a few copies on archival paper using an appropriate printer. Have photos professionally printed on Fuji Crystal Archive or better paper.
Unlike anything digital, we KNOW that paper will last several hundred years with only basic care.
Also, make more than one copy and store in more than one place.
2 External HDs and Blue Ray/dvd disks.
You keep one HD, you get other HD to your mom/bro/close friend to store for you.
Encrypt all data as files, not hd encrypt.
Put tool used to encrypt data on disk with data blobs.
Remember your own password/keys, don't put those on the drives.
Things that don't change like pictures also go on Blue ray or DVD in case grandma wants to see them.
I also have close family friends we exchange family picture backups.
Update at least twice a year if possible.
1 house fire and you lose everything so keep that in mind. Fireproof safe would be nice but I do not think the heat would leave much working either way.
Cloud services only work as long as you pay for them and someone knows where to find them if your hit by the plane that crashed into your house.
There are a number of services that will store your data for you and it's well encrypted. Unless you are a confirmed cheapskate like me, and don't mind the headache of actually performing backups, encrypted online is the way to go.
If online doesn't float your boat or if you really think somebody will be interested enough to break into your stuff, then do what I do. I first back everything up to my NAS, then I backup the NAS to three hard drives. One drive is stored at the in-laws house and gets swapped every time we visit, One is stored in my file cabinet and gets swapped every weekend with the last one that stays in the server and gets a snapshot backup every few days. The NAS is built on OpenMediaVault and has a Raid 5 array with a hot spare. The NAS has a SATA drive bay that I can just push in a bare drive (2.5 or 3.5).
I use "rsync" to push all my data from the various windows and Linux boxes I'm backing up automatically to the NAS, so I end up with a minimum of three copies of all backed up data (the original on the original host, and two in the RAID array) I'll have four copies once the NAS is backed up, and FIVE copies once we visit the In-laws. Overkill, I know. However, it's worth it given that I happened to loose a few years of pictures once and my wife about came unglued. I'm NOT doing that again.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
If you are looking for long term storage, that is what M-Disc is made for.
http://www.mdisc.com/
Writable DVDs and Blu-rays use an organic coating that degrades over time. M-Disc says you only have up to 7 years of reliability before you start loosing data. A pressed CD/DVD will last up to 100 years, but I've had pressed music CDs that the media layer burned from very little use. Also pressed CD/DVD/Blu-rays are not practical for backup.
Hard drives are designed to be spinning. M-Disc claims hard drives are good up to 5 years, but if you don't spin them up every once in a while, they can fail in less than a year.
Flash Memory they say is up to 8 years. Again, if it isn't powered up every once in a while you can only be sure everything will be there for 2 years. If it is an SSD, your information can disappear any second. The SSD will still work fine, but they sometimes just loose everything on them.
Backup tapes have been the tried an true form of long term backup, but even those, people suggest having at least 1 backup of the backup as magnetic material degrades over time.
Microsoft, Apple, Google, Amazon what's the difference? All steal money from devs and control with walled gardens.
One thing I should mention about SSDs is that I have a couple of thumb drives that have been sitting around untouched for years that are still usable and the files are still readable. The only one that doesn't work is one that was sacrificed to a very powerful magnet in a demonstration of how vulnerable flash drives are to magnetic fields. So, perhaps an SSD drive will be more stable than I suspected.
Whew! This water sure is cold!
You might wish to consider using a hard drive cloning tool (drive imager) that makes backups of the entire hard drive (including the OS) to save time and effort. These programs are capable of automatic scheduled backups to make things easier on you. An external USB hard drive is typically the medium of choice for these types of backups since hard drives and USB 3 enclosures are so cheap and portable. Some software versions let you do incremental backups (since the last full backup is made, only files that have changed get backed up) and there is often cloud storage too (which I personally do not trust). Anyhow software like Macrium Reflect (which has a free version and is excellent) or a paid for version like Acronis True Image or others is worth examining. YouTube is full of tutorial videos on how to use these programs but they are very easy to figure out on your own. Now, you will need to make boot medium too (CD/DVD/USB key) too don't forget and it's a good idea to include an ISO image of the boot medium on the storage hard drive too in case you need to create bootable medium once more in a pinch (you lost your original boot medium for example). Why I suggest using these programs is that when restoring after a failure of a hard drive or what have you, it only takes a fraction of the time required to get back up and running to a fully working state (all your programs and customizations are working just how you like them at the time of backup). These programs are capable of restoring file and folder level restoration or entire hard drive contents. Since hard drives are so cheap, saving complete drive images makes sense and you can have several terabytes of data compressed down down into something a lot smaller on the hard drive allowing multiple images to be made (you can even span them over several drives if needed). You can also encrypt them for safety. The only thing iffy about going this route is that OS vendors deprecate their OS's over time (like Microsoft has done with XP) and perhaps the hardware of the future is not able to load the drive images for some reason. But, if you were staying diligent and doing backups at least once a month or more (and keeping your software version current), you'd most likely be fine since you'd be deleting old images and replacing them with current images.
It really doesn't matter what media you select. The important part is your schedule of updating. Ex.. You pick optical discs and you burn a new disc every 3 months and store it securely. The important fact is not the storage media, it's the 3 month schedule of updating it. You can switch media out on your 3 month update schedule, use flash media, external hard drive, tape, whatever. As long as your most recent backup is no more than 3 months old you'll still be able to read it.
Where most people fail at this is the schedule.. And that's why an online backup solution would work better for the majority of people. Schedule your online backups with one of the secure vendors and let it run automatically.
Comment removed based on user account deletion
This is the number one question asked on Ask-Slashdot. In my recollection, I believe this is at least the 7th time I've seen it. The answers are always unsatisfying. Such is life.
Here's a related question I've been wondering about. Assuming that cloud storage is used as part of a solution to archive personal data, what are some easy tools that can do strong encryption on file sets such as a directory tree? 7-zip looks like it may be a good choice, but is there something better for that?
It would also be nice if such a tool automated the upload/download process to/from the cloud, was open source, and was easily to compile on a variety of systems (yes, including Windows) in order to reduce the possibility of any back doors.
By far the best method to ensure long term survival of your data is to send it out in interstellar space embossed on a golden record like the guys from NASA did with the Voyager probes. Now, you didn't ask about retrieving it did you?
External HDs are cheap these days.
Set up a robocopy script to backup to an external. drive Periodically backup to a second external HD.
Periodically cycle the external HDs into your safe-deposit box at the bank.
Accept that every few years your external HDs get cycled out due to age.
Don't try to make some permanent archival solution which will rely on technology in the future working ... keep them active and in the air. Two local copies, and possibly as many as two remote copies.
I think your specific medium over the long term is less meaningful when you can buy a 3TB external HD for under $100 .. especially if archiving those files actually is valuable for you.
Nowadays, it seems like redundant, offline backups for stuff you deem important enough is fairly easy to do.
The advantage of a robocopy is it will only copy what's changed, so your static data doesn't add too much.
Lost at C:>. Found at C.
I have DVDs that I've burned as a teenager kept in a nice, high-quality soft "archival" binder for the last 18 years. Nearly all of them, of varying quality/expense, are unreadable due to degradation.
Same experience as this guy. While I do have a handful of CD's that were written 15 years ago that're still readable, I have a ton of DVD's that have degraded and caused loss of data that were written less than 5 years ago.
Optical media, at least the writables we as consumers have access to are completely inadequate for long term storage. No comment on BluRay consumer writables however, as the loss of data from using DVD's for archival really turned me off of optical media.
Magnetic tapes or sealed HDD's are probably the best bet. I am currently using a pair of 2GB external HDD's for my long term archival of data, which I mirror periodically, keep one in a safe place (usually at least) and the other is my working copy. When one fails, I will replace it with a similar HDD. This to me is the easiest solution, and resilient enough for my needs as well as portable.
CDs/DVDs. No LPs, though :-[p
Why can't all fpga/microcontroller manufacturers just release free optimizing compilers???
Most fire safes are designed to keep the contents at less than 450F for a certain number of minutes. That's based on the temperature at which paper bursts into flames. Media such as tape and DVD will be ruined at 200F or less. So a 30-minute fire safe might last ten minutes with DVDs in it - your data will be gone before the fire department arrives.
A safety deposit box at the bank is cheap. You can also throw a USB drive into your office drawer if your office isn't at home.
As someone who spent the first five years of his career putting hard disks in freezers and all the other old tricks to get data off crashed (and not backed up) client disks I eventually came up with the following perls of wisdom.
1. Archive and Backup are the same thing and its name is called Constant Data Protection (CPD). There is just too much data to do a traditional backup and there is no point in archiving data like it will never change anymore. Maybe with a 6TB HDD you could go back to just doing a start to end backup of all your data to a single media but if that is the case then you don't need to follow any of the advice in this thread.
2. Whatever setup you choose make it simple. Simple tends to work. Not simple tends to not work so well. Also I have noted that much data loss happens because the backup procedures stopped getting followed because nothing bad happened for a while so people could not be bothered following them. Automated is best although I have had clients with automated backup that stopped working and they never noticed because they never did point 2 which is...
3. Manually and periodically validate the backups are working. I remember one client that followed backup procedures religiously but failed to notice all the dust in the tape drive was shredding his tapes. Only when the disk failed did he notice that he had no backup. Take the disks, memory sticks, tapes and do a restore and compare every now and then.
4. Remember that most data lost is because of deletion or corruption, not hardware failure or house burning down. Use a versioned file system (zfs, btrfs, etc) to be able to recover old files. Don't complicate the backup procedures by trying to use it for versioning. You would need an massive amount of extra backup storage space in order to do something that btrfs or zfs is designed for.
5. Don't go nuts on multiple levels of backup. I cannot stress this enough. No need for Raid6, second servers, etc. This just makes things complicated and refer to point 1 about complicated. I mean what is the point of having a second server if you still got to backup offsite to protect against fire or theft anyway. Just means you have even more copies of data you need to manage and pay storage costs for. All you need is online storage with Raid 5 (or 6 if you have 6 or more disks), and an offsite backup solution. Anything more than this is going to cause more hassle than it is worth.
6. Offsite can be done in either two ways. Either backup to external media and store offsite or use a cloud storage service like crash plan or even put a second server with enough capacity at a trusted friends house and rsync. Always encrypt what goes out the pipe though for reasons I don’t need to explain.
Personally I have a ReadyNAS with that is expandable (x-raid) and versioned (btrfs). It works as both a central file repository and a backup for local pc and server files (time capsule, etc). At the moment I simply have a set of old disks that I rotate offsite but I will soon be moving to crash plan (or something like) using a self generated set of encryption keys that I will store offsite on a separate swiss cloud service. This will totally automate the backup and the self generated keys guard against crashplan (or someone else) looking at what I store.
Or you could just use my secret formula which came to me in a sweat lodge fever dream, to convert all your data to a decimal number string, locate its position in Pi, and you're done. For eternity.
The best part, I thought, would be the ability to retrieve work I hadn't even completed yet. But there's a wiggly bit of formula necessary which is evading my thoughts, so that hasn't worked yet... Also, I think I might get in trouble if I succeeded.
Get a job as a contracter/sysadmin, and store encrypted copies of all your stuff on the servers in your client's offices around the country.
I would also suggest their desktop computers. The executives desktop computers always have extra space, but they get upgrades too often. I suggest low-level management's and receptionist's desktops. They never get anything new.
IMO the best (but not the cheapest) option would be to use personal NAS server with some level of mirrored RAID. Configure backup from all machines/data you wish to backup to the NAS server. Then sync it with cloud provider. Of course when picking cloud provider do check to have strong data encryption, 2F authentication, account/data access audit and DO backup your encryption case (in case you loose it there would be no way to acces your data) - just print it in plain text form and store somewhere safe.
If you do it right everything would be automated and you won't need to do any manual actions with it. Just monitor its status. And do test recoveries from time to time.
And YES - I've noticed you are against the cloud which is in my opinion silly. Decent cloud provider's DC will by much more secure (as in physical security, data mirroring) than any homegrown solution. What you are afraid of? If you are afraid of automated attacks like malware they will target your personal machine anyway, not your backup, backup is not the weakest link here. Also any profiled attacks on your person will target your client machine. So what is your practical point against using cloud storage?
Also worth mentioning that NAS server is not mandatory in such setups. Just it speeds up things a little and gives more control. Also it provides the "oops" factor protection (like incidentaly deleting something - which is satistically the most often case to need backup recovery anyway).
Still if you oppose to use cloud just exchange cloud option for offline media stored offsite (like safe at your friends house or bank). Which media to use is entirely up to you. As you haven't stated what your need are (like how much data, how often it changes, what would be your preffered policy as weekly, monthly etc.) I can't recommend anything. An uneducated guess would be to use external HDD drives in enclosures and rotate them. Or for the cheapest option BlueRay discs.
A tiny bit of problem of HDD is that we don't see motherboards are being made with the connectors for HDDs made a few decades ago. Same thing might happen to the controller you might buy today in order to read that HDD in a few decades. So if "decades" is being the time frame then Optical disc might be better because the drives are usually made with some backward compatibility.
I have DVDs that I've burned as a teenager kept in a nice, high-quality soft "archival" binder for the last 18 years. Nearly all of them, of varying quality/expense, are unreadable due to degradation.
OTOH, I've got old 500MB harddrives that read/work just fine and are just as old. I'd expect sealed HDDs to be as good as it gets - tape is nice, but maintaining a supported/working tape drive was always difficult (used to have one). But, unlike every other type of storage, harddrives are actually capable of warning you of an impending failure. (I've been *saved* by S.M.A.R.T. at least twice, over the years.) Add some rudimentary RAID, and you're probably good. The only way I can think of to go further is to use two/three, and cycle them between your PC(often/all the time), a nearby firesafe(When you are heading in that direction), and a safety-deposit box (seasonally?).
It's hard to ignore spinning disks if your archival requirements are in the midrange (2-4TB) where optical media would take up far more room. Just keep an extra drive around for spare parts in case you lose a motor or something.
M-Disk is a dvd disk advertised to last 100 years. The drives cost about the same as a standard dvd drive, but a disk is $4. I can only personally verify 1 year.
For redundancy I'd go with a dvd and another copy on a flash drive.
The obvious answer is silver-in-emulsion photographic film. Microfiche will outlast typical U.S. made copy paper by a long shot. (Cheap copy paper starts to eat itself in about 20 years.) If properly stored--which is easy--it will last centuries. Even certain color film has a typical lifespan of about 70 years.
I have probably incited a riot and will be modded a "troll" but this is a serious answer folks--be objective. Film seems expensive at the outset but it pays dividends every year that it keeps storing your documents/pictures without needing maintenance, repair or energy.
If it's not worth the effort (are we talking heirlooms here?) then why even bother to spend the money on storage? Remember those times you typed up those important documents on your TRS-80? Do you know where those files are? Do you care?
Now if we limit this to a discussion of digital data where we intend to not mitigate, but *eliminate* all types of risk (except force majeure) then forget about single hard disks, DVD/CD recordables, RAID or flash memory. All these devices have no guarantee of ability to perform as archival storage--they could fail at any time from the day you buy them to a decade from now. There is absolutely know way to know when and therefore the risk is incalculable. If you can't calculate the risk of using an individual device there is no way you can even hope act appropriately to prevent data loss. All of these mediums but one require handling. You've never misplaced or dropped anything your life, right?
None of these technologies were marketed as or intended to be permanent storage. Hard disks are made to hold software, DVD/CDs are meant to hold copies of video and music, RAID's purpose is to decrease the length of service outages on a business computer (although it's pretty useless now) and flash memory is used as temporary storage for digital camera pictures. You are nuts if you rely on /any/ of these for storing important documents.
Low-tech minimum: printed on archival-quality paper with high-quality dyes. High-tech minimum RAID-Z array with at least 8 lower-capacity drives and identical units in two disparate geographic locations, replaced every two years.
encrypt it into a zip file, put it on drop box, and external media.
To loose the data now, Drop box and every PC in your home has to die at the same time.
I use Cobian backup to automate it on windows.
Put a USB adapter in the lockbox as well.
As someone who spent the first five years of his career putting hard disks in freezers and all the other old tricks to get data off crashed (and not backed up) client disks I eventually came up with the following perls of wisdom.
1. Whatever you choose make it simple. Most data loss happens because the backup procedures stopped getting followed because nothing bad happened for a while so people could not be bothered following them. Automated is best although I have had clients with automated backup that stopped working and they never noticed because they never did point 3 which is
2. Manually and periodically validate the backups are working. I remember one client that followed backup procedures religiously but failed to notice all the dust in the tape drive was shredding his tapes. Only when the disk failed did he notice that he had no backup. Take the disks, memory sticks, tapes and do a restore and compare every now and then.
3. Remember that most data lost is because of deletion or corruption, not hardware failure or house burning down. Use a versioned file system (zfs, btrfs, etc) to be able to recover old files. Don't complicate the backup procedures by trying to use it for versioning. You would need an massive amount of extra backup storage space in order to do something that btrfs or zfs is designed for.
4. Don't go nuts on multiple levels of backup. No need for Raid6, second servers, etc. This just makes things complicated and refer to point 2 about complicated. I mean what is the point of having a second server if you still got to backup offsite to protect against fire or theft anyway. Just means you have even more copies of data you need to manage and pay storage costs for. All you need is online storage with Raid 5 (or 6 if you have 6 or more disks), and offsite.
5. Offsite can be done in either two ways. Either backup to external media and store offsite or use a cloud storage service like crash plan or even put a second server with enough capacity at a trusted friends house and rsync. Always encrypt what goes out the pipe though for reasons I don’t need to explain.
Personally I have a ReadyNAS with that is expandable (x-raid) and versioned (btrfs). It works as both a central file repository and a backup for local pc and server files (time capsule, etc). At the moment I simply have a set of old disks that I rotate offsite but I will soon be moving to crash plan (or something like) using a self generated set of encryption keys that I will store offsite on a separate swiss cloud service. This will totally automate the backup to be made and relieve me of the need to perform backups.
It is not hard to find large numbers of cheap motherboards with IDE controllers...
Heck, you can still buy addin cards new with such controllers.
This won't be an issue for awhile and the data should be moved to newer drives by then anyway.
Glass is if its obsidian.
Disk interfacing was messy up until IDE became the standard. IDE is no longer an option on my newer motherboards, but I have an adapter that allows me to access IDE drives via USB. Actually, I have 2 of them, come to think of it. One of them also can talk SATA. These days, external USB is popular for disk (and thumb drives) so I expect that it's probably going to have a fairly lengthy run, even as it mutates. I've got an external USB3 drive, but it can be used on USB2 systems.
Optical devices are not immune. Assuming that you're only referring to the CD/DVD branch of the tree, there's some compatibility, although that's another device that used to be available in IDE (and SCSI) and has since migrated on. Other stuff like LaserDisc and WORM devices I'm not so sure about.
Tape is the classic, but tape devices come and go with little inter-generational compatibility. Plus, at least in the case of older tapes, the plastic substrate was prone to become brittle and the oxide to come off.
Even with "scratch resistant external media", I use 10% of the disc space for redundancy and recovery by using Multipar. It's a PAR2 compatible program that handles subdirectories. I've also bought but haven't used in recovery mode ISOBuster, a program that can handle the internal disc structures to try to recovery from corrupted media.
I have manually changed files and parts of files and had Multipar recover the originals; I have not yet physically scarred a CD/DVD/BR to see if it's recoverable via ISOBuster. It's supposed to work, though.
Fair (not archivable quality discs) BDs cost $0.50 for 20GB effective or $25 for 1TB, this is comparable to hard drive prices. They'll handle drops better and if one goes bad, you "only" lose that media (20G) vs terabytes. It's much slower, smaller, and write-once, though.
(OMG -- 20G is "small"?? I remember having things fit on a single 256KB 8" floppy. Much better than paper tape, though.)
If the universe is someone's simulation -- does that mean the stars are just stuck pixels?
Tape media may still be a good bet, and probably better than magnetic HDDs. Tapes are small, store lots of data, and are pretty resilient.
I wonder how long they require/expect the data to last for? Years? Decades? Generations?
The complexity of making a tape drive work has to be at best, 2x that of an off the shelf hard drive given the number of moving parts. And, unless you spring for the really really expensive version a tape cartridge wont come anywhere close to the density of a 2TB 2.5" HDD.
That being said, the only real obstacle to longevity of any medium is maintaining a good backup regiment. How hard would it be to, once every 2 years, purchase a new reasonably-priced USBx flash drive, copy the backup from the last flash drive (assuming your storage needs are modest, currently in the 100GB range) and put the new one in the safe? 24 months is definitely within the safe time range of even the cheapest flash media.
This morbid interest in "time capsule" media that will survive untouched for 50 years so your grandkids can come along and see your vacation photos from spring break after you die from a heart attack at age 30 is really bizarre. If no one has come for your data in 10 years or so, guess what: it wasn't worth anything.
Optical media, at least the writables we as consumers have access to are completely inadequate for long term storage.
What about something like M-Discs? They're a consumer-available optical medium designed for long-term storage. They require a drive with a higher-powered laser to record, but will read in a standard DVD or Blu-ray drive. Of course, their "1000 years of storage" can't really be tested, but the idea of using an inorganic data layer seems like an improvement, and the discs passed some kind of DoD reliability testing.
It is pitch black. You are likely to be eaten by a grue.
How persistent is the storage over time? I've had CDs become unreadable due to media degradation...though I don't know whether it was fading of the dyes or yellowing of the covering...or some other reason...but it wasn't scratches.
I think we've pushed this "anyone can grow up to be president" thing too far.
I'm putting silver bullion into my lockbox.
I remember back when single CD blanks were $30 a pop and pirates still used them to copy games. I thought it was nuts then but people did it.
Don't assume that just because you are a luddite late adopter or intolerable cheap that the rest of us are too.
A Pirate and a Puritan look the same on a balance sheet.
Well, recordable Blu-ray discs use an inorganic dye, so they should last longer than DVD-Rs and CD-Rs. The manufacturers typically claim a lifespan of 100+ years.
If you search you'll find there are "Archival Grade" CD-Rs and DVD-Rs. Read the specs, but some are rated at 100-300 year retention life.
There's usually a 2 or 3 year period between media type transitions where both are available on the same PC. That should be plenty of time to migrate to the new types of storage hardware. For example, I moved all my backed up data from IDE drives to SATA drives with an easy robocopy one day.
This question of storing something for 5 or 10 years or more never made sense to me as we're always collecting more and more data to store. We're not going to stick the data on a hard drive and then hide from computers for 10 years. We're always going to be upgrading to the new hard drive types or optical disc mediums and will be pulling our older data along.
I've got 3 2 TB USB drives (encrypted -- about 700GB free). (1) Main drive with my data. (2) Backup (which get's synced nightly) and (3) one at my mother-in-laws house in a drawer.
Every year I buy a new drive to replace the (1). The old (1) becomes (2), the old (2) becomes (3). And (3) gets wiped and either given to friends/family or ebay. Also, drive (2) and (3) get swapped every week or so the "off site" drive is never more than a week or two out of sync.
I've been following a version of this procedure for 15 years (starting with IDE drive drawers) and increase size as necessary. So far I haven't had a drive completely fail once, although at the tail end of the life cycle of one I was begging to get some SMART errors.
OTOH, I have read tapes after 30 years. If long term storage is what you want, the LTOx is the answer. Make multiple tapes and put them in different places (countries, continents).
Sent from my ASR33 using ASCII
The issue with any backup media is if it is placed somewhere in storage, you have no indication when degradation will make it unreadable.
My strategy is to have at least 2 live systems in different locations connected by a standard network, with active workstations & servers ( laptops, desktops etc) backing up to one of each, and then having the 2 systems backup to each other. These systems are live HDD based (with SMART technology with alerts) so am aware of any impending issues. Each also has external HDD's with a copy of the whole system.
The HDD's are upgraded every couple of years, and old HDD's are stored in yet another location - ( NOT the dump/tip ).
You can buy a 1TB thumb drive (Kingston HyperX Predator), but it will cost you around $1K.
You can buy thumb drives for $20 per, but they'll be 64GB, maybe 128GB if you're lucky and don't mind dodgy manufacturers.
You can buy a "1TB thumb drive" for $40 or so on eBay, but you'll find that it "redundantly" stores the last few gigabytes you wrote across the entire drive. In other words, it lies about its capacity, and just trashes existing data once you exceed its real capacity (likely 8GB or less).
Of course, if you're just trying to save "important documents", you probably don't need anywhere near a terabyte, or even a gigabyte.
The cloud makes a great backup. If what you're archiving is small, encrypt it and upload it to a variety of cloud file companies with free offerings - Cloud Drive, OneDrive, DropBox, etc.
For a moderate amount of data, use (encryption and) Amazon Glacier. If you don't know the trick: Amazon offers mail us a hard drive as an upload format for S3 and Glacier, and it's as good as way to do offsite backups as any.
I wouldn't use the cloud as my only archive, but as the offsite copy it's probably more disaster-survivable than most other choices most of us have available. (And affordable if we're talking a few hundred GB of personal stuff, not the entire multi-TB geek archive of "binaries").
Socialism: a lie told by totalitarians and believed by fools.
I have got data from when 5MB H/Ds were introduced ON TAPE, written with tar that is still readable. Good luck reading your ST506 interface H/Ds written with DOS 4.2.
Sent from my ASR33 using ASCII
If you backup to an external HD and then stick it in a safe, chances are you won't back up very often: your backup routine contains manuals steps you have to remember/set reminders for.
Also, this HD is used only occasionally, and in my experience that's not a recipe for high reliability: I tried using HDs this way (accessing them only once every few months), and of my limited sample, pretty much every one broke down in a few years. Exacerbating factors: flaky USB enclosures (the tiniest nudge of the connector and it'd interrupt the connection, potentially corrupting the drive) and stiction.
I'd want to carefully monitor the backup drive, reading back what it wrote to make sure the backup matches the source. I'd also want to read the entire drive at regular intervals to pick up signs of trouble at an early stage.
I've got an excellent program (Watchdrives) from a fellow Slashdotter that does this for my main drive: reading the drive using dd in a low-priority process, so that the entire drive gets read once every ~2 months.
RAID is not a backup or an archive solution.
If you store a raid it can't detect data and/or media degradation because the system isn't running. I haven't seen many safe deposit boxes that allow you to run a computer inside of them. The drives will most likely degrade inside of 10 years.
To archive something you want archival media. Something like the 100+ or 200+ year gold archival DVDs and Blu-ray discs. The readers for those disks will be available for a couple decades at least. (Look at M-Disc as an example)
Add the data as post comments on slashdot.
It will work froeevr!
Print all of your text documents on acid free paper in triplicate and store them in climate controlled facilities around the planet. Maybe even keep an extra copy on the Moon just in case. All of your digital files can be uuencoded before being printed out.
If you're really paranoid, you can encode everything into the DNA of some organisms and then distribute them throughout local and deep space with rocket ships and comets!
Acid-free paper, otherwise you and your friends will just keep eating bits of your archives.
More seriously, paper's only good for some things, and only if you protect it well enough. Some years ago, my work hard drive crashed, and when I was driving to work a day or two later, my coffee cup bounced off the holder into my briefcase, taking out both the Palm Pilot and the dead-tree copies of my data. There were backups of some of my PC data, but my current calendar was gone.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
How much of your information needs to live significantly beyond your personal lifetime? My guess is you need not consider storage that will live beyond your children, who might have some need to review your papers for personal or practical reasons. Your grandchildren might like a handful of pictures, nothing more than than.
If your information is actually valuable (its creative or philosophical or similar) other people will look to its preservation.
I tend to split stuff up. For my profession life as a programmer I use github and one other git based storage. Anything worth keeping I'd migrate to whatever replaces git. For personal life I keep backups of photos and videos on local and networked (cloud based) storage. For tax stuff I just have a fireproof locker.
I imagine in 20 or 30 years the only stuff of value will be my movies and photos and written personal documents. After I'm dead none of that stuff will meant much to anyone, unless my son wants some pics of our dogs when he was young. And he'd be the last one to care about any of that stuff.
Try not to let possessions become too important. You are going to have it all taken away from you eventually.
Peace, or Not?
Moore's Law is only partly your friend here - storage keeps getting cheaper rapidly, but that also means that not only do devices become obsolete, but the interface specs and data formats also become obsolete. You probably don't have an 8" floppy drive anywhere, or a working 5.25", or the right kind of cable to plug the 5.25" drive into, or a Bernoulli drive, or a 9-track tape drive (800, 1600, or 6250dpi), or the Sun cartridge drive, or anything to plug those MFM drives into, or SCSI-1, or probably SCSI-2. You might have something that can handle IDE / PATA, or an old laptop with PCMCIA, but even those are getting scarcer. If you can connect to that old disk disk drive, you can probably load a virtual machine running NetBSD that'll have drivers for the file system format, but maybe not; you certainly don't want to risk having Windows "update" the format. You might think that FAT 8.3 format will stick around for a long time (and maybe it will for reading, but it's rapidly getting replaced with FAT16, FAT32, ExFAT, NTFS, etc.
Leave aside the question of whether you can read a 20-year-old version of WordStar or WordPerfect format file (unlike my late-70s nroff files, which would be readable if they weren't on a 9-track tape I've probably lost.) You can probably read that 4-year-old TurboTax file, but if you need to get tax data back from when you bought your house, you'd better have everything on paper.
Just for physical format alone, you need to copy stuff every couple of years.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Important to you, yes. Other than that, no one cares.
Your dissertation? Print it out. Send it to the cloud and let that take care of it. Pics of the family? Again, print. And not on your crappy inkjet.
Paper in a folder or photo album is universally, instantly accessible. Anything else (digital) requires a continual update to whatever storage medium is current.
10 years ago, saving to a CD might have been a good idea. 10 years from now, will anyone you know have a device with a working CD drive?
Maybe I have been lucky. I have CDs I made in the late 1990s when CD-R writers were 1-2x speed, and I can still read data from those. I once had to pull some files, so grabbed a DVD from about a decade ago, extracted the files and called it done. Since I use WinRAR for an archiver, I do know if there is bitrot, and if damage did happen, there is a chance that it can be handled by a recovery record.
I've also been lucky with tapes as well. I've restored DLT media over a decade old with zero errors.
Of course, when it comes to hard drives, I have a nice pile of dead ones over the years, including a batch of drives which failed at the same time. Similar with USB flash drives.
I am hoping the Sony and Panasonic ArchivalDisk product gains some steam and the price of drives gets dropped by a factor of 10-20. 300 GB AD, or 160 GB Ultra HD Blu-Ray (yes, that is the name, announced a few weeks ago) would be useful for a long term backup/archive format, especially since the technology is innately WORM driven.
Of course, here is something I wonder about which would help immensely with backups: Why isn't there a decent backup/archive/retrieval program out there that works well with multiple media types? Retrospect used to be good, but doesn't support USB Blu-Ray players (making it worthless for archiving.) In the enterprise, there is NetBackup, Tivoli TSM, ArcServ, Networker, heck, even Backup Exec. These not just do backups and restores, but can transfer stored backup sets between media types, validate backups, retrieve archived files, periodically move data from one pool to another (say from disk to tape), and handle one set of data (documents versus OS files) differently from another.
Why do I have to pay insane prices for an enterprise-tier of software if I want the ability to select some documents, click "archive", have them copied to an archive media pool, then go on? When I want to make sure the backups are secure, I create another pool on an external drive, copy the data there, and flag that pool as offsite. This way, every single file I have is backed up, archived/deleted files are retrievable with just one command (perhaps additional time to attach the media if it is offline.)
This isn't state of the art functionality here... ADSM (now TSM) had this stuff back in 1998. This should't be locked to an appliance either. The Unitrends appliance and the former WHS were nice devices, but it would be nice to have a server handle the backup coordination, then if need arose, separate media servers could be used as well... for a price well under five digits.
Backup software (and I'm not meaning the Acronis TrueImage and the other clones that can copy data to a drive or offsite and back... but stuff that can keep track of multiple media types, move files between them, deduplicate files, and be able to figure out where some spreadsheet from 2008 was, out of hundreds of DVDs burned) just has not kept up with the times for average users. I just don't see why Symantec, EMC, or IBM offers this for home users, as it not just makes data safekeeping easy... but because the server could be installed on a separate machine that accesses local desktops, malware on a client would not be able to destroy the data on other machines.
Tape is king for this type of thing. They are actually fairly affordable; if you don't have the need for robotic loaders and tape libraries you can probably pick up an older LTO-5 reader and some LTO-5 tapes for a few hundred bucks.
This is the medium every serious organization uses for offsite, out of sight, out of mind, its there if we screw something up really bad, sort of archiving.
A friend of mine and I both use crashpan to backup to each others home servers. We each have 2 tb in RAID1. this should protect against drive failures and disasters such as fire. As long as both our houses don't get destroyed at the same time, the data should be safe.
If you don't trust the cloud (which is entirely reasonable), encrypt your data and back it up to multiple clouds. Then put the private key on paper in the lockbox.
A few years ago I resigned myself to the fact that every thing about me, probably even my DNA sequence, is out there on various systems and will get stolen or compromised one day. I do what I can to keep things secure, but there's only so far I'm willing to go. If you have been reading a few security blogs over the last few years, you'd know that more and more experts (whoever they are) are recommending that corporations focus more toward mitigation of security breaches while taking resources away from prevention. It's kinda like getting mugged or in a car accident. Eventually, it's just going to happen to you. This is how we live in 2015.
Backing up to DVD or thumb drives or whatever is so 1990's. You may as well have a stack of Zip disks. Physical media is as stealable as anything else in your house. And BTW, if you do get burglarized, guess what the first thing is that they take? That's right, any box in the bottom of your closet or in your garage with a lock on it.
The cloud is technology agnostic so format doesn't matter, you shouldn't have to worry about that. It'll evolve over time but that's not your concern.
Buy yourself a little NAS, load it with 4-5 fat drives. Put everything important on it. Back your laptops and PCs up to it. Encrypt what you feel like encrypting. Push it all to Glacier, which costs a penny per GB. Done. No trips to the bank. No wasting time burning media that will degrade. No physical items to lose.
It probably is a good idea to have more than one copy of everything, just know that as soon as you make physical media of something, it's outdated. Someone else on this thread mentioned keeping a NAS at someone else's house, perhaps your grandma in another state. I'm personally not that paranoid about my stuff so I don't do that, but it's probably a good practice.
Fewer and fewer laptops are coming with dvd/bluray drives built in. Over the last decade we have seen fewer and fewer models of external drives/burners in stores. The writing is on the wall. In perhaps a decade or two you won't have choice, your stuff will be in the cloud (or whatever they change the name to), may as well get on that business now.
Asking myself the same question, I went with MDisc technology, in the BluRay capacity, in addition to my hard drive backups. MDisc uses an inorganic pigment as opposed to the organic dyes that are common on CD/DVD/BluRay recordables (and degrade over time).
I'll do an MDisc burn every year and move it offsite, to keep with the 4TB ZFS drive I rotate offsite weekly. The MDisc won't get my mp3 or mp4 files, but the stuff I can't recreate.
My best idea currently is to write PAR files of loop-back mounted LUKS volumes and include the PAR software source and ISO of the distro on the disc, in case I need the data in 20 years (emulators should be readily available for 2015 hardware).
I needed a BluRay writer anyway, so I went with this LG and it's been a great drive so far, and at the right price point for me.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Well, recordable Blu-ray discs use an inorganic dye, so they should last longer than DVD-Rs and CD-Rs. The manufacturers typically claim a lifespan of 100+ years.
Beware BD-R LTH media, which use pretty much the same type of organic dyes as are normally used for CD-R and DVD-R as a cost-cutting measure. BD-R HTL uses phase change in an inorganic alloy to record bits, which will almost certainly outlast BD-R LTH media (and probably DVD-R and CD-R, too).
I've been using these for archival recently. (I'm almost out, too...was going to put in an order, but (1) they're currently out of stock and (2) their per-disc price may have gone up substantially since my last purchase. :-P Will need to double-check once they're shipping again, but my last order was about $27 for 25 discs, shipping included.)
20 January 2017: the End of an Error.
John Edward is probably the best medium. He will be able to retrieve your data even after you're dead!
the interface specs and data formats also become obsolete.
You can buy devices for almost any media and format, and, thanks to eBay, these devices are more available than ever before. As time goes buy, obsolete media is becoming less of a problem. You can even find paper tape readers. If you don't want to buy a device, you can use any of numerous companies that will copy your media for a fee.
You probably don't have an 8" floppy drive anywhere, or a working 5.25", or the right kind of cable to plug the 5.25" drive into, or a Bernoulli drive, or a 9-track tape drive (800, 1600, or 6250dpi), or the Sun cartridge drive, or anything to plug those MFM drives into, or SCSI-1, or probably SCSI-2.
No, I don't. But I can have any of those things on my front porch in a week.
I think the point was that after you clone your backup drive to a new one, you can reuse the drive to replace or expand your main system drive, whereas once you burn an optical disc, "reburning" means throwing away the old plastic (or keeping an extra copy around). This effectively makes optical media a lot more expensive than magnetic media.
Check out my sci-fi/humor trilogy at PatriotsBooks.
At more than 8 cents per gigabyte, archival DVDs are horribly expensive. You could cycle your backups across three hard drives for about the same amount of money, and then you have three backups instead of one.
Not to mention... have you ever tried backing up your 4 TB hard drive onto a spindle of 1,000 DVDs? Have you ever seen a spindle of 1,000 DVDs? It's slightly taller than an average person. Yes, if you don't have much data, you can do what you're proposing, but....
Hard drives are really the only viable backup medium unless you have a big enough collection of data for tape drives to make sense—maybe Blu-Ray, but only if you don't have more than about a 100-disc spindle worth of data (2.5 or 5 TB) to back up (and really, most people lose interest at more like ten or fifteen discs).
Check out my sci-fi/humor trilogy at PatriotsBooks.
How does that compare to commercial DVDs that you've bought? I have movie DVDs, PS1 games and PS2 games that still play perfectly. My Dad's CD collection is older than me and it's still fine. It seems that it's the quality of the disc and the way it's burned that makes a difference rather than the medium itself. That may not help much for home backups, but there is plenty of evidence (my house is full of it) that disc based media lasts for decades. On the other hand I too have tried to read discs that I've burned maybe 10 years ago and all are corrupt.
I would go with the cloud services. For the sensitive parts, use encryption. And for the private keys, use stone tablets.
There is no substitute for common sense. Especially, no body of rules will do.
Unless you're dealing with Media Files - plain archival quality paper is the best for any long-term archive. Even images printed on Archival Quality Photo Paper with Archive Quality Inks will certainly out last a bluray disk - keep in mind the burner that created the disk may have it's key revoked, possibly making all archives written by it useless.
Since Electronics have not withstood the test of time like paper, I'll stick to paper.
Well, the problem can be solved by adjusting the "as long as you want it to" input into the equation.
Step back, take a look at the temporary naure of life, appreciate your own insignificance, and ask yourself, does it really matter if, when I'm 95 and cannot remember my own name, I still have photos of the cat I had in college.
It's called the Zen backup plan.
Someone had to do it.
It took my friends months to find working 8" floppy drives they could take to Guatemala to decode the files the police and army had created during the dirty wars there. I don't want to have to buy a 9-track tape drive to read the one 9-track tape I have (if I find it again, and if it's still even readable.) (I gave away the Sun cartridge drive along with the Sun-2.)
Much more reliable to copy the data every couple of years to some current medium, knowing that Moore's Law means that it's not going to cost much and the only problems will be data formats, not media formats.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
What he said.
I have a Mac, so I created a bunch of writable sparsebundle disk images ranging in size from 10 MB (single-PDF tax returns) to 1 GB (car documentation). I save them all directly to iCloud Drive. When I mount the disk image and make changes, only the changed bands are uploaded, avoiding a bottleneck or incomplete sync situation.
Time Machine keeps versioned backups of the iCloud Drive files on my offline backup disk (as of OS X 10.10.2). Periodically, I copy the disk images into OneDrive and Dropbox for redundancy.
Each disk image has a different password, all of which are secure (long strings of random characters) and managed well (saved on my Mac, as well as printed out and safely hidden in case of total disaster). My cloud accounts all have secure passwords, two-factor authentication, and all my computers have encrypted drives so I'm not out of luck if my computer gets lost or stolen.
For disk images that I know will never be modified again (e.g., Taxes 2003), I convert the disk images to a read-only format to save space in my clouds. I haven't paid a dime for cloud space, ever.
It sounds overkill when I type out the procedure, but because I've used only features built into the operating system, I can scan and archive a document in under a minute. On the other hand, this is complete vendor lock-in, so if I switch my primary computers from OS X down the road, I would have to throw out this entire solution and start a useless Ask Slashdot thread like this.
Hope this helps out any Apple nerds.
You sure the dvds were 18 years old? I'm not sure there were even consumer dvd writers available around 1996. Perhaps you meant cds?
I actually had really good luck with my burned cds from around 1998 or so. I was able to read all the data off of them recently. A couple had to slow down a bit to be read, though, so there must have been some sort of degradation. Wonder if it could be more related to the burner you used or maybe the reader you used to try to read them? (Also, I can suggest ddrescue for recovering from cds/dvds if needed. It came in handy when I was recovering a scratched cd with info on it that I really wanted. I used it over and over with a couple of different readers to get a complete image of the data.)
Your hard drive suggestion sounds exactly like what I use. All the data I've read from old cds/dvds I now store on hard drives. I use SnapRaid to create a couple of parity disks. (So I can recover from up to two disks failing. Plus, since it isn't online raid, I can fix deletions and other problems as well.) I occasionally backup onto external hard drives utilizing SnapRaid again and then store them in a firesafe at another location.
Is TrueCrypt still OK to use after it was shut down or whatever happened last year?
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
I would consider a diamond to fit the definition of a stone. The molecular structure is a trade secret and isn't even revealed in the patent. What do you expect? Plenty of molecules have properties of its constituent elements. Why would glassy carbon be inaccurate (even if obtuse)?
That protects ONLY against drive failure or human error. It does nothing for fire, flood, power surges, etc... That is not a backup.
Correction: 2GB should be 2TB lol.. 2GB HD is actually funny.
I have DVDs that I've burned as a teenager kept in a nice, high-quality soft "archival" binder for the last 18 years. Nearly all of them, of varying quality/expense, are unreadable due to degradation.
OTOH, I've got old 500MB harddrives that read/work just fine and are just as old. I'd expect sealed HDDs to be as good as it gets - tape is nice, but maintaining a supported/working tape drive was always difficult (used to have one). But, unlike every other type of storage, harddrives are actually capable of warning you of an impending failure. (I've been *saved* by S.M.A.R.T. at least twice, over the years.) Add some rudimentary RAID, and you're probably good. The only way I can think of to go further is to use two/three, and cycle them between your PC(often/all the time), a nearby firesafe(When you are heading in that direction), and a safety-deposit box (seasonally?).
I do a double backup to external hard drives. One is kept on site, the other offsite. When I backup something, I switch drives and redo the backup. Or, in other words, I take the backup to the second site and copy over the backup to the second one.
I expect sata / usb interfaces to be around for the next 10 years.
Leslie Satenstein Montreal Quebec Canada
A few years ago an English professor contact the Computer Science department to see if we could read some 5 1/4 floppies. She wanted to re-start an old novel she had been working on. I still had a computer at home with a 5 1/4 drive. Her disk format was toast, but I was able to `dd` an image and use `strings` to pull most of the text back. She was happy.
A couple of years after that someone else asked if we could read another 5 1/4. I still had the drive but I had upgraded the motherboard and no longer had a floppy controller to use.
Ignorance killed the cat. Curiosity was framed.
For true redundancy safety three is good but four is even better. Or if you want near total redundancy safety six. . . .
Below the speed of light Special Relativity is one of the most accurate theories in physics - above the speed of light..