Slashdot Mirror
Long-Term Personal Data Storage?
BeanBagKing writes "Yesterday I set out in search of a way to store my documents, videos, and pictures for a long time without worrying about them. This is stuff that I may not care about for years, I don't care where it is, or if it's immediately available, so long as when I do decide to get it, it's there. What did I come up with? Nothing. Hard Drives can fail or degrade. CD's and DVD's I've read have the same problem over long periods of time. I'd rather not pay yearly rent on a server or backup/storage solution. I could start my own server, but that goes back to the issue of hard drives failing, not to mention cost. Tape backups aren't common for personal backups, making far-future retrieval possibly difficult, not to mention the low storage capacity of tape drives. I've thought about buying a bunch of 4GB thumb drives; I've had some of those for years and even sent a few through washers and driers and had the data survive. Do you have any suggestions? My requirements are simple: It must be stable, lasting for decades if possible, and must be as inexpensive as possible. I'm not looking to start my own national archive; I have less than 500GBs and only save things important to me."
Amazon S3. dirt cheap, there forever.
Hard drives, while they may fail, are still probably your best chance. Using RAID-1 or -5, you can keep the drives running (possibly intermittently) and can avoid failure. With the rate of hard drive growth, you can just replace them with bigger drives when the time comes you need more space. It isn't exactly the same as throwing them in a cold room and forgetting them, but it isn't too expensive either.
Waffles rock.
Parchment.
We don't have enough history on this tech to know what, if anything, will "last for decades". Possibly "paper" and "microfiche" might fit in that list, but those aren't the sort of things you're talking about. Best option I can think of right now would be to get a couple 500gig drives, put everything on both, and then put them in different areas. In 3-5 years, back them up to something newer, and repeat that every 3-5 years. Maybe in those intervening years, we'll have more data and newer tech that's demonstrably suited for what your needs are.
creation science book
No storage medium is perfect. Remember paper? That actually roots after a period of time.
Personally a data CD is probably the best most long term solution. Not a DVD mind you, a 700 or 800mb data cd. Get some cases for it and call it a day.
Mind you, there isn't a perfect way to store any type of information for long periods of time. Personally I think a CD would be the safest bet.
Even-though longevity of magnetic tapes has not been explored extensively, it is understood that if they are kept in a stable humidity and temperature environment with no light, that they should last for at least 20 years.
For important stuff I use memory cards in my safe deposit box at the bank. I could see flash being a viable long term storage, some of them coming out with 10 year to lifetime warranties.
Obviously this question hasn't been answered for the general public because this is like the 4th year in a row that this question has been asked on Slashdot.
Long term:
Use quality DVDs. Redo the backup on a schedule such that everything is re-backed up every three years or so. Every month, say, you make one DVD. Keep the backups in a climate controlled, dark, secure place, such as a safe deposit box at the bank.
Short term:
Back up everything you want to save to an external hard drive weekly. Every three months swap it with a drive kept in the safe deposit box.
Daily:
If you have a Mac, use Time Machine. If Linux, some sort of cron job running a Python script that copies /home to an external hard drive. If Windows, I dunno.
Best Slashdot Co
Just get two drives and put the data on both of them.
Controllers can fail,2 drivers failing at the exact same time is unlikely.
Also put your most important data in a location different than where the original is kept.
How many times has this question been asked on Slashdot? I swear, it shows up on the front page at least three times a year.
As for the question itself, the answer is pretty simple, but unhelpful. Basically what it comes down to is that there is no safe place for your data. You're asking for the best type of basket to put all your eggs in. If you look at it that way, the solution is easier to arrive at. Your choices are A) spare no expense and build/buy the world's strongest basket and pray no flaw arises, or B) start copying your eggs around to all sorts of cheap baskets and continuously add more baskets in the expectation that the oldest baskets are going to fail.
Copy all your stuff to all your computers. Burn to DVD and/or CD ROM. Buy SD cards and USB flash drives. High capacity storage devices are so cheap now that you can keep all your valuable pictures of your vacation to Cleveland quite safe by constant duplication. That's the value of digital. Copies are perfect. Make lots.
If a job's not worth doing, it's not worth doing right.
What I have in my home system is a small back-up drive sitting inside the PC. Every night, the drive spins up, personal/irreplacable data gets rsynced to it (therefore very little work) and is spun down.
Easy, cheap, and lets the HD work for the minimal time needed.
Also serves as a sort of recycle bin if I mistakenly delete something.
^_^
Here's the thing. Flash drives will *probably* last long enough. I wouldn't at all be surprised if they were still readable in 20 or 30 years. But a) what's the odds of your current WinUx or iHoloTablet having a usb connector in 30 years? and MUCH more importantly, what's the odds of having anything capable of reading those historic Word 2007, Acrobat 5, or any other type of file format in 20 years? Yes, there are some folks technical enough that they can still read and readily interact with Geoworks, Wordstar, Xywrite, etc. stored on 8" floppy disks. But if you ain't one of them, and I'm happy to admit I ain't, the fact that the flash drive is physically capable of being read in 30 years simply won't matter. That's why I crack up reading various vendors' claims of CDs, DVDs, BDs etc. lasting 50 or 100 or more years. The disks will be readable but you'll have no mechanical or logical way to read what's on them.
It's impossible for guarantee 100% storage integrity, just like it's impossible to guarantee 100% uptime. What you want to ask is what risk of data loss you are willing to take.
This page compares some of the options in terms of Mean Time To Data Loss (MTTDL). For the amount of space you're looking at (~500gb), a three-way mirror is probably sufficient to last for your lifetime.
But there's always the risk of fat-fingering "rm -rf" or having the building catch fire, so maybe you want to have two synchronized sets of mirrors, stored in different physical locations. Only you can decide if that's too paranoid for you (or not paranoid enough).
Paper tape - accept no substitute.
Apart from anything else, the standing waves you can get as it goes through the reader are alone enough to justify it.
But seriously, I've had the same (but growing) data set in my /home for over 15 years, and going. I find the easiest way is to just keep it on my drive, and have a few frequently updated copies on external media (optical or solid or dirve) and to keep it on another PC too, disk space is so goddamn cheap. I also have a large music collection, and instead of wasting time backing it up onto optical media, I just keep it on both my notebook and PC, its unlikely both will fail at the same time, and incase of a robbery, I can also archive it at work.
Don't expect any form of media to last forever, it's multiple, frequently updated copies that will ensure your data lasts forever.
Also, if you have friends and family you can trust, make a copy for them to keep for you, off-site backup is also important.
Obviously this all depends on how important and/or private the data is.
my 0.2
As the tag implies, Gmail is your friend. 7 gigs per account, searchable, accessible from any connected computer, free, and if in the future, google starts to decline, you can transfer to their replacement.
I'm currently evaluating the cost-effectiveness of a DroBo (http://www.drobo.com/) for archival VS ease-of-access storage. It's not an inexpensive item, exacerbated by its storage method which gives you 2.7TB actual storage when fed with 4x 1TB drives.
However, the unit is far more resilient than any conventional RAID solution.
Connectivity is via Firewire or USB, with ethernet via an add-on. I'm more interested in this than any other long-term archival method.
I will feed it with some 640GB drives, with a set of spares taken from different vendors. Job done.
There is absolutely nothing that you can put away for decades and expect to be useful. Your requirements are not simple - they'll actually very, very hard to meet, even if you want to throw a lot of money at the problem.
You don't know that a jpeg, for example, will be readable in 30 years. The format may be so deprecated that there might not even be a viewer available. Like my old Microsoft Works 4.0 documents - although I have the data, I have nothing that can read them unless I want to spin up an old Windows image, assuming that I can generate a virtualized environment that can support an old Windows (Windows XP probably won't even boot on any PC being produced 30 years from now). And some of that data is only a few years old, not decades old.
You should store not only the data, but also the applications that created the data. And the computer you need to run those applications. And backups of those. And then every few years, pull it all back and validate it and update as required.
You may have only 500GB now, but 10 years from now that will be 5TB. And then you need a way to actually be able to find something you added to your "archive".
I deal with this at work regularly. An archive is not a backup that you keep for a long time. It's much, much more than that. Once you start thinking about all of the issues that come up, you'll see that the media is the least of your problems.
This is really close to what I do. Running a home network, all the data is dumped to a RAID-1 and monthly copied to CD/DVD, which are kept in suitable storage space. This gives 3 levels of recovery:
1 - local hard drives have the data - manually done
2 - RAID on the network has a copy - scripted backups
3 - CD/DVD has a copy - manually initiated scripted backup
If I was truly worried, I'd make two CD/DVD copies and store one in Iron Mountain or something similar.
You can substitute USB drives for one of the CD/DVD copies if you like. The only answer to fragility of storage mediums is to make multiple copies and refresh those copies often enough that the inevitable failure is mitigated. I personally choose to use TAR and GZIP for now as I trust these formats will be usable in the months ahead. If they become outdated at some point, I can change that going forward and save a LIVE-CD with those utilities on it with the older data.
You can encrypt the CD/DVD copies easily enough for security, but long term you might want to make sure you write that password down :-)
Support NYCountryLawyer RIAA vs People
I believe this to be a serious problem with no good solution currently. That's the truth. You'll get lots of dismissive posts saying it's no big deal, but it is.
Forget media integrity. The problem is technology drift. Everyone thinks "ubiquitous" (as in every computer has a USB port) is the same as "eternal," and it isn't. Twenty years from now, your USB thumb drives and CD-R's may have their data physically intact, but only museums will have equipment that can read them.
It is a fantasy to suppose that you can successfully perform Sisyphus-like task of systematically recopying your data to new media and formats. The proof of this is the innumerable stories of big, well-funded organizations that have neglected to do this. If the NASAs of the world keep finding reels of tape with important data on it that can't be read due to technology skew, what makes you think that you can do much better?
(What makes me bitter is failure of vendors to give adequate warning when software updates remove the capabilities of reading file formats that were formerly supported. I once verified that my new Mac could read my old MFS diskettes, and did not notice when a software update to the OS removed that capability. Microsoft was less than forthcoming when they removed the built-in ability of Excel to read Multiplan files).
"How to Do Nothing," kids activities, back in print!
Oh and, don't forget to use tools like Gmail drive, you can upload your stuff there, and if I recall you can store up to about 6-7GB on a gmail account nowdays, plus it's free.
get flash cards make at least 2 full copies (3 is better) and do full 100% reads from all the cards at least twice a year or buy space with a major hosting company and store a copy there and have at least 1 copy on flash cards that you check regularly it's not the throw it in a drawer and forget for 10 years solution you want but that solution doesn't exist
juenger1701
You are the analogy of an investor who wants a high-yield, low-risk, completely liquid instrument. The term is TANSTAAFL.
I maintain two (yes, two) USB external drives. Every couple of years, I migrate to a larger, or otherwise better medium. I use an incremental backup system (for me, cpio) that ends up keeping too much stuff, but at least I have the stuff I want if I need to get to it.
In a decade - in my case, four decades - one can accumulate a remarkable amount of crap, along with things one truly wants to save. I have a total of about 90 gig of actual data, plus a far larger amount of music and video, which I consider more or less disposable. It is not difficult, nor expensive, to purchase another external drive and copy the data. My oldest backup is on IBM 2314 disk pack, but the data still held on that disk is also present on my current backup, a WD 160G in a USB-1 enclosure. Sometime next year, I'll go to a 500 G drive in a USB-2 enclosure.
An important consideration is to periodically check to see that the data ostensibly held on a drive (or CD, or DVD) is actually readable. DVD/RW in particular has a tendency to get flakey over long periods of time, expecially if stored under adverse conditions (jammed in back of desk drawer, under sixteen pair of scissors, stapler, a box of pop-tarts, and four old coffee cups. I always keep my last few generations of backups, and if I find an unreadable datum, I make an effort to recover it from the previous backup.
While it may be stating the obvious, it's a Bad Idea (TM) to wait to back up data until you have a problem. I back up all of my data every week or two, and critical data, daily, without fail. Critical data is cached as a three-generation dataset (IBMese).
Good luck. There are no real solutions, just ways to cope.
Don't take life too seriously; it isn't permanent.
1) Encrypt your data
2) Tack it onto the end of some anime or porno mpg
3) upload to kazaa/gnutella/whatever
It'll still be circulating the net long after our grandchildren are dead.
Very durable. Write speed is a bit slow though...
I can't imagine a guy with 500G of personal files. It must take a month just to read the titles of his various files.
But people are different. I'm not prone to believing in the media failure reports that we have all seen. For example I have a pile of floppy disks that are still intact after 15 plus years and I stored them like a barbarian. Hard drives also tend to last for me. And I suspect that any quality CD or DVD will last for quite a long time if handled and stored carefully. I would worry more about the PCs being unable to use ancient formats or OSs that can't cope with older stuff. In order to avoid that issue he would have to make certain to keep trying his disks every couple of years.
If you buy quality DVDs and take good care of them they will quite probably last for decades, perhaps half a century. They are expected to degrade over many years but some of the CDs written back when CDs were first invented are still readable today so nobody really knows how long they might last. There is a similar problem for HDDs, while in constant use MTBFs are well established, but for a HDD that is written to and then left unpowered for many years, well again nobody really knows because we haven't observed it yet. I'd say go for both, obviously HDDs have the massive advantage that you can plug one 500GB and write all your data to it all in one go. To store that much on DVDs will take you days or weeks to write to each disc one or even two at a time. I know, I have over 1.5TB of data backed up on DVDs which number over 500 already.
right up until you have an enviroment diasaster.
(enviroment can be as small as a tiny fire in the power supply of that PC)
Theft of the PC? are you covered?
FFS it is IN THE SAME CASE!
every day http://en.wikipedia.org/wiki/Special:Random
Write the information on stone tablets. If you can't be bothered to do that, then at least use a medium that doesn't require electricity.
I believe that ridulian crystal is your best choice for long-term storage.
www.timcoleman.com is a total waste of your time. Never go there.
More than one copy in separate geographic locations is important but media diversity also is. Suppose you buy 3 500Gb HDs from the same manufacturer just to discover they have the same defect. There is no way to have a catastrophe proof backup without putting a great deal of work in it. Disciplin, automation, location diversity, backup media renewal, media diversity, everything is important.
DVDs will fail on you when you least expect it. Service providers will dissapear over the years.
The only sure method is to maintin the backup, and incure the monthly maintnenence fee (either actual money for backup service, or consume your time maintining the backup server yourself).
My primary machine is Raid5, so data is not lost. My backup server is Raid1 (because drives will silently fail). I use a backup service just in case the other two fail (but sensitive information can not go there).
My experience is that a server can last almost a year without hardware failure. Same with hard drives: I have 6 drives, one or two will fail inside the year.
That reminds me, my backupserver died last week. :(
Actually Microsoft has a free, rather nice tool that functions mostly like rsync called SyncToy. It's a native Windows application so you don't need a Windows-compiled rsync like many other tools do. It's GUI based to make is easy for the most unfamiliar of users but it is useful even to the advanced for periodic synchronization of data from Windows hosts to other drives or a network location. It has a built-in scheduler to allow it to function as as service.
(Note: This isn't an MS add. I recently discovered this tool by accident to backup my wife's laptop and was pleasantly surprised it "just works")
Some people take their .sig way too seriously
Jungle disk frontend for s3. Easy, pay as you go and redundancy and multiple backups are taken care of by s3
If Linux, some sort of cron job running a Python script that copies /home to an external hard drive.
why a python script?
Do you even lift?
These aren't the 'roids you're looking for.
Depends on your definition of 'personal' files. Video can take up a lot of space.
... honestly, Slashdot - and others - have covered this time and time again. Nothing has changed. There still isn't a cheap digital storage medium that we know for sure -will- retain your data -and- be readable (in terms of media -and- the hardware to read that media) down to the very last bit for your grandchildren.
IF and when there's a breakthrough, I'm sure Slashdot is one of the first places you'll hear about it.. but it won't be in an answer to an Ask Slashdot.
http://it.slashdot.org/article.pl?sid=07/04/21/1257249 - Digital Media Archiving Challenges Hollywood .. and so forth and so on.
http://hardware.slashdot.org/article.pl?sid=06/11/20/2036247 - Archiving Digital Data an Unsolved Problem
http://hardware.slashdot.org/hardware/06/12/11/1714232.shtml - How To Choose Archival CD/DVD Media
http://hardware.slashdot.org/article.pl?sid=05/06/26/218250&from=rss - Archiving Digital History at the NARA
http://tech.slashdot.org/article.pl?sid=07/03/31/2141204 - How To Properly Archive Data On Disc Media
Yes, I realize that you stated "I'm not looking to start my own national archive; I have less than 500GBs and only save things important to me". However, it doesn't really matter whether you're archiving hollywood movies, NASA records or just your own random crap. If it is important to you - important enough that you want it to be "lasting for decades if possible" - then your concerns are the same as NASA's... and they're struggling with the exact same question.
The 'best' answer so far is one you will find in each and every single discussion on this - including this thread, so I'll just point you there:
http://ask.slashdot.org/comments.pl?sid=1061489&cid=26102825
You mentioned 'cheap', as otherwise all the answers saying "duuuuude, ditch the digital - go analog!" might have some validity.. take a wild guess as to what it would cost to have thousands of photos transferred to negatives/prints, or video transferred to tape/film, etc. Plus you mentioned documents.. some of those may not transfer to e.g. paper (easily) at all depending on the 'documents' in question; e.g. CAD files.
Look, I know how you feel. I'm a professional writer and journalist, and the digital files I save are pretty much the only result of my life's work, so I try to save them for the future. I'm in the same position, but I seem to have a system that works.
Amazingly, (or sadly, depending on your opinion) I also have all of the documents and material I've ever created (starting with the disks from my Apple //e from high school in the mid 80's, to my emails and my first book's Word files from the early 90's, and everything since including about 300GB of photos) still immediately accessible and viable. It comes from the following understanding:
Hard drives (or basically any storage medium) can and do fail, but they tend to do so in a predictable pattern, but rarely at the same time, so the key is duplication and regular maintenance.
For example, hard drives tend to have a 5-year lifespan, but two identical hard drives in two different machines aren't going to fail on the same day unless there's a fire or natural disaster that wipes out your room/building/city/whatever.
So what I do is have a system where your data is regularly backed up or duplicated at least two times to at least two devices, and then regularly check them. These days I use syncing software that copies my files to a second drive (internal or external - doesn't matter) and then to a second drive to another computer on the LAN. If the syncing software can't read the destination disk during the sync process, then I immediately know something's wrong with that disk, but it's no big deal to replace it and resync, because the chances of the original disk and the first duplicated filestore of both going bad in those couple of days is basically nil. The syncing happens every night. (And I know, I'm already up to three drives, but two drives would be fine, and drives are cheap anyways.)
Every few months or so, I grab a portable external drive from my office a few towns over and make a new copy of my files, and then return the portable drive. The chances of both all my home copies of my files AND the external drive at my office going bad at the same time is practically nil. And the chance of a major natural disaster destroying the disks at both locations is very remote too.
And then replace any drive that fails as the years pass. The new drives will be much bigger than the originals, allowing for more room for more data as the years pass.
Frankly, I think the notion of keeping digital data store that remains inert for decades is silly. You're always making new data. You (probably) always want it backed up. So the data store should always be changing. Once you accept that, and then accept a storage medium that allows for regular changing and updating, and then accept the need for duplication, then data storage isn't risky. Add automated software to do the syncing for you, and it isn't even troublesome.
(And what about those Apple // files, you ask? I have two working Apple //s I still play with to read the duplicated original disks. But those disks were also copied to a Compact Flash card, which I now use on the IIGS as a hard drive. The CF card is backed up to the PC, which an Apple II emulator reads just fine. The backup image of the CF card is always synced between the hard drives. I know, TMI.)
I know M$ isn't that popular here, but skydrive.live.com gives you 25 GB storage for free. And I really doubt MS is going anywhere in the next couple decades. Plus, I'm sure that the 25 GB limit will be increased in the not so distant future.
Paper lasts for centuries
http://www.ollydbg.de/Paperbak/index.html
http://ronja.twibright.com/optar/
How about Solid State storage drives? They are increasing in capacity. Intel currently has an 80gb model out. It uses NAND flash. They are quite expensive right now though. That 80gb model costs about $600-$700. Of course as the manufacturing process gets better the drives will become cheaper and the sizes, larger.
Gmail is only free if you don't care what google (and their CIA/FBI/government overlords) knows about you.
Do you even lift?
These aren't the 'roids you're looking for.
[From Slashwayback]: Dear Keypunch, I have data I want to keep for decades. Should I invest in a good card reader, or should I transfer my data to these far more efficient but newfangled "floppy disks"?
It's pretty ridiculous to expect one storage format to be viable for 'decades'. Not because it goes bad (even though it probably does), but because you're not likely to be able to maintain the necessary equipment for that long. If you find a storage solution, you need a retrieval solution to equal it. What equipment will you be able to find decades on that can access your storage, even if it stays good? You have no idea.
I've been maintaining a collection of Apple IIs and recopying the programs and data regularly (mostly through full HD backup, reformat with error block deletion, reformatting and replacing) to keep it readable. I have machines and data between 20 and 30 years old. I recognized long ago this had become a hobby in its own right, as most of what I had hasn't been of interest to me for many years. The little bits that have been useful have been transferred to newer machines and formats several times. That's decreased as more and more of it can be found easily on the web (previously FTP/gopher/etc.).
Get used to transferring your data to new formats as they come into widespread use, and recopying as necessary to keep them readable. Or else:
[From Slashwayforward] Dear Galactic EM Field Computing, I just found about 20 pounds of aluminized plastic disks that used to have data on them, but I can't read them to tell if I still want it. Is there any museum that might want these? Or are there still any operating plastic recycling centers that might give me a few bucks for them?
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
I would suggest using at least two quality options from:
1. Apple's MobileMe (used to be called .Mac): you get only 20 gigs of storage with a basic subscription, but storage upgrades are cheap enough.
2. high quality managed (or partially managed) hosting - if you need this anyway for business use, get extra disk space. Make sure that their backups are regular and secure.
3. Other paid for storage options.
4. back up to DVD-Rs, and recopy every 2 or 3 years. I buy different brands, and rotate which I use. Redundant copies, redundant copies, redundant copies...
Basically, secure long range backup requires laying out some money.
For video, giving copies to friends and family members is good also, except that DVD-Rs have a short shelf life, unless you pay a lot extra for gold foil DVD-Rs.
Take two hard drives, different makes / models, dump all your important data to them and put them in a fire proof safe.
Hard copies of any important documents. Its been working for hundreds of years and is impervious to technology leaps. It might sound archaic now but in 10 years when your .wpd file or whatever needs a 8 year old program on a 15 year old OS to be read... well... it just makes sense.
The other option would be to save everything as .txt or .rtf files. Universally readable on any OS.
As for graphics files, I would try the .jpg or .png format. All programs still read .gif's, and those are 20 years old at least. I would suggest that most programs 20 years from now will read .jpg files or .png. As for media storage... I dunno. Perhaps a dedicated laptop with SSD drive big enough for all your data would be good. Integrated OS and media reader, should be good for 20 years.
Even quality DVD surfaces (on DVDs you can burn yourself) degrade quickly over a period of time (in my experience 2-4 years). Doing a re-backup every 3 years is too risky, it would have to be every two. In my case, with close to 1.6 TB of personal data (video, pictures, the works) it is not even practical, it would mean doing a re-backup of a DVD every two-three days.
My answer is to periodically copy all of my backup data to a new, larger hard drive. Drives get bigger and cheaper...my data grows...perfect match.
As interfaces, formats, or software become obsolete, I always keep up with the current technology.
I am skeptical of any solution that promises tens or hundreds of years of accurate retention.
I think the issue is that people are thinking about this incorrectly. You don't really want to 'archive' this data -- keep it with you! Keep it with all of the data that you are using day to day and back it up and move it along with that.
My home workstation still has files from 15 years ago on it. I've replaced the computer many times, had a few hard drives fail, etc. but I've always restored both current and 'archive' data from backups and kept going.
I can't imagine a guy with 500G of personal files. It must take a month just to read the titles of his various files.
Off-topic but I thought I'd share how easy it is to accumulate 500G of personal data very quickly as I'm well on my way there: Digital photography. If you're a serious amateur and shoot in RAW format with a 14MP or so digital SLR, each photo can be around 15meg.
I can take between 250 and 400 photos on a night out with friends, still having a blast at the time, and these all add up very quickly. Throw in photos of family, holidays and such-like and you're running into hundreds of gigs of photos very quickly; I've got just over 200G of photos after having my camera for around 10 months. Software like Lightroom, iPhoto or Aperture makes organising and searching them very quick and easy, too.
The only things I'd be mortified about losing are my photos. It'd be a pain in the ass but code can be rewritten (probably quicker second time round, too). Mail is stored on the server. I can't ever again recreate the moment I pressed the shutter on the tens of thousands of photos I have, and I still haven't found a reliable way to back them all up frequently either.
G.
Assume that something will go wrong. So don't just keep one copy - make sure there are at least two. Keep them in geographically separate locations: maybe with a family member, if you can trust them.
Personally, I'd go for two different solutions: maybe one magnetic and one optical. However, whatever you decide on, make sure to get it all back and test it. Even better, rewrite it every few years.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
i use industrial-grade compactflash cards.
Sandisk used to make very reliable stuff, but they don't produce 'em anymore...
nowadays it seems SiliconSystems SiliconDrive is the only product that's actually designed for extreme circumstances (-40 degC to 85 degC; 8% to 95% non-condensing humidity; vibration, shock and altitude according to MIL-STD-810F) and long-term storage (10 years guaranteed). they also have very detailed datasheets publicly available, so you can easily construct a reader in the future.
the advantage of compactflash? small, durable, and very easy to interface with an IDE-bus - you just have to gamble on the fact that there will be legacy-IDE busses on the market in the future...
Because nobody wants to learn perl or bash, and C/C++ is just overkill.
Wow, I'm surprised Slashdot could be so miss-informed when it comes to this. While I agree that there is no way to put data in a spot for 10 years and expect to be able to use it easily, there are plenty of solutions for this depending on your needs.
If you take the technology-skew argument to the next level you could say you will never be able to store your data for more then a year, because a asteroid would destroy it. NASA is not comparable to even a large business let alone the home user who posted this question.
1. Open file formats or least common denominator formats. Convert documents to plain text. Avoid proprietary formats, obviously.
2. Nothing fancy like SCSI. S-ATA obviously today. I'm sure they will still be offering systems with S-ATA capacity in 2018. Use what ever home consumers are using. 2028? Well since you used a consumer-grade system, you will be able to find supporting systems on eBay. How old is the Commodore 64? Even less popular systems can easily be found.
Buy 5 Hd's....You'll have 5 copies and if you see technology changing, put it on the "new" tech....you'll be fine. The chance that 5 HD's all fail at once is zero. If that worries you, make copies of the drive on DVD's or Blu-Ray. I have HD"s from 15 years ago that I just copied onto newer drives. I left the info on the old drive too, I have 5+ copies of all my data at least. Then the copy on my PC. It would take a nuke hitting Detroit to destroy my data. If that happened (and I survive) I'd just head to my family in LA. I shipped them a drive a few years back that I check in on once in awhile. They send it to me sometimes to update but overall it's chillin' in LA safe.
Might be better to start weaning yourself of anything that will not pass the death boundary. In a few years there will be so much personal information in digital form that no life form will ever get to find any MEANING in it.
Philosophical yes, this posting is.
Help end the use of Sigs. Tomorrow
There just aren't any good long term options today for either consumers or businesses. Yes, CDs and DVDs can fail, but there's another problem - in twenty years are there going to be working devices that can reliably read that media? Maybe, maybe not. Earlier this year, the Storage Networking Industry Association (SNIA) started up a Long Term Data Retention group to address this very problem. Perhaps they'll be able to come up with something.
Until then the only "solution" is to migrate data from medium to medium every few years.
Clay tablets? Carving into marble?
Or try digital photographs. Photography is my avocation. I have 1.2 TB of pictures after 6 years of shooting. I could probably cull another 500 GB off if I really wanted to... But managing that much data for long periods of time is an issue (which is why it crops up here from time to time).
It's not just porn and Slackware iso's.
Faster! Faster! Faster would be better!
Well, you can start with the Wikipedia article on Digital Preservation. And then follow links around some. Or go ask your local archivist. Or find a librarian forum and ask there.
Seriously--they're the experts. And there are a lot of digital archives at colleges and such that are on a small enough scale that their solutions would be practical for individuals too.
How about having it printed on acid-free paper?
Seriously, if you want something that can interface with a common personal computer, then what matters is not the storage medium so much as the device interface. Once upon a time everything had 5.25" floppies, printer ports and serial ports.
Just get a couple external USB2 hard drives, (two for redundancy) copy everything onto both of them, and there you go--ten years of good storage.
(unplug them from the computer and the wall power when you're not actually reading and wrriting on them)
If you happen to notice that USB ports are disappearing from new PC's before then, then you know it's time to convert to the next interface.
It ain't that hard.
At a job I had for a while, I would occasionally have to read stuff off drives that were 10+ years old. And I had that job back around the year 2000, so these drives were from the mid-to-late 1980's.
~
Yesterday I set out in search of a way to store my documents, videos, and pictures for a long time without worrying about them.
Try a different approach, why not simply *stop* worrying about them?
Let your own brain be your backup. Lost great photos and videos of relatives and friends? At least you will have some fond memories of them.
And when you die, and thus your backup is gone, then you can *really* stop worrying about them, like totally.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I have been using the free version of SyncBack (www.2brightsparks.com) for a few years now, and just recently decided to try SyncToy. I have personally used both versions of SyncToy to backup my data using Vista as my OS. Neither version of SyncToy was capable of backing up everything that I asked it to.
First of all, I set it to Echo, which should make the backup mirror the source (ex. Deletions on source are deleted on backup). But, that didn't happen. At least, not all the time. Sometimes it would delete all the contents of the folder, yet not the folder. Other times, it would add new files to the folder while leaving the old ones as well. Very strange behavior if you ask me.
So I went back to using SyncBack because I didn't exactly trust the way SyncToy worked. It actually found files and folders that SyncToy never backed up. I think SyncToy has a limit on either the number of characters in a filepath, or the branches in the tree structure.
I don't know whether it was Vista to blame or not, but I kind of doubt it, seeing as how Microsoft says that SyncToy works with Vista.
So far SyncBack has been the best free backup solution I've used for Windows. I challenge anyone to find a better (easier/faster) free backup program on Windows, because I would honestly love to try it. I just haven't found anything better at the moment, so I will keep using SyncBack.
Note: I am not affiliated with www.2brightsparks.com, just a satisfied user.
I have gigs of photos (wedding, long lost family picnics, etc) and music that I can't bare to lose.
Find someone who doesn't want to lose anything either and setup rsync over ssh. Synchronize often, rsync is very friendly to bandwidth.
I have three levels of backup
I have copies of important files stored on two different drives on my PC, then a current archive external HD, and then I fill external HDs and put them away safely.
I just change my archive drives as technology improves such that I can fit more on one drive. Next in the purchase queue is a 1Tb mirrored raid external drive.
If you move your data to whatever the latest technology is, your storage media won't go out of date.
Works for me anyway.
A learning experience is one of those things that say, 'You know that thing you just did? Don't do that.' - D. Adams
"Store ... for a long time" and "without worrying" are mutually exclusive, just like "perpetual care" is an oxymoron (ever seen a historic cemetery?) Somebody has to periodically check the data integrity and correct single bit errors as they creep in, because over time it's "will", not "if". The key to data protection is redundancy. The key to data protection is redundancy. The second mechanism is "diversity" as in "bio": using technologies that will "go bad" at different rates to cross-check each other. Good cheap alternatives available now are Blu-Ray burners and external (eSATA) hard drives; however, you have to keep checking and fixing data integrity every year (so plan to check every six months). If you have enough data, a RAID 5 SATA NAS backed up to BD-R may be optimal. It should be possible to build a BD-R RAID, but I don't know of any market products at the moment.
Nothing will last for decades without some monetary investments. You can get media that will last for decades, but will the computers 30 years from now be able to read them. The only way to get around that is to also pack away a PC with an OS and hopes it boots up 30 years from now. Or every 2-3 years you transfer the data to new media
If you have a DV tape based camcorder, you can use it as a backup drive and keep the tapes offsite. I'm not sure how much data a DV tape can hold other than its in the multi gigabit range. Being tape media its good for years. I've re-watched videos that I made 5+ years ago on DV tape and the quality was still as good as the day it was filmed.
For pictures, you can always get the ones you must absolutely keep printed (Walmart, Costco, etc..) and keep them offsite in an album or 2. If you want to re-digitize them after a loss, you can always re-scan them.
You can buy archival DVDs. Those are a lot more expensive (3-4$ a disc) but are guaranteed for a hundred years (at least mine are.. but I probably won't be around in 100 years to test them). But again, you have the problem of reading that media sometime down the road. DVD readers will one day become obsolete, as will the data format.
It's better to burn out than to fade away
I use drives, DVD-R (I don't use dual-layer) and things flash drives. Then, on top of that, I use Mozy's unlimited backup service. One reason? If I get multiple media failure, Mozy's a good extra option. On top of that, if I have a problem with a single file, it's just a hell of a lot easier to grab it back that way. I'm using my own encryption key for Mozy. Backing that up? Well, I've got it on a couple of different storage media, including two in a safe deposit box. And if Mozy goes away? Oh well, I've got my other copies, and I'll just switch to a different online backup service.
I'm currently scanning in my old film and slides. It's 1 to 2 GB per roll of film, and there are hundreds. So for me, this is a long-term issue. And for any pictures I really care about that were taken with digital cameras, I eventually want to get high-quality prints for each of the kids as well. Paper has a way of lasting. After all, I know how my digital backups are set up, and my wife does, but it's hard to imagine that someone else might in 20 years. It's really easy for a relative cleaning up your house after you're gone to just think "oh, hey, some old hard drive" and ditch it. Besides, there's something about just being able to pick them up and look at it. I notice that on a computer, my kids want to flip between pictures quickly, but an actual print they'll examine in detail. There's just something different about how people look at it.
Seems like ever year we this this same questions on Slashdot. There is no perfect solution that doesn't require some maintenance down the road. Personally I just keep two 500gb HD in software raid-1 mirror, on an free\cheap old pc (from craigslist) and boot once ever year, make sure both drives work, if once fails, swap. Then when future tech changes enough move the data to that equivalent format.
After all, they are basically EEPROMS.
I would imagine if someone wrote infrequently to one, then only used it to read data 10yrs later, there would be no corruption?
Uh, Linux geek since 1999.
It is just not possible to create a digital storage system that will be archive quality and guarantee it will be accessible in a time frame measured in decades.
Burnable CDs and DVDs degrade, the foil comes off and the plastic corrodes. Magnetic tape may be a good idea, but will a computer, 10~20 years from now, support the storage media?
I have stuff on 360K 5 1/4 inch floppies. If I could FIND a drive that worked, could I get the data? maybe. What about my CP/M floppies?
A 10 year time-frame may be doable. 20 year time is problematic. 30 years? Not a chance.
Recently I've been working with a storage cluster technology from Caringo called CAStor. This is ofcourse for larger deployments and similar to Amazon S3. Main difference is that you can buy your own hardware and run in your own basement. Their long-term strategy is to run their proprietary software on top of commodity hardware. As long as they are around you can always upgrade your iron to the latest commodity and upgrade the storage cluster every 3-5 years. Your data is available via HTTP/1.1. The standard "www" protocol of the Intertubes will ofcourse change over time, but CAStor will adapt or you can pull your data out and move it to whatever "eternal" platform you might find in 10-15 years.
Girls are strange. They don't come with a man page.
-- Michael Mattsson
Are you certain of the integrity of every bit on those floppy disks? That's a lot different than not having a problem the last time you pulled a file off of one of them.
I tend to agree that a lot of media and storage failures are due to hammer-hands, but why risk it when the companies selling the products are almost universally competing on price.
Nerd rage is the funniest rage.
... you can make life for your future self easier doing the following:
1) Only store information in formats that are ridiculously ubiquitous right now. Use (in that order): TXT, JPEG, PDF, AAC (or MP3, if you must), MPEG-4/AVC (h.264). The sheer amount of information available in (an on) these formats makes it more likely that you will be ably to recover them in the future.
2) Keep REPLICATING it ALL THE FUCKING TIME, especially when there is a big shift in storage media (like floppies -> CDs or, right now, magnetic hard-drives -> solid state drives) and keep at least two copies in two different places.
3) In addition, move it into the cloud if you can, as redundancy for your two physical locations.
Mod a DVD writers laser to physically etch a pattern onto a disc.
Create a reader to read it.
Buy really cheap computers to run it from, and store them with it as well.
Physical etchings still survive 1000s of years later in tombs and so on, so why not?
True, the lasers probably won't etch very deep, but as long as it is sealed in a room with very little disturbance, its more of a non-issue.
Also, you probably won't need to use an actual DVD or CD.
In fact, there is probably better materials out there that could be cut to order for etching.
I know that in the next year, solid state hard drives will be release and if I remember, with comparable storage size as regular hard drives. You can load your data onto one of those and keep it in a shoe Box if you like.
And the chance of a major natural disaster destroying the disks at both locations is very remote too.
Guess you don't live on the Gulf Coast or any where near New Orleans....
At least for documents, google docs maybe a good option. Also, gmail can hold small attachments (you email to yourself). My strategy is redundancy, to backup as often as at many different media as possible
Take the size of the total data to be backed and split it across the size if the medium (usually dvd-r at this time) Then archive the total mass split at the size of the medium. Then use something like par to create restore stripes. Backup the restore stripes in triplicate and the data in duplicate and you should be good to go.
Warning: Most CDs are just as bad for archival purposes. Buy CD-RWs. They have a higher grade material that lasts 10 years instead of 2-3. And buy media with a data layer out of gold. Not the green or blueish ones. They are organic and die rather quickly. Especially under light. Look for disks with a golden look.
I personally would go with a DVD-RAM for 2x-3x speed. They also use gold, and are specified to survive 30 years. They also have many other advantages like defect-management (like hard disks), sectorization, being rewritable as often as a hard disk, and allowing full UDF usage. If only they were 500GB a piece... :)
I personally recommend buying a good hard disk where all your data fits on, and then two others that are exactly the same.
Then put them in hot-swappable bays, and create a mirror setup with a basic well-known file system. Now write the data to it and put the disks at 3 different locations (eg work, home, car [in a sealed bag!]).
Now put them into the bays every month, and let a tool completely read and write the data on the 3 disks again, thereby letting the error correction do its job. This prevents the motor from sticking and the data from bit-rot.
And just as important: Keep the system as it is, and keep it around, just for that purpose. Because some day, you may have disks, but no system that can read them.
I would recommend making everything 3-times redundant (eg using 3 systems, and replace the failing ones, as soon as they fail), but excanging the system by a new one with new technology and new media every 10 years should also do the job if you're not planning for a nuclear war.
Any sufficiently advanced intelligence is indistinguishable from stupidity.
I have a pile of floppy disks that are still intact after 15 plus years and I stored them like a barbarian.
How does one store a barbarian?
To the OP:
Answer this question: How important is your data? If it's really important, like it is to a Fortune 500 company, the Federal Government or other important data creator then a tape backup system is worth the investment. I don't get your assertion that they (tape drives) have "low storage capacity". LTO and DLT drives have native capacities of 200GB or more these days. Yes, the drive will cost $1000+ and the tapes are about $50 each, but if your data is really that important that you have to hang on to it for decades, then that's your best solution.
As someone that has lost some "precious" data over the years I will also say this: If it's more than five years old, chances are you don't need it anymore and you're just being a paranoid pack rat. But, if you are dead serious about long-term data storage, then tapes are the way to go.
If properly printed and stored, the photos and documents will outlive you.
For the documents, I believe that you want laser prints on acid-free archive quality paper.
For the photos, call a university art library and ask them for recommended paper & ink combinations.
Still use digital storage for easy access; keep the printed material stored. But until the media is proven, it's probably best to consider all optical and magnetic storage volatile, either due to its nature or due to obsolete hardware. Just be redundant --store thumb drives and a hard drive, for example, and copy the files onto your working machine every time you get a new computer.
If you need storage for 500GB I would go with harddrives, they certainly won't last forever, but they can last quite a long long while and they also happen to be cheap and in the case of USB drives they are also very easy to use if you want to check stuff and also highly compatible with every OS out there. Duo to their cheapness you can simply use lots of them, copy data from one drive to another every year or so and in not much time you will have plenty of drives with redundant data sitting around. And in a year or two you might be able to switch to SSD drives instead of spinning plattern for more reliability.
If that isn't enough I would also suggest to go multiple routes if possible, i.e. you might want to backup your emails and other stuff that is small enough to CD or DVD in addition to the harddrive.
Another important issue to worry about is data integrity, since data that is still readable might still be trashed. So keep MD5 checksums or Parchives around for checking and recovery.
In the end the most important point simply is to keep multiple copies of the data around and if possible off-site, so that a burned down house doesn't kill it all.
...Paper! as what ever is on it will always be scan-able and software can be written to decode it.
So how much paper does it take to print out the code of the linux kernel? Depends on how fine the print is.
There is another, but it has not yet been developed to a consumer level. Perhaps its only an idea in a lab yet.
We can make diamonds and other synthetic gems as well as many other things like Integrated circuits.
By permanently growing the data into the crystalline gem structure or Integrated circuit that is then encased.....
In essence establishing a permanent atomic structure that represents the data.
Of course with enough heat or hammer it too can be destroyed.
The Rosetta project is an organization that seeks to document all of the worlds languages before they die out, and their medium for doing this is to use a laser to burn a copy of some book (I think it's Genesis) deeply into stainless steel plates.
In 30 years you might have to rig up something to extract the data and process it, but it will still be intact.
Also, I'd like to point out that analog and physical storage media last a long time if they're cared for, much longer than digital copies. Think about it: record players are still around, laser discs are not. And even if record players vanished, you could buy a used one, or even build your own mechanical phonograph player (you just need a needle, a turntable, and an acoustical horn to magnify the sound). Try building your own DVD player from raw materials! The same thing is especially true for video. Film projectors are not terribly mechanically complex, so if your video is all on archival quality 8mm or 16mm film, you or a handyman could build a projector to play it in 30 years. Those WMV's? Not so much.
And, of course, use printouts on acid-free paper in a dark, dry storage area, and they'll outlast your stupid Word docs by a factor of ten, or more.
Considering the speed with which both data and medium deteriorate with age, perhaps it is not surprising that ancient civilizations continuously relied on stone carvings and other physically substantial means to record information.
Being able to create something that will last down through the ages has been one of the classic challenges to mankind.
Even if you were able to come up with a solution for preserving data integrity, there is nothing to prevent purposeful destruction by third parties. Even the library at Alexandria burned.
Unfortunately, the only information that has ever been successfully preserved involves science, and that is because we appear hard-wired to reverse engineer how the world works. It is rather difficult to forget the laws of physics when they are repeatedly proven on a daily basis.
Reality is prettier inside my head...
I think everyone here has touched on it, but its something we discussed 20 years ago at work, and its still the same "issue".
The *only* solution really is to migrate your data to whatever new form of media exists every 10 years or so. In the 90's you'd have been burning your data to CD's, but I'm not sure the media itself will last more than maybe 20-25 years. Now we have DVD, and most can still read CD's (plus Audio-CD's will keep the basic "CD" idea around a while). But how long until that goes the way of the 8-track and vinyl records?
Tape generally is worse, someone mentioned the old reel 1600/6250 tapes - yup, used those.. then 4mm DAT, DLT... quite honestly CD's have been a better long-term option than any tape format I've encountered in my 25+ years in IT. Tape is meant for *backup*, not long-term archival.
That of course leaves the other side of the equatio, since you are talking about "digital media" - the software side of things comes into play (had this at work as well)... Does Excel still read "visicalc" files? Will it support Quatro Pro format in 10 years? If you have a pile of documents from the early 90's in WordPerfect for DOS, will you be able to read them when you need to? Convert them all to PDF? Will PDF be around in 20 years? If not, will a copy of Acrobat 9.0 still install on whatever hardware/os you are running then? Could you even install a copy of XP on such a system, to then install Acrobat and read your files? Is the data, in 20 years, even still relevant/useful, or are you just piling up data that'll never be needed? How do you decide w/o taking it on a case-by-case basis?
Lots of questions, and as someone said, even NASA and a lot of other companies haven't figured it out...
Unless it is video, paper tends to be more reliable than all those high-tech methods. Books were still readable after thousands of years.
If your data is not in your possession, how do you know others won't see it or edit it without your permission? For archiving purposes, the best technology is magneto-optical. Despite the fact that makers have been exiting the market due to competition from faster and larger-capacity technologies, MO remains the champ for data storage duration. Remember, it is partly based on a Natural phenomenon that lets geophysicists detect which way the Earth's magnetic field was oriented, hundreds of millions of years ago --data retention just doesn't get much more long-term than that. MO disks are removable from the drive, and every modern drive can read any older same-size disk (they come in the standard 3.5" and 5.25" sizes, but have quite a range of capacities), so if the drive fails, just make sure you have spares. Perhaps, sooner rather than later, the manufacturers will realize that archival storage is a niche market that will demand that they stay in the MO business.
Or let other one pirate it.
Its not really that cheap, and not that simple to use for personal backups. Unless you are willing to write your own backup scripts, its going to be a headache.
Querying S3 for a list of stored files is *very* slow, and you only get 1k results per query. This means you have to index what files you put in S3 in a local db. This allows you to ask the db what files are there (and how to grab them).
If you only have a few files you can use the S3 browser extension for Firefox (or one of a many file system mounting, ftp simulating, etc tools). Just keep in mind the 1k file limit per query and box things in folders of no more than 1k items. Otherwise you will have a very slow browsing experience.
I have around 120 GB of family photos and purchased mp3s that I would like to store. To store 120 GB at .15 per gigabyte/month for 1 year would cost me: $216 (at $18 a month).
We use it where I work, with great success, but it would be much to much work for me for a personal backup system.
Considering the cost, I would go with a consumer targeted app (there are LOTS of them). A number of them charge a flat flee for "unlimited" storage. Beware of how you interface them. Some support windows only.
Try Jungle Disk http://www.jungledisk.com/ . It makes S3 easy to use...
The vast majority of our "irreplaceable" personal data will be completely meaningless in a couple of decades. Personal conceit leads us to preserve everything "for our kids and grand kids." However, your grand kids probably don't need fuzzy snapshots of you with a skanky ex-girlfriend on your lap, or a hundred photos of some long-forgotten dog. The best approach is to create and document a "best of" collection of videos and photo. Preserve it in multiple locations and recopy it onto new media as it becomes available. Come to think of it, this is much the same comment I had last time this question slithered onto the /. front page...
You mentioned 'cheap', as otherwise all the answers saying "duuuuude, ditch the digital - go analog!"
The only way to go is to select the photos that mean a lot to you, pick the documents that you don't want to lose, then print them. Many photofinishers use archival papers today, and a 5x7 is pretty cheap. Store in a cool, dry place away from direct light. Paper can be stored in a file cabinet for decades. Multiple copies can be made and distributed to friends and family.
The cost can be managed. Just think of who else would want to keep all 500GB of photos and documents. When you review this stuff again, if ever, you can pick out anything else that should be saved.
I've never used SyncToy, but for my Windows box I regularly use SyncBack, of which there is a free version available.
"Three eyes are better than one" -- Lieutenant Columbo
There's two reoccurring theme's I keep seeing in the posts here. 1) Is the media I make the backups to going to be readable in 20 years? i.e. will there be compatible hardware and software in 20 years? and 2) Off site storage in case of fire, theft, etc.
1) As one person put it
"Psh, its not like he's gonna wake up 30 years later and go "Oh yeah! My data! I need it right now!" And then feverishly try to push the USB slot into the wireless receiver, and then start screaming "OH THE FOLLY OF MAN"
If USB really started disappearing, you would go get your USB drives and copy the stuff off of them "
2) Again put perfectly by a poster, what if an asteroid came along and destroyed it! I could keep all my stuff offsite, but what happens if the offsite location burns to the ground? There is no failsafe from natural disasters, we all plan the best we can, but to me this is a separate issue and not one that I care about when I posed the question.
Several people point out that this shows up on slashdot every few months, and the reason it does is because there has yet to be an answer. Perhaps I was hoping my predicament was different and there was an answer for it, but it doesn't seem so. Maybe I should scribe all my data to papyrus and bury it in a pyramid, that's the only thing proven to last several thousand years.
It seems odd to me though that, as a whole, as a world that's invented so much technology, we don't have any media that will reliably last beyond 5 years, much less 20. Sure, anything could happen to any media, I could have my data inscribed on steel plates and they would eventually rust. Failure is always an unknown, but I'm talking about reliably. We know HDD's are reliable for at least 2 years, most are warranted for 5. In this technological era, why don't we have anything that's reliable in for 20 years? Who cares if it's readable, sure, a fire could burn it down, but we don't have one digital storage solution that's reliable for that long? I guess this was what I was hoping would be answered, but as has been stated, there is no good answer yet.
BTW, the best solution I've seen so far is a 500GB external and a live backup living on my computer. Hopefully someone can come up with something better when this question is posed again in 3-6 months.
Suck it up and purchase backup hosting from Mozy. That's your best bet. $5 a month for someone else to be responsible for your data. You also have the opportunity to add to your backed up data whenever you want with an unlimited cap. It'll be a lot less expensive in the short term than buying 2 external hard drives.
Comment removed based on user account deletion
I realize SSD's are still pretty expensive for large capacity storage, but they will come down in price over time. They are already a great solution for smaller capacity needs. SSDs can detect when a cell of the drive is going to fail, and write the data somewhere else automatically (unlike with traditional HDDs these failures tend to occur on writes rather than reads). If you put two or more of these together in RAID, that's a pretty secure backup.
It doesn't solve the problem of natural/environmental problems of course, but I suppose you could make monthly off-site backups if you really wanted to (to either a standard HDD or another SSD).
There really is a simple way around this - and it is what I've done - I've got data 25 years old and it's still relatively easily manipulated with a little work. I've found floppy disks are relatively resilient, and old hard drives seem to keep their data for a long time. I've got a computer, display, keyboard, and associated peripherals stored for every generation of data that I kept:
1.I have a Commodore 64 with floppy drive and cassette drive stored in a box with the floppy disks and cassettes from that generation (late 70s/early 80s).
2.I have an IBM PC/XT with keyboard, a 5 1/4" floppy, 3 1/2" floppy, internal 20MB hard drive, and CGA monitor stored in a box with a load of 5 1/4" floppies filled with data from that generation (Mid 80s).
3.I have an IBM RS/6000 with display, keyboard, and mouse and internal 500MB hard drive loaded with all my docs and projects from that generation (early 90s).
4.I have a Pentium 2/300 PC * 15" monitor with windows 98, CD R/W drive, 3 1/2" floppy drive, and USB ports - and a crapload of CD's and 3 1/2" floppies full of stuff from that generation (Mid/late 90s).
When the current generation looks like it's going to be moving on, I'll put away a Core 2 Duo system with 1 TB of hard drive full of stuff with the different OS's I used loaded on it with boot manager (Ubuntu, XP, FreeBSD), a crapload of USB keys full of documents, along with burned DVDs etc. That'll take care of the "'00" generation.
The answer lies in not only archiving your data "of the generation" but the essential equipment needed to access it. I may have a heck of a time moving data off of my Commodore 64 - but I can at least see it and access it - I believe I stored a modem with it - so at worse I could set up a terminal server that it could dial into and dump data to. All the other systems I'm pretty sure I could recover stuff from - even if the PC/XT does have an MFM hard drive, etc.
I have data 18+ years old . You're approach is admirable, but why not just move your data forward with technology?
When floppies started dieing, hard drives got large enough so I moved all data off the floppies to hard drives and optical media. When word processing software I used started dieing, I moved all my documents or obtained converters to MS Word format. Also, I don't archive music and movies. I do archive pictures etc.
I believe the best approach is just keep your data moving forward & current and not in some archaic format. This means I have 3 redundant copies of all my data on hard drives using a current OS.
Granted I don't archive "silly things" like music which I can re-create; but rather just personal data (i.e. personal documents, pictures, personal videos) so the total quantity of data after 18+ years is only about 13gb.
Dreamhost has a service they call file forever. It is $.01/4 MB to store your files forever. You pay once and you can read your file forever.
Now the question is what happens if they disapear...
Either that or archival grade tape. That will cost more than 1000USD/EUR though.
There used to be a solution that was adequate: MOD. But nobody cared enough and it was not cost effective to develop it further. It is still used to archive digital X-rays, but apart from that it is pretty dead. BTW, the MOD "successor" DVD-RAM is a bad joke.
As to USB-FLASH, you will still need to check and refresh the contents every 5 years or so.
My personal solution is to have a archive directory on all my machines that is synchronized. Currently I do this manyally, but I plan to move to rsync-backup or an SVN server.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Well, I guess I cannot recommend BackBlaze then. I pay only $5 per PC monthly to have every file backed up, at a remote server.
Very, very easy to set up. I installed it from their website, and that's it. Simplicity is key.
Has anyone considered twibright optar...data storage on paper! woohoo!
http://ronja.twibright.com/optar/
The oldest known written story, The Epic of Gilgamesh, was written in cuneiform on clay tablets, about 8000 years ago. They were probably baked (or perhaps just kept dry). Some of these still exist and are as readable as when they were written. Some combination of current technology with modern ceramics might work well. Store them in a deep mountain cave in Utah.
Stone tablets last a fairly long time.
But if you really must go digital, then you're going to have to put in a little bit of work and money every 5 years or so.
Just store a machine that can read the CURRENT media. Then, 5 years from now, get a controller for the old machine that can handle the NEW media. And transfer your files from the old stuff to the new stuff. And repeat the process every 5 years or so.
Don't go with whatever the cool tech is. Always go with the mainstream stuff.
I have a friend who recently copied over all her Zip Disks to USB sticks. Now she's set until that format is replaced.
Of course, this only addresses the files themselves. Who knows if there will be anything that can read the data from them in 50 years? There are word processors from 20 years ago that created files that it is very difficult to find a reader for now.
Several people have tried; but, noone has done it right. So, here goes
Etch the data into stone tablets. Choose a hard stone unaffected by water immersion; the harder the stone is to work, the better. Include a primer in multiple languages, which explains the content, data format and duplicates the first page or two. Make multiple copies of it all. Distribute in many locations worldwide. Every few years add more copies to distribution.
That should give you a few thousand years to come up with a better method. If your lucky, a copy will be incorporated into a building, monument, or other structure. This will enhance the data's chances for survival.
Other people have touched on it, so I will too. If you have the material in some proprietary document format, even if that format is the most popular format on the planet right now, still convert it into multiple versions of other file types. Here's an example I had to deal with: ClarisWorks (.cwk) was the most popular computer document format before Microsoft Word introduced the .doc format, and I had a lot of docs in .cwk from my old G3 Mac. Apple stopped supporting it a year ago, so I had to save my older (AppleWorks 5 and before) .cwk documents, and resave them as an RTF file for Pages before transferring them to the new iMac. I now have them saved in PDF (that can be read in Preview, Photoshop, and a multitude of PDF reading programs), OpenDocument format (.odt with NeoOffice, Mac's version os OpenOffice), HTML, Word document, and that Rich Text file.
The same could happen to .doc files in a decade or two. MS have decided to push the .docx format in newer versions of Word, so who knows when the older files will require some workaround to be accessed?
Do the same with images (RAW, JPEG, PNG-24), sound files (mp3, AIFF, WAV), whatever it takes to ensure your media won't be unreadable.
Shiny. Let's be bad guys...
For 500 gigs.
A couple of hard disks, stored in different locations (cities, not drawers), that you update+check once a month or a quarter.
Burn DVDS of the really important stuff (pictures, documents) around once a month, and mail them to your parents/family.
What NOT to do:
- RAID is NOT a backup solution, it is a high-availability solution. Of all the problems bakcups need to adress (theft, destruction, viruses...), it solves very few.
- don't keep your backups online and/or in the same spot: viruses, power surges, fires, theft... will destroy them
- don't have only ONE backup: Murphy's law, if your live data disapears, the backup will turn bad also
- don't forget to check that your backups are still good
- don't delude yourself into thinking that any physical media in use today will still be easily readable more than 5 years from now. (except for the consumer type media: CDs, DVDs)
The Cloud - because you don't care if your apps and data are up in the air.
"If Windows, I dunno." NTBackup? http://en.wikipedia.org/wiki/NTBackup
Sorry about the mess.
True. I only backup the stuff that is very important to me, my /home is about 16GB or so, so that's an easy thing to manage, various other things, (like ISOs and stuff I keep handy) i have on multiple machines anyway, those can be re-created from the source media, so those aren't an issue. Media I don't backup since it's a) too big, b) too much effort to do so and c) I don't really care, because it can easily be acquired again.
Personally I'm longing for the day when 64GB flash disks are at the price 4GB ones are currently, because having 3 of those (1 off site) would be a huge benefit to me.
Sure, you can easily shoot hundreds of pictures a night. But, FFS, sort those suckers! Maybe five of those are good shots, and if you're lucky one of them is a great shot.
Just toss the rest! Really! Nobody cares for the reams of out-of-focus or incorrect compositions.
Keep the great shots (one in a hundred, if you're a good photographer) and delete te rest.
Jpeg2000 supports lossless compression. Image won't degrade over time, as with normal jpegs, AFAIK.
http://www.oreillynet.com/pub/a/javascript/2003/11/14/digphoto_ckbk.html
http://en.wikipedia.org/wiki/Lossless_JPEG
Main difference between the BSD license and the GPL license: one is from California and the other is from Massachusetts
I have about 6gb of irreplaceable stuff, that often changes (development source, financial stuff, pix), and I use Karens Replicator http://www.karenware.com/powertools/powertools.aspto sync it nightly to my Windows2K3 server, then once a week it uploads an incremental to Amazon S3 via Jungledisk. The laptop has a copy of the 6gb of irreplaceable stuff, on a Truecrypt container on an 8gb SDcard. Everytime I use the laptop, I sync the SDcard via Truecrypt/Karens Replicator on the desktop machine... Works fantastic for me!!
THANK YOU, Edward Snowden!! Americans owe you a debt of gratitude (whether they know it or not..)
I myself just addressed this problem... It's for a different situation, but it works. I keep all my Data (Even old stuff I don't care about like family pictures from the past 5 years and other stuff) on my storage drive. My storage drive is one of the new 1.5TB seagate drives. I have a external enclosure connected via eSATA with another 1.5TB and I use Synctoy 2.0 to backup and echo the changes made on the internal storage drive every few days or week... and if one of them fails I just buy another and copy it over.
I suppose it depends on the specific type of document or file you want to save, but I have been using Google Docs for years to save my important documents. Its fairly safe storage [I dont see google going down any time soon], and available from any PC.
Long-Term Digital Dilemma
http://blog.longnow.org/2007/12/24/long-term-digital-dilemma/
Apparently, Hollywood is resorting to cellulose.
Hey, let's have those Super-8s reborn!
http://en.wikipedia.org/wiki/Super_8_mm_film
Main difference between the BSD license and the GPL license: one is from California and the other is from Massachusetts
I am doing much the same thing, but with FreeBSD. For long term storage, how about a RAIDz with 2 parity drives, and use the largest SSD thumb drives you can find/afford. They ought to last a while, and up to two of them could fail with no data loss.
If somebody is skilled enough to just think about writing scripts on linux, and knows python, would have to 'learn' simple bash scripting? GTFO.
Yeah - sure - until Amazon goes out of business or gets bought and then the new owner dumps the service and you're S.O.L.
RS
Shoes for Industry. Shoes for the Dead.
Go buy two 1Tb hard drives. ~$300 bucks total. Put your data on both of them.
Wow, that was hard.
Seriously, who lets this get on the front page?
Incorporate yourself as a 'record company', and then call all your videos and documents your product, your intellectual property. Join the RIAA and paste BIG warning labels on the opening screen/page of every file stating a 100-year prison sentence and 100 billion dollar fine for every instance of unauthorized copying of any of these files.
Naturally the files will get copied and stored by thousands of people. The files will get stored on secret third-world sharers dedicated to illegally preserving and presenting the world's culture.
Then in a few years from now, when you want a file, just search for it on BitTorrent or Kazaa!.
The record companies don't realize it, but the internet has changed them from being musical recording distribution companies to being digital storage companies. The whole point of the P2P networks is not to pirate so-called intellectual property but instead to ensure that the digital files (into which this 'intellectual property' has been formatted) don't get destroyed by media damage or obsolescence.
The P2P networks were a gift from the computer nerds to the recording industry; a little transition present to get them from the 20th to the 21st century. It's sad that they don't have a clue as to what to do with our gift to them.
Just ask yourself what destruction is going to happen to your life if you lose the data. If the answer is "no problem", then hit Delete or just let the data to rot in an old RAID-1 setup until both disks crash and then just stop worrying.
If, however, you have some data that you think are particularly important, you can rent a safe deposit box in a bank for about 30 EUR per year (for an A4-sized box) and put copies of your data in there, paying attention to choose a bank that is too big to let fail :)
For text, nothing beats non-acidic archival quality rag-paper or vellum. Make sure to get the right kind of non-acidic paper and ink. Print your stuff out. Either bind it or file it in an organized and indexed manner. It will truely last hundreds of years, will always be a "universal" read-write format, and will never be technically obsolete.
This also means you will probably have to review it to seperate the wheat from the chaff. The paper and the space to store it will be limited. So you will have to cull out the junk and figure out what's worth keeping.
You can't store much information on paper/steel tape. It might be more economical to use stainless steel punchcards.
It is called hard encryption. Learn it. Use it.
You don't know that a jpeg, for example, will be readable in 30 years. The format may be so deprecated that there might not even be a viewer available.
Not bloody likely. PCX is deprecated in favor of PNG, but PCX is implemented in too many Free programs to ever disappear. Likewise, JFIF, the file format defined by JPEG, won't die until C and C++ die because just about every program written in C or C++ that handles photos uses IJG's libjpeg. And given how long Fortran has been in use, it's not bloody likely that C++ will die any time soon either.
Like my old Microsoft Works 4.0 documents
I see where you're going with this: formats that are publicly defined only as "what some piece of proprietary software writes". Convert your documents to a format with a widely recognized public specification, such as HTML or ODF or PDF, and it will be much more likely that you can compile a Free app (even an ancient one) when you need to view the data.
assuming that I can generate a virtualized environment that can support an old Windows (Windows XP probably won't even boot on any PC being produced 30 years from now).
That's what an emulator is for. NES games won't boot on any console produced today, but Nintendo still sells them on Wii Shop Channel bundled with the "Virtual Console" emulator. (And there arrr other ways to run ROM dumps on a PC.) So wrap up your environment in a virtual disk image in a well-documented format, test it in a current Free virtual machine such as VirtualBox, and you'll probably be able to emulate it later.
For long term storage you not have to worry about media degradation but also file formats. So far being able to read old data has been complicated over periods like 20-30 years. For example I've moved from: Apple/Mac Write, Wordstar, Wordperfect, Amipro,
Word, open office write over the last 25 years.
This actually is a good idea. If the porn files were maybe one half sex imagery and one half encrypted private data, and there was no easy way to separate the two halves, then people would download, store, and upload the files in order to view the porn. Anyone who had data in the private section of the file could download it from various P2P sites.
The cost of filming and creating the porn file would be covered by the people who would be using the file for long-term distributed storage. Say a 1.5Gigabyte file that was an hour of MP4 video entertainment and 500 megabytes of distributed storage. The fees received by the producers for the storage would pay for the video production costs. Since porn is cheap to produce, this may solve the problem of piracy and secure storage at the same time.
Having read the comments and having given this issue much thought over the years, I have to say that only useful ideas will be preserved, and no one has 500GB of useful ideas. My point is that the fate of one's personal data archive is ultimately in the hands of others, and they decide what's important.
It could be worse. In the pre-technological era, apart from a small handful of writers, the closest thing to a persistent data archive was a gravestone encryption. On that basis, and with an appropriately skeptical view of the durability of storage media, choose a small handful of critical data items and engrave them onto a stone. This actually works -- there's a beach in Wrangell, Alaska that archives messages from a prehistoric native culture, engraved into stones. The messages make up with persistence what they lack in depth:
http://www.wrangell.com/visitors/attractions/history/petroglyph/index.html
...since DVDs etc. can be retrieved with any high-res optical scanner. You just have to fetch the software to read it.
It seems to me if you bought half a dozen 500GB drives, then in 10 years time at least one (probably most) of them will still work. Get different brands, different sizes to guard against particular problems. Maybe even do the 4GB Flash thing as well, and get some good quality DVDs too. I think the key here is not to put all your eggs in one basket. This is an open research question, and we may be interested how your solutions go in 10 years time.
Oh, if you want to be *really* sure, do the Mormon gold plates thing.
Anyone wanting something to last for 10 years should include the player. Don't bother putting away a CD, DVD, tape OR hard drive. Put the entire computer into storage. It would be foolish not to take it out once a year and fire it up just to keep the oils and lubricants working. If it's important enough to store for 10 years then it's important enough to check once a year. If you put anything like a CD, DVD, or other item in storage for 10 years without a player you take a huge gamble but if you include the device it runs on you have a chance to view it. You may have a hard time retrieving it because there may be no USB or Firewire connectors (there will most certainly NOT be any such connectors). What that ultimately means is you CAN'T easily store electronic documents that long. You must convert them from device to device, from tape to floppy to zip drive to hard drive to holographic drive to biometric drive (I come from the future). A storage medium from one generation can't be read by the next generation so the question is about storage AND being able to access and convert the data - otherwise storage itself is meaningless.
Just get a couple of online rsync hosts. Two, in case one goes down. When you get a new system, just sync the data back to it.
Although, since you do want to store 500 GB, that could be a little expensive at this point. Maybe you can juggle external drives for now, and then do an online host when the price comes down.
Computers are useless. They can only give you answers.
-- Pablo Picasso
We already have. It's called JPEG 2000, using wavelet as opposed to DCT to compress the image. This gets rid of the blocking effect prevalently observed in highly compressed JPEG files. However, I don't see JPEG 2000 making JPEG obsolete for various reasons.
However, there is still much research to be done on inter-frame compression used for motion pictures. I think we had hit a limit in intra-frame compression, so the focus for highly efficient video codec has been on synthesizing interpolated frames from neighboring frames. At this point, I think video codecs, even H.264, still stands a high chance to become obsolete within the next decade.
I once had a signature.
Why store so much data? Because you can?
Storing countless photos and hours of video is a nice idea, but really what is the point? Are you going to review it all? If all you do is maintain data from the past what kind of a life is that?
You should be very careful what you hold on to. It becomes a weight that you have to carry. Filter it down to the very best, and only keep that. You might find it easier to keep a smaller amount of data.
I have thousands of photographs that I've taken over the last ten years that I back up to multiple off site hard drives. I am fairly good at culling redundant or poor photos, but that's still a lot of photos. I have an immediate need to be able to search for specific photos in all that data. Currently I use Lightroom 2.0 for that. It works pretty well but I consider it "expert friendly". If I die and my daughter inherits all that data she will probably throw it all away if she doesn't have an *easy* way to find the hand full that may be important to her. Organization, cataloging, and search tools are critical.
If NAND flash SSD lifetimes are determined by write frequency, then wouldn't this be fantastic for archival storage? Just write the data once, then read it as many times as you like.
Dreamhost has a system where you can store your files "forever". I think you need a Dreamhost account, though.
For truly long term storage you're going to have to leave the realm of computer tech and look at archival methods. Microfilm may have a shelf life of up to 500 years if stored correctly (according to Wikipedia). You could convert binary data to a visual representation with something like Optar (http://ronja.twibright.com/optar/), and then microfilm that. Optar has a data density of around 200kB/page and around 2,400 letter-sized pages can be stored on a spool of microfilm, leading to a storage density of roughly 450MB per spool. Not great, but this is data that you *really* don't want to lose.
Which, of course, refers to the process of getting the data (in binary form) tattooed on your body by tattoo artist Tegan Stadnyck of Seabrook, New Hampshire.
For me, the amount of time required to check back through and move over 10,000 or 20,000 documents "per generation" just isn't reasonable. I move forward email from platform to platform - but that's about it. I figure one device per "8-10"years put away means I shouldn't really have more than 8-10 devices I need to archive throughout my lifespan. I rarely ever fire up or use the devices - I started up the XT about 6 years ago and copied off it 500 or so things I wanted (it had an old 10 base 2 network card in it - I still had an old 10base2 - 10baseT repeater around) - I loaded a packet driver and used an old DOS based FTP to move the things off to my current server. I may fire it up again someday - I may never. It doesn't take up too much room in the office closet.
I find that format shift isn't too bad - my documents that were on my XT from the mid/late 80s were Word Perfect, GIF's, Lotus 123, and a few other applications, which opened just fine in modern apps. I know everything on my "90s" era computer would open fine on any modern application. Again, "Commodore 64" era documents are likely the only thing that I couldn't easily work with now - of those I really only have maybe 200... and I doubt I'll ever actually *really* need anything 25-30 years old.
For my valuable data (documents, family pictures, etc) I keep
(a) the originals on my hard drive
(b) a second set on another, offsite computer
(c) a third set on an external RAID-0 array, which I power up only when in use, and
(d) a bi-annual DVD backup on quality media with lots of error correction
I used to keep all sorts of parity data for the DVD media (QuickPar, etc), but recently discovered a program called DVDisaster which actually lets you augment the ISO/filesystem of the DVD itself with the redundant data. I typically archive 2Gb of files with 2 to 2.4Gb of error correction data per disk. I figure that should give me a hefty boost in reliability, especially if I follow good DVD storage practices.
Obviously, also keep a copy of the DVDisaster program handy. It's open-source GPLv2 and multi-platform. I include it in each ISO.
A preposition is a terrible thing to end a sentence with.
He talks about keeping many 4G Flash disks. Why not use few big SSDs instead? I think they should be as durable as Flash disks. Am I wrong?
Persian Project Management Software as a Service
You can increase your reliability by renting rack space in a data center and putting up your own "backup box" that essentially mirrors a "backup box" at your home/business location. Use something like DRBD (over a VPN connection) to keep the disks in sync, and secure the everloving hell out of the remote box.
It's still not perfect, but does get you around the "house burns down" type of data loss, and you can still periodically replace the "local" and "remote" boxes/drives as time goes by.
The downside is that colocation hosting isn't free, but every "9" you add to the reliability of your backups is going to multiply your cost in a non-linear fashion.
You can have reliability or low cost - pick one.
You guys remember this?
http://arstechnica.com/news.ars/post/20061126-8288.html
See also:
"Can you get 256Gb on an A4 sheet of paper? No way!"
http://www.techworld.com/storage/news/index.cfm?newsid=7432
Overall, it was a scam. But the idea of somehow using a durable physical medium seems pretyy good, no?
PS: OK, this doesn't solve the OPs original question.
Main difference between the BSD license and the GPL license: one is from California and the other is from Massachusetts
1) save it on a HD in yer house and do DVD backups.
2) save it also at another location such as safety deposit box (dvd's and/or thumb drives)
3) save it on google or wherever.
4) transfer it to new media as it becomes availalable.
simple and simple.
So far several of the strategies revolve around the premise that you still need to keep your data safe if your computer, in-house server, house and neighborhood all blow up. Now given this situation, one of two events has happened:
A) World War III
B) Our Alien Overlords are displeased
Now assuming one of the above did occur, having backups, even for a super secrete important business file is going to be moot. The only secure way to store an archive or backup is to take it out of phase with reality such that it is no longer subjected to the dangers of time and space or other things out of phase with reality. This being said, there are some ways to increase the survivability of your data.
First, keep at least one on site copy as far away from danger as possible. Dig a nice big hole in your back yard. Build an air/water tight lead lined box with several hard disks and perhaps a few flash drives and a stripped down computer capable of accessing the media and place the whole thing in said hole. Next run some piping underground from the box to your house or business and through the basement wall. Make sure to have some string already through the pipe to do wire runs or just do the Ethernet and power cables as you go. If the cable is less than the maximum transmission length (300 feet I think) then you are all set. If you really want your data to survive your house being bombed then multiple boxes could be used to extend the distance away from your house along with some form of wireless communication (very low power directed signal to prevent hacking or discovery) to prevent any of the pressure waves resulting from the explosion from destroying your data at the other end of the pipe.
This plan does have a few drawbacks as you are basically hiding in plain site. First digging the hole and pipe trench can be a major pain as would retrieval every few years to replace parts. Second, there is no fast way for retrieval if that data or power connection is damaged. Finally, heat dissipation could be problematic depending on the design of the case. The ground temperature usually stays fairly constant (if you go down far enough) so testing during spring/summer won't mean things will freeze during winter.
Off-site backup has already been explored to death with the main conclusion being that it is necessary. Obviously there is only so much you can do to protect your data especially with regards to cost. You really just have to decide how important your data is, how much it would cost you if it was lost and if that amount of money can build a decent backup strategy.
while(1){sig.get()}
How come these stupid questions make the fron page? Slow day? Of course, start your own frickin' server! and learn to care for it! Problem solved!
PS LTO-3 tapes hold 400 GB uncompressed. Since you are asking stupid questions on slashdot, they will be all you need. If you find them expensive or something, then your data doesn't deserve to survive. Ding! Next stupidity!
For some of my important documents, I've been writing them up in TeX. The format has been around forever. It tends to format everything properly with new releases*. And since you have the 'source code' not only to the software but to your document, you should be able to scan your source code from paper format should your digital copy go bad and still have close enough to the original in 10 years with maybe some slight changes in formatting.
* Compare this, at least to multiple document layout and file format changes with MS Office (which 90% of the PC world uses). I tried restoring a Word 1997 document in Word 2003 and the formatting was really buggered up. I didn't wish to repeat that.
Open and well documented binary formats should be OK, e.g. there are so frigging many devices that read or write to the JPEG format that devices that read the format should be available for decades. As an example, up to about 1980, most consumer grade record changers could handle 78 RPM disks, even though the format was obsolete in 1950. Similarly, it is still possible to purchase turntables that will handle 7" 45 RPM records - a format that was introduced in the late 1940's and passe about 1980.
If you want true long term storage, simple paper is probably the way to go. Invest the time and money to print it, and organize it, and (especially if you use special acid free paper) it can easily last for centuries and you never have to worry about either file format or medium becoming obsolete.
My other suggestion, is to accept that it will become unreadable after a certain amount of time, and simply make at least two copies, stored separately, and every few years (5 is probably a good number), take the time to move it to whatever medium is sota for archiving at that time. You can't forget for decades at a time, but you can forget it for years at a time and the copying should only take a couple of hours. When you change mediums you can also look at changing file formats as needed.
Simple. Massive redundant storage under Cheyenne Mountain, plus blue-green lasers and a simple COBOL program which writes 1's and 0's onto the Moon.
``Tension, apprehension & dissension have begun!'' - Duffy Wyg&, in Alfred Bester's _The Demolished Man_
Archival quality paper. You can keep it in an unheated storage facility, and you won't need to find obsolete equipment to read it when the time comes.
Seems like every three months or so, this question pops up on Ask Slashdot:
Yup, sounds like middle management material to me!
The real answer to the OP's question is that no current consumer-level media is suitable for archival purposes on its own. To really archive something, you need high-quality media that won't degrade easily, a way to read it in the future, and a cool, dark, dry vault to store it in. All of this is (not surprisingly) outside the financial feasibility of most businesses and consumers.
The cheapest, easiest, and most reliable solution is to throw together a network file server with redundant disks and backups. Monitor it continuously for failure and upgrade it every couple of years. If the data is super important, perform periodic audits on it. The advantages to this method are:
1) You have immediate access to your data at all times. Need an old-ass file? Just copy it over the network.
2) You have backups. You can even set up an incremental system if you want, so that you can get that file as it was on Monday that you zapped on Tuesday.
3) You have redundancy. Build the server such that if it one part fails (especially a disk), the system as a whole does not suffer.
4) It scales up and down very gracefully. You can get by with one of those cheap two-disk consumer-level NAS boxes or employ a rack of CORAID boxes, depending on the size of your data set, bandwidth requirements, and how "safe" you want to be from hardware failure.
5) You're actively in control of your data, whereas if you hire it out to a third party, you really have no absolute guarantee that your data is safe.
6) If the system is properly monitored, you are notified when something's going wrong whereas if you write some data to media and stash it on the shelf, you have zero indication if the disk loses its airtight seal, the flash chip is degrading, or the aluminum is slowly delaminating from the plastic disc of the DVD.
If this seems like too much work, then your data isn't important enough to protect in the long term.
p2p neworks might be a really nice solution considering their resilience, although I don't know any that is oriented towards personal data storage. It would take more than half of the internet to go crashing in flames for you to lose your data, and if that happends, you will probably have bigger worries.
What about these new 30,60,120,250GB type solid state drives? Yah, they aren't that cheap yet, but in a year or two it seems like they will be. No moving parts like hard drives. Most of the reviews state their longevity over 10 years, but that accounts for daily use and *WRITING* to them. If it were a read-only archive with minimal writing, I wonder how much longer they would be slated to last?
This is not a perfect solution, but one I have considered... Photo printing services like QOOP (works with Flickr) now print photo books (up to 600 pages). One could (1) Back up to Flickr (does have an annual fee, but then has unlimited back-up) (2) print all pictures to a photo book (~ $13 for 20 pages, then 40 cents per page, 20 small photos per page). Not cheap, but there's nothing like paper...
Seriously, if you really want your data, get mozy home. It requires you keep a copy on your computer (windows only), but it costs $5 a month for pretty much unlimited storage.
Compare that to a) buying equipment, and b) maintaining equipment c) worry, etc. $5 a month is pretty cheap!
Also, you can (optionally) encrypt your data with your own key using their software . Just don't lose your key!
a crapton of paper, and a couple of fireproof safes
...people need to learn to use the fscking search function - http://ask.slashdot.org/article.pl?sid=08/08/27/2119252
The more 'compact' a data storage format, the more likely that just a little bit of 'aging' will degrade it beyond intelligibility.
If it's REALLY important, rewrite it so that it's interesting to more than 'just you,' and publish it on archival-quality paper. If there are several thousand copies around, the odds are that a few will survive for a couple of centuries. Decay that would render any digital media format unreadable, just makes archival paper smell a bit musty.
Beyond the 'clay tablets' mentioned above, the most enduring long-term storage I've found is 'oral tradition' in a cultural context that values both integrity and history. Assuming that whatever you want to save long-term is interesting and important to anyone besides you, tell it in a memorable format to your grandchildren's generation.
A couple of decades ago, I worked on a history project involving both oral tradition and archival material: the elders' memories of events retold through the generations were, in some instances, MORE accurate than archival documents. But, of course, most of the younger generations in that community are paying more attention to TV and the Internet than to their elders.
My dad was a maniac about data preservation: made multiple copies of everything, indexed it meticulously, etc., etc. My sister threw it all away after he died.
I'm sorry but this man should set up mirroring or a hotswappable raid array. I've had raid 5 for many many years and have not been disappointed, the key is to replace drives before they fail and keep the drives cool. I can't see how he can complain given the cost of hard drives at the moment. If data is really that important to you, buying 3 600GB+ drives should do more then the trick for less then $300. If that's "too expensive", I think he's not really serious about backing up his data IMHO.
I keep multiple copies of the same data.
My data is smaller, around 15-20 GB. I think I have 6 copies right now. I constantly use Unison to synchronize them and to compare hashes every month or so. Three copies are in the computers I use all the time; another two are on hard drives that I keep in a media fire safe except for their weekly update. Every time I get a new computer, I just make another copy. Every time a drive fails, I just throw it away.
The last time I lost any data was when the firewire controller on my B&W G3 started intermittently introducing errors of about 10 bytes per megabyte of copying. Comparing hashes showed me there was a problem, but it took a while to sort out which versions of the files were OK and which were corrupted.
I have a system using FreeBSD servers running ZFS as the file system. The servers are built with around 6 disks each in a ZFS RaidZ2 pool. This allows each system to have upto 2 disks fail without losing any data. Data stored on ZFS is also internally checksumed which means that the files will stay 100% pure and not suffer from bit rot. I have 2 physical machines in different locations which are synced using rsync. Each machine runs smartd to run SMART checks on the hard drives weekly and they also run a zfs "scrub" weekly to verify that all of the data is in place and is correctly checksum verified.
For critical files (wedding photos etc.) I use par2 software to create a further checksum and generate additional redundancy. The par2 software allows corruption to be detected and corrected. This was more of an issue before I was running ZFS however it's still better to be safe then sorry.
Daily snapshots are taken of the data to protect against accidental deletes. Critical storage is moved to read-only zfs volumes.
The system has a few flaws, the first is that it doesn't protect against malicious damage. It would be possible for someone with admin access or an axe to "zfs destroy" my files. I have some critical fil2es dumped to DVD however I should probably have a third redundant "offline" site where all of the files are backed upto manually. Perhaps just a collection of removable drives.
I am also concerned about data file compatability. I currently store my digital camera files as RAW camera files. Critical images I also store as Adobe DNG files and as JPG's. One of these formats should be hopefully supported by Photoshop CS428 in the year 2100.
The system can periodically have the hard drives, pc hardware and software upgraded or replaced without disturbing the data so I can see no reason why the system can't keep the data valid for many many years.
One concern is that after I die and stop maintaining the system it may stop working. I guess it won't be my problem.
> ...plus blue-green lasers and a simple COBOL program which writes 1's and 0's onto the Moon.
Looks like all they've managed to write so far is: 010000110100100001000001
"Slow down, Cowboy! It has been 3 years, 7 months and 26 days since you last successfully posted a comment."
Hm.
Printed at a modest 750 DPI, with a half-inch border... ((7.5*10)in * (750 DPI)) =~ 4.22 M dots.
5.27 Mebibytes / sheet, conservatively single-sided
2.52 Gibibytes per ream (500 sheets)
If you move up to 1200 DPI, and if your toner or ink won't cling between sheets you can use both sides: 12.9 Gpr.
Printer drum will last 20,000 normal pages, but let's say 1,000 50%-full (i.e., data) pages. Drums (or printers themselves) are around $100. Thus $50 / ream.
Cartridge will last 9,000 normal pages, and say 450 data pages. Cartridges perhaps around $75. Thus $80 / ream.
Acid-free paper 500 sheets (20 lbs.) ... USD$5.
So...
About ten bucks a gig for simple storage that'll last 500 to 1,000 years.
Good idea.
why not write an application to encode your data as coloured dots and print it on archival grade paper with archival grade ink? then use another application in the future to read back the scanned page and convert the data back (I guess scanners or something like will still be around in 2250), you should be able to get 4mb to a page with some trickery (2048x2048 pixels), thats 2 gig per ream, 100 reams = 200 gig, humanity has developed the art of storing paper information to a fine art, it should last far longer than a lifetime with care
Granted, it's not all personal per se, but I have well over 500 gig of data that needs to be backed up and archived. I'm a freelance graphic designer, my client files end up getting massive. 300 to 600 dpi images, multi-layer PSD's in multiple stages of revision... It's massive. I only retain client data on a "live" disk for a year, then it goes into storage. After 3 years, some if not most of that data gets purged. 500 gig really isn't that much when you consider files types other than text data.
Redundancy (both of media and of types of media), checksums, regular inspections and regular transfer of data.
Nothing's going to last for ever, and even if it does, as other people have pointed out, the file formats may not be easily readable. Spread your stuff out over, say, a few hard drives, flash drives, archival quality DVDs. Store some stuff off-site, in case of house fires. Make sure you have redundancy. Go back every year or so and check everything is still readable, both the media and the actual file formats. Burn new DVDs, check the SMART data on the hard drives, etc., etc.
As time goes on, increasing storage densities should allow you to condense things, assuming you're not adding data at a faster rate than the technology is progressing. As file formats become out of date, convert stuff. You may even find that as time progresses, you realise you don't need to keep all that stuff after all :-)
This is unacceptable! I want my glory to shine in the far future, when my elaborate tomb is found by future scientists. I want my email and pr0n to be fully accessible! They will obviously remmeber my greatness, but they must know the full details. For instance I save the html of every first post I've ever made!
My data must survive the eons, despite flood, fire, nuclear war, the sun going nova. Nothing else is acceptable, after all these great historians who have devoted their lives to my greatness undoubtabily travelled far at great expense. Hyperdrives cost money, hell they probably had to re-evolve back into physical beings!
I purchased 2 small form factor USB external hard drives. I encrypted both with truecrypt. Since I am a Windows user I use MS's SyncToy to 'backup' the stuff I want to them. Synctoy makes it easy for me. One hard drive sits at work plugged into my PC, one at a bank in a safe deposit box. If my home computer dies and I need access to my data, I VPN into work and mount the encrypted drive or go to the bank and pick the other one up. I have a routine where on a set schedule I take the one from work to the bank and swap it out with the one in the safe deposit box. I take that one, synch up my data and then take it to work.
If one of the hard drive dies I still have the data at home plus on the other external hard drive. If one of the drives dies after the 5 year warranty period ends then I will buy another drive. If USB starts to become obsolete I would save some money up and buy 2 more drives that uses whatever technology we will be using then.
You don't know what technology we will be using to access data 10 or 20 years from now. Use what you think will be around and change later if that turns out not to be the case.
Two 1TB hard drives thrown into Firewire enclosures. One stays in a fireproof disk safe at my office, the second goes into a safety deposit box across town. I flip-flop the drives every month, both of which are a mirror of a "live" disk running on my primary workstation.
I'm a freelance graphic designer, my client files end up getting massive. 300 to 600 to 1200 dpi images, multi-layer PSD's in multiple stages of revision, complete bundles of print-ready pieces, digital art with file sized upwards of 300mb each...
I only retain client data on a "live" disk for a year, then it gets archived. After 3 years, some if not most of that data gets purged. 500 gig really isn't that much when you consider files types other than text data.
For any large amount of data, S3 or Mozy or Gmail will NOT work. It's not practical to spend a month to upload just one interation of your data to these services.
Storing critical or even mildly important data only in your house or office is just stupid. Fire, theft, flood, earthquake... any calamity will eat your data and there's nothing you can do. Planning to grab your hard drive on the way out of the house in the event of a fire is pretty short-sighted and unrealistic. If you're fighting to breathe or save your kids from burning to death, you are NOT going to waste time yanknig hard drives out of your computer. Further, if you're not home (which is more likely) your brilliant plan is useless.
High capacity hard drives are cheap. External enclosures are cheap. Don't rely on loading one up and forgetting about it for 5 years because it's likely to fail. Someone above said to keep your data "hot", and you should. Get into a routine and cycle your drives monthly, semi-annually or annually.
I have family photographs that are over 150 years old and quite legible. They're daguerreotypes of one of my 3rd great grandfathers and his grandfather, who was born in 1771. The photographs were taken in 1853 give or take a year. That's pretty good archivability.
"Do the Right Thing. It will gratify some people and astound the rest." - Mark Twain
Chances are, that after you are dead, nobody will care about your gigabytes of data, your holiday photos or your emails to other unknown citizens.
Unless you are exceptionally famous, forget about it. Nobody will care.
Forget about your backups. Get a life.
If it isn't off site, it isn't truly protected, flood, theft, or fire can put an end to your precious data if it is kept locally. The method I employed is to take a second hard drive and use backup/encryption/ftp software to transfer it to the second drive for my initial backup. Then I gave that drive to my brother to put in his computer and he did the same and gave me his.
The software performs incremental backups over the wire in a completely automated fashion.
I've tried some of the online systems such as carbonite, etc and the features and volume just aren't there yet to keep up with my video files. 400gb data. (The kicker for me is that most of these files are static so and doing the initial backup locally made this method so much better).
One very important note is to keep your encryption key backed up somewhere remotely otherwise you won't be able to recover your data when your system dies. I sent mine to myself in an email.
I've been think about this problem as a 30 year archival system and a laptop with dual SSDs running open source software is probably the best way to go. If a drive or laptop fails, the data can still be recovered. If all existing formats goes the way of the dodo, the information can still be retrieved from the laptop. It's not enough to store the bits, you have to make sense of them in the future. -guggs
Some thoughts on this issue:
1. I wouldn't trust technologies that haven't been used for a decade or two yet, so people know what makes them last & fail. See here:
http://findarticles.com/p/articles/mi_m0BRZ/is_8_23/ai_109665179/pg_1?tag=artBody;col1
2. File formats are a more dangerous issue than the physical media. Text-based formats (including XML) is something someone else could write a reader for later. That's a lot easier than trying to find both the software and the machine it ran on. Virtualization helps -- it may be worthwhile to keep a VMWare image with your stuff so that you have a copy of the software required. Emulators exist for most any obsolete system I can think of, and I'm pretty sure you're going to find x86 emulators for span of your grandkid's lifetimes. If paranoid, use an open-source emulation platform instead of vmware, and include the source of the emulator.
3. Archival formats are another problem. I love zfs dumps for their current usability, but I know I can't trust something that system-specific to last terribly long.
So, here's a procedure for you. Note that this is for important data. Don't interpret this as a method for keeping your Hentai collection. Of course, each has their own priorities in this area...
1. Choose something you know will be around "for a while." Easily available, good track record, easy to use. Don't worry about extreme-long term. Step 3 addresses that. More important are the short-term technological concerns: Is it simple & reliable enough that you _will_ back up reliably with it?
2. Keep a Master Image of what you want to back up. For example, a root directory with a per-year, per-project, or per-customer set of subdirectories. When you add stuff to your backup, you're appending to the master image. As storage (capacity & demand) is lightly exponential, you can usually afford to store all your old backups with the new (long term), as the old stuff is relatively small.
Note that I'm *not* saying copy this stuff every time you back up. The master image is append-only. You'll probably want more than 1 copy of the whole thing, but that can be two stacks of DVDs, two boxes of tapes, two RAID arrays, whatever.
3. Plan to migrate this growing master image to new formats. The entire master image moves to the new format. When you migrate, all your backup data is in the new format, and your old-format backups can be stored as a hedge against your new media having surprising failures later.
4. Treat this stuff *seriously*. Don't skimp or cheap-out. Spend a month or two every couple of years setting this stuff up correctly, and make it something you can rely on. A little organization and automation once makes worthwhile. Cron et al for Unix, and a good commercial backup app for everyone else.
5. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES. TEST YOUR RESTORES.
Care about electronic freedom? Consider donating to the EFF!
Definitely stone.
The only media demonstrated surviving more than 2k years... and counting.
I admit it's not very portable, but hey, here we're talking about durability.
You're going to have to tier your data into different (cost of archive, cost of loss) groups, and set up procedures for that.
I generally have two tiers, one for professional/academic concerns, another for large media (video, not music).
The latter's on a spare USB drive, the former's on a few drives and an off-site backup service. Not expensive (~$30/mon) for my data needs.
Care about electronic freedom? Consider donating to the EFF!
As a hedge against physical disasters, I'd also do incremental backups on some far-distant system -- get yourself some cheap hosting (I use 1&1, 120GB for $5/mo.) and dump copies there. It won't necessarily be any more reliable than your own system, but it does provide a remote backup not subject to local disasters.
As the tagline goes.. The four California seasons: Fire, Flood, Riot, and Earthquake. :)
~REZ~ #43301. Who'd fake being me anyway?
Dude, storage tech is in constant flux so your mission is to keep copying your data onto current storage media using redundancy to protect against corruption.
You're constantly adding ever greater volumes to your collection anyway, so that works out cheaper and more convenient in the long term.
Even if some storage media last decades, the devices that read them will be a pain to maintain in years to come and may not be accessible on future computers, I'm keeping an old pc just to use my 10yo scanner for pete's sake, even Nasa can't access some of their own Apollo mission data.
Now, in 2038 will you be able to open a Microsoft Publisher file with any program? Page and text format for example is information, lossless retrieval must represent the original, open formats are your only hope.
Copy it to several different folders on the same harddrive.
Copy that harddrive to another.
Put them in separate physical locations.
At least one of the drives will work 5 years from now. At least one of the directories will be readable.
Copy to latest drive technology at that time.
Repeat until some "Permanent" storage comes along. Even then use two of them. :)
Why on earth use python to copy files?
Well I use a combination of things
Firstly - I use synctoy to sync all my important data between the three live computers with big enought Hard Drives in the house.
One is a linux server running raid 5 and 2 are windows xp machines. When ever my dad buys a new PC - I chip enough for an extra drive and I use an external drive to update that info every 6 months.
for music I keep copies backed up on friends machines and for photos I keep backed up google with paid storage. for documents I keep backed up on my EEE PC which travels with me and on USB key. not perfect means I wont lose much besides video.
BTW - does anyone know a Linux equivalent to synctoy (i mean in terms of user friendliness ?)
Ta
Phill
tar -zcf brittany-sex-tape.mpg /home, then upload on P2P.
Dear Sen. Stevens,
I was recently sent an internet. I'd like to keep this internet safe for a decade or more. Being a technology expert, I thought you'd be the best person to ask. What storage solution do I need; a dump-truck or a semi-trailer?
Warm regards,
S. Palin
... and then they built the supercollider.
Every hour of video I take on my Sony HD camera consumes 8GB. You can easily eat up a few hundred gigs a year if you have a kid :)
Fortunately two things are working in our favor - HD video codecs are pushing the file size *down* as the storage price drop exponentially. e.g. a raw DV stream used to be 13GB / hour... worse than the 8GB for an HD stream today.
You can also now get a 1TB disk for $100.
What I'm really waiting for is for 1TB of SSD in a raid-1 configuration for a few hundred dollars... I really don't want to lose all of our photos and home video due to a lame raid.
Oh, and beware of most cheap raid solutions out there right now. If they don't do disk scrubbing (parity checks or checksums like ZFS) then you really don't know if your data is safe.
Pat
Imagine what it'll be like in 20 years-- anything other than NTSC, PAL, or SECAM will be effectively extinct.
Well, there aren't *that* many standards that are really in consumers hands. I suspect that in 20 years we'll still have VLC or its successor and that developers will dutifully create or recreate codecs for virtually every video format that has ever existed.
Look at people digging through ROMs of old video games and creating emulators... There is a bigger market for recovering old video, I suspect.
Pat
Wow, you have 500Gb of porn?
Your best hope for storage that's going to be more than 20-30 years is optical storage, such as DVD or Blu-ray. most flash drives are only rated to last 10 years.
You are fortunate in not losing data when your flash drive went through the washer/dryer -- mine died the first time.
perhaps the best solution to this:
dude... life is short, nothing is that important. stop worrying about nostalgia, go do something new...
If you have some time to throw at it... Create a virtual machine with the operating system and all the required software to read/edit your data AND your data. Convert this VM to newer versions of the VM software as it arises. Keep the VM duplicated in two geologically different sites.
Just my 2 cents (of an â) :)
After a few years of trying to find the solution to this problem, I looked at my miniDV camera... I was happy to discover that people thought of it before (http://tech.slashdot.org/article.pl?sid=03/03/03/0246245&mode=nested&tid=126&ti%20d=137&tid=106). Same as tape backup, but no need to purchase more (expensive) hardware, if you happen to use a miniDV camcorder. Even if you don't, those are much cheaper than "real" tape drives.
/wrld
> Well, there aren't *that* many standards that are really in consumers hands.
True, but it's not just the consumer formats; the biggest problem right now is proprietary solutions that are deployed over the Web. The number of codecs out there keeps multiplying. People download these videos and players today, and then tomorrow it changes.
> I suspect that in 20 years we'll still have VLC or its successor and that developers will dutifully create or recreate codecs for virtually every video format that has ever existed.
I think we'll still have something like VLC, but it's absurd to expect it to support every codec ever devised. There are already dozens in existence, and many of them are patented and/or proprietary. I expect most of them to just vanish into the mists of time.
I use venti, an archival storage system originally develloped for Plan 9 from Bell Labs and now available on a host of other systems via Plan 9 from User Space. Venti is strictly archival: it stores blocks permanently. This storage is organized into "arenas", or pools of a fixed size. When one arena fills up, it is sealed, never written to again, and the system starts dumping bits into the next one.
/n/dump/2008/0712/usr/a/src/cmd/ngcscatgen.c" to see the version of that file as I was working on it over the summer. Pretty nice.
Primary storage for my venti system is a pair of mirrored SATA disks. Yes, magnetic disks can fail, but with mirroring they're still cheap enough to almost certainly be your first line of defense. When an arena fills up, I burn it to CD (by default, they're 512MB each) and mail that to a friend three time zones away. If my house burns down, I can recreate everything up through the last arena by basically dd'ing the contents of those CDs to a new disk.
Using a real archival system has other neat benefits, too. You don't have to worry about whether you saved the right version of something, or how to organize different versions over time; it's all automatic. I've used this for "work stuff" for a long time with very good results; after my last laptop hard drive crash, I've started using it for personal stuff (although I haven't made that quite as automatic yet). I can now "cat
i speak for myself and those who like what i say.
Set up a single dedicated machine with four large (500GB) drives. Script it to replicate the data across the drives. In case of errors or differences, it should default to the most reliable copy (compare the copies across drives). Make it notify on drive failure or file loss.
Once a year, boot it up, run the script. Leave it for the day or two it takes to run. Remove any bad drives, and replace them. Once the process is complete, shut it down again. Repeat next year. Upgrade the hardware from time to time, and replace any drives that get "too" old, even if they haven't failed.
If you want to do it on the cheap, always use the last-gen drives and hardware on it. Better than throwing it out.
All this talk about making sure data lasts for the future. Well, I'm currently mining old data that was laid down well before my birth date, that's old family photographs back to the 1860's.
For the first time I'm seeing images of my great, great grandfather and his family candidly posed in an Aberdeen studio. My great grandfather and his brothers on an old tintype photo plate.
Some of the images are in a fairly poor state, but half an hour with the Gimp has got them back into serviceable condition.
When my relatives submitted to the new craze of photography some 150 years ago, did they ever believe that one of their descendants would lovingly resurrect the data? On a machine that would have been a complete fantasy to them.
Oh, and how do I know who they all are? Simple; they wrote their names on the back of the photos for posterity.
I have seen the back ups of the College I work for. Excluding Student Data for true data we clock in at under 100G. *Including* student data we barely break 400G And that is historical data going back 30+ years. There is no way on this earth that you have 500G of actual data. I am sure that you think you do, and I am sure that someone will respond to this post with 'proof' that they do have that kind of data. However if you make a "must survive me/must survive a fire/anyone other than would give a damn" list I cannot see anything over 5G.
Slashdot, where armchair scientists get shouted down and armchair theologians get modded up.
Really hope I'm not the first to suggest this.
Build a really simple desktop machine, with ordinary, inexpensive, large hard drives, in a RAID 1 mirror. When one fails, or every few years, swap it out, let the mirror rebuild in a few hours. I've had good drives last for over ten years, but you don't have to trust them if you simply add another drive every year.
Your cost becomes one drive per year -- a whopping $50 - $150 -- and the one-time cost of the cheapest server you can find. Or you could use any number of N.A.S. solutions that support multiple physical drives in a RAID 1 configuration.
You're guaranteed compatible access to your data for as long as hard drives are used -- that'll cover the looming SSD era as well. Basically, for as long as SATA is used.
Hard drives are very stable and reliable over at least 5 years. SSDs, when they become cheaper, will be even more reliable, and you'll be able to simply swap them gradually into your mirror configuration. And, at any future point thirty years from now, you'll easily be able to convert your array from SATA to the new hotness, and you'll have an overlap of at least five years during which it'll be convenient to do on even the cheapest system.
Standard storage options, standard configurations, inexpensive everythings, unlimited capacity, unlimited redundancy.
But you should still worry about fires and floods and theft. Maybe take one of the drives away to some other locations once in a while. Or nightly FTP to your office closet -- incremental to save on bandwidth.
Oh yeah, and you can use any number of encryption methods, just to be on the safe side.
I was looking at some old 35mm black-and-white film a few days ago and thinking that it would actually make a pretty nice data storage medium, if you rigged up a machine to write to it. Sort of a "poor man's microfiche."
You'd need to document the hell out of it, but I think it'd be possible to make a data-storage medium that used film, and would last 100 years easily, and be readable with only a very basic system of sensors and light sources.
I was imagining having eight parallel tracks down the film, although you could probably easily fit many more. Depending on the film grain you could probably get hundreds of tracks, and each "bit" would only have to be a few thousandths of an inch wide, depending on the grain of the film stock. You'd just move the film over the write head, and expose it with short pulses; the result would be like a continuous 2-dimensional bar code. (Maybe you'd want to waste one track with timing information? Probably.) Develop it in some nice high-contrast developer (since you only want two tones, black or clear) and then store it as you would any other film. Modern film stock is very stable; in a cool, dry place in a metal tin, it lasts practically forever. (Old celluloid isn't nearly as good, but chemically modern films have very little in common.)
Reading it wouldn't be that different from decoding a big barcode (and it's very similar to the way digital soundtracks on theatre films are written). But even in the absence of any existing hardware, it's the sort of thing I'd bet any undergrad EE with access to a machine shop and electronics lab could come up with in a couple of days. As long as there's no fancy encoding or compression done to the data, you just read it off as quickly or slowly as your equipment allows.
Anyway, it's totally impractical for a personal backup system, of course, but it might not be a completely ridiculous idea for organizations looking for very long-term archival storage. The advantage I think it has, versus other candidates (e.g. magnetic backup tape) is that reading it would be relatively easy and intuitive. Anyone looking at a piece of film with a magnifying glass or microscope would be able to see that there's something there, plus it's a lot harder to wipe accidentally.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
There really isn't a good solution right now.
After thinking about it a while I realized that:
1) Most of the stuff on my computers could be replaced.
2) The one thing I would really, really hate to loose are family photo's.
3) Hard drives WILL fail sooner or later.
4) Tapes are reliable for a while but even in a climate controlled vault I have had tapes at work end up bad after a couple of years. (not to mention the pain in the neck it is to find a working legacy tape drive after 10 years)
5) DVD's will probably have the same issues.
My solution for now is redundancy.
Digital photos get offloaded to my Linux pc. I use a program called Digicam. .5T USB external disk drives with the "one button backup" program that is set to run nightly.
I have a bash script that syncs the new photos to a Windows share on my wifes pc.
My wife has one of those
When I have a couple of new directories of photos I run another script that compresses the whole directory and splits the output into a bunch of 45 megabyte rar archives.
I then upload them to Microsoft's free "Skydrive". Microsoft just upped their free disk storage to 25GB.
I also have some documents saved on the free AOL Xdrive.
I figure in a couple of years there will be a better long term storage option. It will probably be something like a solid state drive that lasts for two hundred years. At that point I will save everything to that and store it in my safety deposit box at my bank.
Every wrong attempt discarded is a step forward - T. Edison
Same problem, but do not assume I am able to access the media between now and X years from now. Assume also I am flat broke and have no lawful access to privately owned real estate or safe deposit boxes, and have no reliable / trustworthy / willing friends or family to assume physical custody of the media. In other words, I am most likely talking about burying or otherwise concealing physical media in a public place in case I am out of the country or otherwise unavailable for a period of years. Other than encrypting and appending the data to a pron file:
1. What is the longest lasting / hardiest machine-readable media excluding paper tape or other overly bulky media. Flash drives?
2. Assume I have a small waterproof canister containing the media. What redundant waterproofing or other physical media protection measures should I take? Extra layers of waterproofing? Silica gel?
3. What is the most viable and secure long-term hiding place for the canister? Burial in a public park? Near a freeway on-ramp?
I think a "buddy backup" system makes a lot of sense. Keep multiple copies of your data on the hard disks of friends and family through the net (and let them keep their data on yours, of course). Encrypt it if you like. Each participant only needs to invest in a big hard drive every few of years (1.5GB is as low as $133 these days). No need for RAID - geographically distributed redundancy is better.
There is a commercial application to do this called CrashPlan. I'd really love to see an open source project with similar functionality. It should be easy to use for non-technical people and run on multiple platforms.
Stop worrying about the risks of nuclear power and start worrying about the risks of not using nuclear power.
It is designed for archival purposes and has a specified minimum lifetime of about 30 years. It can be written to like a regular hard disk (though at much lower speeds) without any burning software. The drives perform data verification in hardware when writing and have a hardware based defect management. While not as cheap as DVD-R and all the other less secure DVD formats the drives and discs are affordable (about 150$ for the drive and 1,50$ per disc).
I have not once seen a read or write error on these discs and use them to store my digital photo archive. One copy is stored to a hard disk for easy retrieval without searching the disc and another copy gets stored to DVD-RAM. Store the DVD-RAM in a different location.
DVD-RAM is used in libraries around the world so you are sure to find an operating drive in 30 years even if you missed to keep a working one yourself.
I have used computers daily since 1981. From every one of the media on which my data has been stored, I have suffered loss of that data at some time, with the sole exception of the hard disk. Floppy disks, tapes, CD's and DVD's have all become unreadable after a while, sometimes after 3-5 years, and some in as little as 6 months.
I am not saying that I have never had a hard drive failure; however, when it happened there was plenty of warning in the form of frantic clicking, slow reads and program errors. If I did not already have a backup of everything on that disk, I knew it was time to make one. I had no idea that the CD's on which I had stored a large number of movies over a period of 3 years would by the end of that period be about 10% totally unreadable and 50% recoverable with some effort. Those CD's were in proper storage cases , unused and kept in a cool dark place. And yet they rotted away unseen. I could actually see holes through the reflective film in some of them.
I believe that a hard drive, kept unused most of the time, but run occasionally, is very unlikely to fail. The probability of two such drives failing simultaneously is negligibly small. Therefore my solution is to make backups on two separate external hard drives. As the typical cost of HD storage has now fallen to about 10 cents per GB this is also a very inexpensive method, and of course it is also relatively simple and fast.
The big mistake is to assume you can "fire and forget" - there's no such luck. All backup systems need maintenance and migration as technologies age and get replaced. To assume anything else is pure fantasy.
That makes me think about a bunch of "Ashon Tate Framework III" spreadsheet dating back to the 80's, that someone asked me to read and reuse in a modern Excel.
Long term storage in our times and certainly in ever-changing America is utopia. Given a media is found, that won't be supported anymore soon. Given a bunch of stored formats, those won't be supported or even understood anymore soon. Today new inventions replace so fast the older that there's not even time to write history: no one will remember the iPhone or what DRM was in hundred years, for what we are concerned in a hundred years spreadsheet might appear directly into our brain, without any visual aspect anymore...
Long term personal data storage is like trying to save legacy in an attic, where both the things and the attic keep changing shape and use. Impossible.
There is anyways an interesting and ambitious point in the wish to save history. I would even think of a public agency with the only job to save legacy in a retrievable way, after having found that way, because history is foundation in human life.
If you're looking for simple archival storage, get an ink jet printer, load with a high quality metal based ink with an insanely long shelf life and chemical stability out the wazoo. Pick a mylar film, something again that is chemically stable and has a probable shelf life in the hundreds or thousands of years. Use the first dozen of so sheets to print the digital key so that the rest of the data can be decoded by anyone in the foreseeable future, then crank through all your data until it's on mylar sheets. So now your data has been converted into a simple binary code on film. You pick whether ink dots are '1's or '0's.
Store the sheets in a hermetically sealed light tight metal box with a pure nitrogen atmosphere, and super low humidity. Keep the box in a safe place. Check the data every 5 years for integrity. Replace bad sheets if and when they appear. I'm guessing your great, great, great, great, great grandkids will be enjoying your data.
1. Carved in stone. 2. Oil paints on canvas.
store it on the internet
Don't use time machine, it's utterly useless. Freezes while backing up and when it came to restore time when I had a problem the backup was incomplete and everything went the rectum output shape.
Use SuperDuper! instead, it works which makes it good.
001110110010110100101001
``Tension, apprehension & dissension have begun!'' - Duffy Wyg&, in Alfred Bester's _The Demolished Man_
They say that paper is still a pretty good long-term storage medium. There's been talk about using high-quality printers and high-quality paperstock to print out tiny, complex patterns that look like UPS shipping data (those weird patterns replacing barcodes) that's actually a representation of encoded data. The encoded version is supposed to be able to fit hundreds of pages worth of data on one page.
OF course, paper is still capable of deterioration. CD's and DVD's sound like a really good idea with a laser burning the data as areas of light and dark on a special substrate but even that substrate can decay.
When we get right down to it, pretty much all of our storage media come down to moving something past a read/write head, be it magnetic tape, magnetic disk, or optical disk. Flash RAM is really the only one doing it differently. What we need is a media that won't degrade. That makes me think it won't be bendy so any tape-like solution is out. I'm guessing the answer we want is going to come down to a material with very high stability that is unlikely to decay regardless of heat, moisture, etc.
My guess is the true long-term solution will be some sort of glass that can be fed into a drive that burns patterns in with a laser. There's no substrate, the pattern is on the glass itself, and burned deeply enough to account for a little erosion. Now the next question is whether the glass will be cylinders, disks, square tablets, etc.
Now since I'm able to think of this, I'm sure other people have as well, smart people. Since we aren't already hearing about this solution on the market, there's probably bigger pitfalls than I'm anticipating that's holding it up. I'm guessing it'd probably be really, really frickin' expensive.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
I have a consumer grade DSL line and it's unacceptably slow. I tried to use it for backup of a large set of files (audio, family pictures and some video) and it was running for days, and finally timing out or failing (don't remember how large the batch I tried, but it was at least 20GB in many small files). Even if the cost was fine that's not ok. (And of course if you lose your home disk then you have to do this in reverse to restore; if you can't restore it's no good).
My solution so far is to put it on a portable hard drive and store it offsite, encrypting anything that's sensitive.
Every year when this question comes up on Slashdot I know it's time to do my long-term backups.
Since you cannot know what will be used then, you should find the data media with the interface you can recreate from scratch with your almost bare hands. There are only 2 kinds of media with such an interface: 16-bit parallel CF (IDE is excluded due to lubrication aging) and 1(4) bit serial SD/MMS (You cannot easily recreate USB, SATA or any kind of rotating media). Then include the clay tables with description of the interface and the data format. Any future computers will have the ability of some parallel i/o, it's enough.
I have 2 computers, cross-backup every few days. Then every couple of weeks copy everything (copy, not update) to an external HD. The external HD is normally kept off site (at my work). If I accidentally trash a file, I can recover from the other computer. If I somehow wipe out everything on both computers, or if my house burns down, I can recover an at most 2 week old backup from off site. If storage technology changes, I will change the backup media I use. At the moment I use conventional hard disks. I can go to flash, or hyperspace quantum drives or whatever as technology changes. I need 3X the disk space required for a single copy of my data, but disks are cheap. Since my media are regularly used, I know if things are going bad.
For that price you could buy a new hard drive monthly and copy to it. Have one fail? You still have all your other drives. Remember, in this example you're talking about a steady 500GB or less of data, not something being revised monthly.
Get off my launchpad!
01000001 01011001 01000010 01000001 01000010 00110010 01010101!
...If Linux, some sort of cron job running a Python script that copies /home to an external hard drive. If Windows, I dunno.
I use "rsync -aH /home/me /media/usbdisk/homebackup" to preserve file permissions, dates, links, etc..
I suppose you could put this in a cron job to automate it.
I keep an online backup and then mount my backup volume as a network drive. That way there's two copies (local and remote) that I can access anytime. I picked up a huge SATA for about 100 bucks and have OPENRSM CloudBackup take care of the offsite part. It's easy, cheap, and if I replace the home computer I've already got a place to restore from. You can do it with S3 if your techy at all too. The big thing is the way I do it is easy, cheap, and gets the job done without reinventing the wheel.
Amazon S3 has been mentioned before and it's cheaper then all comparable offers. (For typical personal use (<< 100GB) S3 is pretty affordable.) With JungleDisk, a commercial (US$20) client, encrypted backups are trivial.
If you want to beat Amazon's pricing you probably need a good friend working at some large data center.
Buy two different brands of high quality archival rated DVDs and make two backup of everything. One to each type of DVD. Better yet, 4 copies. 2 on each type. For the cost of blanks, this is cheap peace of mind. I'd also burn at less than your max drive speed. Say 4x.
Store them somewhere safe (and dark) and repeat the process every 1-5 years.
As for this question being asked over and over. Well until someone solves this problem, don't complain about the question being asked. It's a basic consumer problem that is ignored by industry. There are at least one of these in every industry. That one piece of the puzzle that everyone wants solved or improved, but no one ever does anything about.
For example, I can buy a car that can e-mail me when it needs an oil change, or parallel park itself (in the event that I suddenly turn retarded), but they all still have the same useless damn windshield wiper technology that's been around for a hundred years.
Asking for simple, reliable, obsolescence resistant backup systems is like asking for windshield wipers that work for more than 3 months, that don't freeze to your window, and don't leave streaks after the first pollution filled rain leaves it's oily filth on your window.
As long as the problem exists, people will want a solution.
Now if I could only invent super wipers that store 5GB of data for 20 years, I'd be a millionaire.
The Real Tachyon
The other alternative is doing it yourself with RAID arrays at different locations. Sync your 'must be preserved folder' nightly to both locations using rdiff-backup or similar. Regularly test recovery and make sure all the disks in both arrays are working without error.
If there is fully redundancy by storing the data in two locations anyway, and this is a back-up solution only anyway, is it really so beneficial to use RAID? I mean, sure, using RAID in one or both locations increases redundancy, but also increases complexity (increasing the number of ways in whcih something can possibly go wrong) and cost.
I do it myself, with cheap Linux software RAID boxes in separate locations, piggybacking on existing Internet services to do rsync mirroring between the sites. The machines are kept alive, running disk scans.
Just fsck scans, or full disk scans in which you read every sector that actually contains data? If the latter, I'd be very interested in any statistics that you may have accumulated about disk blocks going bad. For example, is there a correlation between bad blocks, with blocks close to each other on the physical medium having a greater probability of going bad at roughly the same time than blocks with are further away from each other?
If less than 50gb, I'd suggest a few SD cards.
What's the rate of bad blocks for SD cards, over time?
get yourself a DROBO, problem gone.
Repeat this process every decade or two and you should be in okay shape. This is of course assuming that optical media continues to be relevant. If we end up back at magnetic (or FSM forbid cloud) storage, it gets trickier.
FWIW, it'd be interesting to know more about this data. Any particular reason it can't be printed out into a book or onto microfiche? The only long-term important media to me is stuff that lives on after I do, and that's really means diaries/logs and photos.
Get a Magneto optical drive. These are designed for archiving and I know the government uses them for data storage. I've seen drives on ebay for $20 -$50 and media for a few bucks ea.
Cmon? really? this question still has multiple answers? I'm sure that most of the people here on slashdot have at least a desktop and either another computer or a laptop. set up a script to copy files between them automatically. video files? how long can it take? using vista's normal SMB i can get 55-60MB/s, copying a 1.5gb 720p 45min video takes a total of under 30 seconds. I run a script on my laptop that whenever it is plugged in using a cable to my network it syncs photos, videos, university documents, saved files, downloads, application installers, drivers etc between my main desktop, laptop and a HTPC. using windows its very easy, open the scheduled task, set one to start when a network device is active, and write a small batch file using robocopy (u can get it for xp with the power tools pack and it comes with vista) if u dont like using windows programs, using rsync through ssh and copy it that way, both can do copying resuming so u could do it over wireless too (which i do with small updates). if you have 2-3 computers and all their drives fail on you at once... well you werent supposed to have those files anymore, were you?
Depends on your definition of 'personal' files. Video can take up a lot of space.
I presume the number of space for the video files varies between the duration of being single and the number of ex's one had.
I really don't think it's in your best interest to offload the data to a passive storage media. As quickly as storage is increasing, it's not unreasonable to literally keep every picture/movie you've ever taken, every email you've either sent/received, or any other piece of data you would vomit over if you lost it with you via either a network share or a local resource.
Personally, I keep a RAID-1 mirror established with some non-system internal SATA drives and have an external USB drive that I periodically back up to. When we leave for a vacation, I just lock the drive up at my office. I wish I could say I was disciplined enough to have two external drives and keep swapping them between my office and home, but I'm not there yet.
Bare-minimum, if I get hit with a nasty piece of malicious code or do something just plain stupid, I've only lost the delta between then and the last backup. If it's a simple drive failure (which has happened more than once,) I'm covered.
Anyone tried the Kodak 24k gold cd-r / dvd-r? They advertise them for 100 years I think...
So that suggests the YOU are the ideal backup. Could you kindly provide longevity statistics of your relatives so we can compute MTBF?
Place nail here >+
Basically, you have two fears -- media rot and format rot. Using reliable media, common interfaces, multiple copies, and regular attention to the issue, you avoid any single point of failure.
To deal with media rot, you ought to deal with the idea that no single piece of media is guaranteed, so backup often and in multiple places. Archival quality DVDs are a good bet. I shoot lots of digital photos, and I do a new backup when I have roughly 1/3 new material on my HDD. Thus, I have a pretty good chance of dealing with single point failures on all my archival backups.
I also keep pretty much everything online in a RAID (actually a Drobo... same idea, more flexibility) -- that gives me access to the work at any time. This changes over time, of course... and with technological improvements every few years, the storage medium never goes out of style, and never runs out of space (I'm currently on a 2TB Drobo for this stuff... all my digital photos, all my Dad's digital photos, many scanned transparencies, etc. all in one place, as well as living on DVD backups). In fact, in the case of the Drobo, getting close to full every now and then means swapping out a drive or two for more room, thus automatically cycling in new drives for old, and extending the life. This unit can deal with the total failure of one out of four drives, so it's a fairly painless way to get past the single-point-of-failure issue, even without backups (but still, do the backups, at least of data that's important going forward).
USB drives are cheap, but not ideal... check the expected storage live of your particular unit. Some multi-level flash drives are expected to last only around 10 years without refresh.
Moving to formats, think "consumer product". CD, DVDs, and now BDs are consumer products... they have a pretty long life in the marketplace, and it's currently fashionable to support compatible upgrades. USB is similar... you're far more likely to find USB 3.0 ports on a computer 10-20 years from now than a SATA connector. But as well, these things don't change overnight... you can react to the changes as they happen. That's part of the routine, just like backing things up. I used to do backups on CD, now it's all DVD, and it may eventually be Blu-Ray, or something else that shows signs of a long market life in the consumer world.
Media is also critical... a gold archival DVD is designed to last much longer than the $0.20 cheapies. You're also going to actually see that long life keeping the discs in the dark, under the proper humidity and temperature conditions, etc.
-Dave Haynie
Another form of data rot is the data file. While you probably notice that every audio, video, photo, wordprocessor, etc. format accepts a plethora of file formats, there are some that are virtually certain to be around in 30 years, and others not-so-much.
First of all... any format with an open source reader can be potentially usable in the far, far future... but to be really pedantic, including the source code in your archives every do often.
Next, go to consumer or archival formats. JPEG is a pretty easy win for photographs -- it's a web standard, it's been used in virtually every digital camera since the dawn of digital cameras, etc. On the other hand, you may get in trouble with RAW formats, which are camera-specific... I know a number of RAW readers I have don't support the RAW format from my old Canon Pro90IS. You're better off converting to 16-bit TIFF or perhaps Adobe's DNG format (no reason not to ALSO store your original RAW files if you want, other than storage efficiency).
For video, similar things: consumer formats like DV, MPEG-2, AVC, etc. are likely to be around for a long time, simply because they have such a high level of support. Something Open Source like XViD is another one you can probably count on being usable in some way or another in the distant future... and if it vanishes, you can hack up a new version yourself (well, I know I could) if you keep that source code handy.
For documents, you're probably safe with PDF, particularly now that it's been certified as a standard... but to date, older versions have always been supported in Adobe's reader, and there are many open source readers. ODF is another one that's likely to be safe simply because it's open source, so someone's likely to be able to deal with it in the future. Microsoft formats tend to change every week or two (ok, I guess they're slowing down a bit nowadays)... they're common, which means they're LIKELY to be supported, but at some point, they become uncommon and may well get dropped by current software, with no recourse on such closed formats.
For audio, open sourced formats like OGG or huge standards like the MPEG formats (Layer 2, Layer 3, AAC) are likely to be around pretty much forever. WAV and other very simple computer formats too, well described and ultra common. Odd formats ought to be avoided, particularly if they're proprietary.
And ultimately, you need to be smart about it. Check into your backups every so often, read the release notes on your new software. If the new version of Photoshop or The GIMP drops support for Amiga IFF/ILBM images, I might want to think about updating my archive files to something more modern, while that's still a pretty easy thing to do...
-Dave Haynie
Get a DROBO. Data redundancy, and flexibility to grow the array if required.
Much more readable than bash, infinitely more readable than perl.
Best Slashdot Co
Hey,the only things I care for are pictures. And I've accumulated about 80gb of them in the last 20 years.
Here's what I do:
1) set up a simple website with all of them. This site is basically an html generated from whatever those "html album" proggies. Don't depend on php nor anything else, just plain html. The pictures are stored into folders separated by date.
The advantages of this: /home). So I have two copies of them anyway
- these folders are the same in my machine (where the pictures are in my
- as I don't backup manually very often, I've created some space in my tv's media player (a tvix) with time machine. So every time I connect my computer there to upload some movie I've downloaded, I have a full backup of them.
The media player thing + time machine solves the issues of (a) doing backups regularly, as you will connect your computer to it often (b) you will always want more space for hd movies and music. So you WILL replace the hard drive every years or so.
And the website, besides of being an extra copy (with some searchable information being the album covers with some text), helps you by being found by archive.org and google cache.
And don't forget to copy all your data to gmail. Simple, searchable and free.
And let life keep going on. 500 giga is not too much to have on a backup, and in two years is not too much to have even in your notebook's hd.
You buy/make 2 NAS with RAID 5 or 6. Your friend somewhere far away does the same. One of your NAS's becomes his, and vice versa. You set them up to mirror each other. If your house burns down or the contents get stolen, you buy/make two new NAS and get the data back from your friend. Upgrade in 5 years and continue indefinitely.
Embrace the pain of data loss, taking comfort in the fact that losing something that you really didn't *have* to have in the first place is actually freeing ;)
As far as maintaining your data's integrity such that a retrieval program can have a shot at recovering it you have a few options. You can backup to a physical medium that is indestructible (within the timeframe and conditions you must decide yourself) or the medium can be meta-physically indestructible within the same conditions as above. This would appear to you as a black box since you do not need to concern yourself with the 'how' aspects of its implementation just the means of interfacing with it (physical and communication protocol) which would be done through any number of interfaces which as time progresses might change just as the definition and implementation of a "computer" may change.
Now we know that when we submit something to this black box it will be stored and as long as computers and computing don't change too drastically we be able to retrieve it at some future time. The black box portion of this could be in the form of a company that handles all of the details of looking to you like a black box as described for a reasonable fee but here you run into a problem. The company becomes a single point of failure that all of your data at risk. Perhaps governments could provide the black box duties but I, personally, have several problems with that option.
An alternative way of providing the black box functionality and eliminating the single-point-of-failure problem is to somehow, see that the open source community produces a multi-platform (easily migrated code is what we need here), distributed, well-designed and documented, open source, peer-to-peer, backup program. I know that the topic has been discussed here before but don't know the current status of any of the projects.
This would be a great project for the open source community as it would solve a problem that has been faced by anyone that deals with large amounts of data and doesn't have a fortune to spend keeping it retrievable let alone easily accessed and navigated. There are many products, both software and hardware, that provide or, at least, approach meeting the requirements of the overall solution.
Be as you would have the world become.
http://www.labnol.org/internet/best-online-storage-live-skydrive/5771/ has a positive appreciation of Windows SkyDrive "The Best Online Storage Service"
Extract from the above blog post: "5. SkyDrive is Microsoft service and an integral part of their larger Windows Live strategy so you really don't have to worry about its future existence."