Slashdot Mirror
Long-Term Personal Data Storage?
BeanBagKing writes "Yesterday I set out in search of a way to store my documents, videos, and pictures for a long time without worrying about them. This is stuff that I may not care about for years, I don't care where it is, or if it's immediately available, so long as when I do decide to get it, it's there. What did I come up with? Nothing. Hard Drives can fail or degrade. CD's and DVD's I've read have the same problem over long periods of time. I'd rather not pay yearly rent on a server or backup/storage solution. I could start my own server, but that goes back to the issue of hard drives failing, not to mention cost. Tape backups aren't common for personal backups, making far-future retrieval possibly difficult, not to mention the low storage capacity of tape drives. I've thought about buying a bunch of 4GB thumb drives; I've had some of those for years and even sent a few through washers and driers and had the data survive. Do you have any suggestions? My requirements are simple: It must be stable, lasting for decades if possible, and must be as inexpensive as possible. I'm not looking to start my own national archive; I have less than 500GBs and only save things important to me."
We don't have enough history on this tech to know what, if anything, will "last for decades". Possibly "paper" and "microfiche" might fit in that list, but those aren't the sort of things you're talking about. Best option I can think of right now would be to get a couple 500gig drives, put everything on both, and then put them in different areas. In 3-5 years, back them up to something newer, and repeat that every 3-5 years. Maybe in those intervening years, we'll have more data and newer tech that's demonstrably suited for what your needs are.
creation science book
How many times has this question been asked on Slashdot? I swear, it shows up on the front page at least three times a year.
As for the question itself, the answer is pretty simple, but unhelpful. Basically what it comes down to is that there is no safe place for your data. You're asking for the best type of basket to put all your eggs in. If you look at it that way, the solution is easier to arrive at. Your choices are A) spare no expense and build/buy the world's strongest basket and pray no flaw arises, or B) start copying your eggs around to all sorts of cheap baskets and continuously add more baskets in the expectation that the oldest baskets are going to fail.
Copy all your stuff to all your computers. Burn to DVD and/or CD ROM. Buy SD cards and USB flash drives. High capacity storage devices are so cheap now that you can keep all your valuable pictures of your vacation to Cleveland quite safe by constant duplication. That's the value of digital. Copies are perfect. Make lots.
If a job's not worth doing, it's not worth doing right.
Here's the thing. Flash drives will *probably* last long enough. I wouldn't at all be surprised if they were still readable in 20 or 30 years. But a) what's the odds of your current WinUx or iHoloTablet having a usb connector in 30 years? and MUCH more importantly, what's the odds of having anything capable of reading those historic Word 2007, Acrobat 5, or any other type of file format in 20 years? Yes, there are some folks technical enough that they can still read and readily interact with Geoworks, Wordstar, Xywrite, etc. stored on 8" floppy disks. But if you ain't one of them, and I'm happy to admit I ain't, the fact that the flash drive is physically capable of being read in 30 years simply won't matter. That's why I crack up reading various vendors' claims of CDs, DVDs, BDs etc. lasting 50 or 100 or more years. The disks will be readable but you'll have no mechanical or logical way to read what's on them.
Remember paper? That actually roots after a period of time.
That's why paper is specifically prohibited on my network.
There is absolutely nothing that you can put away for decades and expect to be useful. Your requirements are not simple - they'll actually very, very hard to meet, even if you want to throw a lot of money at the problem.
You don't know that a jpeg, for example, will be readable in 30 years. The format may be so deprecated that there might not even be a viewer available. Like my old Microsoft Works 4.0 documents - although I have the data, I have nothing that can read them unless I want to spin up an old Windows image, assuming that I can generate a virtualized environment that can support an old Windows (Windows XP probably won't even boot on any PC being produced 30 years from now). And some of that data is only a few years old, not decades old.
You should store not only the data, but also the applications that created the data. And the computer you need to run those applications. And backups of those. And then every few years, pull it all back and validate it and update as required.
You may have only 500GB now, but 10 years from now that will be 5TB. And then you need a way to actually be able to find something you added to your "archive".
I deal with this at work regularly. An archive is not a backup that you keep for a long time. It's much, much more than that. Once you start thinking about all of the issues that come up, you'll see that the media is the least of your problems.
Its not really that cheap, and not that simple to use for personal backups. Unless you are willing to write your own backup scripts, its going to be a headache.
Querying S3 for a list of stored files is *very* slow, and you only get 1k results per query. This means you have to index what files you put in S3 in a local db. This allows you to ask the db what files are there (and how to grab them).
If you only have a few files you can use the S3 browser extension for Firefox (or one of a many file system mounting, ftp simulating, etc tools). Just keep in mind the 1k file limit per query and box things in folders of no more than 1k items. Otherwise you will have a very slow browsing experience.
I have around 120 GB of family photos and purchased mp3s that I would like to store. To store 120 GB at .15 per gigabyte/month for 1 year would cost me: $216 (at $18 a month).
We use it where I work, with great success, but it would be much to much work for me for a personal backup system.
Considering the cost, I would go with a consumer targeted app (there are LOTS of them). A number of them charge a flat flee for "unlimited" storage. Beware of how you interface them. Some support windows only.
That is an interesting suggestion, but 500GB would cost $900/year (plus transfer costs) which I don't call "dirt cheap". As far as being there forever... who knows if there will be an Amazon in 10 years? Amazon might be more stable than most hosting options, but forever is a long time.
1) Encrypt your data
2) Tack it onto the end of some anime or porno mpg
3) upload to kazaa/gnutella/whatever
It'll still be circulating the net long after our grandchildren are dead.
Very durable. Write speed is a bit slow though...
Having a tape around for 20 years doesn't do you any good. 20 years ago, I was writing to 1600 and 6250bpi tapes. Today, my data center doesn't even have a drive that can physically read them.
Today's tape technology is no different. 3 years ago was writing to SDLT tapes. By next year, I won't even have an SDLT drive in my data center, having migrated everything over to LTO.
Yeah, I have round tapes in my offsite storage. I have 4mm and DAT tapes out there. We're just wasting money storing the media, since we have nothing that can read them.
If I could read the old media and extract a really old database, would today's database app be able to read it? Probably not. And could I install that app on today's OS? Probably not. And could I install the OS from many years ago on today's hardware? Probably not. Could I compile source from 20 years ago with today's compilers? In many cases, actually I can't. And if it really did all magically get compiled, is anybody around that can still knows how to run the app?
Don't forget that 20 years ago, many systems didn't have TCP/IP installed. In 1988, mine didn't - it was a combination of RS232-attached terminals and XNS-attached graphics workstations. Drive sizes were 80-160MB. A couple of MB of memory was a lot.
For those of you not still in school, ask around and see how many folks in your IT department can name the server that held your financial data 10 years ago.
well the backups storing the questions have been lost.
If your data is not in your possession, how do you know others won't see it or edit it without your permission? For archiving purposes, the best technology is magneto-optical. Despite the fact that makers have been exiting the market due to competition from faster and larger-capacity technologies, MO remains the champ for data storage duration. Remember, it is partly based on a Natural phenomenon that lets geophysicists detect which way the Earth's magnetic field was oriented, hundreds of millions of years ago --data retention just doesn't get much more long-term than that. MO disks are removable from the drive, and every modern drive can read any older same-size disk (they come in the standard 3.5" and 5.25" sizes, but have quite a range of capacities), so if the drive fails, just make sure you have spares. Perhaps, sooner rather than later, the manufacturers will realize that archival storage is a niche market that will demand that they stay in the MO business.
"Hard drives, while they may fail, are still probably your best chance."
I tend to agree, however I'm a bit confused over what exactly is being requested.
"I've thought about buying a bunch of 4GB thumb drives....I have less than 500GBs and only save things important to me."
At first glance I thought you had 500gb you were trying to store, but then you mentioned "buying a bunch of 4GB thumbdrives" and I can't imagine someone buying 125 4gb thumbdrives to use for backup. So exactly how much data are you trying to store?
If less than 50gb, I'd suggest a few SD cards. 8gb SD is ~$11, or 16gb for $30. While more expensive than hard drives per gb, SD cards are remarkably resilient, surviving a week in the ocean, and a few in a ziplock bag stored in a safe deposit box would probably last close to forever.
SD will probably still be around at least for the next decade or longer. SD has already been around since 1999 and all modern card readers read SD cards by default. SD slots are in nearly every form of consumer electronic device, and every manufacture of digital cameras uses SD except Sony and Olympus, almost guaranteeing the card readers will be around for many years to come.
I would suggest against USB anything since they're already discussing cutting the cord on USB and going wireless USB. While I don't predict that will happen overnight you wanted a solution that would be available decades from now, and wired USB might go the way of the parallel port, which was the standard external port in the 80s and 90s but was replaced by USB late 90s. Parallel port only had a lifespan of about 20 yrs and is no longer on modern PCs, and USB has been out just over 10 years so it's feasible in 10 years PCs will no longer have USB ports, everything will be wireless USB.
my karma will be here long after I'm gone
This actually is a good idea. If the porn files were maybe one half sex imagery and one half encrypted private data, and there was no easy way to separate the two halves, then people would download, store, and upload the files in order to view the porn. Anyone who had data in the private section of the file could download it from various P2P sites.
The cost of filming and creating the porn file would be covered by the people who would be using the file for long-term distributed storage. Say a 1.5Gigabyte file that was an hour of MP4 video entertainment and 500 megabytes of distributed storage. The fees received by the producers for the storage would pay for the video production costs. Since porn is cheap to produce, this may solve the problem of piracy and secure storage at the same time.