Slashdot Mirror
Online, Inexpensive and Secure Data Storage?
ThePolkapunk asks: "After years of suffering through floppy demagnetizations, hard drive crashes, CDR bit rot and the click of death, I've become fed up with having to take care of my own backups. Does anyone know of a reasonably inexpensive, secure data storage facility accessible online that can store all of my important files with enough redundancy for me to feel safe?"
http://www.gmail.com/
Compress everything, encrypt it, and share it on emule as "OSX_X86.iso.
/.
For extra redundancy, post an obscure blog about it, and then submit that blog to
I've recently switched from DLT to external USB drives. They're cheaper, more reliable and easier to restore from than tape.
You can throw all your stuff in my server if you'd like. Make sure it's unencrypted, though, I'll take care of all that for you.
Seriously, though, I store all my backups myself, because it's important enough to me to back it up but I'm not a corporation that can pay someone to back it up for me. Assuming you're like me, you only have one option.
Pulp Audio Weekly - Geek News and Reviews
Sage Advice:
;)" - Linus Torvalds
"Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it
"why don't you just slip into something more comfortable...like a coma!"
A decent RAID setup is easy and fairly inexpensive these days.
Linux, you magnificent bastard, I read the fucking manual!
As usual, someone thinks they can have all three.
Here's a hint: Pick two.
GmailFS - The Google File System as discussed on Slashdot ...
Hulk SMASH Celiac Disease
At least as important as the others is stability. You don't want a site that winks out for a few hours because four guys decided to upload 20 meg .mov files at the same time. You also don't want a fly-by-night dot-bomb that is fast, meets legitimate security requirements, and is free.....but it inexplicably vanishes 13 days after you discover and start using it.
Don't blame Durga. I voted for Centauri.
I would like to know too. I have 80gigs to backup. I currently use a friends machine, but it would be nice to pay about $20/month and not have to maintain it mayself.
my_backup_14-6-2005@yahoo.com
my_backup_15-6-2005@yahoo.com
my_backup_16-6-2005@yahoo.com
my_backup_17-6-2005@yahoo.com
my_backup_18-6-2005@yahoo.com
my_backup_19-6-2005@yahoo.com
ad infinitum...
I have found there are just two ways to go.
It all comes down to livin' fast or dyin' slow. -REK, Jr.
You can try these guys: http://www.box.net/ Marc
Check out http://www.evault.com/
Very reasonably priced and they maintain offsite backups of their backups.
One thing to keep in mind is transfer speed. I would have used them but I have 40gigs of data to back up and it would take too long to send daily.
This technology is based on a scientific method used to detect the orientation of the Earth's magnetic field from millions of years ago. That qualifies as great data retention in my book! The drives (3.5in) run $200-$300 depending on source and capacity (current max is 2.3Gb per approx-$20 removable disk). 5.25in drives are also avaialable in capacities up to 9Gb or so, but also are much more expensive.
I wouldn't call it cheap, but I doubt you will find much cheaper than 10 gb for $5/mo.
Distributed Internet Backup System: I backup your stuff, you backup mine. All encrypted.
That way somebody will mirror it and others will provide torrents.
Information wants to be free!
Show me on the doll where his noodly appendage touched you.
Juggling With Packets
(This was posted on slashdot previously: Slashdot 03/10/06/1049250 )
Perhaps gmail is a bit more reliable though
aloha,
dave
Yahoo offers online file storage with a set ammount for free, and a fee-based structure for buying more space. Upload is through the web, and it also allows you to set up public folders for files you need to distribute.
Barring that, you can use an external hard-drive. I use a firewire storage enclosure with a 250 GB drive to back up my notebook, and to provide extra storage for video editing.
It's good to use your head, but not as a battering ram.
What you need is a peer-to-peer type system that allows you to upload your encrypted backups, but only if you offer a certain amount of storage back to the system. And a lot of upload (as opposed to download) bandwidth.
My boss uses a system I don't entirely approve of, but it works regardless. I work at a sound studio, and he does the following: Current projects are on his main computer's drives, backed up to a 1 TB LaCie nightly. Once a week, he brings in a firewire drive from home and backs up recently finished projects. In essance, he dumps projects onto drives as they are finished.
When the terabyte drive gets full, he stores it in a fire-safe cabinet in the studio, so he is able to access it within a few minutes (plug and mount). When his 'home drive' gets full, he puts it in a firesafe cabinet at home and buys another one. When projects run 4-9 gigbytes, this system has a large cost advantage over remote servers. The issue is that he does not back up his operating system, libraries, plugins or virtual instruments. It costs him a few hundred every time he has to pay me to reinstall everything. Also, the system isn't the most orderly to use. The advantage is that he has 3 TB, or over 400 projects easily accessable.
Sig (appended to the end of comments you post, 120 chars)
My advice - make sure you SECURE your data before it leaves the house... ie. run AES over the whole thing. Depending on your parania level - either keep the key locally, or with a DIFFERENT backup provider so that there would have to be a collusion between the two vendors to get your data.
Another solution would be to encrypt the AES key for each backup with your public key - then all you have to do is keep the private key private. Is it small enough to keep on a pair (or more) USB dongles - or again, back it up with a second vendor.
End result - most backup vendors provide physical security, so it is up to you to provide true security for when their physical systems are broken into.
I have mod points and I am not afraid to use them
The really ideal solution would be a p2p backup network. Each client has their own encryption keys, and agrees that for every X kbytes of local data they need backed up, they're willing to store 3X worth of data from others. Your data is encrypted with your key before being sent out to the p2p network, which is designed to keep a minimum of 3 remote copies of your data out there - if less than 3 copies are detected for any decent amount of time, it just makes more copies on other machines. When you need your data, all you need is your key (it's on a USB drive and/or floppy, or whatever) to decrypt the copies you pull from the p2p backup service.
11*43+456^2
Oh sure, it's easy to laugh. But one day, people will not laugh. Weavers unite!
Materials: 1 Linux server, 1 winxp desktop, 2 removable HD cartridges.
The desktop has a removable hard drive cradle.
- Every week I plug in one of the HD's to the XP machine and mount it onto the linux box via samba.
- I tar my entire linux box, piped through GNU Privacy Guard (to protect against unauthorized access) and 'split' (to avoid the 2GB file limit) and store it on the HD in date-stamped directory.
- I store the removable HD offsite.
The backup drive is much larger then the combined total of my server, so each backup drive can store at least 2 entire backups. Additionally, I never have both backups at home at the same time in case of disaster.My windows box stores all of its critical files (ie My Documents) on a samba share on the server.
I also do a daily rsync of the most critical / volatile files offsite nightly.
Get a hard drive. Preferably of another manufacturer, and copy contents to it. Unplug, and store. What is difficult to understand about this????
At my company, we've been using Iomega REV drives in more and more applications. The drives run about $400. The removeable disks run about $40 and they store 35GB. The drives come in USB2 and firewire. They also ship with some iomega backup software. It's no replacement for BackupExec, but for individuals using PCs, it's no half bad. The software does some pretty impressive compression on the data if you want it to, giving you upwards of 70-90 GB per 35GB disk.
http://www.yournetpod.com/
Many people seem to forget the single biggest issue with online backups. That is, performance. People seem to forget that it will take nearly ten days to upload 80GB even on a 1Mbps broadband connection.
What happens when your hard drive fails and you need to restore? Can you really wait for ten days or more while the restore is downloaded? In my opinion, if you can't do a full restore in four hours or less, then you need a faster backup/restore solution.
Online backups aren't really very useful unless you have a 100Mbps or 1Gbps connection between you and the backup server. Unfortunately for them, most people don't realize or accept this little detail until they have a catastrophic failure and then it is too late.
My vote is with USB drives. They're inexpensive enough that you can ghost a drive image onto a couple and, say, store one in a safety deposit box and another with a trusted friend/relative in another state.
'He who has to break a thing to find out what it is, has left the path of wisdom.' -- Gandalf to Saruman
The odds of all the CDs going bad or all the places blowing up at the same time are pretty small.
If you've got more data than CDs can reasonably hold, switch to DVDs. If it's more than that, use tape.
...then upload it to an FTP server and let everyone else mirror it.
Stop the world; I need to get off.
Not sure if that's what you mean or not, But RAID is *not* a replacement for backup. If you don't pay attention and delete a file, a backup should be able to give you that file back.
That's why you should also take periodic snapshots of your data.
I use Linux LVM over four-disk RAID-5 for most of my data, and LVM over four-disk RAID-1 for my most important data. I have a cron job that creates daily LVM snapshot volumes and removes any snapshot more than a week old, unless it was taken on a Monday, and any snapshot more than four weeks old.
The advantage of this solution is obvious: You always have good backups, with zero effort. For most systems, the amount of storage it requires is trivial, too, since the snapshot volumes only store deltas and most data is fairly static over a short period of time.
Nothing is ideal, though, and it does have a couple of weaknesses: First, there are catastrophic failure modes that might kill more than one of hard drives at once. For most of my data, I just accept that risk. For the really important stuff, I periodically rsync a backup to my laptop, one to my desktop machine and once in a while I burn a DVD which I give to my Mom (in case my house burns down).
Second, Linux LVM snapshots have a fixed amount of space allocated to them and they don't grow dynamically. So if too many changes occur on the primary volume, the snapshot volumes will run out of room to store changes and will just stop storing the deltas. This means that an "rm -rf" will wipe out all of your data unless your snapshot volume is as big as your active volume. So I'm careful not to do that, and I have a cron job that e-mails me if any of the snapshot volumes get over 50% full.
Finally, having lots of snapshots volumes does slow down writes, because each modification not only has to be written to the source volume, but any changed or deleted data also has to be copied to all of the relevant snapshots. I typically have 10 snapshots extant at any given moment, so that can add a lot of overhead. In practice, the overhead doesn't affect me but it might bother some people.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Pick two.
Sick of WoW? Try the thinking man's MMORPG: EVE Online
Check it out!
A quick check of my own hard drive gives me about a gig for my /home directory on the Linux side and about 3.5 gigs for the My Documents and Application Data directories on the Windows side; meanwhile, my Media directory has about 110 gigs in it - all ripped from other media, and thus replacable. Even the 4.5 gig figure could probably be pared down considerably, if I wanted to put the effort into it - especially in Windows, applications tend to leave temporary files everywhere. Although I wouldn't be surprised to hear that many people here have far more data than I, I would also be very surprised to hear many non-business users say that the majority of the information on their hard drives is totally irreplacable - that is, there are more than a few non-professionals who have more space devoted to their own work than they do programs, ripped media, and the like.
Why is this important? It makes it a lot easier to back up your data if you can isolate the important bits. Instead of a fairly expensive solution involving removable hard drives, it's probably possible to fit everything on a much cheaper archival-grade DVD or two - or store everything on a remote backup server for a lot less money. In fact, with only a few gigs of mostly static data to consider, an FTP or SSH-based reciprocal backup arrangement with a friend could work well.
The point here is that, while backing up every single bit on your hard disk is an excellent idea, and will certainly get you back up and running quickly, it can be much more feasable to simply back up the small subset of truly irreplacable data. While it's not as good as a full system backup, it's certainly better than the full system backup you never implemented or used because it was such a hassle. While this kind of system might be unacceptably lax in the buisiness world, it is certainly better than what many home users, including many Slashdotters, are using right now.
That's it. I'm no longer part of Team Sanity.
I believe that's supposed to be:
Fast. Cheap. Reliable.
When I bought my first DVD burner, I specifically chose one that was DVD-RAM capable. Little did I know that the media are 5-10x the cost of DVD-RW. So, what is the verdict on DVD-RAM, are they really engineered to last decades and 10's of thousands of rewrites? If so, how do they do it?
RAID is a great way of ensuring that your data is nice and consistent. If you're running RAID and you overwrite a file, delete one by accident etc etc then what you end up with is a perfectly redundant and consistent state: with your file deleted. RAID is for availability, backup is for safety.
You raise a lot of good points, but I'd like to mention that backing up media files offsite isn't a bad idea.
:)
Homeowners and renters insurance may limit how much they'll pay to replace lost media. Even then, you have to contend with finding the media: CDs go out of print, Disney stops selling certain movies for a time... even the companies themselves go out of business. (I've got CDs from labels that don't exist anymore, and were never re-released.)
It's all a matter of priorities.
Yeah. The time required to find a good way to back up media files (even if only once in a while) is far, far less then the time required to re-rip a music collection...
--LWM
http://onlinebackup.connected.com/plans.asp
o rm.aspx
Windows only, but the price isn't horrible.
Disclaimer: I work for Iron Mountain.
http://www.livevault.com/solutions/smb/multiplatf
LiveVault will run on Solaris for Sparc 7,8,9, and several version of RH for x86. I have no idea how much it costs; I don't work for that part of the company.
DISCLAIMER: I work for Iron Mountain.
Streamload offers a free online storage, retrieval and storage service. It takes only a few seconds to sign up for an account and you don't have to pay to store your stuff (they give you unlimited storage with no additional charges), you only pay a nominal charge for the amount of stuff you download back to yourself above 100 MB. Even the subscription plans are cheap (starting at $5 a month). I've used the service for about a year now and it works very well though using just my browser. I would recommend downloading their upload and download manager if you have a windows machine. It makes backing your stuff up much faster.
I use rdiff-backup for actual incremental backups. rdiff-backup is based on librsync and works amazingly well. There are a few spites such as a lack of checksums and non-numerical user ID's which can be dangerous when restoring a full system, but overall it is very impressive.
If you're solely using (external) HD's for your backups, you'll like DAR (Disk ARchive) even better than rsync/rdiff-backup.
Morality is usually taught by the immoral.
Damn right that RAID is not a backup. It might as easily be a mirror image of corrupt data.
Our company has an Apple XSAN with some 5TB of data on it. One day a controller bug (that Apple have since admitted to) led to it duplicating corrupt metadata across all the backup locations for the XSAN. Cue 5TB of completely (and I mean, totally) inaccessible data. Lost forever.
Fortunately we we able to reconstruct from genuine backups. But no-one should make the mistake of thinking of mirrors as a replacement for a proper backup. They are different!
http://www.backup-connect.nl/?pid=6
They offer online backup services in many parts of europe for reasonable prices, and you can go 30 days back (up to 180, if you pay more.) My company uses it (6 people) and we sleep quite soundly : )
- Jynx
A positive attitude may not solve all your problems, but it will annoy enough people to make it well worth the effort.
Selling online remote data storage internally is the toughest proposition.This is the quintessential problem,right? Even if you do find a good remote backup service provider,(this remote backup whitepaper is a good starting point to find remote backup services providers)you are going to find it difficult to convince your bosses who are always paranoid about possible security violations
> rm -f ~/important_file
> # Oh shit, I didn't mean to do that!
Please tell me how RAID would fix this problem. I'm waiting...
Waiting...
Still waiting....
Oh, that's right. RAID doesn't fucking solve the fucking problem. That's what fucking backups are fucking for.
Fuckin' A, you're dense.
"Avoid employing unlucky people - throw half of the pile of CVs in the bin without reading them." -- David Brent
What if your power goes out in the middle of writing a file to disk and the file gets corrupted? What if you lose multiple disks on your RAID at the same time because they were from the same manufacturing run? What if multiple disks are damaged while transporting the computer? What if there is a fire in your home?
Even if you are perfect and never make any human errors, you live in an imperfect world. Prudent computer users keep offsite backups.
I hope none of the things I mentioned ever happen to you. I hope none of the other dozen or so common causes of data loss happen to you. I hope you never rm something accidentally. As for me, I can't remember the last time I accidentally rm'ed something, but I am currently at the office sitting next to my encrypted offsite backup CDs.
"Avoid employing unlucky people - throw half of the pile of CVs in the bin without reading them." -- David Brent