Slashdot Mirror
A Good Filesystem for Storing Large Binaries?
jZnat asks: "I own hundreds of gigabytes of binary data, usually backed up from other mediums such as CDs and DVDs. However, I cannot figure out which filesystem would be best for storing all this reliably. What I'm looking for is a WORM-optimized FS that also has good journaling methods to prevent data loss due to some natural disaster while data is being shifted around. Trying something new for once, I tried using SGI's XFS due to its promising details, but I was met with countless IO errors after trying to write large amounts of data to it. I feel that Ext3 is not optimal for this; ReiserFS is too slow when it comes to reading large data files; and Reiser4 isn't mature enough to entrust my digital assets to. What filesystem would be most appropriate for these needs?"
So lets get this straight:
You need a filesystem that can be "burned" to a medium, yet have error correction capability.
Journaling doesn't do this. Journaling is for when you get a power surge in the middle of a write, you can get some of the data back. Currently no regular FS can do that.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Okay, why is this modded troll again? He misunderstood the direction of the backups, he wasn't trolling! If anything, it's -1 WRONG, not troll! Anyway - he backs up data FROM DVDs and CDs, not TO
journaling methods to prevent data loss due to some natural disaster while data is being shifted around
Journalling doesn't do this!. Journalling helps reduce file system corruption in the event of a catastrophic failure while modifying the file system - ie, it's possible to bring it back to the last clean state before it crashed - journalling does not prevent data loss. You might say "well filesystem corruption and data loss are the same", but they are not. If the filesystem is corrupted, the data is not lost. It just becomes not easily retreivable. If the data is lost then it becomes entirely irretreivable.
I tried using SGI's XFS due to its promising details, but I was met with countless IO errors
Have you considered your hardware is shit? I use XFS on terabytes of raided disks and have been for more years than I remember... 5 or so? I don't see any I/O errors. XFS is very reliable and I trust it with my data.
I feel that Ext3 is not optimal for this
Well not all of your post was dumb!
ReiserFS is too slow when it comes to reading large data files
How is it slow? It takes a few microseconds longer to access the first data sector because it does some extra processing first? Give me a break. Filesystem performance for journalled filesystems is mostly bound by writing speed, and this is a function of how the journal is updated. I doubt you would notice the difference in read speed unless you ran a million tests over a million different files, took some sort of average for the filesystems and quibbled over a few milliseconds.
Reiser4 isn't mature enough to entrust my digital assets to
You entrust your assets digitally? Shit, why do you trust any filesystem? They are all buggy. Give me a break.
If you don't like it, keep backups on other media; buy a tape drive and a robot and get in bed with a good archiving company to securely store the backups. Don't come one here and poo poo all of the file systems known to man then tell me "is there anything better"? About the only 4 in common use you left out were JFS (good for large databases but not much use if you have a lot of small files), FAT[12/16/32] (not much good for anything really), NTFS (see FAT, but more complex) and ISO9660. I'll concede there are others, but if you want something that's in common use so you can actually retreive your data when the world turns to shit...
Anywho!
I drink to make other people interesting!
I doubt that. To run GFS (assuming you have the code), you need to have a big honking cluster, to replicate data across machines. Also, it assumes a different file semantics, so you need to hand-code your apps to use the different reading and writing semantics. It only works well for appending writes and streaming reads. Furthermore, GFS does not have file-locking, and concurrent writes will leave your files in an undefined states.
Han-Wen Nienhuys -- LilyPond
The number of inodes in a filesystem is an option that can only be specified upon filesystem creation. i.e. to mke2fs, not tune2fs.
retrorocket.o not found, launch anyway?
I never run a computer without a UPS. IMO, they should be built into the power supplies (I wonder if you can buy such things...). I guess I've had too many computers die on me due to power issues, but power can be very unreliable in my location. UPSes are so cheap it's not worth it to *not* use them.
Max.