Slashdot Mirror
Reiser4 Filesystem Released
trixie_czech writes "It's finally arrived. Go to namesys for reasons to use reiser4 as a filesystem and benchmarks. Go here to download. Enjoy!" The Namesys homepage in its current stage reminds me of a cross between The Secret Guide to Computers and the GNU Manifesto -- which is to say, there is a lot to read here, not just a bullet-pointed feature list.
Will I be able to convert my exsisting ext3 fs to reiser4 fs withou having to reformat?
No, you will have to reformat. However, I recommend the upgrade; I've seen a number of studies showing that the performance of ext3 is awful compared to reiserfs. The only arguable advantage of ext3 is its compatibility with the baseline ext2.
Tubal-Cain smokes the white owl.
there is rfstool for reiserfs (afaik not v4)
and many for ext2/3
if OTOH, you are looking for a fully featured driver that can be used for production use, then i wouldn't count on it
Marge, get me your address book, 4 beers, and my conversation hat.
Possibly using convertfs, but I have no idea if it works or not.
This page seems to have more info about it.
Reasons why Reiser4 is great for you:
* Reiser4 is the fastest filesystem, and here are the benchmarks.
* Reiser4 is an atomic filesystem, which means that your filesystem operations either entirely occur, or they entirely don't, and they don't corrupt due to half occuring. We do this without significant performance losses, because we invented algorithms to do it without copying the data twice.
* Reiser4 uses dancing trees, which obsolete the balanced tree algorithms used in databases (see farther down). This makes Reiser4 more space efficient than other filesystems because we squish small files together rather than wasting space due to block alignment like they do. It also means that Reiser4 scales better than any other filesystem. Do you want a million files in a directory, and want to create them fast? No problem.
* Reiser4 is based on plugins, which means that it will attract many outside contributors, and you'll be able to upgrade to their innovations without reformatting your disk. If you like to code, you'll really like plugins....
* Reiser4 is architected for military grade security. You'll find it is easy to audit the code, and that assertions guard the entrance to every function.
V3 of reiserfs is used as the default filesystem for SuSE, Lindows, FTOSX and Gentoo. We don't touch the V3 code except to fix a bug, and as a result we don't get bug reports for the current mainstream kernel version. It shipped before the other journaling filesystems for Linux, and is the most stable of them as a result of having been out the longest. We must caution that just as Linux 2.6 is not yet as stable as Linux 2.4, it will also be some substantial time before V4 is as stable as V3.
Will we ever have a Windows port of ResierFS or any alternative filesystems?
I'm not sure about ReiserFS, but there is already a program -- Explore2fs -- which lets you mess around with Ext2 and Ext3 partitions from Windows. Why you would want to do that is beyond me, but there you go.
Of course, you may be talking about a native Windows implementation of Ext2/3 and/or ReiserFS. Which is a totally different kettle of fish...
Tubal-Cain smokes the white owl.
"* Reiser4 uses dancing trees, which obsolete the balanced tree algorithms used in databases (see farther down)"
that's what he meant.
oh, and whoever moderated offtopic didnt rtfa, either. damn, peeps.. what is wrong with this community these days?
pm
** "It's not my job to stand between the people talking to me, and the ones listening to me." -- Pego the Jerk
Well, Mr. Reiser Dude suggests tar in his posting to lkml which can also be viewed on kerneltrap.org.
In other words,
no.
yeah, that, and *stability*. reiserfs has a noteable history of people losing their data because of filesystem problems.
Not over the past couple of years -- the original corruption problems with reiserfs, although pretty severe, are well in the past now.
Tubal-Cain smokes the white owl.
ext3 has a functioning fsck, reiserfs does not.
I myself have never had any problems with reiserfsck -- what exactly is wrong with it?
Tubal-Cain smokes the white owl.
And yeah, you'd better believe it happens. The BSDs use a similar approach called SoftUpdates (basically odrered writes). If power craps out in the middle of a write, you will have corruption. The main advantage is that, because writes aren't scattered all over, you only lose the file(s) you were most recently working on. This focuses the damage, it doesn't reduce it at all.
I'm not sure about ReiserFS, but there is already a program -- Explore2fs -- which lets you mess around with Ext2 and Ext3 partitions from Windows. Why you would want to do that is beyond me, but there you go.
I have a hard drive with Windows installed on it and a hard drive with Linux, and I use both OSes. Explore2fs is handy when I'm in Windows but I need to grab a file on my Linux drive.
Nope convertfs won't work... From the horses mouth:
To upgrade from reiserfs V3 to V4, use tar, or sponsor us to
write a convertfs.
The lkml posting is probably cached all kinds of places, but kerneltrap also reproduces it in full.
Then again, reiserfs v4 and v3 have nothing to do with each other (unlike ext2 and ext3 for instance), so there's no quick fix possible probably.
On the other hand - reiser4 is completely untested (compared to reiser v3 and jfs, xfs, ext2, heck even the wine-dll emulation layered ntfs writing driver...), so do yourself a favour and don't do anything quite so crazy as not just using it for a production machine but also trying to convert an existing system to it with 'smart' tricks... Give it a little while... or make a lot of backups...
Actually, v3 is NOT the default filesystem of Gentoo, Gentoo has no default filesystem, you pick what you want. They give the easy option of ext2/3 xfs and reiser
>MD5 has been proven to have collisions.
Statistically speaking you are more likely to get malaria in Arizona than experience a random MD5 collision.
Um, yes, there is an advantage. That's what the journal is for (duh.)
It astounds me that your post was marked as "Informative," because it's downright wrong.
Now, if you're talking about fsck after a certain number of boots, or a full fsck for whatever reason, then no, there's no advantage over ext2. It's ext2 + improvements + journal, for the most part.
For my money, using ext3 without btree hash dirs is stupid nowadays. Go back and bench reiser vs. ext3. ext3 is usually still slower, but the gap is narrower nowadays.
Stating on Slashdot that I like cheese since 1997.
Err, the point of atomicity w/ journaling in a heirarchical system is that if you lose power during a write, it is data to which no parent i-node or directory points. The data being created or altered is written first, then its updated directory, and then its parent directory on up to the root. Or you have one journal level, where the file is written to journal and then the journal entry is copied over the original location. If power dies when the journal is being written, data is lost but the FS maintains integrity, or if the power goes during the copy, the journal exist. Atomicity means that a transaction either happens all the way or not at all, and Reiser4 does guarantee this. In-flight data can be lost so long as partially written data does not leave the system or some other API-level atomic transaction partially completed.
All hash functions have collisions; that's not the point... It's extremely unlikely for one to occur non-maliciously. The MD5 collision found was only found after some trivial mathematics done as a matter of principle by your network card just to spite you (okay, just kidding).
It's still perfectly fine to use MD5 to check the validity of your files for bit-errors. Then again, so is CRC32.
I do have a question to anyone more knowledgeable in MD5's weakness: although MD5 can now be spoofed , it's not clear to me from reading the news - is it only directly applicable to messages of a certain type/length or to all messages?
Sure MD5 has been proving to have collisions, but that really isn't news. All hash algorithms have collisions simply because of the nature of what they do.
Suppose you've got a 1K file. There are 2^1K possible values that file can assume. If you map those 2^1K values to the 2^160 values a SHA1 hash can assume, you have an average of 2^944 1K files that collide on any give SHA1 hash.
What differentiates hash algorithms is their ability to prevent people from generate a text that matches a given hash. It is currently not possible to do this for either MD5 or SHA1. It has been speculated that MD5 is nearing the end of it's life in this regard though. I don't follow the field closely enough to weigh in on the matter, but I can tell you that the only thing that finding an actual md5 collision will do is demonstrate what was rather easily proved in the previous paragraph.
As far as verifying files is concerned, the cryptographic strength of the hash algorithm is irrelevant. Unless you suspect someone will be tampering with your results, use whatever algorithm you can find a useful tool for, be it md5, sha1, or even crc32.
When deciding which filesystem would be best for our first critical samba file servers, this post and other scattered rumors of unreliability scared us away from reiser3 for the time being:
u ly/msg00418.html
;)".
http://www.redhat.com/archives/fedora-list/2004-J
The date of the post caught my eye. The test was very recent. Ext3 won in this particular case, by a longshot, leading a Red Hat employee to respond "Your investigation proves that we default to the right mode
I haven't seen ext3 (ordered) lose in any reliability benchmarks versus jfs, xfs, or reiserfs, though it's hard to find many such benchmarks.
It is now possible to generate a bitstream that matches a given MD5 hash. It's recent news too... :-).
Some postings can be found here, and google is your friend
--Eamon
"man reiserfsck"
But ReiserFS doesn't need an "fsck" type program in normal circumstances. In power outages, etc., it's rock-solid. But for things like drive failures and the likes that tend to actually corrupt the data, then yes; EXT3 is the better choice. The reiserfsck program isn't intended to be run on the event of just any power outage or failed unmount, because those sorts of things don't tend to damage the filesystem.
I've been using ReiserFS 3 for years and I've really been happy with the results. The only times (once or twice) that I've had problems were when I had severe hardware malfunctions (due to failing mobo capacitors and a dying hard drive), and my own carelessness when trying to repair the bad data.
It rocks. Very, very fast. Whereas an:
would have taken about 20 seconds to run under reiser3, it takes about 5 seconds under reiser4. Very impressive.
I haven't had any stability issues with it. There were ( last time I checked ) 2 outstanding bugs - one nfs ( files on server with reiser4 ) bug, and one strange bug that made adn OpenOffice compile fail miserably. Bug (1) doesn't affect us, as our nfs server is running reiser3, and bug (2) I got around by creating a reiser3 partition, mounting
I just can't get over how fast it is.
Over the past few days (ever since reiser4 was accepted into the mm kernels) I've been looking for a boot CD with reiser4progs 1.0. I want to try reiser4, but need a boot CD to format my new drives and mount my current partitions for copying.
l in ux/rip/
The only boot CD I was able to find was the (R)ecovery (I)s (P)ossible Linux rescue system:
http://www.tux.org/pub/people/kent-robotti/loop
It was released yesterday (22 Aug) and is still warm to the touch.
Stephen
I've successfully recovered a trashed array running ReiserFS after losing a CPU.
reiserfsck is there, and does work.
I've had more problems with the Ext filesystems than I care to mention, and we do not use Ext2 or Ext3 on any production machines that run Linux any more. Everything's ReiserFS v3, and once we start testing Reiser4, we'll move to that.
Ext3 was a hack for compatibility with Ext2. It serves its purpose, which is easy upgrades and backwards compatibility.
Its also true that by default after 6 months or 30 mounts an ext3 volume *will* perform a full fsck. This can be very painful if the volume is large enough/holds enough files that an fsck will take 24 hours plus (I've seen this several times)
Personally, I'm a fan of IBM's jfs. I had some early problems with it but they were real responsive to every issue I found while punishing it as an NFS volume in a test system.
You are in a maze of twisted little posts, all alike.
Reiser4 does not have such a problem, as it operates in data=ordered mode by default.
To get something done, a committee should consist of no more than three persons, two of them absent.
I think you misunderstand, that's the beauty of it. Basically, Linux (and FreeBSD with GBDE) allows you to encrypt a device at the block level. Everything is written to the disk encrypted, including the file system itself and not just the data. This also allows you to abstract the device. It could be a big file sitting on an existing device or the device itself. It's very flexible.
Some of the other advantages of this are fairly important. Here's a few off the top of my head.
On the plus-side, filesystem level encryption lets you choose to encrypt on an as-needed basis (such as with NTFS), but the uses of this are minimal and questionable at best (what about swap, temporary files, and data that you forget to encrypt?)
I think you may have learned from my previous comments how you accomplish this. Hint: you don't encrypt at the filesystem layer.
Using the loopback device to encrypt data has been available for longer than NTFS has had encryption.
Why bother.
Now that I think about it, the main problem would be with programs knowing that files can also be accessed as directories. tar would work fine provided that the readdir() system call (or whatever you'd use to get a directory's contents) returns both a file and a directory for each file.
"I think it would be a good idea" Gandhi, on Western Civilisation
I suppose it's just a coincidence that the reiser benchmarks page doesn't compare it to XFS... or maybe they were too embarassed to show the results?
---
Please quit being a total twit. XFS has its' place, but for now, we are discussing ReiserFS. Just for the record, ReiserFS has been around for years, and does a great job with mixing loads of little to medium files. While XFS does an ok job, it really excells with the large files, in particular, very large sparse files.
I just wanted to add my two cents to this: We had done internal benchmarks at our company, and found XFS to be the fastest filesystem, and seemed to have a good track record with the community. (We didn't consider reiserfs because of its lack of bad block handling).
Either way, we converted ONE of our 2 Terabyte mount points to XFS. Whenever a file would be created on that mount point that exceeded 4G, bdflush would peg the cpu at 100%, commits to the disk would cease, and file system corruption ensured.
This was with kernel 2.4.23.. The problem was fixed in 2.4.25 (maybe 2.4.24, but we never tested that kernel). When we had this issue, and linked it to XFS (through another test system), we quickly migrated away from XFS, back to ext3.
We never had a problem like that was the ext's. We've lost data with both reiserfs and XFS. And if you grep the changelog for the kernels on XFS, you'll see tons of fixes for "deadlocks, race conditions, oopses", etc. These were all fixes AFTER 2.4.23..
Lesson: Stop playing with something that works, and be happy your servers serve. We never made it to testing JFS, and we probably won't. Ext3 might not be the fastest kid on the street, but it has been the most reliable for us.
-- If we don't stand up for our rights, now, there will be no right to stand up for them later.
This is the single most important factor when it came to deciding what filesystem to run, namely, can reiserfs 4 be upgraded to new versions easily?
Yes; as I understand it, ReiserFS 4 is designed with a plug-in architecture, so that future improvements to the filesystem can be incorporated in a non-destructive manner. You can read more about this functionality in the summary of the new features in v.4.
Tubal-Cain smokes the white owl.
Journaled: The data is written to a temporary queue and then copied to the main storage. If the system dies while writing to the temporary queue then the main storage is unaffected; if the system dies while writing the queue to main storage then the system will notice when it reboots and will resume writing the queue to main storage.
PRO: Safer than non-journaled since you can never end up with half a buffer written to disk.
CON: Writes everything twice, causing delays. Very bad things could happen if data and associated metadata are in separate transactions and the system crashes between them.
Atomic: The file data is written to unallocated space on the disk. Once that has completed, the directory record is updated by writing a copy of that record to unallocated space. The directory's parent is then updated by writing *it* to a new region of the disk, and so on up the tree. Since each write doesn't take effect until the next has completed, any interruption results in complete reversion.
PRO: Safe. Faster than journaled since there is no double-posting.
CON: More complicated to impliment, I suppose. I would expect it to be slighly slower than journalled method when writing very small changes to existing files as journalled can optimise the writes in the queue whereas atomic has to finish what it started...
On reliability:u ly/msg00418.html
"After 3 or 4 power cycles, ReiserFS became corrupted to the point that the system would not boot up (the fsck failed and the bootup stopped there)." - http://www.redhat.com/archives/fedora-list/2004-J
On code maturity:r y/l-fs7/
"In contrast, ReiserFS' fsck is in its infancy..." - http://www-106.ibm.com/developerworks/linux/libra
Hans and co's attitude:
"For $25 you get an answer to any question we can answer with less than half an hour of working on it. fsck support sometimes takes more than half an hour" - http://www.namesys.com/support.html
Our atomicity does not provide isolation or rollback, it is only atomic in the sense of whether it survives a crash. That is, a reiser4 atomic set of operations will either all survive the crash or none of them will.
You can say that this is not really atomic, and by database traditions that is correct, but I believe we have implemented the aspect of atomicity that for sure should be implemented by the file system and not by the layers above.
Later we may support more isolation and rollback, but we started with allowing people to define a set of fs modifying operations that would either all be preserved across a crash or none of them would be preserved. I tried using the term "transcrash" instead of atom, but no one but me loved the term.
I must caution though that the API for defining an atomic set of filesystem operations is still being debugged. The core infrastructure is rock solid though, as it is what we use for atoms defined internal to the FS. We shipped as soon as our core code was rock solid, and plan to incrementally finish the other stuff over the coming year.
Our response is definitely not so what. We might have told you that metadata journaling (what V3 uses) provides a level of service in which, like FFS and many other filesystems before it, if you crash during a write the write gets garbled.
Reiser4 is fully atomic though, and a write will either make it to disk entirely or not at all, with no data garbling. In other words, assuming that metadata journaling was what made you unhappy, we listened, but waited until a deep rewrite could allow us to fix it with no significant performance loss.
We are very happy that the use of wandering logs allowed us to make things atomic without losing any significant performance.
If we go for a week in -mm with no bug reports, I send it to Linus.
Our approach is to first get it to where nobody in the developer team using all known filesystem crashing scripts collected over 10 years can crash it, and then ask our mailing list to crash it, and only then ask real users to try it.
That said, if you have a mission critical server, be sensible, wait a bit.
It is in the -mm kernel, if it goes without a bug report for a week, we send it to Linus. I hav been surprised by the lack of bug reports after going into -mm. All we have is one apache 2 bug report that we cannot reproduce yet.
No, atomic is the only option for reiser4, there is no metadata journaling even as an option. So, it both goes faster and keeps your data safer. It is nice when experiments with algorithms work.... :)
Hans
Oh, dear. Bad block handling is not needed on modern drives, all moderns drives have automatic remapping of failing blocks, and if you have a drive which actually has bad blocks which are visible to the OS you should not be storing any data on that drive.
Just to add a data point: I've also had very mixed experiences with XFS. I installed it and it seemed to be chugging along fine for ~1 year (just regular desktop machine, no particular I/O load to speak of) until suddenly the initial root mount showed an empty
HAND.
No, actually, waiting before deleting the old copy is not enough. You need to make sure that there are not two copies.
Think of classic banking example: credit savings and debit checking are a single atomic operation. You must ensure that you don't get the credit preserved and the debit lost by a crash.
The poster above you was right.
NTFS was usually between Reiserfs and Reiser4 in the benchmarks I've seen so far. Reiser4 being always the fastest by about 5-20%.
That was because reiserfs only hat a writeback journal then. The means that it could happen (very often) that the metadata was written before the actual data was and you would end up with random data in the files that were written just before the crash.
I know it because I had the same problems.
But the current 2.6 kernels have a lot of improvements. reiserfs now also has data=ordered and data=journal journalling modes (with ordered-data being the default now). This means that the actual data is written before the metadata is committed. I've never had those problems again.
And, BTW, it's hardly the fault of the filesystem that you lose some data if your system crashes.
The current reiserfs in 2.6 also has a lot of other improvements. A better block allocator, quota and ACL support. If you need those for 2.4, SUSE has had the patches in their kernels for some time now.