Slashdot Mirror
Benchmark Madness
Guillem Cantallops Ramis writes: "In Bulma (Balearic Islands LUG) we've done real benchmarks this time: Mongo benchmarks (designed by Hans Reiser and used to test ReiserFS, slightly modified by me to support XFS and JFS), kernel compilation benchmarks, and a small "database simulation" benchmark. You'll find everything in english this time, with benchmark results and interesting comments by Dr. Ricardo Galli (Universitat de les Illes Balears, UIB). Have fun... and switch to a journaled filesystem now!" The previous article was here.
It would be verbose, slow, and constantly be upgrading itself to use the latest buzzwords.
It would call itself a journaling filesystem, but everyone knows it's just a hack.
One advantage that ReiserFS and XFS are supposed to hold over ext2fs and other ufs based filesystems is the directory lookup time on directories with moderate to moderatly large numbers of files (1 million to 10 million or so). Does anybody know of any benchmarks available on the net that can backup this claim? If you want to test it yourself, you can look into Postmark which is easy to compile and simulates a heavily loaded mail or news server.
Unfortunatly the primary site appears to be down (I just downloaded the file a couple of days ago!), but if it comes back the primary distribution site is: http://www.netapp.com/ftp/postmark-1_13.c
Down that path lies madness. On the other hand, the road to hell is paved with melting snowballs.
I read the internet for the articles.
Journaling may not be that complex, but XFS sure is. The code size the Usenix paper quotes was over half the size of the BSD kernel I was using at the time.
XFS does give a number of other things. Hashed b-trees for file name lookups, multi volume file systems (with draining), GRIO (not in the GPLed version though).
I don't know about the size of RiserFS, but it too offers more then just journaling, it has some size and speed performance bonuses on small files.
FFS's soft updates changes were fairly small, but not simple. The second round of them adding file system check pointing are also pretty small, but again makes things harder to compare.
I like FFS + soft updates a whole lot. I think someone ought to port it to Linux. However if I had to run a Usenet News system XFS would look very interesting since it will do name lookups in huge directories much faster then FFS.
Yes, but because they force balancing work to be done on (some) inserts and deletes. Sometimes a lot of work. And that can require locking and blocking on those operations. They are still good data structures in many cases, but one must remember they get fast search times at the expense of possibly slow insert/delete times (not as slow as a insert on an unbalanced binary tree could be though as that hits linear).
The journaling capability isn't just about improving fsck times. Check out Intermezzo. The company that's working on Intermezzo (Mountainview Data) has some other cool sounding products that take advantage of some tricks only possible with journals (ie, taking snapshots of filesystems).... looks like mostly vaporware right now, though. I'm sure there are some other applications, but I'm not imaginative enough to see 'em. :-)
It's comparing how well these different filesystems can compile a kernel. Do these tests tell you how well ext2, reiserfs, xfs will be able to serve up dynamic php/mysql content on an apache server? Hell no. Use apache's bench mark stuff for that on a controlled network, controlled machines, reboot between tests to eliminate caching, etc. But that's another can of worms. Please don't try reading more into the results than what's given to you.
---
I'd go further than that - for me, compiling stuff is really the only performance benchmark I'm interested in - everything else happens "fast enough" these days. If reformatting my /home partition as ReiserFS is going to increase performance for kernel compiles significantly, I'll seriously consider converting.
Go you big red fire engine!
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
Probably?
Sorry, but "the good old Ext2" has left me crying for lost files more times than I want to count. I can abuse, for example, Solaris systems pretty well, and I have to work hard to get serious filesystem corruption. By comparison, for a long time it seemed every other time I just did an "init 6" on my SuSE box with Ext2 filesystems I lost another couple of files, until it got the the point where something to do with Qt is gone, and the window system won't come up. Greaaat.
"Only" journalling capability is akin to complaining about Oracle's "only" advantage over MySQL etc. being rollback/atomicity/transaction consistency. Gee, what a "tiny" thing.
7 November 2006: The day Americans realized corruption and incompetence weren't addressing 11 September 2001
Yah, those worthless little things; faster recovery, preserving data and faster boot-ups.
--
Linux O Muerte!
Looking back to the old benchmark on a slow CPU with JFS, the reading od 1398 100k files took just under a second, 200 times faster than next "competitor".
This is transfer rate of 140Mbyte/s, some 20 times faster than my current 5200rpm Maxtor and Quantum drives.
What is that?
Typo?
Effect of cashing? Why only on JFS?
Over 80% of file manipulation time is taken by reading. From the rest, 80% is taken by writing.
Deleting, stats or symlinks take less than 4% of time.
What is very suspicious is, that copying, which consists of reading and writing, takes half the reading time. Does it mean, that if I copy loaded binary to a dummy disk area, I'll sppeed application start twice?
The annoying delays are caused by reading larger files, like images, 10Mbyte mailbox, or a large binary. Why does have the XFS and Reiserfs to be 24% and 9% slower than ext2? Reading does not involve any journaling action! B-tree delays? worse cashing? Where is the problem?!
The 10% on reading makes way bigger impact than ten times slower deleting.
Can we make XFS and ReiserFS to read as fast as ext2 or faster?
Because the availability of the data in the Linux cache may affect the time measured for cp -a, I repeated the command a couple of times before doing the real measurements (there was a huge variance with the first time).
I don't know that much about benchmarking. But this statement seems a little off. If there is data in the cache then the disk is not being tested on the reads. Its seems like the first time running is by far the most accurate because no data is in memory. However you have to ensure that the no data is in memory for all tests. This seems fairly easy to reproduce by rebooting and performing the same set of commands at startup to run the tests.
---
Oops, it's not yet in the standard kernel. Oops, the patches don't apply against the latest, most debugged versions of the kernel. Not, it's not time to switch now. I'll stay with stable and standard functions, thanks
... Quota support ... Kernel automounter support ... Kernel automounter version 4 support (also supports v3) ... Reiserfs support.
Let me check menuconfig on what I got from kernel.org... okay, Linux Kernel v2.4.4 Configuration: File systems
It is in the latest stable kernel.
Good god, learn how a journaled FS works! Journaled filesystems only guarentee the consistancy of metadata, not user data. Thus, a well designed journaling FS is just as fast as a non journaling one. Writes to the journal only come into play when metadata is being modified, say when creating or deleting a file. And even then, all it involves is a couple of extra disk writes, which are coalesced into a sequential stream of writes in most systems. The actual performance hit of journaling is pretty negligable when it comes down to it. For example, the non-indexing version of BeOS performs on part with ext2 for metadata updates. Lastly, using B+trees isn't a "trick" its just plain good design. The same goes for allocation groups, storing extents in trees, etc. For example, while ext2 goes all over the disk to find all of the double indirect nodes for a large file, XFS simple does a tree lookup in (usually) one block.
A deep unwavering belief is a sure sign you're missing something...
Yea, things look pretty bleak. BeOS isn't officially dead, but nothing much is going on. eVilla got some attention recently though.
As for OSS development, it would be the best possible thing for BeOS. Even if Be has to strip out all the commerical code and release a non-working version of the source, great technology shouldn't be allowed to die. Something ala the BSD core team applied to the OS in its entirety (including userspace) would be great.
A deep unwavering belief is a sure sign you're missing something...
Softupdates can guartantee consistency in case of crashes, thus providing save yet asynchronous-like performance (i.e. optimal performance).
Details are explained on the website of the author, Kirk McKusick. Also you can find a link there which leads to an interesting (technical) comparison of logging (aka journalling) versus softupdates.
I wish someone would port softupdates to linux (ext2fs). Or better yet, make BSD's FFS+softupdates a native Linux filesystem. It would surely outperform the other currently available filesystems. At least on my computer, when I benchmark FreeBSD+FFS+softupdates against Linux+ext2fs/reiserfs (on the same hardware, disk, disklocation) FFS+softupdates consistently wins hands down. I don't think this is because of FreeBSD's kernel, but rather because of softupdates (and FFS with it's large blocksize combined with smaller fragments to avoid too much slack).
Benchmarks? Compiling the kernel to benchmark a filesystem? Hmmm. Sorry but, No. How about that "real-world" random write program? Give me a break people. These are not valid benchmarks which was clearly stated the first time this story was posted. Why /. seems to think it's interesting is beyond me.
Someone you trust is one of us.
He repeated it many times so the data would be in cache.
This wont affect write times because write-caching is not enabled in linux by default (it's VERY dangerous). So he was only limiting the effect of the harddrive/filesystem of the source disk in the performance measurement, and leaving the test partition (reiserfs, xfs, ext2) the only bottleneck in the benchmark.
This isn't a troll, but I thought BeOS went kaput recently.
If they dont release the sourcecode for BeOS under a liberal license, I dont see the point in tying up your coding effort into a proprietary platform that seems to have a pretty bleak future. Maybe you could correct any of my assumptions if they are wrong? I dont know..
Watch out man, you're eyes are turning brown. Oracle has transactions but that's not all. You forget a bunch of stuff (quotas, speed, scalabity, fail-over, dictionnary etc etc). Let's not compare apples to oranges. While I am a big MySQL fan, there are places where that type of software just doesn't cut it.
The initial poster, the guy you were replying to obviously doesn't have a clue of the value a journaling file system can bring. Make sure you don't loose your point by loosing your credibility.
Just for the record, I installed Linux today on a VA Linux 3500. 3 scsi cards, 1 RAID controller. You know how long this beast takes to boot? A while! If you add to that the time to fsck 26 gig of data, I kill myself. I agree with you, journaling is the shit!
Looking for a great online backup: Green Backup
cough* EXT3*cough*TUX2*cough
"Prefiero morir de pie que vivir siempre arrodillado!"
Did you read the overview in the link? If an EXT3 fs has been cleanly unmounted (or fscked) it can even be remounted as EXT2 again.
And yes, I know TUX2 isn't a journaling fs strictly speaking but it does help illustrate my point that a fs does not have to totally rewritten to add that kind of functionallity.
"Prefiero morir de pie que vivir siempre arrodillado!"
When XFS was initially released, I ran some very simple, for my-purposes-only benchmarks of XFS and Reiser, using Postgres and the dbbench utility. Ran the dbbench with 100 clients connecting a few times with a fresh initdb on a Reiser partition, and with a fresh initdb on an XFS partition. Didn't save any numbers, but it was considerably faster on XFS. Run it for youself and see:) (Note: This was w/out the use of the notail option for reiser) - James
signature smigmature
- James
ok, i can kinda see a kernel compile cause of all the io but what ever happened to good old find and grep. Certainly they could have come up with some better tests.
find / |grep blah
SEE, now that's alot of IO
"...through this door all my dreams come realities, and all my realities become dreams..."
A database really ought to use fdatasync, not fsync if fdatasync is implemented. fdatasync won't update the file access times, thus requiring only a single write operation instead of two.
Of course, since all "benchmark"/tests used fsync it kinda washes out since the same operation is common among all.
Does anyone know how ext3 is supposed to compare with these other fs?
Who does Mr. Galli thinks he's fooling. I mean, come on ...
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Wow. Does this mean it can handle SQL Server? That's always been my favourite database simulator.
--------
Bleah! Heh heh heh... BLEAH BLEAH!!! Ha ha ha ha...
Journaling IS important and reason enough. ReiserFS and XFS were not designed for speed but for far better data integrity. It is very impressive that for file systems that have to do so much more, are on par with the performance of ext2.
I miss the Karma Whores.
The numbers listed for XFS make it seem terribly slow in almost every category compared with either ReiserFS or Ext2, yet the writeup seems to conclude that XFS and Reiser offer comparable performance. What gives? Did something get lost in translation?
"Perfect numbers like perfect men are rare." -Descartes
See RFS Features
--
and most of the world is using FAT.
Excuse me while I cry.
The only advantage the new FSes hold is probably their journaling capability, leading to faster fscks, faster bootups and less risk of data loss. Did we really need a new set of filesystems for that? ( BSD Soft Updates show that the whole speed and reliability advantage can be had with old filesystems as well!) Where's the advantage? Where's the progress? The benchmarks only leave me disappointed.
There is absolutely no reason to panic.