Slashdot Mirror
File System Round-Up Interview
Little Sheep writes: "An interesting round-up interview regarding modern Linux filesystems is published by OSNews, featuring the developers behind IBM's JFS, ReiserFS and SGI's XFS filesystems."
← Back to Stories (view on slashdot.org)
all your fp?
-
Not much about Ext3, even though Redhat seems to prefer it to the others...
Trolling using another account since 2005.
fuck mackga. fuck cm. fuck es. fuck 'em all.
FIRst fucking nigger post you!
first nigger
Dedicated to The O.P.P!!!
OK, many of you have probably noticed a lot of ascii art flooding this board lately. You're no doubt wondering why CmdrTaco's much encountered "lameness filters" are ineffective against it. To answer that question, I'm going to take you on a Perl journey, deep into slashcode. Think of it as being a bit like "Heart of Darkness", or if you're a typical slashdot pleb, "Apocalypse Now!"
The first files we'll be looking at are the infamous "bitchslap" and "modslap". I think these are a pretty decent introduction to how Rob's mind works when he's coding angry: Here's bitchslap [slashcode.com]. Scroll down to the section labelled "main program logic". BTW. "main program logic" is a ridiculously grandiloquent phrase for what we are seeing here.
Note how draconian this is. I like to think of this code as a memorial to slashdot-terminal, it's first victim. You can see how Malda deliberately broke his "self-regulating" moderation system, to give the admins of slashdot dictatorial powers. This code allows any admin using it to drop a user's default threshold to -1 instantly, and drop his karma down to a level from which it is unlikely to recover. I've heard that this script has been used on slashdot in a modified form, with the -defaultpoints set to less than -1, completely eliminating a user's post from normal viewing, unless people edit their query string manually.
On to modslap [slashcode.com]. Scroll down to "main program logic" again.
OK, what you're seeing here is Taco's method of restricting the flow of crack to moderators, to keep them nice and jumpy. If you don't mod the way he likes, your moderating days are over, and your karma plummets to bitchslap levels. An ugly tool.
To recap, the man we are dealing with here is obviously an anti-democratic tyrant. Censorship is his weapon, and he is vigilantly watching his censors, to ensure that no freedom is allowed to enter his domain. I believe he also lives in a hut with a man driven crazy by his proximity. The hut is surrounded by skulls on poles, but the skulls face inward. Remember that.
We're nearing the locus of my investigation now. You've heard the legends, now gaze into the face of comments.pl's so-called "troll detection code!"
comments.pl [slashcode.com]: Scroll down to the section under "here begins the troll detection code".
You may be wondering why it's such an ungodly piece of crap. I feel that we are seeing evidence of an ingrained unwillingness to think before coding.
The first few tests are fairly simple, based on regexps and length. They're pretty laughable, from an information theory perspective. If you don't believe me, the flood of ascii art should supply adequate evidence of what I'm talking about. Language is a complex thing, and a few simple tests are insufficient to distinguish English from ascii art, especially when the ascii artists are willing to take extreme measures to see their work posted. Regular posters do not have the patience for such chicanery.
The final test is my favourite, though. It begins under the comment ending with this charming sentence: "These ratios are _very_ conservative a comment has to be absolute shit to trip this off". An interesting claim. Considering the number of posts I've tripped this filter on without doing anything out of the ordinary, I'd say "conservative" means the same thing to Taco as it means to George W. Bush, nb. "nazi". What we are talking about here, is the postercomment compression test. (The horror! The horror!) "postercomment" is just the name of the field your comments are sent in, by the way. It isn't cool top secret slashspeak. It's just a variable name.
What this does is, it actually compresses your comment using zlib, then checks the change in size to decide if you are a troll or not! Furthermore, the code comments indicate that if you trip this test, slashcode thinks you are a "luser". Code like this makes it pretty clear that it takes one to know one, Rob!
As someone who as actually seen Rob Malda use the phrase "it won't scale" to dismiss questions about why parts of the moderation system weren't done in a more equitable fshion, I'd like to take this opportunity to laugh until I give myself a hernia.
Anyone who has studied information theory knows that the redundancy of english is estimated at about 50%. This value is fairly key in what we are seeing here, it determines a fair estimate of how effective compression of english text can be before we start to lose information. Taco's estimates were based, in his words, on "...testing out several paragraphs of text...". Doesn't sound like a particularly large sample group. What's more, it's indicative of poor software engineering practice. As is the recent bout of outages.
A few final criticisms. Firstly, there are far better, less memory intensive, and above all, less stupid methods of performing textual analysis than checking it's compression ratio. If Taco had any idea about computer science, he might have investigated a few before making a fool of himself in public like this. It's pretty clear that he's getting more and more frustrated with the situation on slashdot, and doesn't realise that if he ruled with an even hand, rather than a bitchslapping script and an army of trained thought police, the problems would not be so grave.
Secondly, I thought of a much more effective method of eliminating asci art posts, and it will never cause problems for genuine posters. What's more, it's extremely compact and doesn't even require regexps. I won't reveal it here, as I am not willing to assist in a reign of terror that I find to be reprehensible.
As further evidence of the lengths Malda and Co. may be willing to go to, you can find a commented out section that enables the deletion of posts and their descendant threads. We have no reason to believe that this will not be employed on slashdot.
Editorial notes: I don't use Perl and this is really the first time I've examined it closely. It's pretty much convinced me that I'm not missing much. I use real languages such as C and C++ and occasionaly asm to do most of my work, and I, along with 95% of the enterprise world, find Java to be the best solution for web programming. For most scripting tasks, shell script suffices. For more complicated scripting tasks, Python provides a more sensibly designed scripting environment. Additionally, as if to provide further evidence of Malda's incompetence as a programmer, I've hit the junk character post every single time I've previewed this comment, and am now forced to resort to edit it. Regretfully, I have been forced to replace the Perl fragments I was using with hyperlinks. Very unsatisfactory.
there are results froms ults/august_2001/filesystems/raid1e/README
http://lse.sourceforge.net/benchmarks/netbench/re
I quote
Hello all,
I recently starting doing some fs performance comparisons with Netbench
and the journal filesystems available in 2.4: Reiserfs, JFS, XFS, and
Ext3. I thought some of you may be interested in the results. Below
is the README from the http://lse.sourceforge.net. There is a kernprof
for each test, and I am working on the lockmeter stuff right now. Let
me
know if you have any comments.
Andrew Theurer
IBM LTC
seems that its all pretty much rocking and they turn out the same ish even tho they do things differant except riser which sucks and alaways has in my eyes (each to their own)
regards
john jones
perhaps it would be a good idea to file this story under irrelevant.
-h.
What cmdrtaco does in his spare time:
|
|
| MMMMMMMMMMMMMMMMMMMMM
|
| _______
| / \
| / O |
| (___ |
|________________\ |
| |
| |
__________ | |
/ __/ |
/ | |
___/ \_______/
| / |
| / |
| / |
| | |
| | |
| | |
(sorry it's not that great, i'm kind of new at ascii art)
|MMMMMMMMMMMMMMMMMMMMMMM
|MMMMMMMMMMMMMMMMMMMMMMM
|MMM/
\ "
) " ___ \MM/
. _./ \\.I.|:\
\ HHH
` HH : \
') .
\` ) .
\ \ : / | |
/
/ \:
/ _
_/
/
/ .
/ : . . |
/ .
| : . . :
| . : . | . .
| . :
| . : : ' . / . .
`| . : :
|\
|.\
|: | \ \:. . .
|: . | \ \ ` . .
| / \ \ :
| :
| . | \ \.:
| . . . | |
| . \\ | |
| |:.
` \ \|:| / \ :
|:
|:| \ | /
|
| : | | :
\
` |:.|
\ : |:| | | : :
\ |
\|: \. | | : . / | \
\|: \. . |
\|:'\ \ | :
`. - \ |
\ '\___.:
\\ .
ascii spork
last post
I knew the comments were getting more useless as time went on, but the majority of the first 9 posts are not only off topic, but offensive as well.
Idol Star Astronomer
I tried once ReiserFS with Mandrake 7.1. That was great and I really felt faster. :o)
But a month later when I tried to recompile my kernel I had a terrible surprise. As ReiserFS wasn't officially supported by the 2.2.x kernels I couldn't make it boot. So I had to draw back. (a month later I installed Slackware 7.1)
Well, I was a newbie then. But this bad expierence with new filesystems make me think twice before install something other than ext2, or any not-officially supported filesystem.
-=-=-=-=
I know life isn't fair, but why can't it ever be un-fair in MY favor!?
Of the three interviewees, Hans knew more about the other guys, he was better able answer what was better about the others then his, and how his was better than the others. The others came off as suits, he came off as an engineer.
www.eFax.com are spammers
last post
I really wish someone would include BFS-style attributes in an Open Source file system. Hell, I really really wish my Mac OS X installation had it. Steve Best kind of dismisses the Live Queries it as "similar to a change notification mechanism," since he admitted he wan't really familiar with it... but it's more than that.
In BFS, (although I'm probably going to butcher this explanation) the system actually retains the optimized parsed tree of the query, and monitors the modification times of the individual indices used in the search. When one of those times changes, the system re-queries just that branch of the tree rather than re-processing the entire query. Is very neat. Oh, yeah, and there's some notification going on.
Bah... I just really want a native file system with arbitrary indexed attributes so I can run SQL/LDAP-like queries against my non-Be machines too...
I find it interesting that BeFS is mentioned so prominently by each of the developers as a goal for an FS to aspire to, yet the OS itself has basically died even though it was given away for free. What does this tell us?
ReiserFS This filesystem has almost no bugs. We can tell that it has very few bugs, becuase we don't get that many bug-reports. Of course that could be due to the fact that no-one can access their e-mail client after their ReiserFS filesystem has crashed. It's also the fastest. It's really fast. Much faster than JFS...those IBM guys just want you to buy all those copies of OS/2 they've got sitting in warehouses. Well OK XFS beats it sometimes. Oh alright...quite a lot of the time. But damn...those Thinkpads are cool...especially the T23.
XFSI only had an old Indy with 32MB of RAM available to develop this on since SGI is fucked. I think it's OK, but I haven't tested it with hard-drives bigger than 500MB, since everyone who got layed off was stealing the larger drives. I wish I had a T23.
Back in my day, real hackers use Minix FS and DIDN'T COMPLAIN. So there.
XXXXXXXXXXXXXXX X
X X
X X
X X
X X
X X
X X
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
X X
X X
X X
X X
X X
X XXXXXXXXXXXXXX
* lease try to keep posts on topic.
* Try to reply to other people comments instead of starting new threads.
* Read other people's messages before posting your own to avoid simply duplicating what has already been said.
* Use a clear subject that describes what your message is about.
* Offtopic, Inflammatory, Inappropriate, Illegal, or Offensive comments might be moderated. (You can read everything, even moderated posts, by adjusting your threshold o
Now that's cool. Projects that appear to be in direct competition, but they all have great respect for each other and actually communicate with each other. And in the end, each product ends up being better.
Corporations take note.
I think the clearest choices for a production environment are XFS and JFS, as they have many years of proven reliability.
Generally, datacenter environments demand reliability over speed, and a good track record wins the day in selecting technologies.
When we implemented our database backend, we decided to go with SGI's JFS based on years of production use. So far we havn't encountered any problems!
-Marvin
For what it's worth, what RedHat says generally goes. Also, I can live with my OS being named after somebody, but I'll be damned if I'll sit back and let my filesystem be named after that guy on Mad About You. What's next, HelenHuntinetd?
AC's cheerfully ignored
but I didn't even think Reiserfs even journaled.
One of the highest-performance, most managable, most securable file systems on the market for the last 15 years has been Novell's NCP file system for Netware (I am sure it has a name but can't think of it at the moment!). The current versions (Netware 4 and 5) are supurb techncial achievements.
Now, it seems pretty clear that Novell is doomed, and when it goes Netware and NDS will evaporate. I just hope that whoever turns out the lights in Provo has the foresight and generosity to release the details of those two technologies under some sort of open source license, so that even if the products disappear the technology might be saved.
But I doubt that will happen.
sPh
Yeah, right! Have you seen the code for it? It is a horrid mess. I tried digging through it, and soon found that I would have to rewrite it from the ground up. I'm not surprised LT et al. don't want it in Linux. It's a nice FS, but it's NOT ready to go into the kernel.
hello. how are you today?
XXXXXXXXXXXXXXXX X
X X
X taco X
niggers X is X
are X a X
smelly X wanker X
X X
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
X X
X jews X slashdot
X are X blows
X theives X
X X
X XXXXXXXXXXXXXXXX
that's nice.
meow meow meow meow taco meow meow
;lkef nlxdifh srg dxnlcv hoerlt.gt
lskdlkjf lkdfjglkdjf oioiuer xfcox
[BBC world news]--
The ongoing turf war among troll warlords has claimed three more trolls today, said Brigadier General John Big Boote, head of the UN peacekeeping forces on slashdot. "The breakdown in civil order since the introduction of slascode 2.2 has left a power vacuum, and now these trolllords are moving to fill that vacuum" Gen. Boote continued. "This is worse tha Somolia in '93 -- or the Congo in 1996" Gen. Boote said. "Hell, with the exception of Egypt and Lybia, all of Africa is a festering hellhole of AIDS, cannablism and Chineese AK-47s" Boote said.
Among the causualties in the recent chaos was "Stephen King found dead". Authorities cofirmed that this promising up and coming troll was found dead in an alley last Tuesday. "The poor bastard never saw it coming" said coroner John Thomas. "He was gang-raped by a mob of hearless annonymous cowards over in a story about the DMCA" coroner Thomas said. "At first it was the usual beating the dead horse troll, but once that mob got ahold of it, they just posted it to every damn story. That's what is so pernicious in this mimetic warfare being waged across the pages of slashdot these days. Once a troll reaches a certain degree of public recognition, it becomes public property, unlike copyright" said Thomas. Another such casualty of late was the "BSD is dying" troll.
In related news, the "I WILL KICK YOUR ASS" troll has not been seen recently, and is presumed to have been killed in the bomb blast at Patrick "Shithook" Bateman's compound. The notorious trolllord, Cmdr. CYIaBCoX SUYA, leader of the anti-Beowulf-cluster troll resistance claimed responsiblity for the blast. Speaking from his headquarters under the 57th street bridge, Cmdr. SUYA expressed regret that his mentor "I WILL KICK YOUR ASS" may have succumbed in the attempt on the Beowulf cluster stronghold. "But," contiued Cmdr. SUYA, "no price is to high to pay if we can rid slashdot of the scourage of Beowulf cluster trolls."
In political news, the Annoymous Coward ASCII art front stated that it will continued carpet bombing discussion threads until Cmdr. Taco ends ethnic cleansing against trolls and repudiates the lameness filter. Cmdr. Taco was quoted as responding to the news with a long tirade about freedom of speech, but only for people who happen to run their own website. For a full account of the story, see page eight.
riser which sucks and alaways has in my eyes (each to their own)
Those numbers sound very suspicious. Even though the "default" config was used, I imagine that possibly the tester used a distro like RedHat which turns on debug mode, slowing down Reiser FS. When testing databases, you see shrinkwrap agreements which forbid you from publicly posting benchmarks for this reason.
He mentions he used:
linux 2.4.7, Samba 2.2.0, and NetBench 7.0.1
and it is unclear whether that was from an upgraded distro or not.
I think it's very strange to see Reiser saying SuSE is stable and Red Hat is not when we all remember the problems people had with it when running SuSE.
What's wrong with this ?
Even I spend some time vomitting against walls.
Or did the glorious artist mean something different ?
I think that my choice would be ReiserFS (which I've been using for about a year), because of its inclusion in the kernel.... and the way that the nice people at Mandrake Soft. have packaged it. I mean from the get-go, a fresh installation, you don't even need to have more than a 20mb /boot partition with ext2, the rest can be reiserFS.
Also, migration to ReiserFS is pretty easy... check out this link.===> An eye for an eye makes everyone blind - MG
I suppose he's Dyslexic, which is very hard for people of that nature to spell.
Most Dyslexics that I have worked with have had better maths skills and are more logical even if their grammar and spelling leave a little to be desired.
the original AC
Interesting review.
On MandrakeForum the latest news about filesystems is that JFS will be pulled from Mandrake 8.1.
There was done a test with a buildup/takedown of 100.000 files.
In the case of JFS the deleting of those files caused a hard kernel crash.
Seems there is some work to be done, despite it being a 1.0 release.
And hey, what's up with this html here?
Seems only plain text works right for me.
Well, don't worry about that. We can get you back before you leave. (Dr. Who)
I remember reading on /. about another filesystem called tux2 .
The web site is here, but I don't really see anything giving a status update.
Anyone know anything about this project?
I had some problems I thought were 2.4 related so I tried the 2.2.19 kernel with reiser. it didn't seem to be able to locate superblocks when it booted up and just froze ;-( so I had to go back to 2.4
this is the only downside I see. I know 2.2/2.4 reiser should work but it didn't for me. and I think I did all the right things when building the kernel AND userland tools.
I've not tried xfs and I did have some hangs with jfs when it reached 1.0.0 - but that was while on an IDE system with the infamous VIA chipset. I'd probably blame the chipset bugs before JFS, but it did shy me away from JFS.
reiser seems stable on 2.4 and I'm quite happy with it. give it a try.
--
"It is now safe to switch off your computer."
info
http://people.nl.linux.org/~phillips/tux2/
ext2 patch
http://people.nl.linux.org/~phillips/htree/
I'd recommend any of these, if you thought not having fsck was the only advantage check this bitch out.
In short my dialup router was using ext2, it got hard rebooted in a storm, and changed my dialup access number it was dialling... later when the bill for £1000 arrived I was enlightened as to why I should have put reiserfs on that box aswell as my workstation....#%$@!
There are no stupid questions, just lots of inquisitive idiots. taflap.
hey no those results are right you just have to put them into context
Reiser is desgned for large amounts of Dir and files which is what this tests for
and yes it achive very high marks but I have yet to see a TB on a Reiser in a live platform yet I see one every day for XFS (streaming video) and I know someone who has AIX with 3TB on it
most large files seem to be of the video nature or are database yes while MP3s are comman and thata is what Reiser did (remember mp3.com was a sponsor and came up in the boot up) but I have to say round here most peoples dir do not contain over 1000 files most have some MP3s and video, documents and the like
really I have been running XFS on a i686 for a while and have not had anything go wrong (we havent exactly pushed it tho)
what XFS and JFS need are ports to other archs and JFS seems to recogise this
remember benchmarks are written to test certain things but we commanly relie on quantum chaotic things and are unable to test for this (because they dont really have random things)
One thing that pleases me about this, is that it looks like all three of these filesystems have (or will have) metadata support. This has been a pretty serious (imho) weakness in traditional Unixes. (BTW, isn't it interesting that the platforms with the best GUIs (MacOS and OS/2 WPS) happen to depend heavily on metadata?)
I haven't used or programmed for any of these filesystems so far, though, so I'm wondering if the APIs for getting at the metadata stuff, are all the same. This is something that will be absolutely necessary before Linux app writers will be able to start using this stuff.
It looks like the teams have good attitudes toward one another. I hope they're coordinating on the APIs so it'll all be consistent.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
this looks to be a great system
I use a nettapp all the time and a filesystem that had the same sort of funtionality would be great
I heard about this in the write up for linuxconf.au but heard little about it since
regards
john jones
http://people.nl.linux.org/~phillips/tux2/
... on about a dozen systems I administer. These boxen are RH7.1+ with 2.4.3 kernels. I haven't taken the ultimate step of moving root partitions onto JFS yet, but for everything else there hasn't been one problem in 6 weeks.
Another poster commented that the source code was rough around the edges and therefore didn't merit kernel status. I disagree. The reason I don't move some of the boxes to using JFS for the root partition is precisely because it's not yet in the kernel. Thus, kernel upgrades become dicey if you can't smoothly apply the patches. Also if something goes wrong you're basically up the creek. Of course, there are backups, but that's an amazingly tedious exercise.
As for the code, well, isn't that part of the beauty of open source? At least you know what you're getting. And I've seen plenty of code - albeit not necessarily in the kernel - that looks like complete garbage but works great.
One thing that's always interested me is the technology behind filesystem designs - are these guys operating on any references, that might be worth studying? "Filesystem design 101" sort of books?
Anyone know?
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Journalling file systems depend on the correct order of data :-)
...
...
written to the disk. Simply said the journal must be updated first and then
the actual data. To achiee this , the linux kernel must support ordered
writes. I don't know if it does or not. It probably does, because
it would be really weird if not
But there is another problem. Write cache in the disk drives. Linux
sends the data in correct order, but then the disk firmware
reorders it and if you pull the plug in the middle of write,
your journalled FS has gone to hell
And the linux developer don't even care about this. I got responses
like this on LKML :
- turn off write cache if this bothers you ( this
will impact performance and is not even possible
on some disks , they keep using the cache to look
better on benchmarks )
- it is not possible to solve on IDE drives , so there
- we don't feel like dealing with this right now
stein
( email address at http://surf.to/stein )
So when do we get to see a complete round up of all journaling filesystems (I noticed ext3 was missing), including comparisons of features (including implementation features) and benchmarks on a variety of hardware configurations (SCSI, IDE, USB, RAID)?
now we need to go OSS in diesel cars
Oh, and with respect to ordered writes, ReiserFS uses the ATT/Bell/USL/Caldera patented ordered write alogrithm so it is obvious you are blowing smoke out your ass.
Niggers are the scum of the earth. Niggers like you should be sent packing back to your "roots". Adios, jigaboo.
From what I know, all the utilities for setting user and group quotas, on a Linux system, are built around the Ext2 filesystem. As a simple, educated guess, that's way RedHat prefers Ext3; because there is less work on their part in implementing quota support. I know for a fact that IBM's ReiserFS doesn't support quota and has no utilities to configure this. Reiser is planning this in the future, but know date has been released on its release. As for SGI's XFS, I have no experience, or idea, of how XFS works. I never got around reading the databooks. I heard a rumor from an AC that it supports quota, but I can only speculate on a rumor which is not worth flapping any more keystrokes over IMHO.
Slashdot didn't mention anything about Ext3 possibly because it is Ext2 with journalling; not much to talk about maybe? It may be less-featured, generic, compared with ReiserFS and XFS.
without prejudice
I don't believe it. XFS is designed totally for performance, and still it pairs with ext3!? Not even ext2!
"Video bona proboque; deteriora sequor." -- Ovid