Slashdot Mirror

http://namesys.com/r4pics/withoutpluginsJ.jpg

Surely with no pluggin, the water goes down the plugghole?

And the answer is cheap! You should view the namesys.com support page:

$25 gets you an answer to a question about anything.

BTW, I'd like to know the answer, too. Could you go pay and ask the question, then come back and tell us all the answer?

"Namesys seeks to raise the dead, and is willing to commit whatever unholy acts that requires."

Straight from the horses mouth - on their Visions of the Future page, which was directly linked from the main document. I'm sure you all read the links too, right? :)

He sure does seem to be on the defensive - maybe time to get out and get some fresh air, take the tinfoil hat off for a bit...

On reliability:
"After 3 or 4 power cycles, ReiserFS became corrupted to the point that the system would not boot up (the fsck failed and the bootup stopped there)." - http://www.redhat.com/archives/fedora-list/2004-Ju ly/msg00418.html

On code maturity:
"In contrast, ReiserFS' fsck is in its infancy..." - http://www-106.ibm.com/developerworks/linux/librar y/l-fs7/

Hans and co's attitude:
"For $25 you get an answer to any question we can answer with less than half an hour of working on it. fsck support sometimes takes more than half an hour" - http://www.namesys.com/support.html

Did I miss something or aren't those benchmarks rather old by now? The most recent is from March, this year. The rest from 2003...

# 2004.03.26 slow.c comparison against ext2, ext3
# 2003.11.20 mongo comparison against ext3
# Bonnie++ comparison of reiser4 and ext3 done at 2003.09.30.
# 2003.09.25 mongo comparison against ext3
# 2003.08.28 mongo comparison against ext3
# 2003.08.27 mongo comparison against ext3
# 2003.08.26 mongo comparison against ext3
# 2003.08.18 mongo comparison against ext3
# 2003.08.12 mongo comparison against ext3

As retrieved from here.

Surely both ext3 and Reiser 4 must have evolved since then.

This is the single most important factor when it came to deciding what filesystem to run, namely, can reiserfs 4 be upgraded to new versions easily?

Yes; as I understand it, ReiserFS 4 is designed with a plug-in architecture, so that future improvements to the filesystem can be incorporated in a non-destructive manner. You can read more about this functionality in the summary of the new features in v.4.

Sure, the text might be helpful, but I think much of the value of this article comes from the images that go with it. E.g., this one.

No, really, they use a bunch of illustrations like this in the linked article. It dumbfounds me.

To the second question, the namespace of SQL is much more complex than a filesystem hierarchy can handle. If you don't believe me, you haven't done enough SQL work.

If you want a filesystem that is really a database back-end and works as fast as one, try using ReiserFS and reading the docs, especially v4.

1. * Reiser4 is the fastest filesystem,
2. and here are the benchmarks.

1. * Reiser4 is an atomic filesystem, which means that your filesystem operations either entirely occur, or they entirely don't, and they don't corrupt due to half occuring. We do this without significant performance losses, because we invented algorithms to do it without copying the data twice.

1. * Reiser4 uses dancing trees, which obsolete the balanced tree algorithms used in databases (see farther down). This makes Reiser4 more space efficient than other filesystems because we squish small files together rather than wasting space due to block alignment like they do. It also means that Reiser4 scales better than any other filesystem. Do you want a million files in a directory, and want to create them fast? No problem.

1. * Reiser4 is based on plugins, which means that it will attract many outside contributors, and you'll be able to upgrade to their innovations without reformatting your disk. If you like to code, you'll really like plugins....

1. * Reiser4 is architected for military grade security (sponsored by DARPA). You'll find it is easy to audit the code, and that assertions guard the entrance to every function.

Anyone? ... Bueller?

But seriously, even though he mentions Reiser, he doesn't seem to consider it's future direction, which is to allow varying degrees of structure, that could include attributes, as the user sees fit. At least that's how I understand it.

...this?

Hans Reiser has some interesting ideas about the role of a modern file system. Here's a recent USENET post describing some of the immediately visible features of reiserfs v3. Some people have said that there was corruption in the past, but I think there are no longer any problems in recent 2.4 kernels. Namesys is now developing Reiser4, which appears to be more flexible (still needs time to stabilize though). If I had to put down my money on a future filesystem though, it would be ReiserFS.

Linux must find a next-generation filesystem to keep pace

What are the winds of change saying? R..E..I..S..E..R...4...

Hans Reiser has written a white paper containing his thoughts on the design of the next major version of ReiserFS.

DARPA is the primary sponsor of ReiserFS 4. There's this other little thing called "the Internet" as well...

While a lot of folk might suggest a database, structuring data is difficult, especially when you're spanning the gamut from "bits of green wire" to "Cray XMP, Serial no 700l33t4u", with and without photos, etc.

A simpler, scalable solution is to see all of this stuff as semistructured or even unstructured data - and point a search engine at it. (lots of people are heading this way - see eg ReiserFS, WinFS.

To create your data, just make web pages and get the search engine to index them. You can even make the whole process very simple by using a Wiki with built in full text search like MoinMoin, or just go for a proper search engine like lucene/

There are disadvantages. In the most basic setup you will not be able to search for "green things" because until you move from unstructured to semistructured data, there are no properties for the search engine to pick on. Even once you do add properties, you won't be able to ask "add up the cost of all my junk" which is easy in SQL. But the speed at which you can add stuff to your inventory is some compensation.

Test 002 is meaningless: the times are too short to have any statistical significance whatsoever.

Some of the comments are foolish; for example, Test 009: "Surprisingly, ReiserFS wins". Why is that surprising? Perhaps it follows from the design of ReiserFS. It's as if the benchmarker does not realise that different algorithms and design goals are involved in each filesystem.

Finally, I agree completely about the graphs, they're illegible. It's indeed amazing nobody has pointed this out. The guy desperately needs to buy and read a set of Tufte's works.

For this to be a useful benchmark, at least twice as much intelligence and effort would need to be applied...

Test 002 is meaningless: the times are too short to have any statistical significance whatsoever.

Some of the comments are foolish; for example, Test 009: "Surprisingly, ReiserFS wins". Why is that surprising? Perhaps it follows from the design of ReiserFS. It's as if the benchmarker does not realise that different algorithms and design goals are involved in each filesystem.

Finally, I agree completely about the graphs, they're illegible. It's indeed amazing nobody has pointed this out. The guy desperately needs to buy and read a set of Tufte's works.

For this to be a useful benchmark, at least twice as much intelligence and effort would need to be applied...

You can see benchmarks of it at www.namesys.com/benchmarks.html

In fact you have not one but three ext3's: data=journal, data=writeback, data=ordered (look here for details) with different performance.
So if anyone tries filesystems comparison, please benchmark all three data modes of ext3 or, at least, provide information which one was used in the test.
To add my 0.02 EUR I would also like to see comparison of linux filesystems against *BSD ufs2 on the same machine. I know that the underlying operating system adds additional variables in the equation, but if you are interested in filesystem-oriented tasks the system's performance as a whole counts, not only its filesystem. I have seen something like this done already for RAIDs but I would like to see this for ordinary single drives (like ATA and SATA). Shall I have enough time I will perform such tests end of July, when my new hardware arrives.

Fast, scalable filesystem with extensible metadata, isn't that Reiser4?

WinFS may be cool, but we have an equivilant on Linux, and it's almost ready for prime time, it's called Reiser4, check Namesys for more info on Reiser4.

First of all, learning code is not a bad thing, unless it is forced. Before you judge us, why don't you try being one of us?

On a less biggotted and egocentric note, try this: I (currently) work for about $10 an hour (I'm still in High School). Windows XP Professional (last I checked) costs about $200. That means that in order to beat XP in price, I have to give you the feature you want in less than 20 hours. For some things this is obviously impossible, but for others, it's quite likely that it'd work. And considering it's being done exactly how you want it, you might be willing to pay more.

If you don't trust me, you can require me to submit patches to the main project, where the project managers and general community can decide if I really did what you wanted, and didn't put in any back doors. As a bonus, if everyone is doing this, it can start having to happen less often, as more features are added.

Note that you don't have to learn anything except where your wallet is (I prefer cash).

Big companies do this anyway -- in fact, Holywood loves Linux because they NEED source anyway. When something fails, they have to be able to fix it fast, without going through the company. This could easily be done on a more contractual basis (rather than an in-house basis). Check out the developers of reiserfs for an example of this in action.

Here, of course, it ends up costing much more up front, but they are getting results that they cannot get anywhere else. Look at reiser4. Even the toy-ish Microsoft ripoff of the idea (WinFS) is rumored to have been dropped from Longhorn, but there is definite movement in that direction in reiser4, and people (Darpa, etc.) can pay for it to happen the way they want to.

I know that old-world people (non-teenagers) cringe at the mention of "new business model", but really, you (the business) have 4 choices here:

1) Religious blindness keeps you from realizing what features you need, insisting that the ones you have work "just fine". There's always a better way.

2) You pay someone to develop the features you need on Linux. You release the changes to the community and everyone is happy.

3) You try to learn to do it yourself. After everyone in your company has either gone to school for a few weeks (or months, or a year), or you give up, you come back and bitch about Linux.

4) You realize that the features you want aren't worth it either way. You don't buy Windows (or it lacks what you need), and you don't get the features, but you save time and money (in the short run). 20 years later, you look back and calculate how much net profit/loss you'd have gotten out of improving Linux, and (hopefully) send me an email and decide I was right.

thanks for this comment. It prompted me to look at the current Reiserfs website which has a really interesting writeup on how Reiserfs 4.0 is implemented. Its a little offbeat but very relevant.

Longhorn is going to be entirely .NET and include things like Avalon, Indigo, WinFS, and so on. I guess what I'm saying is Microsoft is actually pushing to do a revolutionary release--this will be the same kind of change going from Windows 3.1 to 95 was.

Meanwhile there are various Linux/UNIX projects such as Storage, ReiserFS, Cairo, and Keith Packard's Xserver. It will be interesting to see which set of projects finishes first.

Jedidiah.

Wonder if we'll see reiser4 in 2.6.
You can test it now, but it is very experimental.

Maybee they'll merge it with 2.7

Wonder if we'll see reiser4 in 2.6.
You can test it now, but it is very experimental.

Maybee they'll merge it with 2.7

ReiserFS is where I am hoping to see these kinds of revolutionary features emerge. Version 4 is about to be released and it already has a number of features that are hard to find in general purpose filesystems. If you really want to expand your thinking about data storage and access, read Hans Reiser's vision for the future of ReiserFS. Your ideas would definitely be implementable in his model.

I'm sure I'll be hearing from /. about how all the concepts in WinFS would be wonderful to have... just as soon as we hear something new about Reiser4. Seriously, just adding some extra metadata to a filesystem doesn't have to make it insecure; in fact, in Reiser4, they're doing it to make the filesystem *more* secure.

Now, I know about MS' track record with security just as well as the next /.'er, but let's wait until WinFS is actually released to start picking it apart, ok? Until then, it's still vaporware, and there's no guarantee that it'll get released in 2005, 2006, or any other time.

And if it just ends up being a layer on top of NTFS that lets people sort their music and vacation pictures, well, I'm not too worried about it yet. And if it turns out that it's a security risk, then you *turn it off*, or just use FAT32.

Screw NTFS support, get a petition together to have Apple replace UFS and HFS+ with ReisierFS as the primary bootable filesystem!

Slackware + boxen + AOL homepage + foe of Sensitive Claude = Complete fucking moron. If you want reiser info, why don't you look at the goddamn Reiser Homepage you fucking twit. plzfuckoffanddiethx

Or if that is too much to digest, I wrote a fairly easy to follow summary on kuro5hin.

I used to be really interested in what DB2, MySQL etc. could do until I was turned on to Hans Reiser's vision with respect to file systems. In his view, the storage layers above the file system (complex database software) can be replaced by a more intelligent filesystem that itself acts like a database. I'm currently trying out ReiserFS (a filesystem included in the Linux 2.4 kernel) which internally uses balanced trees to achieve much higher performance in large directories. ReiserFS also wastes much less space in the storage of small files.

• reiserfs is a little bit faster than xfs
• xfs gives you 2 times bigger CPU usage than reiserfs
• both are still much better than jfs
• the reliability of both xfs and reiserfs is satisfactory
• the results are still order of magnitude worse than those I get with UFS2 with softupdates on FreeBSD 5.1

I see too many companies take something that has been in an open-source product (like a Linux window manager) for years, and rewrite it, add a few features, and release it on Windows, for money. Probably the simplest example is popup blocking, even though most of them are free -- Mozilla and Firebird supported popup blocking long before even the first freeware plugin for Internet Explorer.

So the way to do this right is to do your homework. Before you even start planning on a project, make sure it is a unique project, and not something that could be a 3 kilobyte plugin to an existing project. If you want to try to sell 3 kilobyte plugins, it has to really be a killer app -- the only company I've seen do that successfully is CrossOver.

I realize that you could make an entirely new product with one killer feature, but if that feature could be implemented as a plugin to an existing product, it would take much less work for your competitor to do that, and it would annoy customers who don't want to have to use three different programs that do the same thing because each one has its own killer feature.

Next, if you think you can do it right and still make money, release your source code. Namesys has done this right, and though they make less money than they would otherwise, if you believe in good karma, they must be basking in it. A somewhat less successful way is to do what Netscape did with Mozilla and Sun did with StarOffice, which is based on OpenOffice -- create an open project, then make and sell a product based on that, possibly with a few added killer features (Netscape had spell checking before Mozilla did).

Perhaps the best way to do this is to make an open product and give it away, but charge for techsupport. I believe Mozilla has telephone techsupport now.

If you can't do that, at least be open about your development process. Release early and often. Document everything. Allow people to extend your product somehow, even if it's not with source code -- Half-life does this extremely well. Only charge for the really major updates, and charge less for an update. Listen to your community, and make them a community, not just a consumer base. You want a dialog with that community.

And remember that it's software, and that while there are real costs to developing it, the cost of distributing it (and making new copies) can be very close to 0, so charge appropriately.

Be independent. By that, I mean don't leech off of other companies -- particularly their weaknesses. For example, Symantec and many others sell products that would not exist if Windows was better, and ultimately make a worse experience for the consumer. There's a lot of websites out there that sell products designed to erase your internet history -- do NOT make a product like that. If people really cared so much about their history, they would use a product other than Microsoft Internet Explorer -- a product that deletes your history when you tell it to.

Whatever else your website is, don't just make it a brochure. Make it informative. Even redhat has, as long as I can remember, made their homepage in a way that's designed to hype rather than inform. I should be able to go to your site and click on "about" or "overview" for a description of what your product does -- if not in technical terms, at least not in marketing terms. I don't want to hear about a product that "enhances productivity" but won't say how it "enhances productivity". I should also be able to click on something like "contact" to send you an email.

An open bug database and a forum are also very useful things. Make sure they are open. Don't censor the forum based on content, and for the love of God don't censor the bug database! Good rea

You obviously don't know anything about modern Unix systems. Directories have not been sequential lists in a LONG time. Get your head out of the sand. B-trees and lots of other data structures have been used before Mr Bill started working on DOS!

You're overlooking something. Actually, quite a bit....

ext3
Resier/Resier4
JFS
XFS

Reiserfs4 advanced file system for Linux will soon be released for Linux. With transactions and rollbacks, Reiserfs4 will be an ideal platform for running MySQL. The combination is both faster and more reliable than PostgreSQL alone. Get ready to put your money where your mouth is.

is that it leaves us without a really easy to install distro for new users.

I think Mandrake fills that hole to some extent, but they're largely a repackaged RH, and I can't help wondering whether they'll be able to maintain rpm, cygwin, and all the other widely used RH products on their own. Will RH still be employing Cox?

It *is* possible to make money off free software - look at Hans Reiser, or MySQL. For that matter, Slashdot and LiveJournal use totally open source software, even if the software isn't where they make their money.

Why hasn't RH been able to do the same?

Read down the article for details on how they can now do things like mount the registry as a drive and walk it like a filesystem. Yegads!

Yeah, how innovative. I mean, heaven forbid they actually use a tree of real directories and folders on disk to store an inherently hierachical data structure.

Yes lets look at some of the innovations:

1. WinFS
Hmm, another layer on top of a file system to slow it down. MS's filesystem is already slow as a dog, How useful. We already know that that MS is 2.5 times slower as a file server then Linux with Samba 3ReiserFS 4, which is in final testing NOW beats the pants of any offering from MS in features and speed. ReiserFS 4 is sponsored by DARPA (Defense Advanced Research Projects Agency) and is architected for military grade security, ReiserFS. There is also a Gnome project that I believe is using GnomeVFS to have a similar function of a database on top of the file system for searching, etc. Ulike the offering from MS, you will be able to REMOVE it if you don't want it and it will probably be ready in a year. I don't remember the name or link for the project off the top of my head.

2. SVG.
Gnome has had SVG themes for a while now with an SVG engine in beta that will literally be out YEARS before anything from MS. The SVG engine draws all the widgets in GTK+ as SVG.
Some SVG themes
Even more SVG candy

3. .Net managed code.
Holy slowness batman! All this to have some bounds checking to prevent buffer overflows? This is already out for Linux Non-executable user stack

4. DRM
Yes, just what all the users want. Someone telling them what they can and cannot do with thier OS and what files they can and cannot run. Thanks MS for putting your customers before the big media compnanies.

Sure, we've seen the comments regarding spindle rpms, cache size/speed, and so on. What about the actual file systems used here? Doesn't ReiserFS have equivalent read access for smaller file structures as the large streaming ones?

That's why you should use ReiserFS.

If you release something under the GPL, you still own copyright, and you still have the right to relicense your code to other people under other (i.e. proprietary) licenses. (Namesys' license for ReiserFS is a good example).

That having been said, if you wish to relicense code contributed to your GPL project by others then you have to obtain their permission to do so -- like the namesys people do.

I should probably add that I am getting quite different bonnie++ results for reiser4 vs. ext3.

They are available at reiser4 benchmarking page along with
hardware specifications.

http://www.namesys.com/benchmarks.html#bonnie++.20 03.09.30

Have a look at Hans' benchmarks at namesys.com. Although he only compares Reiser4 to ext3, and may not be an objective party. But I'm surprised how well JFS performed anyway and that Reiser4 is unusually CPU-intensive.

I wonder what they believe will protect their business from poaching of these ideas?

Perhaps the fact that it's taken many very smart people a good amount of time to implement and tune the original design, even after having come up with the basic layout?

Go take a look at the ReiserFS Future Vision page -- you'll see some more interesting discussion of filesystem design, and overall direction. There are a few solid developers working full-time on the concepts discussed in the Reiser docs, and they still have enough work to keep them busy for years to come.

Google releasing information regarding the structure of their systems is a bit like John Carmack discussing the structure of his graphics engines: there's a hell of a distance between a conceptual description and a fine-tuned, tested, working implementation.

Given Google's history, I'd also imagine that they're on the lookout for up-and-coming young researchers. As such, if some grad student takes their work and extends it, they can certainly benefit.

I believe this is one of the ultimate goals of the ReiserFS project, however it doesn't seem like something on their immediate plate. Building such functionality directly into the filesystem (ie. as a plugin) does seem to be the best way, since it means all applications inherit the functionality and also consistency of the new system, and if done right then theoretically it should also be faster too.

What I'd also like to see is an improvement on the search/find/browse/whatever UI methods for finding data in a more faceted model such as the one from the article, or VFS, or the Storage project. One technique I'm particularly fond of is used by a project called FacetMap, an example of which is available at http://facetmap.com/browse.jsp.

WinFS is one of them, Storage another,reiser4 has a number of interesting concepts (such as a file being at the same time a directory containing the attributes of the file) which could be used in a similar way

It's nice that someone has made a GUI for it, but I wonder if this functionality truly belongs in a relational database rather than the filesystem level.

Nevertheless, the work done here for Gnome will make the functionality available for everyone with a decent relational database rather than just those with the next generation of filesystem handy (BeFS, WinFS, Reiser4, etc.).

www.namesys.com/whitepaper.html describes why the relational model is not the right one for large heterogeneous stores (filesystems), and describes the approach ReiserFS (a Linux filesystem used mostly in Europe) is taking instead.

Hans