Slashdot Mirror

Apple's CoreOS team includes several of the lead engineers from the ZFS project (who fled the remnants of Sun in the Schwartz melt-down), and the architect of the BeFS.

If this (potentially) verifiable information is accurate,

The bit about the architect of BeFS is verifiable if you believe his home page (I do).

"Even if I WANTED to use tree structured semantic filenames I couldn't due to Filesystem path / name limits e.g. things like the following quickly get you beyond the 128 character limit:"

Couldn't agree more, whatever happened to the database file system they were going to introduce in Longhorn in 2004, something similar to what was in BeOS since 1996 ..

Apple is in a great position to leapfrog them in the filesystem area. They hired Dominic Giampaolo, the designer of the file system for BeOS. "Practical File System Design with the Be File System".. That was a great database-like file system.

If you're interested in this, you'll probably also be interested in Practical File System Design with the Be File System (PDF), by Dominic Giampaolo, the designer of the Be file system. There's also a Slashdot review of this book.

And if Apple had bought BeOS it wouldn't be considered a failure.

Pff. You say that as if it's a bad thing. Apple did much better than they ever possibly could have done by buying BeOS. They got a far more mature, Unix-based OS, AND they snagged key BeOS developers to add cool things like Spotlight. The only real advantage that BeOS had was that it was much faster with Multimedia, something that doesn't matter in a day when it's all hardware accelerated anyway.

Ugh. There's a lot to like about C#, but I would never call it beautiful. It's got more keywords than C++, it's chock full of silly features (did you know you can use a keyword as an identifier if you prefix it with @?), and, most egregiously, Microsoft commonly pollutes the language to make up for deficiencies in their toolset.

(Oh, and Spotlight is a lot deeper than just a nice interface on top of the existing search.)

You might also be surprised by what AppleScript is capable of. It has support for very modern features, including inheritance, exceptions, closures, eval... It's not meant to be the shell - that's why OS X ships with bash, tcsh, and zsh. It's more like VB Script, except vastly nicer and easier to write. (Your criticism of its syntax is dead on, but believe it or not, AppleScript really does have a syntax, which as far as I know has only been fully documented once.)

Apple definitely has the in-house people to write an OS. The guy who wrote Mach works at Apple. The author of one of the slickest filesystems ever works there too. Oh yeah, and remember this guy? Seriously, Apple has been out-delivering Microsoft since 2001; you can't pick on them for not knowing how to write an OS.

(And, just to be snarky, Microsoft got their kernel folks from somewhere else too.)

I'm glad my OS was designed by, err, designers, incidentally. Apple has some computer science folk, but a computer scientist designing an OS is like a physicist designing a car.

I'm eagerly awaiting Longhorn, incidentally.

People who think they need to implement spotlight, HFS+ xattrs or Reiser should read Practical FileSystem Design (pdf) then just go away and use BeFS instead.
[Actually, the person who implemented HFS+ xattrs and worked in the Spotlight team was the guy who wrote Practical FileSystem Design, so I think that counts :-)]

Written by the dev who wrote BeOS

You probably meant to say "wrote the BeOS file system".

They're called Smart Folders in Mac os X Tiger.

Yes, we've covered that.

The NeXT developers had almost NOTHING to do with this.

Except build the APIs, the OS, the file system, etc., etc., etc.

The search/index technology was called VTwin before it was called Spotlight and it has been part of Mac OS since Mac OS 8.

You're confused. V-Twin is Sherlock, a slightly different technology. Spotlight is made possible by new OS features that extract, read, index, and categorize file meta-data.

The integration into the file system was done by one of BeOS's engineers (I don't remember his name) who now works for Apple.

His name is Dominic Giampaolo. He joined Apple in 2002 after stints at Google and QNX. He had designed a similar Meta-Data system for BeFS, and definitely brought understanding and experience with the technology to the NeXT team.

I hate to break it to you, but Copeland is dead and Mac OS only lives on as the phatom "Classic Environment". Most of the tech in OS X came from NeXT, with some bits pulled from the Copeland project (e.g. Blue Box)

We've already got RDBMS tech - why reinvent an inadequate version of it?

Because current RDBMS designs are unsuitable for filesystems. Relational theory still holds (just as it does for OODBs), but the physical design should be quite different if it's going to be effecient.

As I said, this has been beaten to death in the research communities. BeOS even included a DBFS design, but it went largely unused. NTFS also has all the necessary stuff in it, but Microsoft constantly removes it in final releases. ReiserFS has DBFS features, but these also go largely unused.

I think the problem is that making effective use of a DBFS requires a very different set of applications. i.e. If the applications are aware of the functionality, then they can assist the user and provide useful support. But without this form of OS and application support, the user will find that the metadata is nothing but added confusion.

As many of you probably already know, Giampaolo currently works for Apple. Giampaolo's book on file system design is available for free online for any of you who are interested in what the future could hold for Apple's filesystems.

I care about what it will provide to the user in terms of functionality and what it will provide windows developer such as myself. When you guys ship something concrete, then I will take more interest.

All that "what we want to accomplish" is as meaningless and superfluous to me as vapourware is.

Given that you work at MSFT, I am perhaps not surprised that you don't know what Spotlight is. You have to maintain your distance to avoid patent infringement entanglements.

If you are curious about one of the main developers of Spotlight, Dominic Giampaolo. Check out his website:
http://www.nobius.org/~dbg/

There is an interesting PDF of a book written by him there concerning filesystem design. You might find it of interest as it was written around the time he was with Be Inc working on the BFS.

I'm having a hard time understanding what is taking the WinFS development team so long considering that NTFS has rich support of metadata, support for alternate streams and indexing services. Do you guys really have to "reinvent" the wheel? Can't you just optimise the indexing engine, expose the API for indexing metadata/writing to alternate streams and call that WinFS?

As a developer who is not a "fanboy" of MSFT, I'm quite frankly sick of the vapourware which MSFT has been promising since the cairo project. Enough is enough. Either refactor and get it out the door or admit you have nothing and give up.

I'm sorry, but this "much more" sounds like FUD and vapourware to me. From what I've seen, all of the functionality promised by WinFS can be accomplished with today's technology.

MSFT seems like they have trouble with defining scope of their projects. They could use some better analysts to define clearly defined functional requirements, better project mangers, a better QA process (testable requirement and regression testing) and systems analysts/lead developers with a firm grasp of the realities of scope creep and the KISS principle.

BeOS , it had file metadata support years ago and worked well with it .

Did you know that Dominic Giampaolo, one of the file system gurus from Be, now works at Apple? you can even download a book he wrote about file systems from his web page.

Cool!

I guess you could blame this one on Google, who pretty much had the first desktop search out there. In fact, they'll even sell you a search appliance for your intranet.

You can probably lay the blame squarely at the feet of Dominic Giampaolo, file system guru and developer for the now defunct Be Inc, currently working in the File System and Spotlight groups at Apple. Giampaolo's work was hugely influential to a similar feature to spotlight in BeOS, which relied heavily on advanced features in Giampaolo's BFS filesystem.

In fact, Apple users probably have him to thank for the marked improvements in HFS since his hiring, including journalling and auto defragmentation.

Perhaps he realised that Dominic Giampaolo was responsible for Spotlight, meaning that Apple had taken the ideas not from Longhorn, but from BeOS?

After all, Microsoft are copying Giampaolo's work on BeOS (and BFS) with their search technology.

You might want to take a look at Apple then, they hired the guy who designed the BeOS filesystem to work on Spotlight. It's a pervasive search interface that indexes everything on your drive(s). The demo video is pretty impressive.

The book you're talking about is "Practical File System Design with the Be file system".

Here's the slashdot article on it and here's a pdf of the book direct from the author's site.

It looks interesting, but it's been on my to-read list for a while.

Oops, just checked and found Dominic's homepage. He is on the Spotlight team, and his book is now available as a PDF.

Oops, just checked and found Dominic's homepage. He is on the Spotlight team, and his book is now available as a PDF.

I'll take that bet since MS has already said that feature won't be a part of Longhorn.

Plus, BFS predates MS and Apple's implementations by years. Luckily, Apple hired the guy that designed BFS (Dominic Giampaolo). :)

"Didn't Apple hire the BeOS filesystem guys?" Yes.

From Dominic Giampaolo's homepage:

"My biggest claim to fame is writing the Be File System [...] At Apple I work in the file system and Spotlight groups. Lately though I've been spending most of my time on Spotlight (which is a lot of fun)."

Just a small info. The brain behind Spotlight is Dominic Giampaolo, the same guru that wrote the fantastic BeFS for BeOS.

I wonder why they didn't integrate it into the filesystem. That was one of the great things about BeOS, total integration. Everything, including queries was updated on-the-fly.

"Integrated" sounds cool, but it doesn't necessarily mean 'better'.
In this case, they are able to achieve the same functionality (with regard to searches) without tying it to a specific filesystem.

Also, Spotlight queries are updated on-the-fly, as content & metadata importers are triggered on every open, save, etc.

Seeing as Apple now has the guy who wrote BeOS's filesystem working for them (probably on this project),


Yup.

I'm really surprised they didn't move to a real database-type file system.

That would basically mean building a whole new filesystem. This is why Windows won't see this kind of functionality for another two or three years.

I think Apple played it safe by going with a different, but functionally equivalent approach with Spotlight, and wrapped it in a much more familiar interface.

running a sql database concurrently with your fs is a terrible idea for just all the reasons you've named. why you would try to do it is beyond me. perhaps you need to look at the problem a bit differently. be inc. did is successfully. how?

try reading practical file system design (pdf) by be's chief fs implementor, it might give you some clues.

Don't forget that Dominic Giampaolo is behind spotlight on the Apple team.