Mac OS X to Get Journaling FS

overunderunderdone writes "According to eWeek, Apple Computer is planning to introduce a new journaling file system code-named 'Elvis' with the 10.2.2 release. Supposedly it will run on top of HFS+ and will be turned off by default. Though it will cost you 10% to 15% performance penalty the article says it is more extensive than NTFS and is on par with BeOS's 64-bit journaling file system. Not surprising since it is being developed by the same person - Dominic Giampaolo." I've been super impressed by OS X having used it as my primary laptop for the last couple weeks. It really is a great unix box- and this is one of the important missing puzzle pieces.

Comments

Re:a bit offtopic, but

[jbarket]

I made the switch over last December. Love it. I was really more of a FreeBSD user, but you get the idea. It's wonderful to be able to have an attractive GUI with all too many bells and whistles to work with, but still be able to throw up a terminal window with bash and work with the real heart of things if you feel the need. Plus the ability to run XFree86 rootless on top of the GUI is nice.

Thisis 100% PURE rumor

[gsfprez]

i wish it would have been explained that way...

the writer of the eWeek article is Nick De Plumme (or something) - he's the guy from ThinkSecret....

hardly a "journalistic" website.

Re:Thisis 100% PURE rumor

[jamesoutlaw]

Remember a few months ago when several web publishers lost their Press Credentials to MacWorld for publishing Rumors? "Nick dePlume" was one of them. Matthew Rothenburg wrote an editorial entitled "Let My People Go" (or something like that) saying that these so-called "rumor" sites should be allowed the same privileges as the "real" press. Since then he's been co-writing articles every now and then with dePlume (that's a pen name, who knows that the guy's real name). I think that it's to try and lend some credibility to Think Secret and dePlume.

Of course, this is pure speculation and all. who knows. haha

Regardless of the truth, Rothenburg's "association" with Think Secret has basically caused me to lose respect for him.

Re:Thisis 100% PURE rumor

[MatthewRothenberg]

We made it very clear that we're working from sources, and that the release of this information has not been sanctioned.

Of course, I stand behind it completely, and I recommend that you check out our track record for accuracy when reporting unannounced Mac news on eWEEK: The end of Mac OS 9 booting and the rise of IBM's 64-bit PowerPC are just two recent examples of stories we nailed to the wall in advance of the official PR.

It's not my place to speak for Think Secret, but Mac stories we put on eWEEK adhere strictly to a three-source rule (and always make a point of offering Apple an opportunity to respond, not that the company often avails itself of the chance). While we'd never burn a source, we make it absolutely clear what's official writ and what's unreleased insider information. This falls into the latter camp, but that doesn't detract a bit from its authenticity.

Check back with me in a month, gsfprez, and we can talk about whether or not this story has legs. :-)

Matthew Rothenberg
Online editor
Ziff Davis Media

Re:10 - 15% ?!

[MaxVlast]

Pish. I have a Pismo (500MHz G3 PowerBook) and a 933 G4. While the G4 is a lot faster, the Pismo is a delight to use and leaves me with no complaints.

Re:10-15%

[toupsie]

Ok, so being I'm not the highest on there terminology totem pole, can somebody expain to me why journaling matters to me, and why its worth 10-15% of my system resources?

So you can have fun yanking out the power plug of your computer while its doing a write operation without the unpleasant experience on reboot. Most people (as in AOL Grandmas) don't need it but for servers, its a must. This will help beef up Mac OS X Server against Linux.

Re:10 - 15% ?!

[Lewisham]

OS X doesn't really need a fast G4, any G4 is good as long as you have a shedload of RAM. That's the real OS X bottleneck, which is easily solved by a quick trip to Crucial.com.

Re:10-15%

[aburnsio.com]

Journaling your filesystem allows you to maintain integrity through a system crash or power outage. This doesn't mean you'll have all the data in your files uncorrupted (a point often missed by many), but rather that your filesystem won't become corrupted (you won't lose your filesystem because of a crash). Modern filesystems like the more recent Linux etxfs and XFS and Windows NTFS support journaling. It's an essential part of keeping your computer crash-resistant.

There is a cost, however. Journaling filesystems are slower than non-journaling because all file metadata update operations have to be written to a transaction log. This makes journaling a poor choice for some high-volume filesystems in scientific computing or other arenas where performance is uttermost (games). In most cases, however, the performance penalty is worth the added integrity.

Note that journaling your filesystem only keeps the metadata intact, not the file data itself. You can still loose data, such as the contents of a document you were editing but had not saved. For full transactional integrity you need the cost and overhead of a transactional database (SQLServer, Postgres, DB2, Oracle, etc.).

I THINK YOU MEANT "SHITLOAD"

[Anonymous Coward]

Re:10-15%

[rgmoore]

Journaling means that if your system isn't shut down cleanly, it won't take forever to fsck your disk the next time you start up. The journal will contain all the information the system needs to get the system into a consistent state after an unclean shutdown. In addition, if the system journals all data instead of just metadata (as most journaling systems seem to do) it will prevent data loss, too.

Also bear in mind that it won't cost you 10-15% of your system resources; it will slow down disk operations by 10-15%, which is a much smaller penalty. If you aren't doing really disk intensive stuff, you probably won't even notice the slowdown. If you are doing lots of disk intensive activities, you'll probably like the fact that you're less likely to be hosed if your system crashes in the middle of one.

Re:Just another reason...

[aburnsio.com]

You get away with it because you aren't banging hard at the filesystem while yanking away at your cables. Try fulling loading your system with data copies back and forth to your iPod and an external hard drive, open your Mac box, and then on full disk load pull all cables (including internal hard drive connection). Try this a few times and you may have more luck. ;-)

Re:Existing Journaling Systems?

[StressedCoder]

This seems like an entirely new system, because the BSD type systems do not have journaling, and there is no such system on the forseable horizon.

FreeBSD provides something called softupdates, which do much to alievate the need for a journaling system. And it does this without the performance hit. When FreeBSD 5.0 comes out it will do something called snapshoting, which will bring even more stability (and background fsck) without much of a performance hit. NetBSD provides (I think) a different implementation of softupdates. OpenBSD might too, I don't know.

Which makes me very disappointed that apple chose this route. Softupdates+Snapshots solves the problem without the performance hit. BSD doesn't need no stinking journaling.

Re:10-15%

[TheMatt]

If you have ever had a Linux system running a non-journaling filesystem, you'd know. I had a box using ext2, a non-journaling fs, go down in a power failure. This baby had about 100 GB of space in ext2. It took at least an hour to get the system up because if a box crashes without journaling, it must check the drives for consistency.

In comparison, that same box using ext3, a journaling filesystem, takes a second or two to recover since it is not dependant on the size of the drives, but the (small) size of the journal (except if your drive hardware fails).

Also, journaling helps with data integrity in cases of failure as well, so you don't get files filled with garbage at the end.

If they are using anything close to BeOS's filesystem, use it. That was by far the best filesystem I have ever seen. Beautiful.

Re:Disk Space.

[Loki_1929]

"The diskspace used by the journal file in NTFS and this new filesystem can be put to much better use."

You mean like empty cluster tips?

NTFS might use a good amount of space, but you make up for allot of that just based on the smaller cluster sizes. Take a large directory (20,000+ files, 10GB+), put it on a Win2k machine with NTFS, then another with FAT32. Right click -> properties. Size on Disk says it all.

Yeah.. but..

[jbarket]

As a few people have pointed out, that's drive performance. I don't think alot of people will suffer from this. I am kind of paranoid however with my older TiBook because of the slower drive. As is, my external firewire drive rapes the internal one speed wise. Be interesting to see though.

Re:Existing Journaling Systems?

[Geekboy(Wizard)]

OpenBSD does have softupdates. Softupdates don't require journaling during regular use. snapshots only afffect boot-fsck. FreeBSD 5.0-CURRENT is getting nicer and nicer every day. =)

(now if only they fixed the bugs that broke X, it'll be "unstoppable")

Re:Just another reason...

[edremy]

Got a call from one of the help desk people yesterday: he was confused about a Disk First Aid message on a Mac here.

Yep, you guessed it... B-tree was basically spaghetti: reformat and reinstall time. I've seen it happen a few times before: the most spectacular being a crash during a defrag. Basically, nothing pointed to the right file: all the icons were there, but the info in them was basically noise.

Re:10-15%

[mickwd]

"Note that journaling your filesystem only keeps the metadata intact, not the file data itself".

That depends on which journaling filesystem you use, and, sometimes, which mode you use it in.

For example, the Linux ext3 file system supports three different journaling modes: "journal", "ordered" and "writeback".

From the "mount" man page:

journal All data is committed into the journal prior to being
written into the main file system.

ordered This is the default mode. All data is forced directly
out to the main file system prior to its metadata being
committed to the journal.

writeback Data ordering is not preserved - data may be written into
the main file system after its metadata has been commit-
ted to the journal. This is rumoured to be the highest-
throughput option. It guarantees internal file system
integrity, however it can allow old data to appear in
files after a crash and journal recovery.

"You can still loose data, such as the contents of a document you were editing but had not saved".

Well unless you've got some special sort of memory, you're going to lose everything you (or the application) haven't saved, whatever type of file system you use.

Re:Existing Journaling Systems?

[jazman_777]

OpenBSD uses softupdates (still in development, need an fsck still). Right here. The linked papers are useful reads.

Re:10 - 15% ?!

[Twirlip of the Mists]

The only place Apple currently lags is in the CPU, we all hope the Power4 will fix that....

I'm sorry to nitpick, but you're talking about the PowerPC 970. (AKA GPUL) The newly announced chip is not the same as the POWER4. They share some architectural aspects, like the instruction set, but they're not the same.

Again, sorry for nitpicking. It's just that this is a really confusing matter, what with the POWER4 chip and the POWER architecture and the Amazon architecture and the PowerPC architecture and the PowerPC chips and... so on.

Whoops! I thought it said on by default.

[BoomerSooner]

Oh well. I don't run OS 10.2 on my G3-333 I just tried it on that before installing on my G4-733 to test it (since I'm new to macs). I also wanted to see if it could live up to the hype on older hardware. I knew if it would run well on a G3-333 it would scream on my G4-733. That being said I run OS 9.2.2 on it and it's faster than hell (10.2 was creeper slow).

I also missed the part about HD performance, however my G3 has SCSI hard drives so I bet it wouldn't hurt that much. Now on IDE it's probably a whole different story.

Re:10-15%

[ahknight]

There is no penalty for games. Games only read. The performance hit is for mass file creation and renaming and so on.

Re:10-15%

[rusty0101]

Both. It depends upon the journaling method used. BFS for example only journaled system information. User data was allowed to become corrupted because a user's data file being corrupt is unlikely to bring the system down.

If you go through the documentation for the various journaling filesystems, XFS, ext3, jfs, Reiser (sp?) others, you will find that each implementation takes a different tactic when it decides what gets journaled, and what does not.

A document you are editing, is almost never journaled, unless it is happening as part of a background save process, or as part of you saving the document to disk specifically.

Then again, I have been wrong before.

-Rusty

Re:10-15%

[Doktor Memory]

Journaling filesystems are slower than non-journaling because all file metadata update operations have to be written to a transaction log.

That isn't necessarily true. At least, not always. In a RAID environment, filesystem logging can actually result in a net performance increase, because queuing up your writes in cache ram while waiting for the journal to commit increases the likelihood that the physical write operation will get spread sequentially over all of the drive heads. See Adrian Cockroft's book for more details.

Apple != Microsoft

[Corvus9]

Sing it comrade! It's not like Apple has ever released any of their OS as Open Source.

Oops, they have!

Well, we all know Apple's just "embracing and extending", they don't ever submit any of their extensions to the IETF and release that code, right?

wrong, and wrong again!

Re:Case sensitive

[Twirlip of the Mists]

I'd rather have a plain old UFS filesystem that IS case sensitive than HFS+journaling+whatever.

Then... use one. You can create and mount UFS filesystems with OS X. You can even install the OS on a UFS filesystem, but legacy software often depends on features of HFS/HFS+. So if you don't have any legacy apps, you can run an entire OS X system on UFS.

Re:a bit offtopic, but

[xtremex]

MOST Linux distros have a couple of Journaling file systems. ext3 being the most common. JFS (from IBM) and XFS (From SGI aka IRIX) are others. I use JFS on my home partition and ext3 everywhere else with naru a performance hit....actually, ext3 has been out for almost 1 years and a half with Kernel 2.4.10

FYI -- This isn't meant for you

[MidKnight]

I'm sure some people out there realize this, but I wanted to point it out: having a journaling file system isn't meant for the typical user who has their laptop to check email & surf the web. Duh! No one using a graphical interface would want to sacrifice 15% of eye-candy processing power just to have a more reliable file system.

This change is meant for people who are using OS X on *servers*... possibly even (gasp!) headless servers! I'm currently running a webserver & IMAP mail server off of an OS X box, and I never actually pull up the GUI on it (why would I need to?). But I'd love to have the added assurance of JFS on it. This is the market that Elvis is meant for.

Apple is trying to edge their way into the low-end server market, which is already over-crowded. Putting this feature into their OS, even though it's turned off by default, is a big feature difference for the XServe-purchasing crowd.

So, unless you're really nervous about losing your porn, your desktop machine doesn't need this.

--Mid

Re:a bit offtopic, but

[Anonymous Coward]

Mandrake started with reiserfs about 1.5 years ago, now defaults to Ext3. Redhat defaulted to Ext3 sometime ago. Most Linux distros have been running Jounaling FSs for 1-2 years. Soon, I expect for somebody to mention that Linux will be getting a GUI (Somebody from MS, or the media).

Re:10-15%

[rgmoore]

They're not as contradictory as you think. Almost all journaling systems journal metadata only, so they only protect the integrity of the filesystem, not the actual data. There are a few systems that actually journal everything, providing protection of the actual data as well as the filesystem integrity, but most ordinary users aren't likely to use them.

As I understand it, full data journaling is not really practical for most user applications. If you journal to the same physical device that the main filesystem is on, there's a big performance penalty because you essentially have to write everything twice. That obviously isn't efficient, and you can still lose your data if the system crashes while you're writing it to the journal. Full data journaling only makes sense if you have another device to use as the journal- say a small but blazingly fast SCSI disk or some kind of nonvolatile RAM. That's not something that most users are going to do, but it would make sense in an enterprise environment.

Umm... no.

[tgd]

Unless Apple is caching its graphics to disc before displaying them, it wouldn't make a different in your "eye-candy processing power".

Thats a 15% hit in disk performance, not system performance.

Re:Swap performance

[GMontag451]

ive never used osX but if it's like linux swap space has it's own partition.

OS X isn't like Linux in that respect. The swap file resides on the main partition.

Re:Existing Journaling Systems?

[sirket]

No there really isn't. LFS has been dead on FreeBSD for years. Soft-Updates is reliable and very fast. With the ability to do snapshots, there will be no reason to use an LFS or JFS.

-sirket

Re:Can this be rolled back into the BSDs?

[Tailhook]

What does SYSV init even have to do with Linux? The kernel doesn't care how the crap it's expected to support gets started!

If you're really so worked up about the trivia of SYSV init, go wander over to FreeBSD's CVS site and grab the etc module. Snag the rc files out'a there and smear them bodily into Gentoo or Debian (or whatever.) You're probably looking at 2-3 hours work to port that rc nonsense over.

This is O P E N source. You are free to have what you want. But, whatever you do, please stop your stoopid whining... k thx.

Nazi-esque.... wut a dink. You badly need to spend some time with real nazi folk.

We need better tech writers

[Wesley Felter]

A journaling filesystem does not "provide corporate Mac sites with a new, historical view of their data"; all it does is increase reliability.

Swap Partition HOWTO

[red5]

I found this howto through google. It shows you step by step how to set up OSX to use a different partition for swaping. I've never used it myself so I can't comment either way.

Re:Swap Partition HOWTO

[weatherbee]

I found this howto through google. It shows you step by step how to set up OSX to use a different partition for swaping. I've never used it myself so I can't comment either way.

Some of us techs were talking to an Apple rep the other day, and we asked specifically about this procedure. We were told that extensive testing by Apple had determined that there is NO advantage to having a separate swap partition for OS X.

We were also told that, despite OS X's unix base, it's still running on HFS+ (usually), so it's still a good idea to defrag every once in a while.

Re:a bit offtopic, but

[IamTheRealMike]

what other important features has OSX that Linux has not.

huh? Linux has quite a few journaling filing systems, in particular it has the rather fine ReiserFS. In the next kernel series, RFS 4 will be out, which seriously kicks ass from what I've been seeing. In particular it has very high performance, esp for small files. XFS has attributes too.

I am thinking about getting a Laptop with OSX so I was wondering how OXS compares to Linux.

A quick comparison:

• OS X easy to use. Linux, not there yet, check back in a few years
• OS X single vendor, Linux multivendor
• OS X good hardware integration, but not much range in terms of machines you can buy, Linux will run on anything, not so good hardware integration.
• OS X expensive, Linux cheap
• OS X brand name software, usually high quality, most of the important stuff made by Microsoft, most software relatively expensive, Linux no branding at all, no MS software, practically all software is free speech/beer.

But the most important of all, OS X is proprietary and has all the lockin nastyness you'd associate with Windows, Linux is free. Nuff said.

Re:Just another reason...

[Pengo]

Hmm... I disagree.

I use my tiBook as my unix dev server for our server platform. though, we deploy onto unix, I can do 99.9% of the operations on my tiBook and as I work 50% from my home office, it's a perfect fit.. I always have my dev server with me.

I have managed thus far to achieve a few crashes, mainly due to a resource leak with file handles and also a strange bug in Java. I have had my software crash litterally thousands of times in development cycles. I have had to hard-reset because of a hard lock w/the quartz engine....

Anyway, I -abuse- this laptop (which I am typing on right now). It's has been an absolute champ. I haven't had anything go corrupt, and never had an error i couldn't recover from.

Note, that I did re-partition my drive and stuck /usr/local/ mounted with a 3 gig UFS partition, but not problems with that in the last 3 weeks.

My G4 tower (466) has been a mysql and pgsql database server and a host of my personal webserver for almost 2 years now. It survived moving over-seas twice and my abusive MP3 harvesting tools to newsgroups *gasp, I know*.

Anyway, to say that a 'real user' will experience problems is utter bullshit. I don't know what's going on with your stations but we have 5 people on the team using G4's (some laptop+tower) and we never have problems.

How it works...

[emil]

Imagine that you have a library, and a librarian is filing away new books. When she is done filing them, she puts entries into the card catalog downstairs for the new books. The card catalog represents a filesystem's metadata.

Now imagine that the librarian falls out of a 2nd story window into a dumpster and is carted away before she finishes filing the books and updating the catalog. You have no idea what books were filed; you have to perform an exhaustive search of the library to ensure that the card catalog is correct, which takes a long time. This was fsck before journaling.

Servers with large amounts of disk space cannot afford extensive fsck times after a crash. It can take hours.

Now imagine that the librarian keeps a small notepad of the books that she is filing, and when she meets her sticky end, the new librarian can read the notepad, check and verify the new entries, then update the card catalog to a consistent state. We assume that the notepad is updated before the book is filed, so if we have an incomplete notepad entry, the librarian died and the entry can be disregarded. The notepad corresponds to the journal in a journaling file system.

It takes time to write a journal, so journaling filesystems will always be at least a little slower than non-journaling equivalents, design improvements aside.

Most journaling filesystems will only guard the card catalog (metadata). Some, such as VxFS and ext3, can also be made to journal the books (data), but performance goes down because so much more goes through the log.

Another feature to look for in journaling filesystems is dynamic inode creation. ext3 does not have this feature - you can only have so many card catalog entries, and when you exceed them, you can't add any more new books. XFS, for example, can create new inodes on the fly as long as you have disk space.

For Sun people, it is always a surprise to find that Sun's UFS does journaling (you don't have to buy Veritas VxFS), but you have to turn it on with an option in /etc/vfstab.

Re:more than just a pretty face

[aiabx]

Except it isn't a rainbow apple any more - it's a white one that lights up. The old rainbow logo was an emblem of the touchy-feely-UI-is-everything old Apple. Now that they are trying to win on technical merits, they want a cooler, high-tech looking logo.
-aiabx

Re:Focus Follows Mouse

[Rand Race]

It does in OS X... sorta.

Window controls focus as they are moused over, but it does not switch to that window. The main problem with focus follows mouse on a Mac (X or Classic) is that the menu bar is not attatched to the window but is, rather, at the top of the screen. They would be impossible to use if the focus switched on your way up there.

HFS+ explained in detail

[anarkhos]

http://developer.apple.com/technotes/tn/tn1150.htm l

Re:10-15%

[legis]

> Wrong, Journaling filesystems insure a sane metadata state, they do nothing to protect data.

This is true with the exception of ext3 which can journal data (data=journal mode) as well. Even in its default data=ordered mode ext3 can mostly guarantee data integrity by using transactions.

Please refer to:
http://www-106.ibm.com/developerworks/linux/l ibrar y/l-fs7/

Impatience

[gabe]

That's what the reaction will be. Because they'll be waiting for Mac OS X to finish booting. OSX fsck's the file systems itself while it boots up. first it fscks the main drive/partition and then after the boot process starts up (and you see the pretty OSX start up screen with the progress bar) it checks other drives/partitions.

So, thankfully, AOL Grandmas running OSX all over the world would not ever have to see the command line.

Re:Swap performance

[GMontag451]

It is apparently possible to put the swap on its own partition by fiddling with rc, but it is kind of pointless. The OS X swap file is dynamically allocated, so most of the time, the rest of that swap partition will be sitting there useless. It does prevent swap fragmentation, but with more than 256 MBs of RAM, this really isn't an issue.

With a journaling filesystem, it may become more of an issue. It would still be a performance vs. disk space trade off though.

Why AOL Grandmas need journaling

[dmoen]

Most people (as in AOL Grandmas) don't need it

I know of one AOL Grandma who has only one troubleshooting strategy: she power cycles her iMac whenever she has a computer problem.

Doug Moen

I would just like to congratulate Dominic...

[Anonymous Coward]

...on being the first member of the ex-SGI thursday poker crowd to get a headline on /.

Journaled File system a welcomed step

[Anonymous Coward]

I welcome the Journaled File System to OSX and certainly hope they are able to improve performance on machines. I'll gladly give up HFS+ for a good file system that has all the benefits of XFS. I'd like to see it make improvements over the much touted BFS and I hope it will be a 64-bit Journaled File System that allows attributes to be searched quickly much like BeOS and if possible clustering like SGI does with their XFS disks in CXFS.

Here is a great article by Wilfredo Sánchez on Mac OS and Unix. In it is an extensive explanation on HFS, HFS+ and UFS.

http://www.mit.edu/people/wsanchez/papers/USENIX _2 000/

-Diganta

Re:"More modern framework"?

[aburnsio.com]

What I mean by "modern" is "modern design, e.g. designed with modern features (such as journaling)", not necessarily "modern implementation". It seems that HFS+ was just an extension of HFS, not a complete rewrite for OSX, and so you're just hacking on top of a hack. It seems that integrity, reliability, and speed of the filesystem would be better if they started from scratch and designed a completely new, OSX-centered filesystem rather than extending HFS+ which is really MacOS9-legacy.

Re:10-15%

[rixstep]

With the risk that you are getting more than you bargained for, here goes.

Journaling is the process whereby a file system keeps track (keeps a journal) of its operations, along with a 'pointer' to where it is in its 'todo' list. Cutler called this 'transaction-based' on his NTFS. When an operation, such as a copy or move, is planned, the system makes an entry both for the operation as planned and for what would be needed to reverse it. The system holds an 'atomic' pointer that is only advanced when the system knows an operation is complete and flushed to disk.

NTFS has been historically impossible to screw up because of this. (NT's Registry runs the same way, another layer of journalism on top of the file system's.) If the power goes, the system will see at next boot that an operation was underway but not completed. It can then 1) undo what has been (partially) done; and 2) complete the operation properly.

Hope that helps.

Re:a bit offtopic, but

[Feral Bueller]

OsX almost no games whatsoever!

http://www.apple.com/games/features

Count: Games for OSX: 132 Loki Games: 20

Loki Games is out of business.

A quick jaunt over to the Transgaming site and a search of the database of games supported gives the following results:

5 games with a rating of 5
7 games with a rating of 4 TOTAL: 12 Games

There were only 2 games listed that the Mac doesn't have: Grand Theft Auto 3 and Half-Life.

Transgaming's ratings are for usability -- Apple doesn't have (need) usability ratings.

I can only assume there are no ratings lower than 4 because of mathematically challenged winex fanbois such as yourself.

Try again.