Cross-OS File System That Sucks Less?

An anonymous reader writes "I recently got an external hard disk with USB 2.0/Firewire/Firewire 800/eSATA to be used for backup and file exchange — my desktop runs Linux (with a Windows partition for games but no data worth saving), and the laptop is a MacBook Pro. So the question popped up: what kind of filesystem is best for this kind of situation? Is there a filesystem that works well under Linux, MacOS X, and Windows? Linux has HFS+ support but apparently doesn't support journaling and there's also an issue with the case-insensitivity of HFS+. Are we stuck with crummy VFAT forever or are there efforts underway to bring a modern filesystem (I'm thinking something like ZFS, BeFS, or XFS) to all platforms? Or are there other clever solutions like storing ISO images and loop-mounting those?"

Comments

Network it, or NTFS

[Anonymous Coward]

NTFS-3G

Re:Network it, or NTFS

[my $anity 0]

sudo apt-get install ntfs-3g

Re:Network it, or NTFS

[m95lah]

It runs in userspace, so it should never hit the kernel.
I'd be surprised if it wasn't in Ubuntu already.

Re:Network it, or NTFS

[Bishop923]

By default OS X only has ready-only NTFS support, but there is a Read-Write plugin (ntfs-3g) available as a plugin for MacFUSE:

http://code.google.com/p/macfuse/
http://www.ntfs-3g.org/

Here is a set of instructions to get it working, it mentions much older versions, but the idea is the same:

http://www.lifehack.org/articles/lifehack/how-to-r ead-and-write-ntfs-windows-partition-on-mac-os-x.h tml

Re:Network it, or NTFS

[fd0man]

Not really. While ext2/3 are, as I understand it, not *completely* cross-platform (I don't think the Mac can read ext2/3 drives, but I could be mistaken), between Windows and Linux it works just fine. Google for "ext2ifs", which I have used with high degrees of success in the past. The way that I keep my data on USB drives, I make a small FAT partition to house the ext2ifs installer for Windows systems, and the remainder of the device formatted as ext3. ext2ifs doesn't do journaling, but when the FS is attached to a Linux box it'll take advantage of ext3 features on the drive.

Re:Network it, or NTFS

[hahiss]

Actually, OS X can read and write ext2/3 with this bit of software:

http://sourceforge.net/projects/ext2fsx/

Like the Windows software you mention, it works as ext2, which means no journaling; it read and wrote to my ext3 partitions just fine when I used several years ago (IIRC).

Ext3

[Anonymous Coward]

http://www.fs-driver.org/

I just use a external drive formatted in EXT3, and for windows files i just install the Ext3 driver.

Re:Ext3

[Anonymous Coward]

and http://sourceforge.net/projects/ext2fsx/ for max os x.

ext2 supported everywhere

[markybob]

ext2 is supported everywhere and it's far better than fat32 or ntfs. for windows, http://www.fs-driver.org/ and for osx http://sourceforge.net/projects/ext2fsx/

Re:ext2 supported everywhere

[klazek]

This is the *ONLY* bit of software I have ever used that gives me a consistent kernel panic. Granted, it is a kext, they can be risky. I don't know of another solution for using ext2 or ext3 on a mac.

Re:ext2 supported everywhere

[Blakey Rat]

ext2 is supported everywhere and it's far better than fat32 or ntfs.

ext2 is better than NTFS? Seriously? Have you been eating the yellow snow or something? (I'll give you that ext2 is better than Fat32, but then again nearly everything is.)

for windows, http://www.fs-driver.org/

You have an interesting definition of the word "supported." From the FAQ:

Access rights are not maintained. All users can access all the directories and files of an Ext2 volume. If a new file or directory is created, it inherits all the permissions, the GID and the UID from the directory where it has been created. With version 1.10a of the software there is one exception to this rule: a file (but not a directory) the driver has created always has cleared "x" permissions, it inherits the "r" and the "w" permissions only. See also section "What limitations arise from not maintaining access rights?".
The driver treats files which have got a file name beginning with a dot "." character like other files, but not as hidden files.
The driver does not allow accessing special files at Ext2 volumes, the access will be always denied. (Special files are sockets, soft links, block devices, character devices and pipes.)
Neither different code pages nor UTF-8 encoded file names are supported. The driver always uses the current code page of Windows.
Alternate 8.3-DOS names are not supported (just because there is no place to store them in an Ext2 file system). This can prevent legacy DOS applications, executed by the NTVDM of Windows, from accessing some files or directories.
Currently the driver does not implement defragging support. So defragmentation applications will neither show fragmentation information nor defragment any Ext2 volume.
This software does not achieve booting a Windows operating system from an Ext2 volume.
LVM volumes are not supported, so it is not possible to access them.

and for osx http://sourceforge.net/projects/ext2fsx/

I've tried that before. It kernel panics my G5, so I uninstalled it post-haste. Maybe it's better now, but it used to suck ass.

Re:Quick answer: No

[Hatta]

There are ext2 drivers available for windows. ext2 is just ext3 without journaling. It should be a viable option.

Windows is the limitation

[halfloaded]

The biggest problem here is the lack of file system support in Windows. On a linux box, it is trivial to add support for virtually any file system type: NTFS, HFS, FAT, etc... The list goes on.

Since MacOSX is BSD based, I would be willing to bet that similar projects and support can be found (but, I Am Not A Mac Fanboy).

On Windows, you are pretty much stuck using either NTFS or FAT. FAT volumes can not be created in windows larger than 32GB. Although, you could create the partition using 3rd party tools to get beyond that limitation. I have had some success mounting ext3 partitions using Ext2 Installable File System For Windows or Ext2 File System Driver for Windows.

Personally, from my experience, VFAT or NTFS are about your only options.

Re:Windows is the limitation

[quux4]

Goddamn, will people stop saying this?!? I've formatted a 200GB hard drive as FAT with the Windows XP installer. There is no 32GB limit.

OK, here is the Real Deal:

• FAT itself can be up to 2 terabytes in size. FAT32: 2TB (theoretically 8 TB) FAT16: 4GB FAT12: 16MB.
• Large FAT partitions can be hugely wasteful of disk space, because FAT has a limited number of possible entries in the file allocation table itself, and therefore must use ever-larger cluster sizes (think extents) for file storage if you wish to have a large partition. Much disk space is lost to the many resulting partially-filled clusters. We used to call it 'slack'.
• The 32 GB limit (which MS admits is arbitrary) was imposed in the GUI partitioning tool in Windows 2000, and has persisted since. This only applies to partitions created with that formatter; W2000 and above will happily use much larger partitions.
• But the gotcha is that if your FAT filesystem is larger than about 124 gigabytes and it breaks, you will not be able to fix it. Scandisk is the repair tool for FAT filesystems, and it simply cannot process a partition larger than 124.5GB.
• If you want to create a >32GB FAT partition from within the Windows GUI, you can use fat32format.exe.

Other references: Limitations of the FAT32 File System, Raymond Chen, NTFS vs FAT.

Been there, Done that

[dfn_deux]

Having been in the exact same situation I've tried all sorts of different solutions and I'd say the best current solution is NTFS, which is out of the box natively supported on both OSX and Windows (natch) and also available R/O in the default linux kernel as well as having strong R/W support now via ntfs-3g. Of course fat32 still works just fine for this application, but it's getting a little long in the tooth as far as advanced features and modern storage needs go (c'mon what is up with those weak filesize limits)!?!? And I've had some limited success with using ext2/3 on windows and linux but found that the windows kernel driver for ext2 was not very stable in my config and the userspace tools to read/write ext3 in windows was far too kludgy for my tastes; I haven't had a chance to try ext2/3 on OSX.

Re:Been there, Done that

[Anonymous Coward]

Caution! NTFS read only is the default on Mac since 10.3.1 (Panther). You can get limited out of the box write support by using mount_ntfs from the command line, and can get much better write support from the stable-since-Feb 07 ntfs-3g project by using their FUSE/ntfs-3g installer. (This is the project you want to use on GNU/Linux, too.)

If you go from lab computer to lab computer, and the terminal is restricted on Macs, you can try writing an AppleScript wrapper for the a Bash session that runs mount_ntfs. I have not tested this limited write capability and do not know if it works (or is disabled in the default binary).

One could create ISO (image) files, but those often need special permissions to mount in GNU/Linux and a mount program in Windows; this is a difficulty in restricted environments.

From the mount_ntfs man page:

WRITING
There is limited writing ability. Limitations: file must be nonresident
and must not contain any sparces (uninitialized areas); compressed files
are also not supported.

SEE ALSO
mount(2), unmount(2), fstab(5), mount(8), mount_msdosfs(8)

CAVEATS
This utility is primarily used for read access to an NTFS volume. See
the WRITING section for details about writing to an NTFS volume.

HISTORY
The mount_ntfs utility first appeared in FreeBSD 3.0.

AUTHORS
The NTFS kernel implementation, mount_ntfs utility, and manual were writ-
ten by Semen Ustimenko .

BSD November 11, 2004 BSD

HFS+ can be case-sensitive

[Jimithing DMB]

For quite some time now (10.3 Panther I think) there has been a case-sensitive variant of HFS+. The Linux kernel has supported mounting it for some time now since I contributed a patch after realizing I couldn't access my filesystem. Unfortunately, it does not support HFS+ journaling so you have to make sure OS X gets shut down properly. Also, the last time I looked, the open source HFS+ utilities like fsck did not handle case-sensitive HFS+. I looked into fixing it but it was such a god-awful mess of code I decided I didn't trust it anyway.

On Windows you should be able to use MacDrive but you may want to check with them to make sure that case-sensitive HFS+ is supported. I only say this because for instance Alsoft's DiskWarrior product didn't support case-sensitive HFS+ until very recently. Why, I don't know since case-sensitive HFS+ simply omits the case-folding step before determining b-tree position. It's all documented in TN1150.

Re:Ext3^H2

[F-3582]

...which is perfectly fine, because Ext3 is backwards compatible and Windows wouldn't make use of the journaling feature, anyway.

Re:Doesn't work with a Macbook.

[Simon80]

See above.

The NTFS-3G driver is an open source, freely available read/write NTFS driver for Linux, FreeBSD, Mac OS X, NetBSD, and Haiku. It provides safe and fast handling of the Windows XP, Windows Server 2003, Windows 2000 and Windows Vista file systems. Most POSIX file system operations are supported, with the exception of full file ownership and access right support.

Re:Quick answer: No

[Teun]

ext2 is ext3 without the journaling.
There is no problem what so ever accessing an ext2/3 partition or disk from XP, it's just not journaling when writing.

UDF

[Ant P.]

It's not just for 12cm frisbees.

Re:Doesn't work with a Macbook.

[linhux]

NTFS is exactly what I use for my portable hard drive that I share between Windows, Linux and Mac computers. The main reason for choosing NTFS was that I need to store big virtual machine disks where files are sometimes many gigabytes in size. In Mac OS X and Linux, I use NTFS-3G to access the drive. It works, but it's very slow when transferring many and/or large files, so I would love to have a better alternative.

UFS / FFS

[nbritton]

Linux and Mac both have native UFS (a.k.a Fast File System) support, windows can also support UFS: http://ffsdrv.sourceforge.net/

Re:Why not just use ext2?

[Stalin]

Because it doesn't work. The ext2 driver for OS X is _VERY_ unstable. The last time I tried it, about five months ago, the driver caused a kernel panic. After rebooting, OS X wouldn't read the drive anymore. It was unable to seek the disk. I thought it had caused a head crash until I hooked it up to a Mac without the driver installed; that one was able to see the disk and format it. Needless to say, I removed the ext2 driver.

FAT is really the only viable option at the moment. The problem there is that you will be limited to files 2GB in size. Have a DVD image you want to access from all three platforms? Forget it. You'll either have to burn it to a DVD or use FTP, because SAMBA is limited by the same 2GB limit.

Someone else posted a response about using UDF. I'll have to look into that, but I'm not sure OS X or Windows will format a hard drive to UDF. Well, at least not with OS X's "Disk Utility" application.

Re:Moving Target - ntfs-3g

[Enderandrew]

The ntfs-3g website says you can boot from it, and run Linux of it, so apparently you can. Will there be any issues? Quite possibly.

Re:Doesn't work with a Macbook.

[Thagg]

The authors of the free NTFS claim that they've found and worked around a number of bugs in Microsoft's NTFS implementation, bugs that Microsoft has later acknowledged and still later fixed.

All experience I have had, and have heard of, shows it to be robust and bug-free.

Thad

Re:You're not very smart, are you?

[Smeagel]

I'm working from personal experience here. I did tech help for two years when I was in university, and I've had to "fix" so many peoples computers by doing a backup, complete reformat and restoration of backup because the FS was fragmented to crap. The simple fact is, that process would ONLY radically speed up a computer if the underlying file system was incredibly fragmented. Doing a complete backup/restore with a reformat in between does nothing but clean up after a crappy FS. But please, continue to feed me a load of sh*t about how great NTFS is, and how it's the most elite FS and never fragments. I know it's BS, most educated people reading this know it's BS, and you probably even know it's BS.

And I understand perfectly well that windows is built off NT now. In fact, that virus infected, featureless, resource hog Windows XP was based off of NT too. Didn't seem to do much for it.

Re:You're not very smart, are you?

[TheNetAvenger]

and how it's the most elite FS and never fragments. I know it's BS, most educated people reading this know it's BS, and you probably even know it's BS.

See here you are already showing your inexperience. It isn't that a FS won't fragment, even if the FS is good and 'tries' not to. The issue comes down to how the FS deals with fragmented files and lookup processes that are required to access additional file fragments. For example some FS are horrible at this because they have to play 'where is next chunk' or let's follow the maze around the HD.

The thing here is that NTFS fragments just like all other FS, even though it does try to anticipate file usage and tries to write to a unfragmented area.

However, the performance knock with fragmentation is in how much it 'costs' to access a fragmented file versus one that isn't, and in this regard NTFS is very good and the cost is very 'light' in comparison to several types of file systems, especially older ones like FAT.

So even if NTFS is fragmented to hell, the decrease in performance is going to be MINIMAL, that is why it was 'never' important and on modern HD and hardware is still less important for average desktop users.

Yes MS started shipping a defrag tool for NT in Win2k, but it is honestly more important and used more for high use and load files and PROPERLY ordering them on the HD as the OS and applications that use them would benefit from the placement.

So this has more to do with boot optimization and file layout and with XP and Vista with regard to prefetch and superfetch and less to do with the 'tiny' performance difference from a file that has 10 fragments and moving it so it only has 1.

Truly go look up fragmentation, this whole post is getting really old.

It is also getting old that you 'magically' would fix computers because 'you' knew what was wrong with them by 'reinstalling' a backup.

You may have solved problems, but it would have been from tweaking and restoring the installation from an OS install than ANYTHING to do with NTFS and you are either fooling yourself or your friends.

Take this advice from someone that has 'truly' worked in the tech industry with NT and NTFS since it was Alpha, not just a couple of years at college.

As you will find with most people at SlashDot, you may be your parent's computer genius, but you can easily find yourself out of your league here quickly.

Was discussed couple of years ago.

[ThePhilips]

That was discussed couple of years ago and there were no solution found. I mean FAT32 is no solution - more of a problem. Albeit being read by most if not all OSs.

Many people in past had recommended for OS specific stuff to use ZIP archives (since they are also universally available). Additionally to preserve verbatim information from *nix/MacOS volumes you can create disk image (laying on FAT32 volume). All decent OSs allow you to mount such disk images. Formats are different so it is not portable solution to preserve not portable OS-specific information about files.

Just to reiterate FAT32 is more or less only such solution.

P.S. I have looked also into ext2 support. In MacOS 10.3.x there were no official drivers (nor such drivers materialized in 10.4). Second party solution (I found only one) crashed my MacOS during installation and didn't worked in the end. For Windows there are multiple working ext2 solutions. Though not nice, yet allowing you to extract your files from ext2 volume. Not fitting for usual everyday work - but passable.