Filesystems For Removable Disks?

An anonymous reader asks: "I have recently (as in today ) acquired a 250GB external HD with both USB2 and Firewire ports, with an eye to using it to carry around all my stuff (my humungous e-mail archive, ISO images of whetever distro I'm running, music and work files - I do a lot of database work, so I often need to move 40GB+ database dumps). The thing is, In order to make proper use of it I have to be able to mount and write to it on all three platforms I use: Windows (easy, it comes formatted as FAT32), Linux (trivial mount syntax) and Mac OS X (it just works as is, since it also supports FAT32). However, I'd like to get rid of FAT32." What filesystems, aside from the FAT varieties, have decent support across the major operating systems?

"The disk comes factory-formatted (Windows doesn't allow you to format a disk this big as a single FAT32 partition), and even though I'm not running against any FAT32 limitations yet, I was wondering if there was a better filesystem to use. NTFS would be perfect (given its rock-solid transaction support - always useful on an external drive), but the Linux versions are far from reliable for large file writes and Mac OS X lacks it. ext3 isn't supported on Windows or the Mac (as far as I know).

In short, my requirements are:

• The filesystem must be read/write for Windows, Linux and Mac
• The disk must have a single partition
• There must be tools available for all three OSs to format the HD with that filesystem in case something goes wrong and I'm away from home base

However, if I'm to be stuck with FAT32, I'd appreciate pointers to utilities for reformatting the HD with a single 250GB partition for Mac OS X and Windows (the built-in Disk Manager only lets me format 40GB partitions in FAT32, to force people to move to NTFS)."

Comments

hm

[Anonymous Coward]

Fat32 works across all, so I cannot use it.

OK.

Re:hm

[shaitand]

FAT32 has some severe limitations, one I'm sure that is ringing in his head is the 2gb file limitation.

not to mention other things (although for the purpose carry crap from one place to another it's probably good enough), speed, stability, security, etc. Although NTFS sucks as well, at least it's better than fat32...

The question reworded

[Anonymous Coward]

Hello, I currently have a portable storage system working just fine with FAT32 across three different platforms. However, this is much too easy to me and I'd like to bash Windows at the same time, so I'm asking for advice on how to make my life difficult and go with some obscure filesystem which won't have many third-party tools available to alter it if something goes wrong.

Have you ever heard "If it ain't broke, don't fix it?"

If not, can you at least tell me WHY you want to break a good thing? It works FINE for you in all 3 OSes! Is your question a troll? If so, damn fine work!

Re:The question reworded

[Zocalo]

Actually, he *can* reformat the drive as a single FAT32 partition, and use it on all OSes, since he's not running into a physical limitation of FAT32 but rather a deliberate design limitation. "The built-in Disk Manager" bit means he's running NT/XP and IIRC Microsoft has deliberately limited Disk Manager's FAT32 partitions in an effort to encourage people to move to the more advanced NTFS system. Despite this NT/XP is perfectly happy to access FAT32 partitions *much* bigger than 40GB, as long as they are created elsewhere.

I have successfully created FAT32 partitions in excess of 100GB and mounted them under XP using Partition Magic, Linux's *fdisk tools, and Windows 9x. We're talking a removable drive here, so it's not going to be too much hassle to partition the drive on another OS (it's the partitioning that's the problem, not formatting).

A simple process of elimination shows that FAT32 is the most portable filesystem that offers a realistic level of confidence that your OS wont trash the data. It may not be the most sophisticated system out there, but unfortunately that's the price you pay for portability at present. Plus it has the added benefit that it's accessible from a single DOS/Linux boot disk in emergencies - something that's save my ass on numerous occassions.

Windows is your limiting factor

[TheSHAD0W]

With Windows, your choices are FAT16, FAT32 and NTFS. NTFS isn't amazingly portable, so you're pretty much stuck.

Re:Windows is your limiting factor

[GiMP]

Linux supports ext*, fat*, iso9660 xfs, jfs, reiserfs, efs, isofs, ufs, udf (experimental), minix, VxFS, HPFS, HFS, HFS+ (limited?), QNX, ntfs (limited), BFS, Amiga FFS, ADFS, BeFS, and finally System V/Xenix/V7/Coherent FS (that is a long one!).
MacOS 10.x supports HFS, HFS+, UDF, ISO9660, AFS, and FAT*
Windows XP supports ISO9660, NTFS, FAT*, and UDF.

I believe that MacOS and Windows both require 3rd party software to use UDF.. but I could be wrong about that.

The solutions would be FAT*, ISO9660, and UDF. ISO9660 is read-only and I've never heard of someone using UDF on a harddrive (it is for those 'direct cds' you might have seen). FAT* sucks, but it works everywhere. It might be worth the effort to see if UDF could be used at all, but a small FAT32 partition would have to be made to accomodate the utilities for using it on the target system.

Before everyone flames the story submiter for being bias against Microsoft, the issue is that FAT really does suck and it would be great if there was something else that everyone supported.

Personally, I'd like to have a 6 gigabyte external (usb/firewire) harddrive that I could boot MacOS9 from AND share it between Linux and Windows computer. I guess I'll keep dreaming for a while :)

Re:Windows is your limiting factor

[beat.bolli]

But I guess its considered fratenizing with the enemy.

The problem is not so much the fraternizing, but the fact that the installable file system interface documentation is not available to your average open source hacker. The IFS Kit costs $899 + S&H. You just can't integrate other file systems cleanly without these docs.

In fact, there are utilities to read ext2 and ISO9660 FSs, but they are stand-alone and require you to extract the files to your native partition before they can be used.

Re:Windows is your limiting factor

[kasperd]

plus, you can't boot MacOS from ext2

Linux OTOH can boot from almost anything. However I allways boot Linux from ext2. A 31MB /boot partition is more than enough. I don't consider it any problem to have a small boot partition for each OS, even if none of the other OSes can access that partition. Some OSes might even want to (or at least be able to) boot from a partition with no filesystem at all. Linux versions prior to 2.6 could boot from a raw floppy.

NTFS

[Anonymous Coward]

Supported on variety of operating systems, including Windows 2000 Professional, Windows 2000 Server, Windows 2000 Advanced Server, Windows 2000 Datacenter, Windows 2003 Server, Windows XP Home, Windows XP Professional, Windows 2003 Server Web Editition and many, many others.

Re:Give it a rest!

[torpor]

Right, exactly. This isn't a 'switch', its a 'compare'.

I myself am very interested in the answer. I wonder if the solution may be to have an ext3 (or xfs, or jffs) shim for Win32, also?

Man, if only there were an *open*, *journaled*, *fast* and *efficient* filesystem which all 3 OS's were allowed to play well with.

Seems to me if the answer to this "Ask Slashdot" ends up being "just use FAT32", then there's an opportunity for a decent OSS project: completely open, cross-platform, fast, journaled filesystem, with code tarballs for all major platforms.

Hmmm...

fat32 is your best bet.

[dougmc]

Like it or not, fat32 is the only option that works on all the OSs in question and doesn't cost extra money.

Your other options include Paragon's Mount Everything and their Ext2fs Everywhere (which is really just a subset of `Mount Everything'.) These programs let you mount ext2/ext3 under Windows, or let you mount NTFS under Linux (I don't know how good that is -- I know that Linux has some NTFS support itself, but know it's not very mature.)

If that's not clear enough -- if you want to spend some money, spend it with Paragon and you can use ext3 or NTFS. If not, stick with fat32.

... followup ...

[torpor]

- Maybe use FAT32, but mount .ISO files?

- Windows is the problem: what open source filesystems are there for Windows, anyway?

ext2 is a valid option

[nsrbrake]

as well as using ext3 under linux.

See here for general info:
http://www.ibiblio.org/mdw/HOWTO/Filesystem s-HOWTO -6.html
And here for windows tools, but read the link... First.
http://www.it.fht-esslingen.de/~zimmerma/s oftware/ ltools.htm
And finally for OSX:
http://www.versiontracker.com/dyn/moreinfo/m acosx/ 18619

Play nice and have fun!

So don't do the formating on Windows. Duh.

[Anonymous Coward]

The important fact that you're missing is that it IS broke.

He can't reformat the drive as a single partition. Perhaps you missed the parts where he said "the built-in Disk Manager only lets me format 40GB partitions in FAT32" and "I do a lot of database work, so I often need to move 40GB+ database dumps". That's a serious usability problem.

He's obviously using two other OSes without that artificial 40GB limit. He could just use one of them when he needs to format.

Re:I think you're stuck...

[babbage]

But then, Jaguar is the current version of OSX, and it does not AFAIK have NTFS support. Maybe Pathern (10.3) will? ;-)

I think you've about nailed it though: while there are a lot of valid criteria for selecting a good filesystem (security, permissions, metadata, etc), one of them in this case has to be portability, and without the help of third party software, no version of Windows has support for anything other than FAT* or NTFS. And while NTFS isn't such a bad filesystem, incomplete support for NTFS's security mechanisms has meant that there are few if any non-Windows drivers that can both read and write the format. Maybe, as you say, Panter will have this, but that still leaves some maturing on the Linux side.

My best idea -- which maybe isn't viable to the original "anonymous reader" -- is to let the single partition requirement slip. If the drive has one or two FAT32 partitions, that can meet the portability requirement. Then a more advanced filesystem can be used for OSX/Linux interchange. I know you have more flexibility if you just need to support those two, but I'm not sure what your best options are. OSX will let you use either HFS+ (modernized version of the old MacOS filesystem) or UFS (Unix FileSystem, which AFAIK is also what the *BSDs use. UFS might be a pretty good choice then -- it should be mountable on not only OSX & Linux, but also Solaris, BSD, Irix, and other *nix variants. I don't know what support OSX has for ext2, ReiserFS etc, but if it isn't there already then the architecture of the system is such that adding third party support should in theory be easier than adding any other drivers to Windows.

How about 2 out of 3? ;)

[Yaztromo]

Okay -- this one is purely for the "FWIW" file...

You could run NT's other original filesystem, HPFS. Linux has decent HPFS support available, the allocation unit is a 512b sector, and it's organized for fast searches and minimal fragmentation. It can also format up to 64GB partitions (although it still has a 2GB filesize limit).

The trick is to get HPFS support for Windows. To do this, you need to get the driver files from Windows NT v3.x (something that, admittedly I'm not sure works with Windows versions > NT 4. I don't do Windows personally, so I haven't tested it -- like I said above, FWIW). That will give you two of the three OSs supported. HPFS has been around for a while, (circa 1988), so you might be able to find something from the Intel FreeBSD world you could port to OS X.

I use HPFS for my 100MB ZIP disks (which I admittedly rarely use anymore for anything than quick archival purposes). It's not journalled, but it uses a bidirectional sector pointer system, so chaining errors are amost always fixable. The big downside is that if the filesystem is dirty, checking it can take a huge amount of time.

It's probably not a practical solution (I didn't and won't claim it is), but it's still a slightly more constructive answer than "Your stuck with FAT32" :).

Yaz.

ext2

[Ann Coulter]

You could use the ext2 filesystem. Mac OS X Ext2 Filesystem is a beta and Explore2efs is available for Windows.

Re:repartition or you're stuck

[kasperd]

Unfortunately, same goes for ext2 in general. You can compile your kernel with large file support, but your applications and filesystem drivers need to support it as well.

It is true that applications need to support large files, however the most important applications already do. And ext2 does support files larger than 2GB. I just created a 17247252480 bytes file on my /boot filesystem (the only with ext2). This is the maximum with 1k blocks. The limit is caused by the maximum number of blocks addressable with three levels of indirection. It is slightly larger than 16GB because you can also use the direct, single, and double indirection. With three levels of indirection you get 2^4 times as large files by doubling the block size. In other words with 2kb block size you can create files as large as 256GB. With 4kb block size you run into another limitation at 2TB file size.