Slashdot Mirror
Filesystems For Removable Disks?
An anonymous reader asks: "I have recently (as in today ) acquired a 250GB external HD with both USB2 and Firewire ports, with an eye to using it to carry around all my stuff (my humungous e-mail archive, ISO images of whetever distro I'm running, music and work files - I do a lot of database work, so I often need to move 40GB+ database dumps). The thing is, In order to make proper use of it I have to be able to mount and write to it on all three platforms I use: Windows (easy, it comes formatted as FAT32), Linux (trivial mount syntax) and Mac OS X (it just works as is, since it also supports FAT32). However, I'd like to get rid of FAT32." What filesystems, aside from the FAT varieties, have decent support across the major operating systems?
"The disk comes factory-formatted (Windows doesn't allow you to format a disk this big as a single FAT32 partition), and even though I'm not running against any FAT32 limitations yet, I was wondering if there was a better filesystem to use. NTFS would be perfect (given its rock-solid transaction support - always useful on an external drive), but the Linux versions are far from reliable for large file writes and Mac OS X lacks it. ext3 isn't supported on Windows or the Mac (as far as I know).
In short, my requirements are:
- The filesystem must be read/write for Windows, Linux and Mac
- The disk must have a single partition
- There must be tools available for all three OSs to format the HD with that filesystem in case something goes wrong and I'm away from home base
Fat32 works across all, so I cannot use it.
OK.
Hello, I currently have a portable storage system working just fine with FAT32 across three different platforms. However, this is much too easy to me and I'd like to bash Windows at the same time, so I'm asking for advice on how to make my life difficult and go with some obscure filesystem which won't have many third-party tools available to alter it if something goes wrong.
Have you ever heard "If it ain't broke, don't fix it?"
If not, can you at least tell me WHY you want to break a good thing? It works FINE for you in all 3 OSes! Is your question a troll? If so, damn fine work!
With Windows, your choices are FAT16, FAT32 and NTFS. NTFS isn't amazingly portable, so you're pretty much stuck.
With Linux, your choices are ext2, ext3, jfs, reiserfs, vfat, xfs. None of these are portable at all, so you have to go with the portable filesystem.
Supported on variety of operating systems, including Windows 2000 Professional, Windows 2000 Server, Windows 2000 Advanced Server, Windows 2000 Datacenter, Windows 2003 Server, Windows XP Home, Windows XP Professional, Windows 2003 Server Web Editition and many, many others.
Dynamic Volumes are a partitioning scheme, not a file system, i.e., you can run NTFS on top of a dynamic volume instaed of the usual DOS partition table...
To all of you saying "FAT works fine", consider that the OP may be asking the question because he wants a filesystem that's faster, more reliable or perhaps more secure (encrypted?) than FAT. Just because FAT works doesn't mean it works the best. He's asking if there are any alternatives, besides what he's already using.
This space intentionally left blank.
If you wanted to share the disk only between Mac OSX and Linux, you'd have lots of different choices, since both are easily extensible and lots of people have done that extending. But between Mac and Windows, or Linux and Windows, your choices are limited. Therefore, Windows is the limiting factor.
Your other options include Paragon's Mount Everything and their Ext2fs Everywhere (which is really just a subset of `Mount Everything'.) These programs let you mount ext2/ext3 under Windows, or let you mount NTFS under Linux (I don't know how good that is -- I know that Linux has some NTFS support itself, but know it's not very mature.)
If that's not clear enough -- if you want to spend some money, spend it with Paragon and you can use ext3 or NTFS. If not, stick with fat32.
In other words, look at the problem from a tools perspective, not a filesystem perspective.
You are being MICROattacked, from various angles, in a SOFT manner.
- Maybe use FAT32, but mount .ISO files?
- Windows is the problem: what open source filesystems are there for Windows, anyway?
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
as well as using ext3 under linux.
m s-HOWTO -6.htmls oftware/ ltools.htmm acosx/ 18619
See here for general info:
http://www.ibiblio.org/mdw/HOWTO/Filesyste
And here for windows tools, but read the link... First.
http://www.it.fht-esslingen.de/~zimmerma/
And finally for OSX:
http://www.versiontracker.com/dyn/moreinfo/
Play nice and have fun!
Bah!
I too have an external (FW + USB2) drive that I like to keep quick backups on from my WinXP desktop and my MacOS X laptop. Unfortunately, while FAT32 works across platforms, it doesn't support any of the permissions, etc. that are native to each platform. One more thing to consider is that FAT32 doesn't support large files (>4GB per file).
What I've done so far is to use FAT32, but then use "disk images" on the MacOS X side to emulate the HFS+ file system (to keep permissions, etc.)
There is hope, though, as I read somwhere (can't remember where) that MacOS X has NTFS support planned in Jaguar. While, I don't think that will address the permissions issue across platforms, at least I can start making 4+ GB disk image files in MacOS X.
"Sometimes the only thing left to say is 'Oops'" -- debbers
Windows is the limiting factor. Microsoft supports FAT in all it's incarnations and NTFS and that is it. That would tend to make it the limiting factor. Windows does not support the Mac OSX filesystems, or the Linux EXT2, EXT3, Reiser, XFS, JFS, etc. filesystems. So yes, Windows is indeed the limiting factor and there is NO DOUBT about it. Period. With that out of the way, Dynamic Volume support is something you run FAT or NTFS on top of, so you have no idea what you are talking about. So sorry. Thank you for playing. It's ok to defend Microsoft, but please know what you are talking about. VFAT is what I use to play nice between different OS's.
For every annoying gentoo user, are three even more annoying anti-gentoo crybabies. Take Yosh from #Gimp for example.
The important fact that you're missing is that it IS broke.
He can't reformat the drive as a single partition. Perhaps you missed the parts where he said "the built-in Disk Manager only lets me format 40GB partitions in FAT32" and "I do a lot of database work, so I often need to move 40GB+ database dumps". That's a serious usability problem.
He's obviously using two other OSes without that artificial 40GB limit. He could just use one of them when he needs to format.
you are indeed stuck with fat32 as others have stated. It is the only filesystem that works on all 3 platforms. If you want to format the whole drive as one partition here's a hint. FDISK. I'm tired of all these people using partition magic or other pussified partitioning programs. Get a win98 boot disk or a linux disc and use fdisk. *gasp*!!!
The GeekNights podcast is going strong. Listen!
I bet there is a way to trick it into being formatted like a giant CD-RW or DVD-RW or something.
I suggest looking into that, as all oses should be able to read one of the standard CD formats...
However, I'd like to get rid of FAT32.
Why? It's the only real choice you have.
...what filesystems, besides FAT...?
UDF and iso9660, duh! That's all that's left.
-I like my women like I like my tea: green-
windows sucks. FAT32 can't have files larger than 2gb and its allocation unit is *FUCKING HUGE*. You can experiment with ext2 access programs and drivers for windows, etc. but they're not quite up to snuff yet. If you do that; create a small FAT partition containing the drivers for various OSes, and make the remaining partition ext2.
(why ext2 and not ext3? many tools for other OSes can't handle ext3)
I've used a tool on windows to read ext[23] partitions before. You can find it by doing a search for win32, linux, etc. on google. This was quite a while ago and it could read ext2 just fine. At the time there was a beta or talk of such that supported writing to ext2 from windows. Maybe by now it might be worth checking out. I think this was the program:
f s. htm
http://uranus.it.swin.edu.au/~jn/linux/explore2
MS has you cornered.
As soon as you add windows to the mix you are limited to a FAT variant or NTFS.
Think about it... its relatively easy to have support for different filesystem types on a unix machine. How easily can you do that on a windows machine without buying something ugly and installing it??
Ext2 may work- see my sig for windows ext2 driver (somewhat mature), and ext2fsx for a OS X driver. Since the OS X ext2 driver seems somewhat unstable, I would guess that fat32 is your best bet. Since I've never had any reason to work with a partition size bigger than 20GB before, I don't know what would allow you to format the disk that way (unless a mkvfatfs or suchlike under linux does the job).
I really wish people would give more support to those who are trying to develop cross-platform filesystems. If you're interested in being able to share data and swap partitions between windows and linux, please consider helping the projects in my sig. (No, I am not affiliated with either of them, I just happen to use them.)
Here's what I think the original poster was trying to get across, based on skimming the MS KB:
So, given that (and the obvious lack of a journaled FS across the three OSes), I guess instead of ranting on about Windows being the limit and proposing the use of Gentoo LiveCDs and magical incantations, we should cut our losses, accept that the guy will have to slice his files with split, and at least try to find ways of formatting very large hard disks in FAT32 under both Mac OS X and Linux - something no-one seems to have managed to provide an answer to, and which was left dangling off the post...
PartitionMagic? Any free alternatives around?
(How many people actually READ the full post text, I wonder...? Maybe it's the banner?)
I can't speak for OS X, but there is a piece of software available for Windows that will let you access an ext2 or ext3 drive - Paragon's Ext2FS Anywhere (http://www.ext2fs.com/). I've heard it works quite well, but I've never seen it tried with a removeable drive. Be careful to shut down cleanly, though, or you risk screwing the filesystem up big. It says it has just added support for files of >4GB, although I thought the ext2 limit was 2GB. Something to look into.
Any reason (other than being a retard/zealot) that it has to be directly connected and not networked?
Opportunity knocks. Karma hunts you down.
Other than that, any of the other platforms available should be able to format the partition if you so desire.
is... don't you feel like an idiot lugging a hard drive around?
You could just leave your hard drive at home and share it over the internet securely.
I have an 80GB file server and have 4 partitions on it. Hey partitions get fragmented, become bad, get crashed etc. You dont want all your data to be at risk there. Say somehow a large block gets deleted or bad sectors, the worst that can happen is the partition falls with it. And I'm sure your data is classified like emails, mp3s, games so you could divide them among the partitions easy.
Still need a more elegent solution than FAT32, put zipslack in a smallish FAT32 and boot into Linux there (or knoppix linux) and you have access to the remaining big partition formatted as XFS.
Yes I too wish someone gracious would make win32 drivers for ext2. That will help the Linux community far more than Bill Goates.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
Okay -- this one is purely for the "FWIW" file...
You could run NT's other original filesystem, HPFS. Linux has decent HPFS support available, the allocation unit is a 512b sector, and it's organized for fast searches and minimal fragmentation. It can also format up to 64GB partitions (although it still has a 2GB filesize limit).
The trick is to get HPFS support for Windows. To do this, you need to get the driver files from Windows NT v3.x (something that, admittedly I'm not sure works with Windows versions > NT 4. I don't do Windows personally, so I haven't tested it -- like I said above, FWIW). That will give you two of the three OSs supported. HPFS has been around for a while, (circa 1988), so you might be able to find something from the Intel FreeBSD world you could port to OS X.
I use HPFS for my 100MB ZIP disks (which I admittedly rarely use anymore for anything than quick archival purposes). It's not journalled, but it uses a bidirectional sector pointer system, so chaining errors are amost always fixable. The big downside is that if the filesystem is dirty, checking it can take a huge amount of time.
It's probably not a practical solution (I didn't and won't claim it is), but it's still a slightly more constructive answer than "Your stuck with FAT32" :).
Yaz.
I'm curious to know why you think NTFS sucks. Because Mister Bill owns it? Not that it really matters -- you can't access it from any non-NT OS.
Yeah, Windows sucks. I think I may have heard this mentioned once or twice before. What I don't get is how that's relevent to the discussion. We're trying to solve a technical problem here, not fantasize about some perfect world where the problem doesn't exist.
I'm probably gonna buy a copy myself. Thanks for telling me about it. Keep up the good work, and please read more carefully.
Microsoft supports FAT12, FAT16, FAT32, and NTFS. NTFS is in many
ways superior to FAT, but Linux support for NTFS is still limited
to read-only unless you want to use highly experimental write support
that is officially labelled DANGEROUS (i.e., more bleeding edge than
code that is merely EXPERIMENTAL) and guaranteed to corrupt the
filesystem. Then there's the issue of NTFS support in OS X; I'm
not aware of any options there. Also, if you need to access it
from Win9x/Me, those versions don't support NTFS either.
The ideal of course would be a journaling filesystem like Reiser.
But Windows doesn't support those, last I knew.
Now, if you're willing to forego native support by some OSes, there
are third-party apps to bring support to Windows and/or Mac for
various filesystems that aren't supported natively. Explore2fs and
the like. These are a kludge and a pain to use, and they don't
integrate with the OS much less with apps, but they do (mostly)
work. But when you go down that path you lose most of the nicer
advantages of those filesystems. The real advantage of these apps
is for multiboot systems, so that if you happen to be booted into
the other OS and somebody on usenet asks how you got foo to work,
you can pull your config file off the other filesystem and look.
It's not really practical to use this as a regular filesystem.
FAT32 may suck, but it's supported by everything. Go with it.
If you need to store things FAT32 can't store directly (symlinks,
permissions, other attributes...), wrap them up in a ZIP archive.
Cut that out, or I will ship you to Norilsk in a box.
You can format a larger than 32GB partition to FAT32 using window's built in tools. The Ask Slashdot questioner said FAT32 is OK provided he can format it using Windows. Just do the following:
Could you put together a custom hardware solution? A disk plus a small embedded Linux SBC, run whatever filesystem you want, put a fast ethernet connection on it and export the drive as a samba share?
I guess this would be a lot slower than USB or Firewire, even with gigabit ethernet.
You could use the ext2 filesystem. Mac OS X Ext2 Filesystem is a beta and Explore2efs is available for Windows.
I remember reading that a lot of card-type storage devices were write-limited. That is, they have a preset range of write operations they can achieve before the card becomes non-writable (one user apparently had an issue with this when putting a swap or log file/partition on an MMC card).
Not to knock journaled filesystems, but wouldn't they pull extra writes in order to store the journal on the removable device? I'm not 100% sure about journal writes, but I think this require at least 1 extra write to the card for the journal per operation? Not a huge thing, considering they are capable of writes in the millions, but still a little more detrimental to the life of the card - these things aren't meant to replaced often-written-to devices such as hard-disks anyhow though.
I've experience with two JFSs: NTFS (because I use a lot of Windows boxes, and I avoid FAT where possible) and XFS (because I used to work at SGI). Both were originally released without any defragmentation utility, because they were thought not to fragment. Both Microsoft and SGI eventually gave in and released the tool.
I have a 80 gig hd in an external firewire encloser. I partitioned it using partition magic (one of the greatest tools created) in FAT32 specifically so win xp and linux could read and write to it easily and so that I could plug it into pretty much any other computer and get it working with little pain. I use it for storing gigs upon gigs on mp3's and divx files and linux rpms.
My question is why is this really so bad? For my purposes and the original poster's purposes what real world benefits would either one of us see if we use a more modern and efficient file system. I'm sure there are lots of reasons dealing with speed and file permissions. I'm the only one using it so I don't care about permissions. But really? Is any other file system that requires tweaking on each OS going to be worth it when playing a movie or song? And how many people have have files over 4 gb on their home machine anyway? Maybe these are considerations for the database, I don't have much experience in that so I may be wrong.
I'm not trying to seem that I know the answer to this and I'm trying to give this guy an answer. I'm really just asking why is FAT32 so bad for these specific purposes?
then there's an opportunity for a decent OSS project: completely open, cross-platform, fast, journaled filesystem, with code tarballs for all major platforms.
Windows is again the limiting factor. Last time I checked (within the past couple months), a license for the header files and libraries required to compile a filesystem driver compatible with Windows 2000 and Windows XP cost $1000. Has this changed?
Will I retire or break 10K?
it is no more difficult to support a non-Microsoft filesystem in Windows
The www.microsoft.com web site is not responding as I type this, but it lists the price of the IFS Kit (the headers and libraries required to compile NT filesystems) as $1000. Are the "redirector services" any different? And even then, is an additional FAT## partition the best way to get the particular redirector driver onto a particular machine?
Will I retire or break 10K?
So can he get a drive and a Samba server in a reasonable-size enclosure with a connection significantly faster than 100 Mbps? Does Samba work over high-speed USB or over an IEEE 1394 serial bus?
Will I retire or break 10K?
Well, since its possible to get read-only access to NTFS partitions from Linux, and to get read-only access to ext partitions from Windows, the solution is simple:
Create two equal partitions, one of NTFS, and one of ext*. Depending on the OS being used, one partition will be readonly and one will be read-write. Therefore, you always have the ability to read and write data.
I'm not an expert, but I think the assumption has to do with active management of free space. Perhaps it would be more correct to say that defragmentation would be performed on the fly, just as memory managers defragment RAM. Probably that turned out not to be as easy as they originally though.
Just because it's USB or Firewire, doesn't make it portable.
I would recommend NTFS. It actually works better than FAT (A copy of Windows 2000 on my system works faster on NTFS than on FAT) and is interoperable between Windows NT 4, 2000, and XP. NTFS also offers better file management (Less fragmentation) added security (NTFS5 is required to have File Encryption), and Limitless files sizes.
The Linux NTFS Project is at Source-Forge.
http://linux-ntfs.sourceforge.net/
Frink: Nice try floyd, but you were designed for scrubbing, and scrubbing is what you shall do.
it works just fine. the issue you're bringing up is glibc, not the filesystem. you need to be using glibc >=2.2. its really really pathetic that glibc versions prior to that did not define size_t as a 64 bit value. the bsds have been doing that for ages.
any correctly written application compiled with a glibc >=2.2 will support large files.