USB Flash Drive Comparison Part 2 — FAT32 Vs. NTFS

Dampeal writes "Ok, a little while back I ran a somewhat large USB Flash Drive Comparison with 21 drives compared, today I got part two of that comparison. I've taken the 8gig and 4 gig drives, nine in total, and formatted them FAT32, NTFS and ExFAT and ran all of the tests over again for a comparison of how the file systems work on the drives." Good news — after some exhaustively graphed testing scenarios, the author comes to a nice conclusion for lazy people, writing "[I]n my opinion the all around best choice is FAT32, or the default for most all USB drives out there today, it seems to give us the best average performance overall."

Comments

Same applies to SSDs?

[gEvil (beta)]

The question about filesystems has come up a few times over on the Dell Mini forums. Basically the question is which is better to use on machines with SSDs? If you're not dealing with >4GB files, several people have suggested that you're better off formatting the drive as FAT32. I'll need to take a better look at this article when I get a chance, but it seems to be suggesting the same thing.

NTFS patten?

[jgtg32a]

I thought NTFS had a patten on it which is why they used FAT32 instead.

incomplete tests

[PatentMagus]

I wish there were tests covering "typical user mishaps". Things like inopportune powerdowns and flash drive yanking. My anecdotal evidence is that I've never had issues with FAT32 but have had entire NTFS partitions become unreadable. It's just anecdote though. Now throw a truecrypt file into the mix ...

Read vs Write

[Hyppy]

It seems like the write time is the most variable out of all these. FAT32/NTFS/ExFAT scores for reading are all within a few % of each other.

I wonder what makes NTFS so slow for writes? The journaling alone reduces it that far?

Re:Read vs Write

[Hal_Porter]

I'd guess it's because NTFS sucks on a removable device. On Windows, by default, hot pluggable devices are mounted with write through caching. NTFS supports this but not very efficiently.

FAT32 and exFAT are simple enough that you can do safe access to a disk even without much write caching. FAT (and probably exFAT) actually defines a way to mark the volume as dirty in the first FAT entry at the start of each transaction where the FAT will be modified.

If someone pulls the drive right in the middle of writing some clusters may be marked as used in the FAT but not actually in use by any files. Next time you insert the drive Vista checks the volume dirt flag and asks you for if you want to run chkdsk. If you run it it will find the 'lost clusters' and convert them to files in the root directory.

Of course this scheme only ensures filesystem metadata consistency, recover user data that was being written when the drive was yanked. Mind you, NTFS journalling has the same limitation. Of course scanning for lost clusters on FAT is a painfuly slow process - you read the FAT into memory and make a bitmap of allocated clusters. Then you read every single directory and tick off the clusters used by each file. Any that are left over are lost. A journaled filesystem is much simpler - you just rollback any incomplete transations in the journal.

Of course if you have to block waiting for write transactions to complete creating the journal entries, updating bitmaps, indexes and inodes and writing data, which you would have to do on a removable device with write through caching, a journaled filesystem like NTFS has a hefty overhead. NTFS structures are much more complex so plausibly extra disk writes are necessary to keep them updated and on small writes those extra writes dominate disk time. Write through caching makes this situation even worse.

Important for me

[david.given]

I'm currently on site at a customer's office. I have one of their PCs and one of ours; legal restrictions mean I can't copy our source onto their machines or their source onto ours.

The solution to this is to put a copy of our source onto a USB stick and plug it into their machine, and then use a NTFS junction point (aka a symlink) to let their Windows-based build system see our source. This works very nicely, and I can just unplug the USB stick whenever I leave and the lawyers are happy.

However:

- I have to use NTFS. This is because the two machines are set to use different time zones, and frickin' FAT stores timestamps in the local time, which means that if I were to touch a file on one machine and the move the USB stick, the build system will go horribly wrong.

- I have 'optimise for performance' turned on; the non-Windows world calls this write caching. This boosts performance on NTFS *hugely*. I see no mention of this in the review. I now have to remember to unmount the stick on the Windows machine before pulling it, but it's worth it.

- You have to use the command line format.exe to format a removable drive as NTFS, because frickin' Windows doesn't let you do it from the GUI.

- If you turn NTFS compression on, you get a tiny bit more speed boost. But while Linux will read compressed NTFS files, it won't write them.

- You need to do something obscure with NTFS file permissions if you're going to move the stick between two Windows machines, because otherwise you'll be creating files on one machine you won't be allowed to edit on the other. Linux, of course, just ignores NTFS ACLs.

I have investigated the Windows ext2 driver, but while it does work reasonably well, it's still pretty clunky, and ext2 isn't much better than NTFS. What I'd really like is a decent Windows JFS or XFS driver --- NTFS is *so* last century.

You can partition flash drives

[Rix]

So you could just have a small partition holding the ext2 driver. Not really worth the effort for that, but it makes sense for things like truecrypt.

Filesystem for Slashdotters

[Bromskloss]

I don't care for compatibility with Windows. I use exclusively free *nixes and so does all my friends (otherwise they wouldn't be real friends, would they?). So having this richer buffet of file systems than just the two in the article, what should I choose? I have heard someone say that ext2 means less wear on the drive than ext3 (something with journaling?).

The first two things I look for in a jump drive

[Doghouse Riley]

(1) Does it have a cap I am likely to lose?

(2) Can I attach it to my keyring? (no silly lanyard clips please)

Both far more important to me in daily use than a 20% speed difference between one drive and another.
It's not like I'm running terabyte database sorts on these little guys.......

Re:Installing the ext2 driver?

[RiotingPacifist]

I use USB-pens to securly transfere my data from A -> B, gpg keys, documents, etc. Just because you use your USB-pens to spread viruses between windows pcs doesn't mean everybody does! I'd be quite interested to see ext2 vs reiserfs vs jfs vs fat.

Re:Size matters

[TemporalBeing]

FAT32 doesn't support single files over 4GB.

True, but it will happily install (at least using the Linux Kernel's driver for it) on any size drive. I did that a few years ago for a backup drive that had to be accessible from Linux as well as Windows. Windows would only allow the FS to format to 32GB; while the Linux driver let it take up the whole drive (120GB? can't quite remember). The real funny thing was that Windows was happy to work with the drive afterwards and didn't complain whatsoever about the larger than 32GB FAT32 FS on it - and I filled more than the 32GB.

However, FAT32 does come at a high overhead price. I know I lost a few GB just to the formatting alone.

P.S. I no longer have access to that drive, otherwise, I'd pull up the real size/usage for it.

Re:Great

[afidel]

If you don't need to write to the drive from XP then UDF is a possibility, it's about 3x faster than even FAT16 under Vista for small files and is supported by almost all current OS's (OSX 10.5, Linux 2.6.10+, Vista/Win7, AIX, etc) so eventually it shouldn't be a problem unless you need to use it with an embedded type device. If MS asks too much for exFAT I can see embedded players supporting UDF for large filesystems.