Fatal WeaknessWith High-Capacity MMC/SD Cards?

jvaigl writes "I think I've spotted a pretty fundamental problem with those little Secure Digital and MultiMedia Cards. Since so many people are starting to use them as portable storage and file backup, I thought I'd try run this by the Slashdot readers and either get some corroboration or else have someone knowledgeable point out my misunderstanding." Read on for the trouble -- which has to do with FAT32 formatting and those high-capacity cards -- and the conclusions that jvaigl draws from his experiences.

"I am working on an embedded project where I am using Secure Digital and/or MultiMediaCards to store data. For convenience in developing and updating, I have decided to use a Windows FAT-type file system. This way I can create them, debug them, and update them on Windows development machines using USB card readers.

Since I have to keep around 25,000 files on the card, and since I'd like to minimize disk fragmentation that would result from large cluster size, I would like to use FAT32 with 512-byte clusters. This is no big deal, and certainly supported on windows - "format f: /fs:fat32 /a:512". Done and done.

The interesting thing was that I bought 4 256MB SD cards (Three from SanDisk, one from Lexar Media), and quickly killed 3 of the 4. The SanDisk cards report that track 0 is not readable when I try to format it. Snooping the SD bus shows the card inits OK, and allows writes, but returns an error whenever track 0 is read. The Lexar card's failure is a little more subtle: a format looks like it works, but subsequent chkdsks always fail. The 4th card I'm afraid to repeat this on.

SanDisk (after some weeks of running around) will replace my cards, but hasn't addressed the cause of the failure. I'm also still waiting for a reply from the Lexar 2-day-turnaround support, after 7 business days, including a reminder email.

My theory goes like this: on FAT32, in the first sector (sector 0), there's a field that gives the sector number of the File System Info Sector (FSInfoSec). Every indication I've seen puts this in sector 1, the second physical sector. This sector contains updated counters of used and free clusters on the device. The 256MB cards have about 499,000 512-byte clusters on them. These flash devices have a lifetime of 300,000 write per block, so if I copy 25,000 files to fill the card, the FSInfoSec has been updated either 25,000 times or 499,000 times (depending on when the filesystem updates the counters). If it's the former, I've just eaten up 8% of the lifetime of the card. If the latter, I've killed it before even finishing my write, since a write anywhere also causes a write to sector 1. At best case, once I update this card 12 times, I have to throw it away.

There is some Microsoft documentation that says the FSInfoSec pointer in sector 0 can be set to 0xffff to indicate it's not used. When I used dskprobe.exe from the Microsoft Windows Resource Kit to patch this pointer, Windows 2000 Professional (with a fresh Windows Update applied) blue screens so frequently when I do a dir or chkdsk on the card that I can't do anything useful before I need to cycle power on my PC.

To test my theory, I replaced the dead Lexar card, and repeated the experiment, this time formatting the card FAT16 (no FSInfoSec anymore), and the minimum supported cluster size of 2K. The bad news is of course that I lose about 26MB on the card to fragmentation since the clusters are so large. The good news is that I can write the disk full as many times as I can put up with, and it never fails.

So there are two conclusions: 1) There's a staggeringly high defect rate in the 256 MB cards (SanDisk denies this) and all my ideas about the large cards ever working well with FAT32 are groundless, or 2) even though FAT16 on a 256MB card is hugely wasteful, it's the only way to get the cards to work for very long at all."

Comments

Have you tried...

[KDan]

any other file systems on these (eg NTFS)? Or am I asking a stupid question (better to ask a stupid question and be stupid for a /. thread than to not ask it and be stupid forever!).

Daniel

Same problem with CD-RWs

[andfarm]

I haven't had any experience with this myself, but I've heard about some programs that supposedly allow one to use a CD-RW as a dynamically rewritable drive. (Like a floppy, in other words.) I would expect similar problems with such a configuration. Anybody tried this?

Re:Forgive my Ignorance...

[andfarm]

He's using Windows on the host machine. No dd or loopback there ;-)

Good idea for Linux users, though.

Re:Forgive my Ignorance...

[KDan]

That's a neat idea though. And surely those cards' drivers support some way of direct-dumping bytes onto them...

Daniel

Re:Forgive my Ignorance...

[babbage]

This is at least partly inaccurate: if you install Cygwin then you can also install DD on a Windows machine. Moreover, I know that even without Cygwin there are ways to do this with native DOS tools -- I know this because the first time I installed Linux, the RedHat documentation described how to write a disc image to the floppy drive from a DOS prompt. I want to say the DOS equivalent was actually something like "dd.exe", but it's been years now and, running "which dd" from a Cygwin shell just gives the POSIX dd that Cygwin itself installed, masking any system version that may also exist.

As for the loopback trick, that I don't know about. Someone cleverer than me might be able to do this within the Cygwin environment (or some other way?) but I have no idea where to even start...