Low-Level Format For a USB Flash Drive?

Luyseyal writes "I unwittingly bought one of these terrible flash cards at Fry's and have managed to nuke two of them, successively. I have a USB flash card reader that will read/write the current one at USB 1.0 speed, but it locks up every Ubuntu and XP machine I've come across in high-speed access mode. I have read that if I low-level format it that it could be fixed, though my current one doesn't support it. My Google-fu must be weak because I cannot seem to find a USB flash reader that specifies that it will do low-level formatting." Can anyone offer advice for resurrecting such drives?

Comments

The official utility, perhaps?

[Anonymous Coward]

http://www.sdcard.org/consumers/formatter_3/

Re:The official utility, perhaps?

[imp]

This formatter won't do anything to help you out. It will just put a new filesystem on the part. You can't 'reformat' FLASH. Bad is bad, and you lose.

Re:The official utility, perhaps?

[dgatwood]

And it's probably not the fault of the flash part anyway. I'm assuming by "hang", the original poster meant that accesses to the device stall indefinitely. Since the device stalls only when accessed at USB 2.0 speeds, that almost completely rules out the flash part as the culprit. Although flash drives do support multiple protocols, there's really no good reason for a flash controller to implement support for more than one protocol, as flash parts have to support all of them. Thus, they're going to pick the fastest protocol and use it every time, without regard to what speed the USB side of the bridge is using for communication. Therefore, the only way the USB speed should realistically trigger a failure in the flash part is if the flash part can't handle high throughput.

It's important to understand that actual cameras write data to flash parts as quickly as the flash part can take it. Thus, a failure caused by high throughput (in the absence of specific workarounds in the camera) would cause the card to be completely and totally nonfunctional in basically any real-world camera hardware. Therefore, the fact that this card is even on the market is a pretty strong indication that the problem is on the other side of the USB bridge---either the USB bridge silicon itself, the USB cable, the host silicon, or drivers.

Yes, lots of people are having trouble with those cards, but in every report, the failure was an outright controller failure, with all data lost, not problems accessing it at certain speeds. I'd be very surprised if that sort of failure were anything other than a junk flash reader.

Now I know what you're thinking. USB card readers "just work". No, they don't. I was rather miserable using dd to manually work around bugs in a USB flash reader just a few years ago. In that particular case, throwing large requests at the thing over a high speed connection caused the device to randomly return a copy of block zero instead of the expected data. Did I mention that this particular controller silicon was used for dozens of products by at least half a dozen major manufacturers for a couple of years before the flaw was discovered? Or that the bug was never fixed in firmware or silicon? :-) So yeah, a thoroughly broken flash reader would not be at all surprising.

First thing I'd do is grab yourself a new flash reader and make sure it doesn't use the same chipset as the one you have. If that doesn't help, *then* you can blame the flash part, and there probably isn't anything you can do about it other than ripping it open and making cufflinks out of it.

If you have access to the flash reader firmware, you might try using a different access mode. I'm guessing that's not an option, though.

Worst comes to worst, if it worked okay with a camera, you could try using the camera's flash controller to read/write the flash part. It will probably be dog slow, but at least you can copy the data off the card before you toss it out.

Re:Hmm...

[marcansoft]

They don't abstract the filesystem, but they abstract the hell out of the lowlevel Flash blocks and just present you with a pretty array of sectors that all work perfectly... until they don't, and then you're screwed.

Good luck, but I wouldn't hold my breath. These things are essentially disposable, and probably more trouble than it's worth reviving.

The specifics:

• Any USB reader is going to present itself as a USB mass storage device. This is the worst possible option, because USB mass storage is horridly generic, and pretty much guaranteed to abstract away any chance of lowlevel operations on particular memory card interfaces. Some readers might have undocumented lowlevel operations, but then you'll almost certainly need a tool specific for your reader or chipset. There's no generic lowlevel format for USB mass storage card readers.
• SD/MMC cards themselves have a high level controller. As far as I can tell from skimming the standard, all you get to do besides read and write is "erase" sectors in blocks, which may or may not help. There may or may not be proprietary commands to do other stuff. Your best bet for a reader is one embedded in a laptop that is not internally SD - that is, one that's a standard SD Host Controller attached to the PCI/PCIe bus. Or an embedded system like some hackable cellphone or a Wii.
• SmartMedia and xD are bare Flash chips in a pretty package, with no controller. If you can get a reader that will expose the low level flash block I/O, you might be able to get somewhere.
• CompactFlash cards are basically IDE/ATA drives with some extra modes. IDE low level format commands may apply. Your best bet here is an actual CF to IDE/ATA converter.
• I don't know about Memory Stick and other cards, but most are along the same lines as SD

dd of course

[v1]

sudo dd if=/dev/zero of=/dev/rdiskxxx bs=1024000

or whatever variation you need for your distro. The above is for mac os x. yes, rdisk is a character device I know I know, but for some reason os x io's a LOT faster o that than the block device. (double or better) No idea why. Block works too tho, whatever works for you. Just plug in the correct disk number for the xxx. Careful which device you're nuking, dd is both swift and unforgiving.

I'd also like to get slightly pedantic and point out that this is NOT a low level format. Low level format refers to laying down the address blocks, and also the data headers and trailers. All dd does is write zeros to the meat of the data block, and update its checksum. There's no such thing as a low level format for non magnetic media because flash drive blocks are electrically addressed, not physically.

FWIW, you can probably tack on "count=20" to make things go much faster. I assume all you need is the partition table completely zapped, and the first 20mb should do it fine. Without this it will wipe the entire device, which for a flash drive may take a little bit. But then again your distro or whatnot may try to find a backup copy of the boot block and partition table etc at the end of the device in which case just wipe the whole thing to avoid it "fixing it" for you.

Re:RTFM

[Donniedarkness]

He said "low-level", which I suppose is what's getting you confused. I know it's subjective, but to be considered "low-leveled" for me, it'd have to be under level 5.

Haven't done a LLF in a while, but...

[Dahan]

Here's what I used to use for a low-level format:

A>debug
g=c800:5

If you've got a fairly speedy machine, set the interleave to 1:1. Don't forget to input the list of bad blocks so the drive won't try to store data in them. There's some more info in this KB article

HTH, HAND

Windows quality file transfer, no easy test tool

[beachdog]

I am responding to your post on the chance that you are seeing a photo import bug because you use gthumb.

The 16GB flash card you link to in your Ask Slashdot question looks like the 8GB flash card I use in my digital camera.

If you are doing digital photography and using Ubuntu or a Linux, take note that the photo import utility in gthumb is broken in Ubuntu 9.10. The gthumb version is 2.10.11 and the specific thing broken is photo import of jpeg images. Photo import fails if there are .avi movie files on the flash card.

I have had a series of flash card aggravations and here is my version of the preceding AskSlashdot comments:

1. Digital cameras format flash memory cards with minor variations or they store image data with minor variations. I work around potential glitches by keeping the card in the camera and connecting the camera to the Ubuntu computer.

2. Use gthumb (note bug above) or the graphical file tool Nautilus. The top level menu item "Places" in Ubuntu starts Nautilus. Copy the files from the camera to the computer.

3. Speaking about USB flash memory, I feel they have devolved into a Windows quality file transfer device = WQFTD That means, they work using the supplied file system. The success of the same devices using Ext2 and Ext3 file systems is problematic.

4. Measuring the read and write reliability of these WQFTDs at the bit level is a difficult problem. As I mention in my journal, I have a big name DVD drive that is a WQFTD. I know it fails when reading huge 8 bit data files. But, building a tool to prove when and where it fails is beyond my available time as an evening hacker.

5. So one answer is "simplify and work around your WQFTD" without challenging it's limits.