Slashdot Mirror
Slashdot Asks: Do You Need To Properly Eject a USB Drive Before Yanking it Out? (daringfireball.net)
In a story earlier this week, Popular Science magazine explored an age-old topic: Do people need to safely eject a USB stick before they pull it from their computer? The magazine's take on it -- which is, as soon any ongoing transfer of files is complete, it is safe to yank out the flash drive -- has unsurprisingly stirred a debate. Here's what the magazine wrote: But do you really need to eject a thumb drive the right way? Probably not. Just wait for it to finish copying your data, give it a few seconds, then yank. To be on the cautious side, be more conservative with external hard drives, especially the old ones that actually spin. That's not the official procedure, nor the most conservative approach. And in a worst-case scenario, you risk corrupting a file or -- even more unlikely -- the entire storage device. To justify its rationale, the magazine has cited a number of computer science professors. In the same story, however, a director of product marketing at SanDisk made a case for why people should probably safely eject the device. He said, "Failure to safely eject the drive may potentially damage the data due to processes happening in the system background that are unseen to the user."
John Gruber of DaringFireball (where we originally spotted the story), makes a case for why users should safely eject the device before pulling it out: This is terrible advice. It's akin to saying you probably don't need to wear a seat belt because it's unlikely anything bad will happen. Imagine a few dozen people saying they drive without a seat belt every day and nothing's ever gone wrong, so it must be OK. (The breakdown in this analogy is that with seat belts, you know instantly when you need to be wearing one. With USB drives, you might not discover for months or years that you've got a corrupt file that was only partially written to disk when you yanked the drive.)
I see a bunch of "just pull out the drive and not worry about it" Mac users on Twitter celebrating this article, and I don't get it. On the Mac you have to do something on screen when you eject a drive. Either you properly eject it before unplugging the drive -- one click in the Finder sidebar -- or you need to dismiss the alert you'll get about having removed a drive that wasn't properly ejected. Why not take the course of action that guarantees data integrity? What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
John Gruber of DaringFireball (where we originally spotted the story), makes a case for why users should safely eject the device before pulling it out: This is terrible advice. It's akin to saying you probably don't need to wear a seat belt because it's unlikely anything bad will happen. Imagine a few dozen people saying they drive without a seat belt every day and nothing's ever gone wrong, so it must be OK. (The breakdown in this analogy is that with seat belts, you know instantly when you need to be wearing one. With USB drives, you might not discover for months or years that you've got a corrupt file that was only partially written to disk when you yanked the drive.)
I see a bunch of "just pull out the drive and not worry about it" Mac users on Twitter celebrating this article, and I don't get it. On the Mac you have to do something on screen when you eject a drive. Either you properly eject it before unplugging the drive -- one click in the Finder sidebar -- or you need to dismiss the alert you'll get about having removed a drive that wasn't properly ejected. Why not take the course of action that guarantees data integrity? What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
one should always eject before pulling out
No mention of the OS or file system. Assuming Windows - there's a setting for "Quick Removal," which disables write caching and makes it so "you can disconnect the device safely", and another for "Better Performance," which doesn't and may cause grief.
"National Security is the chief cause of national insecurity." - Celine's First Law
I can't count the number of times I've tried to eject a USB drive, and Mac OS tells me it's "unable to eject the drive because it's in use."
Usually it's because Preview.app held onto some file descriptor for its stupid thumbnail of recent documents - not the list in the file menu, the one that pops up when you right-click on Preview in the Dock.
Apple used to promote the idea that the user is in charge. When I click "eject," the damn thing should eject!
Depending on the write method in use from the OS, you either have transferred the full contents of what you wanted, or you haven't. In the later, ejecting will finish the write cycle and in the first you're good to go. There is one other rare problem that can arise and that's file-system corruption. and you run the risk of it by just pulling your USB key out, although with any modern file-system, such as EXT4, BTRFS, ZFS, even NTFS, that chances of seeing corruption are rare.
I repair computers and macs for a living and have been doing so for more than 20 years.
Is it safe to yank a USB drive?
No, but the severity varies by OS.
On Linux or Mac? Somewhat, or at least safer than Windows, but I'd go ahead and unmount it anyway just to be safe.
On Windows? No. You can do it, but you're taking a gamble every time that doing so will break the partition tables. You might be able to fix that and get the data back off of it... or you might not.
Now, will I do it anyway, knowing the risk?
Absolutely, and especially on windows. I keep a drive just for moving diagnostic tools or software to windows computers while inside the OS and occasionally, windows decides it doesn't want to unmount it, so I'll yank it anyway. But that's with the understanding that everything on that drive can go 'poof' any time I do it and everything on it is backed up.
In short:
If you value what's on that drive, you shouldn't... but you should have more than the one copy on the drive of whatever you're moving around anyway.
If you don't have backups, it must not have been important.
If you haven't written anything to the USB, yes, it's completely safe. Any filesystem that's writing data just because of your passive reading of the disk is a) stupid, b) should have recovery for such, c) shouldn't be writing anything important (e.g. a last access time, maybe?)
If you have written anything to the disk, and it's synced to disk (i.e. activity light is idle) then, yes, it's safe. This may be dependent on OS and whether you can see any activity light. Modern OS should mount without write-caching for external drives unless told otherwise, and any half-decent filesystem should survive a forcible unmount in such circumstances
If the USB is still busy churning after you copied files to it, and you yank it? Yes, you're gonna lose data.
That a bunch of "nerds" can't figure this out after all the years we've had USB disks etc. (I remember starting with them in 95 OSR2 / Linux 2.0 personally?) really worries me.
Same as floppies before it. Floppy only being read-from? You can yank it. Floppy was written to but has now gone idle? You can yank it. Floppy was written to and it still pulsing / flashing? Leave it alone.
I used to write a lot of 8GB+ file system images to 16GB SanDisk devices using an Ubuntu 14.04 system. The caching it did was immense. The dd or the cp command finished in less than 60 seconds but when I did a umount command on the volume it would block for about 5 whole minutes or more while the cache emptied.
These particular USB drives have a blue activity LED on them so it wasn't hard to figure out what was going on.
In that use case yanking the USB would have been a big no no.
all the time. Have carpeting too. The case is open often and a beer is balanced somewhere nearby that it probably shouldn't. Not the same as forgoing a seat belt at all.
Depends how many places "confirmed" saved data can be cached in RAM.
Spinning hard drives have their own cache, which isn't written to the actual disk until either the cache is "full enough" or you instruct it to do so.
This data will be lost if you power the device down before the cache is flushed to disk, after the drive reported the data saved.
Flash sort of depends, an SSD for example tends to do the same thing, but there are different ways it can go about it.
Older SSDs can only write 4k blocks, it wasn't possible to write less data.
So to write for example 300 bytes, the controller has to pull in a 4k block to its RAM, edit those 300 bytes, and write out the entire 4k block again.
Pull power before this is done and your 300 bytes are gone.
"Removable" USB flash drives, the better ones at least, tend to not report data as saved before it really is, just to help avoid this problem. There is little hope for a non-technical person to know what their particular flash drive is doing however, and not even obvious to technical people either.
On top of that, your OS likely caches data to be written to any disk in RAM, completely independent of what the disk itself is doing.
If one is absolutely certain how all of these caches function, and can be completely assured all data is written in a method that doesn't rely on the disk claiming it is or isn't, then in that case it would be safe to power down the device.
For average and above average users, that will just not be true.
Even for experts, outside of a small set of cases like highly customized and tuned systems, it may be true the expert knows what is going on, but would tell you from that knowledge it wouldn't be safe and is a silly risk to take, with a high cost of data loss in exchange for a couple of seconds of time saved.
Hell, even I am still in the habit of issuing three 'sync' commands in a row before an unmount command, and that's despite the knowledge the unmount command will do a 'sync' call of its own!
But as this advice is for average or below people, as bad as it is, isn't the worst things commonly done.
Average or below people rarely even make backups, which has a far higher cost when (never IF) a drive fails. Corrupting a couple files on a single USB drive compared to not having any backups is like complaining your car only has 5 airbags instead of 6 while driving it off of a cliff...
My thoughts are that this is bad design and will hopefully be fixed in some future revision.
Ever think of testing this for yourself?
It's easy. Create a half a dozen or more files of random numbers or use existing large files. I created six files of a million random integers in Python. End result, six files of about 6.9MB each. Create an md5 checksum file when you make them.
Copy them to a USB stick, and then yank the stick right when the light stops blinking. Plug the USB stick back in. Watch and learn. Easily reproducible phenomenon are:
* Not all files even appear when the stick is plugged back in
* Some of the files might appear, but give I/O errors and won't even be complete
* A Few might pass the checksum integrity test
I'm on Linux Mint, but I have seen the results on other OSes as well. The OS caches the data to be written sometime, presumably to speed up file operations. There's a reason why eject exists.
"What lies behind us, and what lies before us are tiny matters compared to what lies within us." Ralph Waldo Emerson
As you may know, risk = damage x chance.
Only if the damage is monetary and you are in the insurance business. Otherwise risk and chance are two orthogonal axis.
E.g. the typical prisoner dilemma: you are sentenced to life time prison. You get offered to either stay and accept the sentence or walk through two doors: one door gives you freedom, the other door execution.
The chance is 50% for either freedom or death. The risk is death, and not an abstract number multiplied from two other numbers.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
First of all: an annoying dialog pops up and an annoying beep comes, you have to close the dialog, probably even with the mouse because ESC or ENTER does not work.
Secondly, you probably have forgotten that a file is open in Excel or whatever and it is half modified but not fully saved.
Why not eject it and be safe, respectively be not annoyed?
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
You're kidding, right?
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Safe eject/removal is the only way to guarantee that file buffers are flushed and metadata is in a consistent state.
If you're OK with losing files, or having chunks of your free space marked as occupied, then don't use safe eject.
Err on the side of caution. Etiher:
You don't need to because the system isn't caching writes, in which case the result is instant
The result is slow, which means you need to do the 'eject' procedure.
Practically speaking, modern OSes have gotten the point, a small usb attached storage device is at high risk to be yanked out, so they disable write caching by default.
XML is like violence. If it doesn't solve the problem, use more.
Well, here it's a bit of an evolved situation, and one that evolved with the general state of storage.
Once upon a time, OSes didn't bother to cache writes, they just wrote to disk and were done. Computer could just turn off at any time (parking hard drives manually aside). Memory was too scarce.
Then memory was more plentiful and OSes became more sophisticated, disk IO then presumed caching, and shutting off your system without proper shutdown was exceedingly high risk. Removable media came into fashion with this sort of accepted behavior, so they had to be very verbose about users doing it wrong.
Fast forward a bit and the industry realizes that they can't expect clean ejects/shutdowns, so they make better filesystems, maybe losing data but at least not be corrupted and disable write cache on usb disks by default, causing slower apparent behavior, but resilient to yanking out.
It's always a good idea to err on the side of caution, but the current OS designs around ejecting media are less critical because they've decided it is a lost cause.
XML is like violence. If it doesn't solve the problem, use more.
The physical damage occurs when you go "Oh, fuck!" and try to ram it back in before the computer notices.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Windows is funny like that.
[($)]
I don't have a clue, but I'm sure Wikipedia does!
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
...The magazine's take on it -- which is, as soon any ongoing transfer of files is complete, it is safe to yank out the flash drive ...
The problem is that you do not know when the transfer is complete. The UI's representation of it shows when the UI is done with the transfer, but that does not necessarily mean that the OS is finished with the transfer. So Popular Science is correct in that you have to wait until the transfer is complete, they are just incorrect about what tell-tale to use to determine that status.
Me: *Tries to eject flash drive.*
Windows: "Cannot eject drive as files are still being used."
Me: *Closes Explorer window that was open to a folder on C and not any folder on the flash drive, and tries to eject again.*
Windows: "Drive can be safely removed."
Me: "WTF?"
Windows: "Windows is applying Updates. Do not turn off your computer."
The FAT is a table (bit map of sectors in use) which says which parts of the disk are in use, as well as where the files and their parts are stored. There are also flags saying if the file system (partition) is in use, and if it has been modified - and probably other things (I was last involved in this in about 1990).
When you eject, the system stops making the changes you requested (reads and writes), and updates these tables to a consistent state (well maybe not all that consistent, if its Windows, and completely inconsistent if DOS 4.0), and then clears the dirty (modified) and Open (in use) flags for each partition.
If the eject procedure fails to complete, and the dirty flag is set when you next try to use it, the OS will try to clean up the mess - it may succeed, or trash the data, and report success, or it may fail and tell you the system is dead.
With USB sticks, the data you see is not the real data. Because of the need to hide the bad blocks, and hideously long write time, the OS sees virtual data, and operates on virtual data. The real data is different. A single bit change (eg clear the dirty bit) could end up requiring three or four copy and write operations, operating on massive amounts of logically unrelated data, unknown to the OS. If you trash those, the OS for the CPU in the USB stick will be utterly bamboozled, and you have no interface to tell it to do a factory reset, so you USB stick is dead.
(Actually, the underlying interface to the USB stick is SCSI, and there probably is a SCSI command that would unbrick it, but you can be damned sure that the manufacturer won't tell you what it is, because if he did, you would not go out and buy another USB stick, would you?)
Sent from my ASR33 using ASCII
I always eject it, but sometimes it says its in use and won't let me. But I want to pull it out NOW, so too bad its getting yanked out.
I also use a program called Hotswap! that ejects alot better than the windows one, AND lets me do sata drives and things that normally don't show up in the windows eject.
You mean making them work like CD-Rom drives?
You mean where the button ignores you and does not eject the disk. Or starts to and teases you and then closes again? Like that?
Copy them to a USB stick, and then yank the stick right when the light stops blinking.
Two functionalities are key: Sync and unmount.
- Sync forces the filesystem to complete all in-RAM updates to blocks, write the blocks to the driver, and forces the driver to flush all pending writes to the backing store. Issuing a "sync" kicks off activity that insures the data for the current state is all on the backing store when the activity is completed. (And manually issuing two of them insures the first one is completed before the second command returns.)
- Unmount also tells the filesystem to get all files closed, go to such a state (including updating a flag saying it isn't mounted, if that is part of the filesystem) and write this all to the store.
Unixes and linux are usually configured to issue syncs periodically. So if you write new stuff to your filesystem, and your filesystem/driver combination isn't one that tries to always force the data to the store right away, you'll see the activitly light go out when there's still important stuff in a last few buffers which AREN'T written yet. (For a new file, on some filesystems, that will include the metadata about the file, which got finalized when it got closed and thus is the last thing changed.) Pull the store now and important stuff about these new files is incomplete or missing, even if they've been closed (so the last buffer of data is complete and queued to be written).
When this happens, if you wait around a short time, and the periodic sync will force it out to the store. But it might be so few blocks that you don't see the activity light blink. Pull the store after that and it will have the file data and metadata for the new, and closed, files. But the filesystem image will still be in a "mounted" state, so remounting it will require at least some filesystem scan (very short for journaling filesystems) and may generate a gripe from the OS.
Unmount (then sync;sync, though unmount seems to do that for you these days) and you guarantee that the backing store is clean and ready to go. Eject does that for you (or gripes if it can't because there are sill things in use by live apps) before it actually ejects the storage medium or tells you you can safely remove it.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
... so just eject the goddam thing.
It little behooves the best of us to comment on the rest of us.
The computer should simply have a symbol on the taskbar/similar area that says when it is safe to remove the drive or not.
excitingthingstodo.blogspot.com
Because all it does is change the choice to "Do I eject the drive properly, or do I check to make sure "quick removal" is enabled and then yank the drive?" Unless you already know it's been set (i.e. you personally set it previously for this drive for this instance), the latter takes more time than ejecting.
The #1 cause I've seen for corruption of data/partition information on USB flash and external HDDs is yanking it out too quickly, just before the copy has finished. In theory a journaling filesystem like NTFS should be immune to damage from this. But for some reason it occasionally seems to corrupt the partition table making the entire contents of the drive inaccessible unless you're skilled enough to repair it manually (usual cause seems to be the partition type number got changed).
That's a 10 minute or so repair process if you know what you're doing. If you don't, it's probably 30-60 minutes downloading the tools and stumbling around trying to figure it out. And if you obeyed Windows when it popped up the "You need to format this disk before you can use it" message when you plugged the drive in again, and formatted it, now you're looking at several hours for file recovery with no guarantee you'll be able to recover everything. (They really need to add a second line to that message saying that formatting will destroy any existing data.)
All this risk and time wasted just to save a few seconds by not ejecting. So I advise people to always eject the drive before yanking it. The seatbelt analogy is very appropriate. It takes very little time to do, but the rare consequences if you don't do it can be devastating.
The solid state USB drives essentially have their own mini OS that controls things. You have to give that a chance to get to a reliable state.
The OS should set those flags whenever the write operation has completed, not wait until a "clean shutdown" happens. There are lots of things that can happen between the file write and the time the partition gets unmounted (eg power or controller failure), not accounting for technical issues is a flaw.
Custom electronics and digital signage for your business: www.evcircuits.com
Eject anything that there's a provision in the OS to eject (meaning un-mount from the filesystem, in the UNIX sense,) before electrically disconnecting it by unplugging it from the system. Anyone who tells you to do otherwise should not be listened to, because he/she/they/it/etc. does not know what he/she (etc.) is talking about. Even if the device has an activity light on it, data could become corrupted, or the device could be damaged, or both, if you unplug it unexpectedly.
The problem most people fail to consider is that on a device with an indicator (an LED, generally) you have no idea when the device is actually working, and no idea when it's about to receive data. For example, you think your USB device is not in use, not being written to, not receiving data, and that it is therefore safe to unplug it. So you grasp it with a thumb and forefinger, and begin to pull. Your reflexes are typical of a human being, meaning you are capable of discerning events of as short as between a 10th and a 30th of a second in duration, and also that limitation restricts the speed with which you can respond to visual stimulus. So let's say you start to tug on the device. At that moment, your computer decides it's a good time to write data to the device. The computer sends a signal and the device realizes the computer is talking to it, (if I can anthropomorphize it a little,) and it then dutifully sends a pulse of electricity to light up the LED. HOWEVER, by the time the light from the LED reaches your eye, strikes your retina, travels to the occipital lobe of your brain, where it is processed and passed to your cerebral cortex, and you become conscious of the fact that the light has come back on, it's too late to stop the muscles of your fingers, hand, and arm, which are already yanking the device out, as data was being written to it.
Further, you have no way of knowing absolutely that you can depend on the little LED to come one ANY amount of time before writing to the device commences. None. Beyond that, some USB thumb drives, like some PNY and SanDisk I've seen, literally HAVE no indicator LED so there's no real way to know whether the thing is being written to, or if it's even mounted somewhere on the filesystem or not, without actually consulting the filesystem itself through, for example, looking in your file-manager/browser to see what's mounted, or dropping to an xterm and issuing the "mount" command, or doing something similar with a disc-management utility.
It may seem unlikely the thing would spontaneously start writing without the computer actievly working on something, but you don't actually KNOW if it is working or not, LED lamp or no, and it would only be safe to assume if you somehow knew exactly what your computer's kernel's filesystem driver was doing, knew at any given moment what was being prioritized and what put off until later, etc. Also, there could have been files that couldn't be FINISHED being written until some portion of the middle of the file was ready to be written. Perhaps a file began being written, and then some issue arose and the program or application concerned had to pause to ask you something, and couldn't proceed, but that application had been relegated to the background, was minimized, etc.
There's a saying that goes, "only floss between the teeth you want to keep." Similarly there's a saying that goes "only pull a flash drive out of a USB port when there's no important data on either the computer, or the flash drive, and you don't care about what happens to either." (Never heard that one until today? Well, it's a relatively new saying then, but it is a saying now, so...)
Incidentally, computers and peripherals used to come with instructions to shut everything OFF before plugging or unplugging ANYTHING, even a peripheral like a printer or a keyboard. Not powering everything completely off could result in damage to the equipment, or risk of electric shock, fire, or rattlesnakes armed with sub-machine guns suddenly falling from the ceiling and atta
Our reign has gone on long enough. Indeed. Summon the meteors.
any modern file-system, such as EXT4, BTRFS, ZFS, even NTFS, that chances of seeing corruption are rare.
That's all well and good until the "rare" event actually happens to some data that mattered to you.
If you place any value on your data at all, you will not want to chance even a "rare" data loss or corruption.
Save rolling to dice to power or hardware failures, not easily protected actions like removing a USB device from a computer.
Although I do have a humorous aside - I have an external USB reader that loses connection every time my Mac goes to sleep, so every time I wake it up I am greeted with a message that I didn't eject a card properly if I left one in the reader. Since those are just reading images from a camera and I'm never writing, that doesn't really matter much (plus I reformat in-camera after comping images off, so even file system corruption does not matter...)
"There is more worth loving than we have strength to love." - Brian Jay Stanley
You can get away with risks sometimes. But if you push your luck too often, your "luck" will get used up!
And there is no law that says it won't go bad the very first time, or the second, no matter how long the odds.
Murphy's Law rules the world... 8-}
Yes, unmount the drive before taking it out.
If the usb driver doesn't cache writes, then unmounting the device before removal will not take any extra time, and has cost you nothing, while if it does cache writes, removing before unmounting could corrupt the data on the device.
Seriously, it takes like 2 seconds if it would have been safe to do the other way and will save you untold hours of grief later if it wasn't.
File under 'M' for 'Manic ranting'