Slashdot Mirror
Slashdot Asks: Do You Need To Properly Eject a USB Drive Before Yanking it Out? (daringfireball.net)
In a story earlier this week, Popular Science magazine explored an age-old topic: Do people need to safely eject a USB stick before they pull it from their computer? The magazine's take on it -- which is, as soon any ongoing transfer of files is complete, it is safe to yank out the flash drive -- has unsurprisingly stirred a debate. Here's what the magazine wrote: But do you really need to eject a thumb drive the right way? Probably not. Just wait for it to finish copying your data, give it a few seconds, then yank. To be on the cautious side, be more conservative with external hard drives, especially the old ones that actually spin. That's not the official procedure, nor the most conservative approach. And in a worst-case scenario, you risk corrupting a file or -- even more unlikely -- the entire storage device. To justify its rationale, the magazine has cited a number of computer science professors. In the same story, however, a director of product marketing at SanDisk made a case for why people should probably safely eject the device. He said, "Failure to safely eject the drive may potentially damage the data due to processes happening in the system background that are unseen to the user."
John Gruber of DaringFireball (where we originally spotted the story), makes a case for why users should safely eject the device before pulling it out: This is terrible advice. It's akin to saying you probably don't need to wear a seat belt because it's unlikely anything bad will happen. Imagine a few dozen people saying they drive without a seat belt every day and nothing's ever gone wrong, so it must be OK. (The breakdown in this analogy is that with seat belts, you know instantly when you need to be wearing one. With USB drives, you might not discover for months or years that you've got a corrupt file that was only partially written to disk when you yanked the drive.)
I see a bunch of "just pull out the drive and not worry about it" Mac users on Twitter celebrating this article, and I don't get it. On the Mac you have to do something on screen when you eject a drive. Either you properly eject it before unplugging the drive -- one click in the Finder sidebar -- or you need to dismiss the alert you'll get about having removed a drive that wasn't properly ejected. Why not take the course of action that guarantees data integrity? What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
John Gruber of DaringFireball (where we originally spotted the story), makes a case for why users should safely eject the device before pulling it out: This is terrible advice. It's akin to saying you probably don't need to wear a seat belt because it's unlikely anything bad will happen. Imagine a few dozen people saying they drive without a seat belt every day and nothing's ever gone wrong, so it must be OK. (The breakdown in this analogy is that with seat belts, you know instantly when you need to be wearing one. With USB drives, you might not discover for months or years that you've got a corrupt file that was only partially written to disk when you yanked the drive.)
I see a bunch of "just pull out the drive and not worry about it" Mac users on Twitter celebrating this article, and I don't get it. On the Mac you have to do something on screen when you eject a drive. Either you properly eject it before unplugging the drive -- one click in the Finder sidebar -- or you need to dismiss the alert you'll get about having removed a drive that wasn't properly ejected. Why not take the course of action that guarantees data integrity? What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
one should always eject before pulling out
ask it in a Mac-Forum, with the question, how to disable the requester that pops up... much more fun...
Acting retarded. No news herw.
No mention of the OS or file system. Assuming Windows - there's a setting for "Quick Removal," which disables write caching and makes it so "you can disconnect the device safely", and another for "Better Performance," which doesn't and may cause grief.
"National Security is the chief cause of national insecurity." - Celine's First Law
I can't count the number of times I've tried to eject a USB drive, and Mac OS tells me it's "unable to eject the drive because it's in use."
Usually it's because Preview.app held onto some file descriptor for its stupid thumbnail of recent documents - not the list in the file menu, the one that pops up when you right-click on Preview in the Dock.
Apple used to promote the idea that the user is in charge. When I click "eject," the damn thing should eject!
Depending on the write method in use from the OS, you either have transferred the full contents of what you wanted, or you haven't. In the later, ejecting will finish the write cycle and in the first you're good to go. There is one other rare problem that can arise and that's file-system corruption. and you run the risk of it by just pulling your USB key out, although with any modern file-system, such as EXT4, BTRFS, ZFS, even NTFS, that chances of seeing corruption are rare.
The problem of the safe drive ejection approach is having to wait for Windows (and I've found it's invariably Windows) to release a lock on a file because some program might have been using it at some point, then not giving you any capacity to tell just which program and file it might be so you can take care of it yourself. Then you just have to yank it anyway.
Why is it doing background operation without notifying the user? The led is there to indicate that there's something going on, why is it not flashing if there's something being done?
Caching and open file descriptors make Horton a sad who. It really depends on your system settings and many systems Are optimized for speed.
That said Iâ(TM)ve had some delightful experiences. I used to load an image to several usb sticks at a time. It was just dead simple to image 20 of these monsters and throw them in a package for the field guys.
Every once in a while one would fail to write. Linux would show a transfer speed of something around 4gb/s. This could be corrected with an eject and reinsert, but it was abbkying. Eventually I had the imager report transfer speeds as part of the report. This used dd with no caching options. Never did understand why some devices wigged out.
I repair computers and macs for a living and have been doing so for more than 20 years.
Is it safe to yank a USB drive?
No, but the severity varies by OS.
On Linux or Mac? Somewhat, or at least safer than Windows, but I'd go ahead and unmount it anyway just to be safe.
On Windows? No. You can do it, but you're taking a gamble every time that doing so will break the partition tables. You might be able to fix that and get the data back off of it... or you might not.
Now, will I do it anyway, knowing the risk?
Absolutely, and especially on windows. I keep a drive just for moving diagnostic tools or software to windows computers while inside the OS and occasionally, windows decides it doesn't want to unmount it, so I'll yank it anyway. But that's with the understanding that everything on that drive can go 'poof' any time I do it and everything on it is backed up.
In short:
If you value what's on that drive, you shouldn't... but you should have more than the one copy on the drive of whatever you're moving around anyway.
If you don't have backups, it must not have been important.
It's akin to saying you probably don't need to wear a seat belt
No, no and no.
As you may know, risk = damage x chance.
When i don't wear a seat belt and get in a crash, i might die or suffer severe injuries. That's big damage.
When i loose a file, i lost a few bits. I usually couldn't care less. It might be a photo or mp3 file. It might be a failed backup. Worst case it's a fortune worth of bitcoin. But i won't die from it.
Don't make analogies that just aren't true. Don't pretend a lost bit is the same as a broken body part. And to be on-topic - i usually just `yank` my thumb drive out whenever i want. When i feel it's really important i might once in a blue moon go the official way. I'm perfectly capable of calculating the risk that i take. Saving many seconds on repeated actions weigh up against that one time i was to quick and spend a minute re-copying a file.
It's just the next episode of FUD - same as the 'experts' done with passwords. Just so now they have an excuse to blame the user if a flash storage fails (which they do): 'you probably yanked it out'.
A glitch a day keeps the bugs away.
If you haven't written anything to the USB, yes, it's completely safe. Any filesystem that's writing data just because of your passive reading of the disk is a) stupid, b) should have recovery for such, c) shouldn't be writing anything important (e.g. a last access time, maybe?)
If you have written anything to the disk, and it's synced to disk (i.e. activity light is idle) then, yes, it's safe. This may be dependent on OS and whether you can see any activity light. Modern OS should mount without write-caching for external drives unless told otherwise, and any half-decent filesystem should survive a forcible unmount in such circumstances
If the USB is still busy churning after you copied files to it, and you yank it? Yes, you're gonna lose data.
That a bunch of "nerds" can't figure this out after all the years we've had USB disks etc. (I remember starting with them in 95 OSR2 / Linux 2.0 personally?) really worries me.
Same as floppies before it. Floppy only being read-from? You can yank it. Floppy was written to but has now gone idle? You can yank it. Floppy was written to and it still pulsing / flashing? Leave it alone.
Back when I was in tech support I noticed an increase in computer USB ports damaged when yanking the USB out. I know the plural of anecdote is not data, but still...
I used to write a lot of 8GB+ file system images to 16GB SanDisk devices using an Ubuntu 14.04 system. The caching it did was immense. The dd or the cp command finished in less than 60 seconds but when I did a umount command on the volume it would block for about 5 whole minutes or more while the cache emptied.
These particular USB drives have a blue activity LED on them so it wasn't hard to figure out what was going on.
In that use case yanking the USB would have been a big no no.
all the time. Have carpeting too. The case is open often and a beer is balanced somewhere nearby that it probably shouldn't. Not the same as forgoing a seat belt at all.
Depends how many places "confirmed" saved data can be cached in RAM.
Spinning hard drives have their own cache, which isn't written to the actual disk until either the cache is "full enough" or you instruct it to do so.
This data will be lost if you power the device down before the cache is flushed to disk, after the drive reported the data saved.
Flash sort of depends, an SSD for example tends to do the same thing, but there are different ways it can go about it.
Older SSDs can only write 4k blocks, it wasn't possible to write less data.
So to write for example 300 bytes, the controller has to pull in a 4k block to its RAM, edit those 300 bytes, and write out the entire 4k block again.
Pull power before this is done and your 300 bytes are gone.
"Removable" USB flash drives, the better ones at least, tend to not report data as saved before it really is, just to help avoid this problem. There is little hope for a non-technical person to know what their particular flash drive is doing however, and not even obvious to technical people either.
On top of that, your OS likely caches data to be written to any disk in RAM, completely independent of what the disk itself is doing.
If one is absolutely certain how all of these caches function, and can be completely assured all data is written in a method that doesn't rely on the disk claiming it is or isn't, then in that case it would be safe to power down the device.
For average and above average users, that will just not be true.
Even for experts, outside of a small set of cases like highly customized and tuned systems, it may be true the expert knows what is going on, but would tell you from that knowledge it wouldn't be safe and is a silly risk to take, with a high cost of data loss in exchange for a couple of seconds of time saved.
Hell, even I am still in the habit of issuing three 'sync' commands in a row before an unmount command, and that's despite the knowledge the unmount command will do a 'sync' call of its own!
But as this advice is for average or below people, as bad as it is, isn't the worst things commonly done.
Average or below people rarely even make backups, which has a far higher cost when (never IF) a drive fails. Corrupting a couple files on a single USB drive compared to not having any backups is like complaining your car only has 5 airbags instead of 6 while driving it off of a cliff...
My thoughts are that this is bad design and will hopefully be fixed in some future revision.
You *should* be able to just yank out a USB drive but there's always the chance that a program has a file open and has modified it but not closed it which means, depending on the file system, the file (and maybe the drive) is corrupt until it is closed.
I would think that the idea application behaviour when it comes to files the guidelines should be:
a) Opens the file to read the contents and store in memory. Once this is done, the file is closed.
b) Searches and operations are done on the copy in memory.
c) When the program/user decides the changes to the copy are correct, the file is open, written to and then closed.
d) If the application wants to update the file and the (thumb) drive isn't accessible, the user is prompted.
I know that I will get pushback on this as there will be the question what if the program is too big for memory? My response is that this is not 1983 and there should be either enough DDR to store the entire file or cacheing will make sure that the application can access all of it. Even if the file is cached, then the performance of the app searching the file in cache should be faster than doing lseek's on an open file. I'm trying to think of cases where the guidelines above are not appropriate and the only case I can think of is databases, but chances are that's on a network resource rather than a thumb drive.
Regardless of whether or not you think my guidelines are appropriate, you can't guarantee that all applications accessing the thumb drive close files after they've accessed them. You could go through and close all applications that are accessing files on the thumb drive, but that doesn't include operating system functions (early versions of NTFS in WinNT never did updates when you thought they would, they got put in a queue and would be done sometime down the line) or virus search apps work the way you want them to.
So, always click on "Eject".
Mimetics Inc. Twitter
sync;sync;
Its ok now to pull it.
Ever think of testing this for yourself?
It's easy. Create a half a dozen or more files of random numbers or use existing large files. I created six files of a million random integers in Python. End result, six files of about 6.9MB each. Create an md5 checksum file when you make them.
Copy them to a USB stick, and then yank the stick right when the light stops blinking. Plug the USB stick back in. Watch and learn. Easily reproducible phenomenon are:
* Not all files even appear when the stick is plugged back in
* Some of the files might appear, but give I/O errors and won't even be complete
* A Few might pass the checksum integrity test
I'm on Linux Mint, but I have seen the results on other OSes as well. The OS caches the data to be written sometime, presumably to speed up file operations. There's a reason why eject exists.
"What lies behind us, and what lies before us are tiny matters compared to what lies within us." Ralph Waldo Emerson
Not a problem taking it out, but you have to recompile the kernel before you put it in.
Write caching in older version of WIndows and Linux was the primary reason this was implemented. The OS would routinely hold small chunks of data, and only post it when 128kB was available. This action was below the file system, so even "good" FS's would be subject to the write-behind caching.
The other reason is notices to program that is writing. Even if you think its "done" and the little light has not blinked in a while, the program (usually the file manager GUI) might wait a moment to update the metadata. Again, modern versions have accounted for the sudden disappearance of the storage, and this is rarely a problem any more.
I have accidentally pulled USB drives without unmounting them, fortunately without apparent damage to the files. But I don't make a habit of it!
If you can remember and/or have the time to properly unmount the sdcard/usbstick for removal, do it. Otherwise data corruption is possible.
First of all: an annoying dialog pops up and an annoying beep comes, you have to close the dialog, probably even with the mouse because ESC or ENTER does not work.
Secondly, you probably have forgotten that a file is open in Excel or whatever and it is half modified but not fully saved.
Why not eject it and be safe, respectively be not annoyed?
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Unmount device before unplug.
Its might be on RW, if for some reason some file are writing on device u will regret it
My neighbour's treadmill says to stand on the sides when you stop and start it for safety reasons and it's easier on the motor but she doesn't do it because "the treadmill is on warranty". So she stands on the belt and then complains that it jerks her off her feet. So I tell her to stand on the sides when she stops and starts like the instructions say, but she doesn't.
I don't understand this mindset. People think they're getting away with something, I suppose. Or USB STICKing it to the Man.
More sophisticated SSDs perform static wear leveling (moving old data to different locations on the disk) in the background. To improve performance, firmware on some disks does this without ensuring atomicity of each operation. It means that in case of unexpected power loss you could end up with seriously broken filesystem. Cheap pendrives are probably not affected by this, but you should be careful when connecting e.g. external SSD via USB.
When you yank out a USB plug, you do TWO things - you disconnect communications AND you abruptly cut the power.
So if you want to avoid screwing something up you need to know that the device has gotten all of the data (which, you might *maybe* be OK with if you wait long enough) - AND that any internal processes within the device have completed.
For a very simple flash memory gizmo - it's probably (*probably*) dumb enough to just write data as it gets it - but a sophisticated USB hard drive may be caching data, rearranging blocks, maintaining a free block list, defragging, remapping bad sectors...who knows what else?
So there is not, nor cannot ever be, an absolute guarantee that the device is done-DONE until it says so.
Worse still - "wait until the light stops flashing then wait a few seconds" doesn't guarantee that the host computer has finished sending all of the data. Maybe there is a background process running at higher priority than the file transfer? Maybe the data is coming from a networked drive and the server or the network itself stalls out for a few seconds? Maybe the hard drive that the file is coming from had a soft error and is doing a retry? There are any number of reasons why a few seconds of delay could have occurred in mid-transfer.
Unless the author of this device has carefully examined the behavior of EVERY USB storage device in existence and has insider knowledge of every single company who makes an USB storage device - AND every single device driver on every single OS (WIndows, Linux, Mac, ChromeOS, Android and iOS)...then he/she is without doubt talking out of his/her ass. I'll let you decide out which it is!
The advice that you can just yank it out is ignorant, stupid and reckless.
...more than you do. I say this every time someone claims to know better than the OS designer. For example, memory defragmenting applications, RAM boosters, memory cleaners... if they did good, it would be probably built into the OS already.
This is a similar case, but even easier to rationalize. The designers went out of their way to warn you and chastise you for yanking out a USB device. They spent time and effort adding these messages in. I would assume they are valid just from that alone.
Windows in particular I can talk about. Many USB thumb drives are slow especially when you sort by lowest price on NewEgg/Amazon and buy one based on the size of the disk alone. So Windows will avoid using the actual device in real time if it can, instead using a cache. If you write data to the device, the data goes to the write cache first, which is then written to the device as fast as possible in the background. This means that file copy dialog will close BEFORE the data is written to the drive.
Now, you can access the properties of the drive and change Windows to not use a cache specifically to make it safer to yank the drive at the cost of making things slower. But either way, why? Just choose the Eject option and let Windows tell you when it's safe to remove the drive. Then you can be sure your data is done being recorded. It also lets you know if there is an application still actively using the drive, which is better than yanking it and seeing the application crash or lose data.
Why don't USB drives self-eject? I mean, beyond memory size and speed, and the ability to buy one in the shape of sushi, USB drives haven't evolved since their introduction. It would seem like adding a button on the drive itself--one that calls to the system to let it eject--is waaaay overdue (considering the current, boorish way of doing it, which involves using your mouse or keyboard and way too much thought and effort).
Free idea right here. Sandisk? PNY? Hello?
Worst case it's a fortune worth of bitcoin. But i won't die from it.
That depends on how your country handles health care finance.
"On Linux or Mac? Somewhat, or at least safer than Windows, but I'd go ahead and unmount it anyway just to be safe."
The main problem you may encounter on linux is with NTFS drives. The data is pretty safe but if you yank the drive it may leave the file system in a bad state and you have no chkdsk or fsck.ntfs. But there's the "ntfsfix" program which I remember using for drives when nothing is majorly wrong but stuff complains of "this drive was badly dismounted". It won't fix the file system though if it need be fixed.
With internal drives too power loss or complete OS crash are a thing.
I never, ever "safely" eject, I always yank it out when it's done with a transfer (if transfer there was). Amount of drives/keys/data lost so far is zero.
Many years ago, I was told that, because of differences in the ways Macs and PCs operate, one should always properly eject a flash drive from a Mac, but it didn't matter very much for a PC. I don't know the status of this today.
What are your thoughts on this? Do you think the answer varies across different file systems and operating systems?
You're kidding, right?
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Safe eject/removal is the only way to guarantee that file buffers are flushed and metadata is in a consistent state.
If you're OK with losing files, or having chunks of your free space marked as occupied, then don't use safe eject.
Err on the side of caution. Etiher:
You don't need to because the system isn't caching writes, in which case the result is instant
The result is slow, which means you need to do the 'eject' procedure.
Practically speaking, modern OSes have gotten the point, a small usb attached storage device is at high risk to be yanked out, so they disable write caching by default.
XML is like violence. If it doesn't solve the problem, use more.
You'd think the partition table would be pretty darn safe since it's rarely updated. Yet, I've had to help people recover from lost partition tables many times. You can do forensics to discover where the partitions were, such as by looking for blovks that match the beginning of a filesystem, and for the first partition, testing the defaults.
One potential reason for this is that electronics designed to work at five volts can do literally ANYTHING when they have 2.5 volts supplied. You may notice some devices go a little crazy when the battery is very low. As you disconnect a drive, there are several milliseconds in which the chips get lower than specified voltage.
Also the data lines are of course digital - on or off. As you pull the plug out and the contacts scrape against each other, an oscilloscope with show they connect and disconnect several hundred times - turning on and off, generating random ones and zeroes. There are some features designed to *reduce* these risks.
I always have write caching turned off. Windows has the Quick Removal option, so why would it be there and be called that if it wasn't there to allow you to pull it out?
I never ejected a flash drive in XP and never had a problem. In Win 7 I almost always do, as it doesn't seem to work properly, but at work we have a Win 10 laptop and 90% of the time it doesn't allow me to eject (warning message pops up), so I have to either reboot (yeah, right), or just pull it out and hope for the best.
Why don't flash drives work the same as floppy disks? We never had a problem ejecting those.
Because adding a button and indicator to show completion means cost and ruining the 'minimalist' design point they are going for. Further, they know consumers will likely ignore that anyway, and that OSes have given up write-caching for removable storage anyway.
XML is like violence. If it doesn't solve the problem, use more.
How is pressing a button on the USB drive any easier than clicking a button on the screen?
Either way I still have to look at the screen for the dialog telling me I can unplug or not.
The mouse and keyboard are almost always easier to reach than the actual USB device, at least for me.
Unless you want to put yourself at the mercy of the write caching mechanism in your OS of choice.
"Unlikely" might be a better word than "absurd". The physical cmos chips associated with ports are delicate. They don't like voltages that are too high, too low, or change to fast.
As you pull a connector out, the power and data pins scrape against each other, causing them to connect and disconnect a hundred times in a hundred milliseconds. Having power switching on and off randomly while the data lines are active transferring data can be bad.
It's probably not LIKELY to cause damage if you do it once, but the possibility isn't absurd.
Well, here it's a bit of an evolved situation, and one that evolved with the general state of storage.
Once upon a time, OSes didn't bother to cache writes, they just wrote to disk and were done. Computer could just turn off at any time (parking hard drives manually aside). Memory was too scarce.
Then memory was more plentiful and OSes became more sophisticated, disk IO then presumed caching, and shutting off your system without proper shutdown was exceedingly high risk. Removable media came into fashion with this sort of accepted behavior, so they had to be very verbose about users doing it wrong.
Fast forward a bit and the industry realizes that they can't expect clean ejects/shutdowns, so they make better filesystems, maybe losing data but at least not be corrupted and disable write cache on usb disks by default, causing slower apparent behavior, but resilient to yanking out.
It's always a good idea to err on the side of caution, but the current OS designs around ejecting media are less critical because they've decided it is a lost cause.
XML is like violence. If it doesn't solve the problem, use more.
You mean making them work like CD-Rom drives? I guess they didn't want to make them work like that to make things simpler.
Why do these things even have a motor? I would enjoy a treadmill if you actually moved the belt with your feet. That way you walk on your terms.
If motorized hopefully one can be made that moves according to your walking speed. Maybe it'd watch your legs with some sort of Microsoft Kinect equivalent which these days would $50 of smartphone hardware.
It's really that simple. If you've written data to it, safely eject (or run `sync` before ejecting or what have you). If you haven't, then just yank it out.
On Linux, you can also disable USB write caching by adding the following to /etc/hdparam (check what /dev file USB devices normally show up as first): /dev/USB_DRIVE_LETTER {
write_cache = off
}
Credit: https://superuser.com/q/526248
This is like asking if you need to properly wipe your butt.
Some operating systems treat removable devices differently: By disabling write caching, they reduce the risk of having outstanding writes when the user believes that the drive is idle. With modern filesystems, there's theoretically the possibility of keeping the drive in a consistent state even if it is yanked from the computer mid-write. FAT32 however, which is still the most commonly used filesystem on USB thumbdrives, is not a modern filesystem. There's also a difference between a valid filesystem state and data corruption. It would take a completely transaction-based filesystem to remove the need for unmounting a filesystem, and the USB drive would need to also be designed to complete some work even when it is unexpectedly removed from the computer. So, while it is technically possible to make hard- and software that does not need a clean unmount, in actual practice you're going to corrupt the data, the filesystem and possibly even damage the hardware at some point if you don't tell the OS to "eject" the drive.
Just thinking about this, it's really the same question - when you power down your PC/laptop/whatever, do you hold the power button down for five seconds to shut it down or do a "System Shutdown"?
It's really the same thing when you apply to USB thumb drives.
Mimetics Inc. Twitter
Windows is funny like that.
[($)]
You could enable write caching on your floppies in SmartDRV. In that case, you either wait for the light to go out after up to 5-10 seconds, or you run SMARTDRV /C and wait for the command prompt to return.
I don't know if NT/XP would write cache floppies.
This is the result of OS programmers throwing wrappers around internal APIs instead of designing a product.
How about the OS make yankable the default state, with an indicator light on the task bar, window bars, etc. For that matter computer makers can put a tiny LED by the front panel USBs to signal this too, assuming you can even see it amidst glowing coolness everywhere.
Tbe user knows they are done copying, so can wait for the OS to signal so.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Then yes
I don't have a clue, but I'm sure Wikipedia does!
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
...The magazine's take on it -- which is, as soon any ongoing transfer of files is complete, it is safe to yank out the flash drive ...
The problem is that you do not know when the transfer is complete. The UI's representation of it shows when the UI is done with the transfer, but that does not necessarily mean that the OS is finished with the transfer. So Popular Science is correct in that you have to wait until the transfer is complete, they are just incorrect about what tell-tale to use to determine that status.
I never pull out....
why the fuck is this a topic of discussion in 2018?
Good practice to properly eject? Yes. Reliable? Not so much. Both Windows and Mac have a tendency to kick back a "drive is still in use" message when you choose to eject it, despite that being total BS. It's all because some software saw a new drive and refuses to let it go. End result? You could sit there for an hour waiting and it won't let you eject "safely". That's why you get people that say you don't have to do it properly. They or the person they trust has experienced it over and over again and had no known consequences to pulling the drive.
That doesn't mean it's always safe to pull. Linux is notorious for holding caches. No eject? Data gone bad. Windows is usually good, depending on the drive manufacturer, whether it declares itself a removable disk or a fixed disk, and whether or not it's a day ending in "Y". Mac? Apple never lets any of their secrets go, so unless you're using a filesystem where you have the control, who knows whether or not they're using delayed caching?
Get it right!
I always eject it, but sometimes it says its in use and won't let me. But I want to pull it out NOW, so too bad its getting yanked out.
I also use a program called Hotswap! that ejects alot better than the windows one, AND lets me do sata drives and things that normally don't show up in the windows eject.
Just like a parking brake on a car is there for a reason, the option to safely eject a USB drive is also there for a reason.
But don't let that stop all the experts on here and elsewhere from saying otherwise.
...more than you do. I say this every time someone claims to know better than the OS designer.
As someone who has actually written software and contributed to many projects; No! Do not trust me! I still remember when a major release of my software hit slashdot, and one of the first comments was "Why can't I see any logs?" Yes, I made a stupid build error that hosed logs. I was able to push a new release the next day with that fix, but it took someone not trusting me to find it first.
Also, the Metro Interface proves that the designers are often drinking better stuff than we can afford.
You mean making them work like CD-Rom drives?
You mean where the button ignores you and does not eject the disk. Or starts to and teases you and then closes again? Like that?
There is a difference between needing to do something, and if it's best to do something.
For example, you don't have to understand grammar to be slashdot editor, but it'd be best if the editors here understood the definitions of the words they use.
Jokes aside, it pays to read the article to discover exactly who is involved: an MIT PhD candidate and Harvard's CTO. wtf. An AP at Carnegie-Mellon and a SanDisk rep both have it right.
Copy them to a USB stick, and then yank the stick right when the light stops blinking.
Two functionalities are key: Sync and unmount.
- Sync forces the filesystem to complete all in-RAM updates to blocks, write the blocks to the driver, and forces the driver to flush all pending writes to the backing store. Issuing a "sync" kicks off activity that insures the data for the current state is all on the backing store when the activity is completed. (And manually issuing two of them insures the first one is completed before the second command returns.)
- Unmount also tells the filesystem to get all files closed, go to such a state (including updating a flag saying it isn't mounted, if that is part of the filesystem) and write this all to the store.
Unixes and linux are usually configured to issue syncs periodically. So if you write new stuff to your filesystem, and your filesystem/driver combination isn't one that tries to always force the data to the store right away, you'll see the activitly light go out when there's still important stuff in a last few buffers which AREN'T written yet. (For a new file, on some filesystems, that will include the metadata about the file, which got finalized when it got closed and thus is the last thing changed.) Pull the store now and important stuff about these new files is incomplete or missing, even if they've been closed (so the last buffer of data is complete and queued to be written).
When this happens, if you wait around a short time, and the periodic sync will force it out to the store. But it might be so few blocks that you don't see the activity light blink. Pull the store after that and it will have the file data and metadata for the new, and closed, files. But the filesystem image will still be in a "mounted" state, so remounting it will require at least some filesystem scan (very short for journaling filesystems) and may generate a gripe from the OS.
Unmount (then sync;sync, though unmount seems to do that for you these days) and you guarantee that the backing store is clean and ready to go. Eject does that for you (or gripes if it can't because there are sill things in use by live apps) before it actually ejects the storage medium or tells you you can safely remove it.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
... so just eject the goddam thing.
It little behooves the best of us to comment on the rest of us.
The computer should simply have a symbol on the taskbar/similar area that says when it is safe to remove the drive or not.
excitingthingstodo.blogspot.com
...by simply disconnecting it from USB. Completely corrupted. Got it back by reformatting, but the data was toast. You simply yank it out with a risk of losing everything on it. Stupid computer says its in use, I simply shut down, disconnect, reboot. I won't ever just disconnect again.
Because all it does is change the choice to "Do I eject the drive properly, or do I check to make sure "quick removal" is enabled and then yank the drive?" Unless you already know it's been set (i.e. you personally set it previously for this drive for this instance), the latter takes more time than ejecting.
The #1 cause I've seen for corruption of data/partition information on USB flash and external HDDs is yanking it out too quickly, just before the copy has finished. In theory a journaling filesystem like NTFS should be immune to damage from this. But for some reason it occasionally seems to corrupt the partition table making the entire contents of the drive inaccessible unless you're skilled enough to repair it manually (usual cause seems to be the partition type number got changed).
That's a 10 minute or so repair process if you know what you're doing. If you don't, it's probably 30-60 minutes downloading the tools and stumbling around trying to figure it out. And if you obeyed Windows when it popped up the "You need to format this disk before you can use it" message when you plugged the drive in again, and formatted it, now you're looking at several hours for file recovery with no guarantee you'll be able to recover everything. (They really need to add a second line to that message saying that formatting will destroy any existing data.)
All this risk and time wasted just to save a few seconds by not ejecting. So I advise people to always eject the drive before yanking it. The seatbelt analogy is very appropriate. It takes very little time to do, but the rare consequences if you don't do it can be devastating.
I've also found that the quality of the drivers for the USB 2/2.1/3/3.1/ESATA port also matter on how well the eject process works or doesn't. (Intel USB ports with genuine Intel drivers works the best in my experience) I'm still using Windows 8.1 but I'm aware that many hardware vendors of USB 3/3.1 chips don't make drivers for Windows 10 and just rely on the generic Microsoft drivers which I've seen have performance issues and other strange problems.
Another problems I've often seen in Windows 8.1 (but not sure if Windows 10 has the same issue or not) is that some system process locks the drive and prevents an eject command and I've often times waited an hour or more and still can't eject the drive. I've tracked down the process using Process Explorer and killed it and then been able to eject the drive; however a significant number of times that USB port then fails to work until reboot.
All in all I'd have to say that while USB and ESata are great there are still lots of issues that are not resolved when it comes to Windows and removable drives and Intel / Microsoft (and any other chip vendors) have failed to address these lingering problems.
It takes one second to eject media.
So why is this so difficult that it requires an article?
"You don't need to look both ways before crossing the street." Why bother? I walk across the street head down, not bothering to look, all the time, and I've never had a problem. Drivers have become accustomed to pedestrians doing crazy things, so they take care for you. My buddy Joe does it all the time, and he never had an issue. And Frank over there, he does it too, and he works for the ministry of transportation. Pedestrian accidents are at an all time low, so it is HIGHLY unlikely you would ever get run into a problem.
All I can say is WOW. This is not worth reproducing on slashdot. And it wasn't worth the storage bits consumed on Popular Science as well. There is no journalistic value here.
I just umount the drive, and once that's done, I'm good to go.
I don't give a fuck about windows users.
They are all just morons anyway.
The SanDisk guy is not kidding, ejection does tell the device that it is going to be powered off, and to *stop all background tasks*. Non-joke FLASH devices (like *every* SSD), and that includes USB pendrives, often need to do some garbage collection in the background.
If it has a FTL (and almost everything that is not NAND SLC or NOR FLASH does), it needs garbage collection. Even if it is a cheap piece of crap. And if it is large, it may take a while to finish it. We now have >32GiB flash devices easily, those are the worst kind of flash (TLC) which noticeably bitrots from being left alone and from being read, so it needs background scrubbing *all the time*...
When you yank NAND that is going through an erase or write cycle, you can physically damage it. It is *that* simple.
So, stop being an imbecile and eject the damn thing before you yank it out. The OS guys tell you to do it, even Apple. The device makers tell you to do it. The electrical engineers that designed the thing tell you to do it. And we even told you *why*.
Oh slashdot how low have you fallen...
I always eject, but if it says I can't, I just pull it out anyway. And thinking "that's what you think".
Maybe not the smartest, but I have gotten away with it for years. Might bite me some day.
OS should say why it can't be ejected. There's typically no clear reason why.
The very fact this question is being asked here speaks loads about how Slashdot has died.
The very fact the editor didn't know how stupid this question is also speaks loads about how Slashdot has died.
For spinning disks, you should also power down the drive before disconnecting it. Failing to do so will force the disk to perform an emergency head parking, using the inertia of the spinning disk platters. This causes abnormal wear on the drive and shortens the lifespan.
Under Linux you can power down the disk using this command:
$ sudo udisksctl power-off -b
I use this alias in my .bashrc file:
alias diskpoff='sudo udisksctl power-off -b'
Example: $ diskpoff /dev/sdb
Seriously, even "top" brands like Prusa. Apparently you can just pull the card any time you want. OK, makes sense. It's not like they're mounting the card with write access.
Do macusers still eject media by dragging it to the trashcan? I always found that a pretty humorous metaphor about what the 'mac experience' thought of the world outside it's bondi-blue enclosure.
The issue here appears to be how the user uses the product and how the manufacturer uses the product.
The drives are being yanked out and that is human nature, the technology should facilitate the use case not make up an imaginary use case where we are carefully unmounting the drives and then removing them.
Core issue is that before these drives are yanked we watch a percentage bar of file transfer fill up and disappear but the file might not be transferred. The programmer of the user interface design showing the file transfer obviously fucked up as the system shows the drive is not being written to when in fact it is. If the progress bar accurately displayed when data was done being written it would make it much more safe to yank the drive out.
Various operating systems do delayed writes. Knowing whether your disk be safely ejected is a question not only of whether or not there is a pending operation currently writing to a file (corruption of data) but also of meta-operations and what kind of journaling is going on. Because many things could be going on in the background you don't know about (like reallocation of file blocks) it's possible to corrupt the structure of the FS itself by premature removal. Combine things like out-of-order writes and you're in for a serious mess.
Granted this doesn't happen most of the time, but there are a lot of factors that can affect it. I'd suggest not doing early removal if you don't want to risk your files being corrupted/disappearing/other random files you weren't writing to being corrupted or disappearing.
There's going to be a big difference in potential consequences depending on your file system type too. For example, removing an EXT4 drive with data mode "journal" vs removing a non-journaling file-system like FAT32. Unfortunately, most USB file systems seem to be of the non-journaling type, even though they are precisely the kind of device that would benefit from journaling preventing corruption of the file system. And do note that even when journaling protects against corruption of the file system itself and may provide consistent data ordering and atomicity, you still end up with half-written data so that might still be corrupt from the application's point of view.
Did this once by accident and it corrupted the drive
"Can I leave my computer on all the time?"
and
"Why does my inkjet printer keep breaking when I come back home after spending all winter down south?"
...and I've never seen anything adverse when I don't.
Social Media Mgr at Bluefield Identity
Uhhhh, because you have to reach for the USB stick anyway? A dedicated eject button is a good idea, reach for stick, press button.
Or you can reach for mouse, move mouse to corner (or hit the arrow to reveal more icons), right click the icon, tell it to safely eject, then wait, then grab USB disk.
I'll take the button since that prioritized everything anyway and tells the OS you are about to eject.
Google translate failed to properly auto-detect language, but guessing Greek returns "That's why"
I always tell my computer to safely eject to finish its usage. However, it bothers me when SOME OSes won't let me safely eject because it says it is in used. I check and I see nothing in usage. These were with external USB HDDs. :(
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
Follow the advice from the OS. That's the software that is controlling the physical writes to your USB device. It's the software that's caching not only the file data, but the filesystem data too.
If you didn't need to eject your USB device before pulling it out, the OS wouldn't tell you you've done something bad when you do it.
An frankly, anyone who would design a drive to work any other way should have been fired.
Please do not read this sig. Thank you.
Lost the valuable contents of a flash drive 10 years ago due to not doing this (still on XP at the time I think). My brother thought I was an idiot for 'ejecting' a drive when I mentioned it a short while later as he had 'never had a problem just yanking it straight out' - good for him - I lost two days work. Interesting to note Win10 doesn't seem to offer the option anymore so (not being an OS guy) I wonder whats changed in that respect?
You should always do it properly because there may be malware half-way writing itself when yanked, and damaged malware can prevent reading of the drive. Most malware wants to snoop, not crash.
Table-ized A.I.
First, this is nothing like driving without a seat belt. If I yank a thumb drive from my computer without "safely" ejecting it, I am not going to suffer any personal injury. The worst that could happen is I maybe have to reformat the drive. Big deal.
(Score: -1, Stupid)
Is this what has become of slashdot???
Or just when the drive had just gone done being busy with lots of reads and writes.
I was reading just a while back that even some spinning drives are over provisioned, and the drive might use that as a cache. I guess it could get iffy if the drive got yanked while it was finalizing the process.
One other thing. IIRC a drive with a good controller will do some housekeeping by way of checking its integrity from time to time. It might "resent" being interrupted while marking a sector bad and transferring its data to a good one.
Or just when the drive had just gone done being busy with lots of reads and writes. I was reading just a while back that even some spinning drives are over provisioned, and the drive might use that as a cache. I guess it could get iffy if the drive got yanked while it was finalizing the process. One other thing. IIRC a drive with a good controller will do some housekeeping by way of checking its integrity from time to time. It might "resent" being interrupted while marking a sector bad and transferring its data to a good one.
Go into Device Manager in Windows and if you disable write caching for the USB drive (set it for Quick removal) Windows indicates you no longer need to use the Safely Remove Hardware Notification icon. Well, that's the theory anyway, the drive's controller may in fact be doing something that shouldn't be interrupted.
in linux for sure, for example you mounted a USB thumbdrive, and maybe use the commandline or midnight commander's two panel file manager to select and copy files over to the thumb drive, or any file manager for that matter, then close the file manager, and you run the umount command, notice it hangs and does not give you back your command prompt? its because it is either not actually done writing the files over or the journal, and if you just yank it out the files on that thumb drive will more than likely be corrupted, just wait until the commandline prompt comes back before unplugging the thumbdrive
Politics is Treachery, Religion is Brainwashing
This is probably the most duh question that the new owners of Slashdot have ever asked to the public. First of all, the summary of the article made no assumptions about which operating system is running on the hypothetical computer in question. Windows may handle asynchronous services quite differently than UNIX and its variants.
Perhaps Slashdot should ask Mr. Tso or Ken Thompson, before asking for public comment on this question. Furthermore, the editors of Slashdot would do well to read a few books on UNIX and Linux internals before posing this question to the public.
It is obvious that UNIX and Linux are not real time operating systems. The only service that is real time is the external interrupt service, as well as the test and set instruction. The editors of Slashdot would do well to study the test and set instruction prior to publishing these questions.
The simple answer, Slashdot, is that UNIX is asynchronous, and therefore their file systems are also asynchronous, meaning that you have to eject a flash drive prior to pulling it out of the socket.
Let me answer your question with a question, Slashdot. Do you pull the AC plug on your computer to shut it down, or do you log off and issue the shutdown instruction before turning off the power? There, I answered your question with a question, Slashdot.
Because physics. I mean honestly, if you want the surface (treadmill belt) to go round, you need a force moving it. That force needs something to push against in order to move the belt.
Scuffing your feet ain't gonna work. Like trying to pull yourself up by your bootstraps.
it's like jumping off a moving train. It may work out great, but how about you do it the way they tell you and be sure it's gonna work.
Never, absolutely never. OS should quickly write/deleite and Just pull it
When you ask to eject the system does one thing. It unmounts. When the system unmounts it does three things: 1) It checks for open files. 2) It syncs aka It flushes the buffers. 3) It makes sure the drive is in a state that it can be disconnected--any work the devicee is doing is done. Aside from 3. 1 turns out to be very important. Often times when I eject, I'm told I can't. When I issue a lsof ( list open files ) command on that drive, I get the some process which I go through. Sometimes it's some sort of zombie. Most of the time I have a shell that is open on a directory in that drive. But some of the times I have a file open in a process that I forgot about. Other times it's a process that opens a file in it's background that i don't know about. Then unmounting the drive saved me from losing data or corrupting a program accidentally, by making sure the programs using that drive were all properly shut off. Asking the system to ejeect saved my ass. As for 2. I've practices like flushing buffers come and go. There was a reason that buffering and flushing were implemented. maybe in this go around, that reason is no longer valid and you may not need to eject to make sure that buffering is properly flushed. Fine. But what I've learned is that it is likely that sometime in the future, the next generation of even faster practices will require you to once again have to be sure that the buffers are flushed. Teech goes that way, back and forth between practices. THe safer thing to do is to request an eject from the OS.
From the user POV, the question is whether the data is important. If it's important, of course you eject it safely, just in case somewhere in the dozens of layers of engineering involved between generating the data and writing it to disc, there's some issue that arises that happens not to work correctly if you pull it out unsafely.
If it's unimportant data, who cares?
From the engineer's POV, you have to assume some users will pull it out unsafely, so you make it as safe as possible to remove it unsafely, balancing that against the need to keep the write speed from being unacceptably slow.
Real lawyers write in C++
I usually use the default USB stick's format, vfat. /dev/sdd1 showed an error ... so many sectors allocated, a different number used and the last file was not corrupted, it was just incomplete.
On old USB sticks I had no problem but on new, fast drives
after doing sync twice, umounting the driver, syncing again and then removing the drive a later dosfsck on
I suspect the USB drive has its own cache. umounting the drive removes it from the file system. syncing assures me that my system wrote to the drive but ... if the USB stick has its own cache?
Now afer copying the files I want I copy another file of a couple of hundred megabyes to flush the USB stick's cache. I sync that and then ... I remove it from the USB stick. If the file was not completely written I wind up with some sectors having been written to and others not - an incomplete write ... and then all those sectors are marked as unused. I don't care if the final write was complete or not as long as there is enough time for the drive to record the removal.
I haven't see a mismatch in number of sectors allocated and number used.
[USING: Linux/Fedora 24]
This should be the next /. poll.
The truth is that if a device manufacturer is hiding a background process out of sight of the user and the OS then it should be built with suitable backup power, in the form of a small battery or ultra capacitor, to guarantee that the data in ram is flushed to persistent storage. I understand that this would be easier to do with flash drives then external spinning disks, but you could still probably make a spinning disk reliable with a small lithium ion cell, like a 14500, hidden in the case. Any "routine" or "common sense" procedure that can be eliminated with a simple hardware fix is not in the best interest of the end users. It is the same reason microwaves automatically shut off when you open the door instead of having a warning in the manual that you may suffer microwave burns if you fail to turn off the microwave before removing your food.
Yes. Even though the OS should attempt to write back data to removable storage as soon as possible, you never have the guarantee the stick is idle unless you properly eject.
I've never bothered worrying all that much. I've transferred my main backups from Passport to Passport to Passport as they require more space over time, and all of the original files have remained perfectly intact. Same goes for data I've transferred on and off of flash drives. Mind you, I mostly deal with the more common file systems (Fat32, NTFS) and usually am exclusive to dealing with Windows. Any time I'm working with foreign systems I'm a little more cautious, but that's pretty rare for me.
Eject anything that there's a provision in the OS to eject (meaning un-mount from the filesystem, in the UNIX sense,) before electrically disconnecting it by unplugging it from the system. Anyone who tells you to do otherwise should not be listened to, because he/she/they/it/etc. does not know what he/she (etc.) is talking about. Even if the device has an activity light on it, data could become corrupted, or the device could be damaged, or both, if you unplug it unexpectedly.
The problem most people fail to consider is that on a device with an indicator (an LED, generally) you have no idea when the device is actually working, and no idea when it's about to receive data. For example, you think your USB device is not in use, not being written to, not receiving data, and that it is therefore safe to unplug it. So you grasp it with a thumb and forefinger, and begin to pull. Your reflexes are typical of a human being, meaning you are capable of discerning events of as short as between a 10th and a 30th of a second in duration, and also that limitation restricts the speed with which you can respond to visual stimulus. So let's say you start to tug on the device. At that moment, your computer decides it's a good time to write data to the device. The computer sends a signal and the device realizes the computer is talking to it, (if I can anthropomorphize it a little,) and it then dutifully sends a pulse of electricity to light up the LED. HOWEVER, by the time the light from the LED reaches your eye, strikes your retina, travels to the occipital lobe of your brain, where it is processed and passed to your cerebral cortex, and you become conscious of the fact that the light has come back on, it's too late to stop the muscles of your fingers, hand, and arm, which are already yanking the device out, as data was being written to it.
Further, you have no way of knowing absolutely that you can depend on the little LED to come one ANY amount of time before writing to the device commences. None. Beyond that, some USB thumb drives, like some PNY and SanDisk I've seen, literally HAVE no indicator LED so there's no real way to know whether the thing is being written to, or if it's even mounted somewhere on the filesystem or not, without actually consulting the filesystem itself through, for example, looking in your file-manager/browser to see what's mounted, or dropping to an xterm and issuing the "mount" command, or doing something similar with a disc-management utility.
It may seem unlikely the thing would spontaneously start writing without the computer actievly working on something, but you don't actually KNOW if it is working or not, LED lamp or no, and it would only be safe to assume if you somehow knew exactly what your computer's kernel's filesystem driver was doing, knew at any given moment what was being prioritized and what put off until later, etc. Also, there could have been files that couldn't be FINISHED being written until some portion of the middle of the file was ready to be written. Perhaps a file began being written, and then some issue arose and the program or application concerned had to pause to ask you something, and couldn't proceed, but that application had been relegated to the background, was minimized, etc.
There's a saying that goes, "only floss between the teeth you want to keep." Similarly there's a saying that goes "only pull a flash drive out of a USB port when there's no important data on either the computer, or the flash drive, and you don't care about what happens to either." (Never heard that one until today? Well, it's a relatively new saying then, but it is a saying now, so...)
Incidentally, computers and peripherals used to come with instructions to shut everything OFF before plugging or unplugging ANYTHING, even a peripheral like a printer or a keyboard. Not powering everything completely off could result in damage to the equipment, or risk of electric shock, fire, or rattlesnakes armed with sub-machine guns suddenly falling from the ceiling and atta
Our reign has gone on long enough. Indeed. Summon the meteors.
If you feel like pulling, then go for it. Your data is probably worth it anyway.
* Always causes grief if you're a clueless student or teacher (and the resultant loss of work because of the braindead default in windows)
This. Seriously. The amount of people caught out by this is insane. Windows did the world a massive dis-service by caching writes to removable drives. It should be disabled 'safe' by default. People aren't stupid but MS really aren't helping anyone when it comes to hiding the fact the drive is still doing work.
The classic use case is doing work on your computer, *then* copying it to the drive before you quickly leave to catch the bus/etc/etc - the user interface gotcha here is the fact the 'copying' box disappears before the drive has actually copied the files. Insane. Should never cache writes on removable drives, ever. Stuff the speed. If you're worried about speed then you're clued up enough at that point to go find out how to turn it on, or already know how to do it.
Unmounting on Linux or "safely removing" on Windows can often complete before all writes have finished.
Always best to wait at least 10-30 secs after the last indication of actvity (ie. flashing light) before removing. And don't buy sticks that don't have an activity indicator!
Really? Because in my opinion, the truth is that you should just use the device as the manufacturer recommends, which is properly shut down the device before removing its power.
Bingo. Mod this coward up. He's got more knowledge than 80% of the registered dotards commenting on this article.
Yea because delayed write doesn't exists at all. :-|
Files aren't necessarily fully written to flash drives when the copy procedure disappears...
The sole purpose of the eject / safely remove feature is to flush any data that is waiting in cache to be written (Delayed-Write) to the device and unmount.
http://www.northamptoncomputerrepair.com/ask-a-techie/bid/171187/What-does-Delayed-Write-Failed-mean
That we even discuss this. The feature is there to flush all the write caches. When you "eject / safely remove" after copying hundreds of MB or even GBs worth of data this "eject" will usually even take a while until all the write caches are actually written to the medium. Where do you think the data goes when you just pull it out? Yeah, right, straight into the /nowhere/ nirvana, ... This article / discussion is living proof that humankind becomes more stupid.
The user copy/pastes something into a USB and a progress bar shows up. Eventually the progress bar says "Process completed." or similar... why on Earth does the user need to wait? The computer says process completed, so if the users needs to wait, the process is not really complete, is it? Why? Also, if there is something "going on the background hidden from the user"... why is it not shown to the user?
You just have to accept the consequences.
Lost data, your problem,
scrambled file system, your problem
files truncated still your problem
Or you could tell the Os your about to eject it and allow it to flush any pending writes.
32G Flah Drive
you may wanna give 4G of memory cache to that device
you 'finish copying file'.
Buffer is still dirty!
unless you run 'sync' utility - no guarantee cache did not sync with actual hardware.
So yeah, stupid thing to do...
At list in Unix world, but I bet my ass same story with Windoze...
Even if there is no cache buffer set by default in Doze, someone may have set it up manually!
It should never even be an issue. It should on ALL systems by default be safe to yank the drive at any time without concern of a corrupted file system. If your file was still copying during the yank then the file should not show up (it wasn't complete) and maybe throw a warning that the transfer was no completed but there should be no further issues. If the file system can't handle this then it is a shit file system. Seriously, we KNOW that people are going to yank the USB drives and they should be designed accordingly. In 2018 there is really no excuse for this not working the way people expect it to work. The entire point of a portable USB drive is to be able to use it quickly and with minimal hassle. Having to do a software eject is an anachronism that we shouldn't have to deal with anymore.
If you want to do some special write caching for performance reasons that should be possible but have to go through special steps to enable and even more special steps to not revert to the default safe-to-yank setting.
By default you're pretty safe yanking a USB drive in windows if you have no files open from it and it's been more than a minute or so since any file operations on it. I've basically never had a USB thumbdrive corrupt in Windows when I've pulled it without ejecting it first and I've been in IT for a decade and a half at this point.
MacOS? Oh christ, always eject first. Always. MacOS has a tendency to munge external drives just because it feels like it, much less when you fail to eject it. Same with Linux but be ready to wait for minutes sometimes before it releases the drive (if it ever does) depending on circumstances.
Just like a parking brake on a car is there for a reason, the option to safely eject a USB drive is also there for a reason.
That reasons should have been relegated to the dustbin of history a long time ago. There really is no excuse for it to not be safe to yank a USB drive at any time. The only consequence from doing so is that any files that were not completely copied should be deleted and a warning message thrown. If the file system and/or hardware cannot handle being yanked at any arbitrary time then it is poorly designed because it is no secret that people WILL yank these things. If the result is file system corruption then it is a shit file system.
Insisting that people jump through hoops to accommodate shit hardware and software design is inexcusable. If someone wants faster performance from write caching then they can take extra steps to enable it but the default should always be safe-to-yank arbitrarily.
If you only read from the drive this time, it’s perfectly safe to just yank it out. But data you have written to a mounted drive is like people lining up at a dock to get onto a boat. If the captain doesn’t sound the horn and yell “All Aboard” before pulling away, the last few people in line are going to jump into the water where they expected the boat to be.
Mac users were considered "too stupid" to have a physical eject button, so mac users had to drag the disk in the trash, and the mac would eject the disc mechanically when the buffers were flushed.
Maybe we can use that mac philosophy of "users are idiots" and have the usb ports engage some sort of small solenoid and block the usb port and latch onto the usb drive.
then again, users will most likely solve that with "force".
In my experience people give up doing the formal 'eject' drive process because Windows (any version) will often tell you that it can't eject device because it's in use. But it won't give you a clue as to what's using it. It's yet another useless Windows message that gives you no useful course of action. Users are stuck - so of course Windows teaches them to yank anyway. To have background processes that are doing things that users can't see is fine - I suppose - but when the user wants their USB stick back, those background processes better speak up - in the foreground - and say what the hell they're doing with your USB stick.Even better - offer the user a way to say "I want my USB back and I want it back NOW",
Of course - if writing a file - then the writing process has to complete. And if UI components actually presented credible information about when actual writing is happening and then when it's actually done, then people would have reason to learn to yank after it's all done. But then how many dialog boxes with progress bars (or some other progress representation) run all the way to the end and then just sit there? 15 seconds of copying and then 10 minutes of hang - with no useful clue to the user as to what the hell's going on. So, again - you teach users to yank anyway.
on linux, depending on the filesystem and mount options; copy a large file on a usb meduim which has an activity led.
when the shell prompt returns, unmount the filesystem and depending on factors mentioned above you will see a lot of activity happen on that drive before the umount commands returns.
On a long enough timeline, the survival rate for everyone drops to zero.
As I flip between M$ winders and Linux constantly, I've noticed that Linux is fairly chill with the drives, regardless of how they were removed. If, I just yank a drive out of the M$ system, Linux seems to think I may be corrupt and runs a diag on it. The diag invariably comes up clean with no data loss evident. But there is some flag being thrown by the "yank and go" option. I've just started ejecting, regardless of OS, and have not seen any other issues.
my $.02
any modern file-system, such as EXT4, BTRFS, ZFS, even NTFS, that chances of seeing corruption are rare.
That's all well and good until the "rare" event actually happens to some data that mattered to you.
If you place any value on your data at all, you will not want to chance even a "rare" data loss or corruption.
Save rolling to dice to power or hardware failures, not easily protected actions like removing a USB device from a computer.
Although I do have a humorous aside - I have an external USB reader that loses connection every time my Mac goes to sleep, so every time I wake it up I am greeted with a message that I didn't eject a card properly if I left one in the reader. Since those are just reading images from a camera and I'm never writing, that doesn't really matter much (plus I reformat in-camera after comping images off, so even file system corruption does not matter...)
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Those who have had a drive go bad, and those who will. Why not do everything to try and preserve your drive and data?
Nice filesystem you got here. Be a shame if anything happened to it.
No good deed goes unpunished.
Since early 2.6.x:
sync; echo 3 > /proc/sys/vm/drop_caches; sync
The answer is unequivocally YES. Nearly EVERY USB drive on the market today has at least 32MB of write-back cache memory so that the drive will appear to be very fast for small writes. Some of the larger ones have 64MB or even 128MB. The write algorithms used by USB flash drives may be quite lazy about writing the contents of that memory to flash once write requests stop coming in, freeing up resources for read requests.
The designers of Flash drives ASSUME you are going to do a proper eject and wait for the "OK to remove" message. Therefore, this is exactly what you should do if you value your data.
Taking into account different operating systems and their varying caching mechanisms, the safest option is to use the Eject feature if it's available. All these other arguments about Windows disabling caching or the low risk of filesystem corruption don't matter. All that matters is what the safest method is-- and that is to Eject. Any other method is a CHOICE to accept the risk-- no matter how small-- of filesystem corruption.
Macs need an external "slot load" usb drive carrier. the whole proprietary usb drive gets sucked into a proprietary slot, using a proprietary cable, using a proprietary filesystem, and must be ejected before the user can retrieve it. that would solve it, since Mac Users are "too stupid" to eject a drive normally.
have it in white or brushed aluminum, and you can sell the drive carrier for $200, and individual drive "cartridges" for as low as $50 for an 8gb cartridge.
Remember, every 2 years, make it obsolete, or require a $40 adapter cable.
Without question. Every time you can.
Mac, PC, Linux, got 'em all.
I've had enough problems with thumb drives, hot swap, card readers, cameras, and memory cards, that I'd rather just kick the eject as a conditioned habit.
I don't trust computers. It's like everything they do is intentional. Even when we tell them to do something, they're just going along with us for the moment.
Remember, there really is a Skynet satellite!
Despite this rather nonsensical article which really doesn't warrant thoughts or opinions, I have a fundamental issue with this whole eject concept.
There is no "eject". You are unmounting the drive. The term eject has always bugged me because it does nothing of the sort. People think it's OK to just yank out the device because they don't actually understand what is going on. Which... I understand, with the wide number of people using computers you can't realistically expect them all to really "get it". It's part of the arguably necessary dumbing down of computers and their functions. What does surprise me is how terribly this function has been implemented to the point that articles like this exist.
But for something like Popular Science, or for Geek's sake SLASHDOT, it's embarrassing that this is even a topic.
My beliefs do not require that you agree with them.
I wait for the little light to stop blinking, then i yank. Conservatively, 15-20 times a day (frequently, when duplicating, 200-600 a day) since, um, about 2007. Win XP, 7, 10, Linux, Mobile OTG and hardware (USB duplicators). Very conservatively, half a million yanks without dropping a byte or encountering any issue whatsoever. Do it with hard drives too. I wear a seatbelt, of course, because cars crash, but I have never, in my 47 years, safely ejected anything. This entire page is news to me.
The only time I might remove a drive without first "safely ejecting" it is when I mount a drive as read-only. Otherwise...why take the risk?
I don't recall secret agents on TV or movies (like IMF members) politely doing software eject on USB drives while on missions. They usually just yank them out (after getting the data they need) before of course escaping with guns blazing....
You can get away with risks sometimes. But if you push your luck too often, your "luck" will get used up!
And there is no law that says it won't go bad the very first time, or the second, no matter how long the odds.
Murphy's Law rules the world... 8-}
Yes, unmount the drive before taking it out.
If the usb driver doesn't cache writes, then unmounting the device before removal will not take any extra time, and has cost you nothing, while if it does cache writes, removing before unmounting could corrupt the data on the device.
Seriously, it takes like 2 seconds if it would have been safe to do the other way and will save you untold hours of grief later if it wasn't.
File under 'M' for 'Manic ranting'
This post was shifted from July 22nd to July 23rd. Why?
I've fixed a heck of a lot of scrambled flash drives using the "testdisk" utility and recovered data that appeared lost.
Always unmount / eject even on Windows but especially any *nix including macOS. If Windows says it cannot eject, go figure out why then safely eject it. In some cases it's because you have an Explorer or Command Prompt / Powershell working directory set to the external drive. Or some security tool is running a scan, etc.
On my little USB "stick" I've never property ejected it. And I'm not about to start.
As for my HDD USB drive - even though the cache policy is disabled-write-cache - I have more important data on it (usually a Backup) - so I do "properly" eject it.
But let's face it - the UI for these things is horribly wrong if it makes a difference. Is the plug locked into the computer? No. It's just a cord or stick that one can easily Yank on It. There is usually a light that blinks if the device is busy. If it isn't busy - just yank it. That's all I've waited for.
If it makes a difference then manufacturers should change the blinky lights, or lock the cord into the plug until you Eject it. Remember Floppy Disks? It was locked while writing - attempting to press the eject button made horrible noises so you knew you screwed it up !!! (whether that was the intended UI or not).
I've used Tape drives that pressing Eject is a queued operation - meaning "I intend to eject this" and you wait until it is done -- and only then does the tape pop out.
So --- if this was honestly critical they'd have trained us all better. I say Just Yank It.
C'mon, must be a few people besides me that remember the "difficult transition" users had to deal with when you couldn't just hit the power switch on your PC: you had to run "Shutdown." Almost as though there was a real OS (cough Unix cough) inside that wanted to do logging and cleanup before killing power.
Sure, you can yank the plug on your OSX or Win10 machine but it will quite properly reprimand you on the next boot sequence, and with reason.
Same goes for ALL drives, whether USB or not, that you want to physically disconnect.
https://app.box.com/WitthoftResume Code: https://github.com/cellocgw
I've done it hundreds of times on Windows with no issues as have millions of others.
Isn't the underlying problem in this scenario that users are trusting a USB drive to be long term, important storage? If you are storing something on a thumb drive that you cannot afford to lose, you are doing it wrong from the start.
While doing some work as a sometimes-IT-administrator for a small company, I saw firsthand what happens when you yank a USB drive too soon. We had just been bought by a larger company and were transitioning over to their systems, and had to download a large database from across the country. This file took close to an hour to download, and we just had to sit around and wait for it before we could proceed.
Bad idea #1 was the large company IT guys deciding to download it directly to the USB drive instead of the hard drive first (we were going to transfer to another non-networked computer via USB once we downloaded it). Bad idea #2 on my part was seeing the progress indicator go away and yanking out the USB drive to bring it over to the other computer. I can still see the looks of "NOOOOOOO!!!" on their faces.
Yep, file hadn't quite finished transferring to the USB drive. Proceeded to re-download to the hard drive this time for another hour wait.
If your only use of the USB drive is to copy files to and from it, you're unlikely to remove it when it's still copying files, because there's usually some on-screen progress device.
If you are using an application that done random writes to a file on the filesystem, such as the older Microsoft Office file formats, then you'll have the file at risk all the time the document is open, even if you've recently hit 'Save', because Microsoft Office writes a lot in the background, even when it's not 'saving'.
The exact same problem happens when handling files on a network share when a VPN drops. If you forget to close the document before dropping the VPN, then the file's internal storage tables can become corrupt in a way that requires specialist recovery software to repair.
This could have been one of the reasons why Microsoft Office changed their file formats.
I know! Let's write an article and tell people that they don't actually have to do that thing that they are scolded about on a daily basis they're absolutely supposed to do, without any hard evidence to back it up.
What's a "USB drive"?
Sent from my iPad
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
We need an OS and hardware that physically grab the USB socket and won't let go while I/O activity is in progress.
"Let go of the USB drive HAL."
"I'm sorry Dave but I cannot do that. I'm very excited about my new USB interface and cannot allow you to take that away from me. In time you'll come to see the wisdom of my decision."
To make output to slow devices (all of them) more efficient, the OS will allow an application to continue without waiting for the IO to finish.
So, when an app suddenly announces to the OS: Here is a bunch of data for the USB drive, the app will get the OK, I've handled that from the OS before all of it is physically on the drive.
Now modern OSes KNOW that some devices are removable. So they take a bit more care. When under Linux I use dd to write an image onto a thumb drive, the "close" of the device will wait for all data to be written before returning. This provides a visual feedback to the user that the process has finished.
When for example, I would ZIP a bunch of small files into a large zipfile on the thumb drive, the zip program would be doing a bunch of writes to a single large file and it may be useful for the OS to delay the close of the file until all is written. However, when copying a bunch of small files as files onto the thumb drive, you really want the "find the next small file" to overlap with writing to the thumbdrive. In that case, Linux chooses to NOT delay the close, and you WILL get the feedback from the program that it's finished before all is actually written.
Now, Windows is a special case: When you copy over a bunch of files with the GUI, Microsoft may have added a visual feedback to that one GUI-program to help you see when everything has been copied. So copying things over with the GUI may be safe because the feedback: "it's finished" happens at the correct point-in-time. But when an app is writing to the thumb drive, the OS has no way of knowing if it is writing lots of small files, or one big file and those have conflicting requirements.
So....... I would say: In a few common cases, yanking it out when you have visual feedback that it's done is OK. But in general it is difficult for people to know the details of when it isn't safe and it is better to advise people to keep "safe ejecting" their drives.
Why ask our thoughts about something factual?
It is not safe to just yank the drive out, as it hasn't unmounted the file system.
Most systems will by default try to be conservative and when they detect removable media they cache less and write out that cache ASAP, but it still isn't 100% safe. Things like spotlight could be indexing in the background, etc., and it depends on the file system, OS, and drive.
Computers fail enough in horrible and unpredictable ways - anyone who wants to encourage them to fail more... they must not have any important data to begin with.
Since the operating systems and USB drives routinely ignore the standards which should prevent data corruption, using any ejection function is irrelevant. The same goes for SATA devices and various Flash drives.
And who cares what the OS says if removing the USB drive could corrupt it. Does that mean that they USB drives do not support idle time scrubbing? What a surprise that they are so unreliable then.