Slashdot Mirror
Microsoft Drops 'Safe Removal' of USB Drives As Default In Windows 10 1809 (betanews.com)
Mark Wilson writes: Since the arrival of USB drives, we have been warned that they need to be 'safely removed' using the correct method in Windows, rather than just being yanked out — but now this changes.
With Windows 10 1809, Microsoft is changing the default setting that's applied to USB drives and other removable media. The change means that the default policy applied to removable storage devices is Quick Removal rather than Better Performance — so you can now just pull it out without a second thought.
With Windows 10 1809, Microsoft is changing the default setting that's applied to USB drives and other removable media. The change means that the default policy applied to removable storage devices is Quick Removal rather than Better Performance — so you can now just pull it out without a second thought.
Finally this annoying stupid misfeature will go away. Now if they only could with the horrible fake scanning for "fixing errors" after you've used a USB stick on Linux...
Sometimes customer and software availability does not give you a choice. But you already knew that answer. Being opposed to Windows use for anything is fine, I am too, but your childish stance is harming the cause.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
"Quick removal" means the OS will sync all data to disk BEFORE telling you the copy is complete. So if you wait until the OS says the data has been copied, you will be fine.
This is how floppy disks used to work. As soon as the copy completed the light would go out and you could eject the disk. It really should have been that way by default from the start with thumbdrives.
The way operating systems handle removable USB media is retarded. Data should never get corrupted when you remove a device; or if that's not possible, the OS should warn you upon disconnect, give you a chance to plug the device back in, and continue where it left off after you plug the device back in. Why can't this happen? Looking at Linux too. Why after an accidental connection/reconnection does the device not come back and continue operation as the same device? If a USB drive is accidentally disconnected, it's device (say /dev/sdb) is hung/deadlocked/otherwise fucked, and the device re-appears as a new device (say /dev/sdc) when reconnected, with no way to recover operations progress. File systems have to be remounted, checked, etc. It seems silly we are still dealing with stuff like this.
So think it through. People always just yanked the drive out. That is how they operate. Very few people used the safe removal function. Hell, I don't even use it and I am in IT. I just wait until the write should be done and yank the drive. So they changed it to make it more reliable in that mode of operation. Sounds like they CARE about your data. But then you probably knew that.
They aren't likely to unplug the drive during the copying. Before this change, Windows would copy the data to RAM and then lie and say the copy was done. Now it won't, and Windows will say the copy takes longer.
Just pulling it out is not safe. Protect yourself.
This essentially just shows that MS does not care about your data at all.
To the contrary, they are lowering performance to improve data safety.
A larger write will take time, and data will be corrupted if you just "yank it out".
Users yank it out anyway. This change will make it safer for them.
Since nobody uses thumb drives for high performance computing, this change is a sensible improvement.
AmigaOS had this feature, if you ejected a floppy that was in use it would tell you to put it back in and wait for you to do so...
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
And how would the average user know? This is an unsafe default, plain ans simple. It is asking for people to get hurt. It is exceptionally bad design.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I know that. You know that, But the average user? Not so much.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Autorun of has been disabled for many years on writable media like flash drives in windows, it only does it on read-only media like CD-ROM
I think the point is that writes are no longer cached, so copies and other writes will now take noticably longer, but when it's done it will actually be done.
I argue that this is how it should have been from day one, rather than the idiocy they had done up till now.
Maybe because there's a dialog telling the user that the copy is still going on?
so you can now just pull it out without a second thought.
I get that the writer is trying to provide a simple description of the changes, but that is not good advice. Honestly? Just "No." If you yank your drive out in the middle of a write transaction, you're taking your chances. Not having the caching enabled just reduces the risk. Besides, wasn't the default for removable devices always "Quick Removal"? I could be misremembering, but I believe that's been the default setting when I've inspected a device for as long as a care to remember.
MS does not care about your data at all
This would rather be good news from some point of view.
As for data corruption, some changes might be made to libraries, file manager and kernel FS subsystem in order to wait for the complete data flush to the removable device.
Much as I hate Microsoft, I don't think you are correct. I don't think you understand the change. If you read the MS page (not the summary, or the article on betanews), this become clear.
I think that what is changing here is that the OS will attempt to write to USB drives as quickly as possible, instead of caching the data and delaying the write in a manner that would improve performance.
The net result is that, at any given moment, your data is more likely have been written to a USB drive, and hence it is more likely to be safe to remove the drive.
The real "Libtards" are the Libertarians!
Use any filesystem that supports transactions, and you can have both safety and performance. It's especially easy to do for whole-file writes which are the usual way of using USB sticks.
Not sure if NTFS transactions work sanely, but if they do, here's compat with any version of Windows since Vista.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
Because people were pulling the damn USB sticks out anyway. They care, but they were tired of dumb people complaining 'why do I have to safely remove hardware when I've finished with it', not realising the OS hadn't flushed buffers to the device.
What they've done is disabled write-buffering by default; so the write won't report back to the software as 'done' until it's actually on disk. Thus, the 'file saving' progress bar in the software will take longer but is going to be actually flushed to disk when the application software reports "save finished'; whereas previously the application software would report 'saved' when it had simply been pushed to the write buffers of the OS.
Users know not to pull out the USB while their software is saving, but (in reality) they (incorrectly) assume they can pull it out once it has been "saved". Problem is, 95% of the time, you can get away with it.
This is actually the more conservative option, trading off performance for reliablity.
In unix/linux terms, they've turned on the "sync" option when mounting.
EVERY SINGLE WINDOWS since XP defaults to "Quick Removal" for ALL removable drives, including every iteration of Windows 10. I have yet to see a single computer running XP, Vista, 7, 8, 8.1, or any build of 10 where I plug in an SD card, USB flash drive, or USB hard drive that did NOT automatically default to "Quick Removal." I have ALWAYS, as in 100% of cases, had to manually switch the performance setting through Device Manager. Anyone who says that the default policy is different is flying directly in the face of every single computer I've ever plugged a USB or memory card storage medium into over the past 17-18 years, and that's literally thousands of machines.
The only exception is when a drive is not the system drive but is connected to an internal potentially hot-swappable interface such as an AHCI SATA port. Those get set to "Better Performance" by default because they're almost always not in a removable tray nor connected by eSATA, even though they're technically hot-swappable. Of course, that's not what this Slashdot post is talking about at all, so again...WHAT IS THIS POST EVEN TALKING ABOUT?!
So did early macs. It was particularly fun if you only saw the message *after* you'd done passed the disc to a friend, who had, say, completely overwritten it.
Mac: please insert disc x.
User: but it no longer exists.
Mac (refusing to do anything else): please insert disc x.
User: I can't!
Mac (still refusing to do anything): please insert disc x.
User: Oh ffs... here's the disc, though you probably won't recognise it.
Mac (still stuck): please insert disc x.
User: Oh ffs *yanks cable*.
The point is that when the dialog indicating a copy disappears, then the copy is complete. Since Windows 95 that hasn't been the case, even on slow media like floppies, Windows would have a write cache and finish a write when the system was "less busy".
Custom electronics and digital signage for your business: www.evcircuits.com
When you look at how Microsoft implemented USB initially it's really painfully obvious that they didn't expect the "surprise removal" failure mode. Not just USB drives, things like USB serial ports would screw up pretty badly if not closed and ejected properly first.
Worst still they decided to use FAT as the filesystem, probably because that's what they had. FAT lacks journaling so a surprise disconnect can corrupt it easily. Their crappy solution was to have two copies of all the important FAT data so usually at least one would be uncorrupted, but of course you had to run fdisk to figure out which one.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Your sarcasm detector is broken.
I have done this hundreds if not thousands of times on Windows and I've yet to experience any data corruption. Do that on "user-friendly" macOS and it has a nervous breakdown.
So what do you expect then, for MS to magically lock the USB drive into the port while the copy is happening? The setting they are making the default is the safest of the two options to use. In what way does this "shows that MS does not care about your data at all."??
I browse on +1 so AC's need not respond, I won't see it.
It's not really practical today.
AmigaOS could do that with floppies because back then the computer was 100% in charge of the drive and knew exactly what had gone through and what not.
A modern disk runs an entire OS of its own, and very possibly lies to the OS about its internal state, because lies look better on benchmarks. Just because the drive says "this has been saved", doesn't necessarily it has been.
That means the OS can't really do what you want reliably. It might work with some drives, and fail miserably with others.
If every hard disk was truthful about what's on the platter, and every SSD had the capacitors needed to finish work, this would work nicely. But we unfortunately don't have that.
Windows 98 killed my pappy!
Stop complaining about how windows worked 15 years ago.
Socialism: a lie told by totalitarians and believed by fools.
well for example, EVERY cluster you write in a new file needs to mark that cluster as used, and that's a change in the free space map and the free cluster count, as well as the cluster list and directory's size indicator for the file. Obviously, updating it every time you write one more cluster would be grossly inefficient, so it's done in blocks, hopefully adjusted to the write speed of the device, so that most of these things are only updated maybe once a second. So if you yank a drive while it's saving a file, it may have written 150 clusters, but only 140 of them are accounted for, but hopefully it's accounted for EVERYWHERE. (since it can't update the cluster map, file size, etc etc all at the same time, you could easily pull it DURING one of those saves, and only get SOME of them done)
Journaled file systems try to cover for this by doing things in stages, where each stage is noted, performed, and completed, before moving to the next. If the device is pulled, and reattached later, it can see the "open transactions" and roll them back, so the file system is consistent again. (even if you do lose a little data)
I've seen older systems though that clearly have a long delay on updates. I'd save a file, and see it flash and flash and flash, and then go dark. and then like 5 seconds later, another little burst of flashing. I'm sadly assumign that was a final write operation, that if I had unplugged it when it stopped flashing, would have messed something up.
I work for the Department of Redundancy Department.
and manufacturers didn't want you to know that. These days they're fast enough it doesn't matter, so Microsoft can drop the charade.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
A smart programmer would cache the copy and even IF the use yanked the USB out, the program that I wrote would know to warn that the data isn't completely copied and if they stuck it back in, it would pick up where it left off and copy the rest. Today's DVD players do something similar when you yank a DVD in the middle of play.
That's what I/we did way back when. But hey, I'm just a programmer and not an "engineer" or a "scientist". So, ignore what I say about some stupid issue that was solved 30+ years ago.
My bad.
Gee, if only Windows would throw up a prompt if you pull the drive out during a copy. Maybe title it "Interrupted Actions", explain that the copy was interrupted because the drive was removed, and give the user options like Try Again, Skip, or cancel. And if you put the drive back and click Try Again, it would continue the copy.
I browse on +1 so AC's need not respond, I won't see it.
No it was fixed a very long time ago
That may well be true, but Microsoft's implementation of how I can change the option leaves a lot to be desired.
You can change the policy setting for each external device, and the policy that you set remains in effect if you disconnect the device and then connect it again to the same computer port.
I have multiple ports on my machine. I may plug a drive into any of them. I want a device to be treated the same regardless of which port I happen to plug it into "this" time. But MS has decided that I have to set things for each port separately. Why? (and this isn't the first time I've seen that the same device in different ports is treated a as different object).
I find it easier to use the sysinternals command line tool handle rather than procexp for this use.
C:\Temp>handle /?
Nthandle v4.21 - Handle viewer
Copyright (C) 1997-2018 Mark Russinovich
Sysinternals - www.sysinternals.com
usage: handle [[-a [-l]] [-u] | [-c [-y]] | [-s]] [-p |] [name] [-nobanner]
-a Dump all handle information.
-l Just show pagefile-backed section handles.
-c Closes the specified handle (interpreted as a hexadecimal number).
You must specify the process by its PID.
WARNING: Closing handles can cause application or system instability.
-y Don't prompt for close handle confirmation.
-s Print count of each type of handle open.
-u Show the owning user name when searching for handles.
-p Dump handles belonging to process (partial name accepted).
name Search for handles to objects with (fragment accepted).
-nobanner Do not display the startup banner and copyright message.
No arguments will dump all file references.
For example:
C:\Temp>handle VBoxSharedClipboard
Nthandle v4.21 - Handle viewer
Copyright (C) 1997-2018 Mark Russinovich
Sysinternals - www.sysinternals.com
VirtualBox.exe pid: 6088 type: File 50: C:\Program Files\Oracle\VirtualBox\VBoxSharedClipboard.dll
VirtualBox.exe pid: 8008 type: File 50: C:\Program Files\Oracle\VirtualBox\VBoxSharedClipboard.dll
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading
Windows in a VM requires twice the RAM of Windows on metal. If you use a VM, you need RAM for both the host operating system and the guest operating system. If your laptop is already maxed out, that's not fun. Plus you're still paying $200 for a Windows 10 Pro license, plus data overage fees to your ISP when the PC downloads big updates twice a year on Microsoft's schedule.
It takes a dumbass to recognize one.
Because the manufacturer of the device cheaped out, and didn't give each device a unique serial number. The OS can't really tell the difference between different devices.
Transactions don't help if the OS claims the write is done before the write has been committed to hardware.
This isn't a transaction problem. It isn't a file system problem at all. It's a write-cache problem. In other words, a problem in the communication between the OS and the physical hardware. In things like hard drives, write caching makes sense for performance reasons, and literally every OS does it. But when the OS treats the USB as a temporary hard drive instead of as proper removable media, and delays the final writes, you run into the inevitable issue where someone pulls a USB prematurely and corrupts the write because they have no way of knowing that the write wasn't actually completed.
IMO USB is also partially to blame in this issue. It should be mandatory for USB keys to have LEDs that indicate activity.
Ummmm
[($)]
I yank mine constantly.
Don't we all. It just feels so good.
that you even know about "Try again, Skip or Cancel?
I get this condition, especially after editing a large MS-Word file on a USB drive and exiting, that Windows never tells you it is OK to pull out the drive.
I am in a hurry, late for a meeting or to catch a ride home, and I never get the "Safe to Remove" message. So I end up having to do a Shut down on the computer to remove the drive when Windows has saved ev-er-y-thing it has cached, but wouldn't you know it, I have to wait for an Update to complete before Windows even shuts down.
This is when one wants to throw the computer out the window, but I never know if it is OK to yank the USB drive before doing this?
Whether it is safe to just yank your USB drive out at any random time, or equivalently, unexpectedly lose power, depends on whether the file system employs an accurate, reliable atomic commit, such that even if you pull the drive out in the middle of a DMA transfer, the data and metadata on the drive will always be consistent as at some recent point in time, even if incomplete.
You can rest assured that Microsoft has no such thing, but evidently that does not slow them down a bit.
When all you have is a hammer, every problem starts to look like a thumb.
If there is a RAM cache in the drive, then the computer may not even know that the data has not yet been committed to the disk. So It can't warn the user. And this is unlikely to change.
But this change is not to allow users to remove the disk, it is to make it less likely to cause problems. It means that the computer will make sure that, as far as it is aware, the information is written to disk before telling the user that it is done - by closing the explorer copy animations, or by signaling to running programs that the write is done so they can complete, by closing the program or the save dialog. With USB hard drives, they signal that it is done while much of the data is still in the O.S.'s cache, which is a good idea for fixed disk but not for removable ones. USB flash drives are already used in this way, this is extending this to USB hard disks.
Prediction for end of Universe #42: Fencepost error in Quantum_bogosort.cpp
The current default treats USB connected hard drives like internal drives, and assumes that they will be safely shut down before removal. This allows the computer to store data temporarily in the computers memory, allowing the programs to continue while it writes out the data in the background.
This change assumes that the drive could go away at any time, so makes sure that data is written to disk before the applications close or the animations disappear. But you can still get caught out if there is data in the devices internal RAM cache, and 'safe removal' will still be recommended.
Prediction for end of Universe #42: Fencepost error in Quantum_bogosort.cpp
This change extends that behavior to USB connected hard drives.
Prediction for end of Universe #42: Fencepost error in Quantum_bogosort.cpp
Write protect tabs as standard would have been nice, too. Floppy drives had that feature even before viruses and malware even existed.
The real problem is that Windows writes to drives all the time by default (updating metadata such as "last accessed"). I hate having to mess with the automount settings every time I need to do some data recovery, because otherwise Windows will trash everything in sight once it gains write access to any device, even if the filesystem is known to be unclean. I once plugged a Win10 formatted hard drive into a Win7 machine, and in an instant, every file was invisible because Win7 can't handle Win10 security descriptors correctly, and promptly updated the filesystem to lock out every file. It took me hours on a Win10 machine to fix the mess.
I doubt changing the "safe removal" policy will accomplish anything. Flushing buffers as quickly as possible doesn't help much if your philosophy revolves around constantly writing data all the time when it's unnecessary, thus, there's always something in the damn buffers!
Look be fair, just like that other user, I have also felt the impact of the questionable functioning of M$ write behind caching, where it pretends to write data to any media and well, any tiny hiccup and that data gets written all over the place or not at all or anything in between. The only safe way to run M$ write behind caching, turn it off, seriously. It just did not work that well or reliably and that is very problematic.
I found that windows machines run a whole lot more reliably when 'ALL' write behind caching is disabled, a bit slower but way more reliably. Touch wood, I have never had to reinstall the really old windows installation, the only time ever, every other version of windows reinstalled repeatedly. I think it was windows ME that had a bug fix for it's version of write behind caching, it didn't fucking fix it, it just disabled it, seriously that was the bug fix, to disable a feature they fucking advertised and still advertised it after the so called bug fix disabled it.
Chaos - everything, everywhere, everywhen
Worst still they decided to use FAT as the filesystem, probably because that's what they had. FAT lacks journaling so a surprise disconnect can corrupt it easily.
Even with journalling, you can easily corrupt flash media, due to internal operations of the flash media firmware (wear levelling). Also, USB sticks are optimized for the particular preformatted FAT system. It's best not to change it.
The default policy for USB devices has been to disable write caching and set the "Quick Removal" policy since the days of Windows XP. MS even published a lengthy document about it in 2001 describing how drivers would need to override this behaviour.
This has remained the default policy through Windows XP's various service packs, Windows Vista, Windows 7, and I figured maybe they introduced a regression at some point recently so I decided to plug a whole lot of devices into my vanilla Windows 10 pro 1803 machine. All USB devices be they USB2.0 memory sticks, USB 3.0 HDDs, or USB3.2 external SSDs were set to "Quick Removal" with write caching disabled as the default option.
Someone at MS doesn't understand how their own OS works anymore.
In which case they wouldn't be able to handle it in the same port either!
No, this is MS daftness.
Yes, I agree that these problems already had solution a lon....
Disk access failure.
Abort, Retry, Fail? _
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
The progress bar, like all progress bars, is a damn liar.
Wanna buy a shirt?
https://www.redbubble.com/people/stealthfinger/shop?asc=u
I am not an anonymous coward. Slashdot lets me turn off ads.
Better to phrase it "abort, retry, fail?" I think.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
It's still better than the current solution, where data might be cached in RAM to be written unto the flash driver later.
So this change is an improvement.
You missed April Fool's day by a week. Try to keep up.
That really only works for OS-initiated disk activity. Think about what would have to happen with applications, which can (as they should) access disks in any way their programmers need or want to. Also, where does the state for an incomplete transfer exist? At what point is it invalidated, and for what reasons? In the real world, this is a lot harder than you're thinking it is...
"The future's good and the present is nothing to sneeze at." - Roblimo's last
No, programmers should be using system APIs, not making shit up on their own.
I browse on +1 so AC's need not respond, I won't see it.
Be thankful you do not need to put it into a caddy before inserting it into the drive.
It's not just the OS lying about a write being complete. The hard drive controller can lie and say the write is complete when really just queued up to be written. Same for the hard drives themselves. Most modern hard drives have an internal write cache that's enabled by default.
Normally, it doesn't cause too many problems, but for things like transactionaly consistent databases, it's a really really big deal when the H/W lies about a write being complete.
At work, I actually had to test this using a remote logging disk write test driver. Under high write load, we could pull the plug and the test driver could detect that a write operation was logged to the remote machine as completed, but the write never made it to disks. This could cause a lost transaction log write would corrupt the database about 90% of the time (we used diskchecker.pl to test this. See https://brad.livejournal.com/2... ). We found the obscure disk controller command that turns off the hard drive internal write cache, and it completely fixed the problem.
Note that modern SSDs have this problem too. Some "enterprise quality" SSDs have a capacitor power the SSD for the few seconds it takes to flush it's write cache. (See https://ec.kemet.com/ssd-hold-... )
But MS has decided that I have to set things for each port separately. Why?
The USB standard specifically enumerates the same device on different ports differently. This is nothing to do with Windows. Plugging any device in any port on any OS will see it as a "new" device if never seen on that port before.
Indeed since Windows 95 that wasn't the case. Since Windows XP on the other hand it was. You're complaining about something that was fixed 18 years ago, which incidentally is how long this option that for some reason is incorrectly shown in the 1809 changelog has actually existed. And no they weren't fixing a regression, the default is the same on 1803.
But it is daft to treat that "new" device differently when it is clearly the same device.
In my experience, the command line won't return to a prompt until the copy is complete.
Cheap storage VM.
Yeah millions of lucky people do regularly without any negative consequences. I would suggest that your students have been very unlucky instead.