Slashdot Mirror
Data Loss Bug In OS X 10.5 Leopard
An anonymous reader writes "Leopard's Finder has a glaring bug in its directory-moving code, leading to horrendous data loss if a destination volume disappears while a move operation is in progress. This author first came across it when Samba crashed while he was moving a directory from his desktop over to a Samba mount on his FreeBSD server."
Normally while moving you ensure the copy completed before deleting the original. Apple must be using some discount programmers.
A feeling of having made the same mistake before: Deja Foobar
I lost a huge amount of data being MOVED (not copied) from one volume into a virtual volume DMG file. Lost and gone forever, lots of important files. What happened? I simply bumped the laptop Mac Book Pro during the move... zap... gone forever. The DMG file was blank! Yes, complely zero bytes except for a bit of header non-file data. It sucks bad.
...As if millions of fanboys suddenly cried out in terror and were suddenly silenced.
But entirely fixable.
Knowing it exists means you can also work around it by doing copies every time and only manually deleting the source entry when you're sure the copy has completed properly. Now all you need to do is be mindful every time you want to move a file anywhere.
Sucks to be a Mac user (I am one, I know) but I'm sure Apple will have this fixed pretty quick.
I drink to make other people interesting!
Exactly which decade did you fall into your recently awkened from coma in? Testing? Testing? Nobody tests anything anymore, they just go play with all the new toys and stare at the eye-candy. Actual mundane, humdrum testing? That's an SEP if ever I didn't see one.
A feeling of having made the same mistake before: Deja Foobar
All fanboys were just happy with how blazing fast file copy was compared to Vista. The non-fanboys tried to check the real size of the dir by copying it to a Vista machine afterwards, but the progress bar got stuck on 413 hours left and counting, so they couldn't file the bug in time.
This is the new Leopard "iLostMyFrigginFiles" feature, next version they will add a badass black hole effect when it does that!
No offense meant here, but normal move/copy operations are traditionally highly destructive events on MacOS anyway. For instance there is absolutely no simple way to merge two folders contents together on the mac. If you drag a folder called "Documents" into your home directory and click on "OK", the Mac OS will happily delete your entire documents folder. I was reminded of this enormous frustration while recovering from some multi-volume backups recently, having to resort to an obscure OS X commandline tool 'pax' and Leopard's newfound support of hardlinks to make some simple file copies play nice and not unnecessarily consume 3 times the disk space they should have.
For all of the flack the Windows file copy interface gets, it is both safer and more flexible than trying to use the Finder: an interface that makes file management so stupefying it becomes impossible.
Not to be glib, but.. This would be a great demonstration of the value of "Time Machine" backups. Time Machine is not perfect, but it is a good start on a backup system well integrated into the OS. The example problem, data loss, would be really easily recovered via Time Machine.
Beyond the basics that every decent backup app does, the things I like about Time Machine are:
- Integration into Applications. For example: "Show me what my iTunes library or iPhoto library looked like last Thursday"
- Integration into OS install. In the case of disk failure, recovery to previous state is simple - rather than multi-step with a separate backup app.
Some things that need improving:
- Better handling of file exceptions. I keep work data in encrypted disk volumes (DMGs). If I change one byte, the whole huge file needs to be backed up as each change is detected (generating MANY copies of that big DMG). The only other choice is to say "ignore this file/directory". Same thing applies to any large file, such as a VMware VM file. A better option would be to say "Back this file up, but only keep 'n' versions".
- Time Machine has gotten twice, pegging the CPU/fans on my MacBook Pro.
Unbelieveable. Forgot to check the result of the copy operation eh? So basically this is a catastropic defect for people who deal with very large media files to and from remote stores or people who deal with virtual machine images.
Back in the day when I used to use my mac I dropped a directory (A) into another directory (B) but there was an existing directory (C) with the same name as (A). The finder asked me something, I clicked OK. I was dismayed to find that the dialog had asked me "Would you like to replace directory C, with A?" - Why on earth would that ever be the default option for a directory move? From the users perspective you aren't really moving the directory, the intention is to move the files, thus the sane response would be to merge A with C not replace it.
Whatever.
Advert on Amazon Mechanical Turk:
Write OS-X compatible application to Move a file between two filesystem devices..
Time Allotted:: 6 hours. Reward: $10.00..
"A nation that forgets its past is doomed to repeat it." - Churchill
I guess, more bugs are to be revealed when the number of users continue to rise, but they will also be fixed, so the system will become more and more stable with the time.
The difference is, how the media and people in general reacts to an error of such a kind. Could yu imagine what people would scream and cry is Vista happened to just loss a bite of information? Oh, christ, I don't want to even THINK about this. Now, Abble, does that mistake, and... that's OK, nobody is perfect... Will get fixed.... People have double standards, but in the end, Vista was not THAT bad, and Abble's OSX was not THAT good. The true, is as always there, somewhere in the middle.
It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
Move is usually a destructive operation, on almost every platform I have ever used. Here are my experiences
Target diskete failure doing a move from C: to A: on DOS, yep your data is gone.
Network error moving from one windows box to another, yep your data is gone.
NFS write failure on Linux 2.4, check your data is gone.
Maybe move should be implemented as copy, completly then delete but its often not. I don't think there is any convention that demands it be that way. If you care about the data, at all you should always copy, check(maybe cursory, maybe md5 depending on how much you care) then delete.
I tell my users all the time, "move it only if you can lose it."
I don't think this is really a "bug" so much as a behavior, ie there is no handling of media exceptions when doing a move. Now if you data sometime went bye bye with two working devices that would be bug, and that is not what is being described here.
I don't think its fair to single MAC OS out for this either. As far as I know most mainstream OS seem to handle move operations with media exceptions badly. I am also not a MAC appologist. I don't have nor have every had any Apple products. Sure maybe the OS should copy, check at least no exception events happened durring and then delete but its not a bug. I don't think you can blame the OS for problems when the hardware under it be it a disk, NIC card, or memory flakes out. If it handles it gracefully then that is a virtue of the system, if does not handle it then its room for improvement in terms of features but not really a "bug".
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
If it's true that Apple's products seem less-tested these days, it's because they've tossed lots of seasoned customer quality-assurance testers to the wayside.
Many years ago, think System 7, Apple had this great Customer Quality Feedback (CQF) program. We tortured our systems between the alpha-testers and the great unwashed masses. There were Apple staff who (gasp) listened to our bug reports and got back to us reasonably quickly. It was grand.
Then someone got fired, or promoted, or whatever, and the CQF program got lost in the shuffle. Every few years I get an email from someone at Apple telling me that they're reconstituting it, but nothing ever comes of it, and - you know - it's hard to understand how they can ignore a free, highly-motivated bunch of fanboys.
I was born not knowing and have had only a little time to change that here and there. -- Richard Feynman
This may be a bug in the Finder thread code. Why?
Think about it: safe data movement has been around since filesystems existed. However, the new Finder is multi-threaded. It could be that the error handler is doing the wrong thing with the thrown exception...after all, what -do- you do with an exception in a subthread? What mechanism do you use to throw it upwards to the parent thread?
That's the joy of error handling, which is totally separate (though completely integral) to your normal architecture issues.
Okay, this is a terrible bug, but to me the real news is that the Finder has a move operation. I've only ever used the simple drag (ie copy) when transferring files to other disks. That might also be why I've never had this disconnection-delete problem.
The workaround is trivial - copy files until you're certain. In fact, I'd recommend doing that in all OSs anyway. Moves or cut-pastes are fraught with potential badness. I've lost files in Windows doing that, and always wondered what's wrong with just moving and deleting manually later on.
Why is every destructive computer bug that happens tagged with "haha"?
Data Loss Bug In OS X 10.5 Leopard
bug, macosx, apple, haha
Symantec Updates Cause Chaos in China
haha, security, bug, windows, feature
Banner Ad on Myspace Serves Adware to 1 Million
haha, myspace, pwnd, security, adware
Ubuntu May Be Killing Your Laptop's Hard Drive
linux, haha, storage, bug, spam
Islamists exploit buffer overflow, hack U.S. nuclear command; world doomed
eschaton, religion, waronterror, haha
OK, I made one of those up. But it doesn't even matter what OS or company is responsible for the problem - whoever makes the tags seems to take great delight in all computer snafus. How does the tagging system work anyway? It's always been mysterious to me.
Grr! Arg!
Someone should point out that by default, the Finder will copy files dragged to a different volume. You have to hold down a modifier (Command) to force a move operation. So while this is a severe bug that should be fixed immediately, it's much less likely to happen by accident than the article implies.
Hunter Kressel has been warning people about this for years now.
I don't always use unix-like operating systems; but when I do, I prefer FreeBSD.
You have to pay to be able to participate in the program
Wrong-o!! You absolutely do not have to pay anything to involved in the Appleseed program.
http://slashdot.org/~tf23/journal
This has been standard Mac OS operation for a long time. I know for sure that the same happens under OSX 10.4, and as I recall, older versions of the Mac OS (7/8/9) all did the same thing. Granted this is a stupid way of implementing file moving, the alarmist headline is a little unnecessary.
Go Here: http://developer.apple.com/bugreporter/ And File a bug. And They'll fix it.
There are 10 types of people in this world, those who can count in binary and those who can't.
That behavior is by design and has been the way Apple handles directory overwrites since 1985.
There's absolutely no reason that a modern "move" command shouldn't already be implementing the exact process you've just described, no matter where it is moving to.
You have to word the dialog box more precisely. "Do you wish to not allow data corruption or success on loss of destination meda?" Cancel? Retry? Abort? Ok?
There is a good chance the disk drive or the networked computer is giving the finder an "OK I got it" signal BEFORE the data is actually written to the physical disk drive. So it gives this signal, then the finder deletes the old data and then the disk drive "goes away" before the data is written out of the cache and to the drive. Don't blame Finder, blame those huge disk caches where many megabytes sit and wait to be written to the drive.
If this is true then the bug is im the copy of Samba running on the file server. We do not yet have enough information to know.
That's it, moving a file, network failure and poof, moved files deleted from the source. Most probably a bizarre coincidence, but after it happened a few times i now copy and then delete when working on a SMB network.
(Or maybe Samba has something to do with it?)
It's not logical at all. Why should moving one folder delete an existing one?
As for the warnings etc, how many users actually read those warning pop-ups? And if they do read them do you reckon they'd understand exactly what was meant?
Mac's are usually pretty good for usability. I'm surprised they'd make such glaring errors, such as not protecting a User's work, not letting them undo, and having strange behaviour which is difficult to learn on the basic file-system interface, which in theory any user should be able to figure out how to operate on the first go.
See: http://www.asktog.com/basics/firstPrinciples.html for more info on usability.
Simon.
Physicists get Hadrons!
In any case, you should be using Shadow Copy...er...Time Machine which would have protected you from going and losing track of your own files.
Great! So now not only don't we know where our data is but when it is. Perhaps in a week or two's time the data will materialize in the folder it was supposed to be moved to with an accompanying "whorping" sound coming from the speakers?
Speaking as one of those IT people, NTFS is probably one of the coolest pieces of software ever to come out of Redmond. ACLs, alternate data streams, directory junctions, single-instance stores, shadow copy, the list of useful features is huge. Even more surprisingly, it works pretty much as advertised. Frickin' cool.
There's another angle, though. On paper, Vista's NTFS-based backup technology walks all over Time Machine. However, the USABILITY of Vista's technology is crap. This morning, I enabled Time Machine by plugging in a USB drive and clicking "Use as Backup Disk" when prompted. To do restores, I launch the cleverly named "Time Machine" application. I've already used it twice today just because it's fun to watch the spacey animations.
Compare that to Vista's clunky "Backup and Restore Center", which you have to use if you want to backup your files on an alternate volume. I guarantee you that using "Backup and Restore Center" is beyond most average users. Sure, it might be "better", but what good is it if it never gets used?
This
How can you have a 5-digit UID and still have never seen the 17MB troll? Did you buy your account on ebay? P200. How old do you think the troll is?
Put identity in the browser.
Apple charges good money for Leopard, so let them fix it.
Finder, and any GUI parts of OS X in general, are not part of Darwin.
Someone already mentioned that the bug is not in copy or delete functions of the kernel - it is in the Finder GUI that is not open source.
https://appleseed.apple.com/cgi-bin/WebObjects/SeedPortal.woa/wa/faq
OR, there's:
http://developer.apple.com/products/select.htmlI don't think this is an OSX specific bug. Just this weekend I moved some video files over from a Windows desktop pc to my also Windows media pc (Windows based custom box) to a shared folder, then I went out for a tea, came back in, and I forgot the move was in process, and I switched off the media pc. About half an hour later I went back to my desktop pc, to see an error message about a failed write, then all the files I wanted to move have been deleted from the desktop pc. Checking the media pc, the first file was there, but damaged.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
First of all, files should never be deleted, they should only be hidden, unless the disk is full, of course.
Secondly, "move" across different devices should be copy and then delete.
Thirdly, if you copy a folder over another one with the same name, the computer should ask you what the purpose is: merge or replace? merge is often as catastrophic as replace if merging results in undesirable file combinations.
Forthly, files should be versioned by the O/S, as in VMS. It was a great feature, I don't know why it's missing from all modern O/Ses.
This is a bug with the FINDER ONLY, just so everyone is clear. The Unix "mv" command in Leopard is NOT affected by this.
http://www.macintouch.com Obviously this is the front page story today.
This only affects "move" operations between logical volumes. You have to hold down Command while you're doing this, inside the Finder, to get this to happen. Yes, it's a bad bug. It's not something, however, that you're going to run into if you're thinking sensibly about how you treat important data, or if you didn't know that the Command-drag functionality was built into the Finder (which I didn't, and I can't think of a time when I would use it now that I do, even if it was working correctly).
The fact that someone mentioned complexity associated with command-line actions means the conversation is over. Apple's model for file copy has NOTHING to do with the command-line, switches or flags. That complexity is abstracted from the customer.
rsync is VERY powerful. VERY powerful. In order to glean benefit of that power, you have to be educated about how to use it. Could Apple have defaulted the app to use the "include apple-specific-stuff?" Sure. Should they have? Probably.
Regardless, rsync is HARD and complex for "joe sixpack" and it's not simpler for him to run
rsync -e ssh --recursive -l -t -g --compress * joe@destination:dest_dir
than it is to run
rsync -e ssh --recursive -l -t -g --compress * joe@destination:dest_dir --includeAppleSpecificStuffHere
It's the same to them - all gibberish.
But Herr Heisenberg, how does the electron know when I'm looking?
The Finder is not open source and the problem is in the Finder.
FYI the kernel has a syscall called rename which only works if both source and destination are on the same file system. If they are not, it fails and sets errno to EXDEV. Moves between file systems are simulated by userland programs doing a copy then a delete. I'm pretty sure that cp issues a rename and then falls back to copy-delete if it gets EXDEV back. The Finder must already know that the destination is on a different file system because default behaviour is to copy when a file is dragged onto a different file system and move iif it is dragged onto the same file system.
All I want is a secure system where it's easy to do anything I want. Is that too much to ask ~~ Randall Munroe