Data Loss Bug In OS X 10.5 Leopard
An anonymous reader writes "Leopard's Finder has a glaring bug in its directory-moving code, leading to horrendous data loss if a destination volume disappears while a move operation is in progress. This author first came across it when Samba crashed while he was moving a directory from his desktop over to a Samba mount on his FreeBSD server."
Normally while moving you ensure the copy completed before deleting the original. Apple must be using some discount programmers.
A feeling of having made the same mistake before: Deja Foobar
I lost a huge amount of data being MOVED (not copied) from one volume into a virtual volume DMG file. Lost and gone forever, lots of important files. What happened? I simply bumped the laptop Mac Book Pro during the move... zap... gone forever. The DMG file was blank! Yes, complely zero bytes except for a bit of header non-file data. It sucks bad.
...As if millions of fanboys suddenly cried out in terror and were suddenly silenced.
Exactly which decade did you fall into your recently awkened from coma in? Testing? Testing? Nobody tests anything anymore, they just go play with all the new toys and stare at the eye-candy. Actual mundane, humdrum testing? That's an SEP if ever I didn't see one.
A feeling of having made the same mistake before: Deja Foobar
All fanboys were just happy with how blazing fast file copy was compared to Vista. The non-fanboys tried to check the real size of the dir by copying it to a Vista machine afterwards, but the progress bar got stuck on 413 hours left and counting, so they couldn't file the bug in time.
This is the new Leopard "iLostMyFrigginFiles" feature, next version they will add a badass black hole effect when it does that!
Yes, that's easily avoidable. If you know it exists. And hopefully the first time you get bitten by this it isn't something critical. Would be garbage to have to work around that one.
TFA looked decent in it's details. Even step by step recreation. But it's a pretty serious bug, that as you mention, *needs* to be fixed quick and I didn't see any other sources confirming it.
No offense meant here, but normal move/copy operations are traditionally highly destructive events on MacOS anyway. For instance there is absolutely no simple way to merge two folders contents together on the mac. If you drag a folder called "Documents" into your home directory and click on "OK", the Mac OS will happily delete your entire documents folder. I was reminded of this enormous frustration while recovering from some multi-volume backups recently, having to resort to an obscure OS X commandline tool 'pax' and Leopard's newfound support of hardlinks to make some simple file copies play nice and not unnecessarily consume 3 times the disk space they should have.
For all of the flack the Windows file copy interface gets, it is both safer and more flexible than trying to use the Finder: an interface that makes file management so stupefying it becomes impossible.
Another thought springs to mind... is this even that critical? Doesn't leopard have the time machine in it? Can't you just go back and get your files out of the time machine if they were that important?
I haven't "upgraded" yet so I don't actually know much about this Time Machine thing and how it works.
I drink to make other people interesting!
Not to be glib, but.. This would be a great demonstration of the value of "Time Machine" backups. Time Machine is not perfect, but it is a good start on a backup system well integrated into the OS. The example problem, data loss, would be really easily recovered via Time Machine.
Beyond the basics that every decent backup app does, the things I like about Time Machine are:
- Integration into Applications. For example: "Show me what my iTunes library or iPhoto library looked like last Thursday"
- Integration into OS install. In the case of disk failure, recovery to previous state is simple - rather than multi-step with a separate backup app.
Some things that need improving:
- Better handling of file exceptions. I keep work data in encrypted disk volumes (DMGs). If I change one byte, the whole huge file needs to be backed up as each change is detected (generating MANY copies of that big DMG). The only other choice is to say "ignore this file/directory". Same thing applies to any large file, such as a VMware VM file. A better option would be to say "Back this file up, but only keep 'n' versions".
- Time Machine has gotten twice, pegging the CPU/fans on my MacBook Pro.
Unbelieveable. Forgot to check the result of the copy operation eh? So basically this is a catastropic defect for people who deal with very large media files to and from remote stores or people who deal with virtual machine images.
Back in the day when I used to use my mac I dropped a directory (A) into another directory (B) but there was an existing directory (C) with the same name as (A). The finder asked me something, I clicked OK. I was dismayed to find that the dialog had asked me "Would you like to replace directory C, with A?" - Why on earth would that ever be the default option for a directory move? From the users perspective you aren't really moving the directory, the intention is to move the files, thus the sane response would be to merge A with C not replace it.
Whatever.
Advert on Amazon Mechanical Turk:
Write OS-X compatible application to Move a file between two filesystem devices..
Time Allotted:: 6 hours. Reward: $10.00..
"A nation that forgets its past is doomed to repeat it." - Churchill
It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
You're asking if a bug wherein entire folder hierarchies can go *poof* in the event a network share drops should be considered critical? Are you serious?
If it's true that Apple's products seem less-tested these days, it's because they've tossed lots of seasoned customer quality-assurance testers to the wayside.
Many years ago, think System 7, Apple had this great Customer Quality Feedback (CQF) program. We tortured our systems between the alpha-testers and the great unwashed masses. There were Apple staff who (gasp) listened to our bug reports and got back to us reasonably quickly. It was grand.
Then someone got fired, or promoted, or whatever, and the CQF program got lost in the shuffle. Every few years I get an email from someone at Apple telling me that they're reconstituting it, but nothing ever comes of it, and - you know - it's hard to understand how they can ignore a free, highly-motivated bunch of fanboys.
I was born not knowing and have had only a little time to change that here and there. -- Richard Feynman
Why is every destructive computer bug that happens tagged with "haha"?
Data Loss Bug In OS X 10.5 Leopard
bug, macosx, apple, haha
Symantec Updates Cause Chaos in China
haha, security, bug, windows, feature
Banner Ad on Myspace Serves Adware to 1 Million
haha, myspace, pwnd, security, adware
Ubuntu May Be Killing Your Laptop's Hard Drive
linux, haha, storage, bug, spam
Islamists exploit buffer overflow, hack U.S. nuclear command; world doomed
eschaton, religion, waronterror, haha
OK, I made one of those up. But it doesn't even matter what OS or company is responsible for the problem - whoever makes the tags seems to take great delight in all computer snafus. How does the tagging system work anyway? It's always been mysterious to me.
Grr! Arg!
I agree. I always copy then delete, especially when dealing with network shares.
Despite its many shortcomings, Windows ME (*NOT* the more recent ME2) had this truly wonderful feature where if you delete something from a network share it would *copy* the data across the network into your trash folder.
Really handy when you delete 10G of data on a network share and your local hard drive has 5G available and you are on a 10mbps network.
In the free world the media isn't government run; the government is media run.
Someone should point out that by default, the Finder will copy files dragged to a different volume. You have to hold down a modifier (Command) to force a move operation. So while this is a severe bug that should be fixed immediately, it's much less likely to happen by accident than the article implies.
I call bullshit on Windows and Linux, and I'm pretty sure you're wrong even for DOS.
Maybe move should be implemented as copy, completly then delete but its often not.
Why on earth wouldn't you? Doing it any other way is not only obviously dangerous, it's far harder to implement! What would you do, map the file on disk, unlink the file and then copy and wipe each raw disk block? I could see doing something crazy like this in specialized applications, where freeing storage at the source is priority one, but in the general case it's insane.
The Apple issue is clearly just a stupid exception handling bug.
NFS write failure on Linux 2.4, check your data is gone.
Uhh no. In linux mv's always make sure the data moved then delete the original file (as far back as 2.4). You can test this by dd'ing a large file (use if=/dev/urandom), then run a md5sum on it, then do a mv within the same drive, over nfs, over samba/cifs, to floppy (not sure what happens here because linux caches writes to floppies until umount), to usb drive, whatever. While the mv is in operation just pull the plug on the system (target or source system). Your old file will still be there!
You have to pay to be able to participate in the program
Wrong-o!! You absolutely do not have to pay anything to involved in the Appleseed program.
http://slashdot.org/~tf23/journal
is this even that critical? Doesn't leopard have the time machine in it? Can't you just go back and get your files out of the time machine if they were that important?
Only if a backup of the files was run, which is a requirement of Time Machine.
If this was Vista, then there would be a good chance to use 'previous versions' to recover the folder data, as it does not 'solely' rely on external backup for timeline file recovery. Vistas use volume level file version snapshots(a feature of NTFS that HPS+ doesn't support), so there are backups even on the drive if it hasn't ever been backed up.
(Remember Time Machine usually only runs once an hour, and all versioning or changes made in that hour are never kept or tracked.)
-PS Not trying to troll, but this is a perfect example of the difference between Vista's previous versions and Apple's Time Machine I have tried to point out in the past.
Vista does both volume level backups and external backups, unlike just external backups like Time Machine does.
See why IT people prefer Vista's method, even if a backup hasn't run that hour, even users themseleves can access files and folders rather easily that got deleted or changed. And since this has been around Since Windows 2003 Server, in corporate environments, even XP users accessing 2003 servers have had this feature for over 4 years now.
Vista's claim to fame is that it enables these features on the local hard drive, and also integrates with the Vista backup system, so file versions appear from the backups in addition to the 'snapshot versions' on the main hard drive.
Go Here: http://developer.apple.com/bugreporter/ And File a bug. And They'll fix it.
There are 10 types of people in this world, those who can count in binary and those who can't.
Did you miss the part where Leopard reported an error on the copy? (No, really... I'm not being snarky, did you actually miss that screenshot?) Leopard correctly recognized that the transfer had failed, and still deleted the source.
To me, that's a pretty clear bug. What stuns me is that Apple isn't using the underlying "mv" command - since it should certainly deal with this situation. They rolled their own defective version.
Never re-invent the wheel. You'll have a square wheel with 13 lug nuts of all different sizes. Just go to Goodyear and take the tried-and-true.
--Somebody infect me with a
There's absolutely no reason that a modern "move" command shouldn't already be implementing the exact process you've just described, no matter where it is moving to.
You have to word the dialog box more precisely. "Do you wish to not allow data corruption or success on loss of destination meda?" Cancel? Retry? Abort? Ok?
There is a good chance the disk drive or the networked computer is giving the finder an "OK I got it" signal BEFORE the data is actually written to the physical disk drive. So it gives this signal, then the finder deletes the old data and then the disk drive "goes away" before the data is written out of the cache and to the drive. Don't blame Finder, blame those huge disk caches where many megabytes sit and wait to be written to the drive.
If this is true then the bug is im the copy of Samba running on the file server. We do not yet have enough information to know.
That's it, moving a file, network failure and poof, moved files deleted from the source. Most probably a bizarre coincidence, but after it happened a few times i now copy and then delete when working on a SMB network.
(Or maybe Samba has something to do with it?)
Error code 51 doesnt neccesarily mean the transfer failed. It can also mean lost connection to the share - which could have happened (in OSX's "mind") after it thought/was told the transfer was complete.
For instance, what if in stopping the share on the Windows session (incorrectly listed in the /. article as a FreeBSD Share), the SMB crap in Windows is "cleaning up" while shutting down and generating a "complete" code that it sends back to the Apple machine, which then in trying to communicate with the share, realizes it no longer can, and generates the correct error code as noted (51).
OSX did NOT generate a disk write error or any other error that would have been more applicable (like 43 or a bunch of other possibilities). It generated an error stating it lost communication with the other machine.
Again, still not enough info without knowing more about what the WinShare does when closed.
StarTrekPhase2 - The Five Year Mission Continues!
Simon.
Physicists get Hadrons!
In any case, you should be using Shadow Copy...er...Time Machine which would have protected you from going and losing track of your own files.
Great! So now not only don't we know where our data is but when it is. Perhaps in a week or two's time the data will materialize in the folder it was supposed to be moved to with an accompanying "whorping" sound coming from the speakers?
Speaking as one of those IT people, NTFS is probably one of the coolest pieces of software ever to come out of Redmond. ACLs, alternate data streams, directory junctions, single-instance stores, shadow copy, the list of useful features is huge. Even more surprisingly, it works pretty much as advertised. Frickin' cool.
There's another angle, though. On paper, Vista's NTFS-based backup technology walks all over Time Machine. However, the USABILITY of Vista's technology is crap. This morning, I enabled Time Machine by plugging in a USB drive and clicking "Use as Backup Disk" when prompted. To do restores, I launch the cleverly named "Time Machine" application. I've already used it twice today just because it's fun to watch the spacey animations.
Compare that to Vista's clunky "Backup and Restore Center", which you have to use if you want to backup your files on an alternate volume. I guarantee you that using "Backup and Restore Center" is beyond most average users. Sure, it might be "better", but what good is it if it never gets used?
This
How can you have a 5-digit UID and still have never seen the 17MB troll? Did you buy your account on ebay? P200. How old do you think the troll is?
Put identity in the browser.
Apple charges good money for Leopard, so let them fix it.
Finder, and any GUI parts of OS X in general, are not part of Darwin.
Someone already mentioned that the bug is not in copy or delete functions of the kernel - it is in the Finder GUI that is not open source.
https://appleseed.apple.com/cgi-bin/WebObjects/SeedPortal.woa/wa/faq
OR, there's:
http://developer.apple.com/products/select.htmlActually, Time Machine runs every two minutes. You can see a countdown timer in the system preferences. Also, Time Machine is dodgy by itself. Apple confirms that after backing up about 10 GB of data, Time Machine may stop working. See here: http://docs.info.apple.com/article.html?artnum=306932
This is a bug with the FINDER ONLY, just so everyone is clear. The Unix "mv" command in Leopard is NOT affected by this.
http://www.macintouch.com Obviously this is the front page story today.
This only affects "move" operations between logical volumes. You have to hold down Command while you're doing this, inside the Finder, to get this to happen. Yes, it's a bad bug. It's not something, however, that you're going to run into if you're thinking sensibly about how you treat important data, or if you didn't know that the Command-drag functionality was built into the Finder (which I didn't, and I can't think of a time when I would use it now that I do, even if it was working correctly).