Slashdot Mirror
Data Loss Bug In OS X 10.5 Leopard
An anonymous reader writes "Leopard's Finder has a glaring bug in its directory-moving code, leading to horrendous data loss if a destination volume disappears while a move operation is in progress. This author first came across it when Samba crashed while he was moving a directory from his desktop over to a Samba mount on his FreeBSD server."
I lost a huge amount of data being MOVED (not copied) from one volume into a virtual volume DMG file. Lost and gone forever, lots of important files. What happened? I simply bumped the laptop Mac Book Pro during the move... zap... gone forever. The DMG file was blank! Yes, complely zero bytes except for a bit of header non-file data. It sucks bad.
But entirely fixable.
Knowing it exists means you can also work around it by doing copies every time and only manually deleting the source entry when you're sure the copy has completed properly. Now all you need to do is be mindful every time you want to move a file anywhere.
Sucks to be a Mac user (I am one, I know) but I'm sure Apple will have this fixed pretty quick.
I drink to make other people interesting!
No offense meant here, but normal move/copy operations are traditionally highly destructive events on MacOS anyway. For instance there is absolutely no simple way to merge two folders contents together on the mac. If you drag a folder called "Documents" into your home directory and click on "OK", the Mac OS will happily delete your entire documents folder. I was reminded of this enormous frustration while recovering from some multi-volume backups recently, having to resort to an obscure OS X commandline tool 'pax' and Leopard's newfound support of hardlinks to make some simple file copies play nice and not unnecessarily consume 3 times the disk space they should have.
For all of the flack the Windows file copy interface gets, it is both safer and more flexible than trying to use the Finder: an interface that makes file management so stupefying it becomes impossible.
Why don't you just use rsync?
Give me Classic Slashdot or give me death!
Oh yeah, rsync. Is that one still broken on 10.5? Apple's build of rsync on 10.4 consistently choked on a few files when my dad started using it on his Mac Pro.
then some metadata on the disk is modified to reflect your move.
sigfault. core dumped.
Typically if you are moving within the same logical device the file pointer is moved and no copying need take place.
When moving to another device your code reads and writes, within a loop and traps exceptions (such as the device suddenly vanished, where the OS should raise an exception and your application traps it.) A wide variety of errors could occur while moving and in the event any of them happen the user should be notified in an appropriate manner and the original data not deleted.
I've written a number of applications which moved files or data between databases and it's fundamental your application is on the watch for any problems. Not to have an exception raised or to trap any and all, well, that's simply an inexcusable lapse.
This sort of thing is extremely critical if you happend to be defragmenting a disk drive. Long before Macs and PCs we had to defrag our mainframe drives and the applications which did the work were quite careful. Often the best practice, if you had the resource of a second drive, was to simply defrag to a new drive then re-assign the new dist as the original.
A feeling of having made the same mistake before: Deja Foobar
Someone should point out that by default, the Finder will copy files dragged to a different volume. You have to hold down a modifier (Command) to force a move operation. So while this is a severe bug that should be fixed immediately, it's much less likely to happen by accident than the article implies.
NFS write failure on Linux 2.4, check your data is gone.
Uhh no. In linux mv's always make sure the data moved then delete the original file (as far back as 2.4). You can test this by dd'ing a large file (use if=/dev/urandom), then run a md5sum on it, then do a mv within the same drive, over nfs, over samba/cifs, to floppy (not sure what happens here because linux caches writes to floppies until umount), to usb drive, whatever. While the mv is in operation just pull the plug on the system (target or source system). Your old file will still be there!
You have to pay to be able to participate in the program
Wrong-o!! You absolutely do not have to pay anything to involved in the Appleseed program.
http://slashdot.org/~tf23/journal
This has been standard Mac OS operation for a long time. I know for sure that the same happens under OSX 10.4, and as I recall, older versions of the Mac OS (7/8/9) all did the same thing. Granted this is a stupid way of implementing file moving, the alarmist headline is a little unnecessary.
Um... no. REALLY no. Please don't do that.
The syntax for "||" is:
If command 1 fails, do command 2 - otherwise exit (where you used "command1 || command2").
In this case, your command will either copy all the files from $from to $to or delete all files at $from.
What you probably meant is cp $from $to && rm -r $from, which only performs the second command if the first succeeded. This solution is far from perfect for reasons mentioned by other posters, but it's still significantly safer and more useful than yours...
In case you are serious about this question - I believe it is a reference to the Simpsons. A quick google found this...http://www.allposters.com/-sp/Nelson-Posters_i424903_.htm
Go Here: http://developer.apple.com/bugreporter/ And File a bug. And They'll fix it.
There are 10 types of people in this world, those who can count in binary and those who can't.
That behavior is by design and has been the way Apple handles directory overwrites since 1985.
Have you ever used a computer? Youre quite simply wrong. Ive had many move operations fail and never lost a file doing it. You can test by moving a large file, pulling the network cable out. The file is still on your PC. With your "bad disk" example, well yeah if the disk lies to the OS saying that files are copied when theyre not then that could happen. This would be very rare as bad disks tend to advertise rather loudly (to the OS) that they are going bad.
I'll just use my special getting high powers one more time...
You're an idiot. I have been using Linux for 15 years. I have had network shares die and even drives die while moving files in Linux and not once have I lost the originals.
In fact if you use verbose mode with mv you can see exactly what it's doing and it doesn't delete the old file until it confirms the new version was successfully copied. This is for volume-to-volume transfers, a move on the same volume is instantaneous and generally can not fail for most file systems (ext, xfs, etc).
That's it, moving a file, network failure and poof, moved files deleted from the source. Most probably a bizarre coincidence, but after it happened a few times i now copy and then delete when working on a SMB network.
(Or maybe Samba has something to do with it?)
right now version 2.6.0 is the only one worth using. It still can't copy pipes, and isn't smart enough to set ownership on symbolic links, but it doesn't mangle anything else.
You have to be using 2.6.0 on both ends too.
I work for the Department of Redundancy Department.
You want the resource fork... from Apple's rsync manpage: -E --extended-attributes copy extended attributes, resource forks
- passion
How can you have a 5-digit UID and still have never seen the 17MB troll? Did you buy your account on ebay? P200. How old do you think the troll is?
Put identity in the browser.
Finder, and any GUI parts of OS X in general, are not part of Darwin.
Actually I meant this but posted too soon. It pretty much guards against most things (the shell will do all the escaping necessary). The only thing i foresee a problem with is if names that are legal on one filesystem that are illegal on the other. I don't think rsync takes care of the either.
cp -aRp "$from" "$to" && rm -r "$from"
- A dialog appears with the title Moving... and it says it's calculating how much time it'll take. This took a while itself.
- Then the dialog says it's moving the files and the progress meter shows the progress
- After the progress meter fills up, it goes to empty again and then says it's deleting the files it moved
I did this with most of the folders until I reached one that had about 18GB of data in it. I performed the same procedure as above to start the move and after it was copying files for about a minute or two I realized that I was moving them to the wrong location (my desktop instead of a directory on my C drive). At work our desktops are retargeted to a file server and not our local hard drives. I knew that by copying to the desktop I would hit my quota limit on the server whereas if I had copied to the C drive like I did with the other folders I had no limit aside from the free hard drive space. Not copying there in the first place was just a mistake. I clicked cancel to stop the transfer. When I did that, the dialog that had the progress bar changed to the empty progress bar and said it was now deleting. By the time I realized what was going on, it had deleted that folder and all of its content from the FTP site.Now this could have been some weird bug or interaction between the fact that I was using a machine with SP1 instead of SP2, that my desktop and profile were retargeted to another machine, or that I was moving so much data. It wasn't a lot of files as these were data files for desktop publish programs for some brochures and catalogs, along with large print-ready images. I don't know what items worked together to cause the problem. I do know that it did happen and that I had to deal with the IT group at our other site (that had the Solaris server), open the helpdesk ticket to get them to restore the files from backup, wait for them to get around to it, etc. It was a huge pain and delayed what I was working on, causing grief for myself and my internal customer. One thing's for sure; I'll never use the built-in Windows FTP client again.
Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
https://appleseed.apple.com/cgi-bin/WebObjects/SeedPortal.woa/wa/faq
OR, there's:
http://developer.apple.com/products/select.htmlThere have been resource fork patches for some time, but somewhat unreliable. Version 3.0 is going to support resource forks (and other types of extended attributes) out of the box. The setup we have been using extracts resource forks to a separate file on the mac and then backs them up alongside the original files. The restore process just performs the reverse. It will be nice once we can switch to rsync 3.0 and get rid of that step.
I don't think this is an OSX specific bug. Just this weekend I moved some video files over from a Windows desktop pc to my also Windows media pc (Windows based custom box) to a shared folder, then I went out for a tea, came back in, and I forgot the move was in process, and I switched off the media pc. About half an hour later I went back to my desktop pc, to see an error message about a failed write, then all the files I wanted to move have been deleted from the desktop pc. Checking the media pc, the first file was there, but damaged.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
This is a bug with the FINDER ONLY, just so everyone is clear. The Unix "mv" command in Leopard is NOT affected by this.
http://www.macintouch.com Obviously this is the front page story today.
This only affects "move" operations between logical volumes. You have to hold down Command while you're doing this, inside the Finder, to get this to happen. Yes, it's a bad bug. It's not something, however, that you're going to run into if you're thinking sensibly about how you treat important data, or if you didn't know that the Command-drag functionality was built into the Finder (which I didn't, and I can't think of a time when I would use it now that I do, even if it was working correctly).
The Finder is not open source and the problem is in the Finder.
FYI the kernel has a syscall called rename which only works if both source and destination are on the same file system. If they are not, it fails and sets errno to EXDEV. Moves between file systems are simulated by userland programs doing a copy then a delete. I'm pretty sure that cp issues a rename and then falls back to copy-delete if it gets EXDEV back. The Finder must already know that the destination is on a different file system because default behaviour is to copy when a file is dragged onto a different file system and move iif it is dragged onto the same file system.
All I want is a secure system where it's easy to do anything I want. Is that too much to ask ~~ Randall Munroe