iTunes 2.0 Installer Deletes Hard Drives

Cheviot writes: "It seems Apple's new iTunes 2 installer deletes the contents of users' hard drives if the drives have been partitioned. I personally lost more than 100gb of data. More information is available at Apples Discussions board. (registration required). Apple has pulled the installer, but for hundreds, if not thousands, the damage is already done." The iTunes download page has a nice warning about the problem. Ouch.

Comments

How the hell does this happen?

[dimator]

Well, here's the pseudo-code:


if(installDrive->hasEnoughSpace()){
return startInstall(instalDrive);
} else {
installDrive->formatRecklessly();
return startInstall(installDrive);
}


Hard-to-spot bug, actually.

Re:How the hell does this happen?

[Jace of Fuse!]

No.

Sorry.

Extensive testing went out of style around the mid 90's.

Oh -- and actually writting REALLY good code, that's been out of style since at LEAST the 80's.

I blame colleges for letting so many graduates think that they are instantly coders.

Real coders are born, not made.

And anybody who got into computers for the money, and not the thrill of writing code -- well -- they're worse than lame. And they probably use AOL.

Re:How the hell does this happen?

[cshotton]

Apple has had a greatly diminished QA organization over the years. I think this is due in large part to the serious turnover the company experienced in late '97 - early '98. Nearly every engineer that worked on the original Mac OS left during this time. The Developer Technical Support organization was gutted, and quality assurance became an exercise in million monkey button pounding.

Having been the author of a 3rd party product bundled and shipped on Apple hardware, I can tell you that the extent of their QA process doesn't go much beyond making sure the software installs and runs on an out of the box system, followed by some mediocre mashing of buttons and menus. They really don't understand or implement the concept of actually testing on live, deployed, end user (like) systems. They have racks of off the shelf machines with standard software loads. If they install and run and stay up over the weekend, it's shippable.

We would get reams and reams of complaints about how dialog boxes weren't formatted just so, etc., but their QA department never caught a single defect that most would consider a bug in the code. And there were certainly bugs to catch.

This is a chronic problem that most commercial software houses have. They tend to put junior people with little product experience in the QA organizations and assume that by acting like reasonably competent users, they will somehow uncover logic flaws, data errors, and other engineering foibles. The only time I ever saw QA done right was on a NASA project with life critical software systems. The project was staffed with the very most senior engineers running the QA department and all of the junior engineers were slinging code.

It was up to the gray beards to make sure the junior guys wrote code that was to spec, integrated properly, handled all of the possible input scenarios, and actually performed in a live environment. These senior guys were also the architects of the system, so they knew what the software was supposed to do, how it was supposed to be constructed, and what it should take to break it. I doubt that 1 in 100 commercial shops today have an engineer working in the QA department that actually understands the code they are testing down to the module level. When was the las time you saw a QA guy in a design session, learning about how the system he's going to test is going to be architected?

This is so far from the current practice in commercial industry today as to almost have the flavor of a fairy tale. Apple's no different than any number of other companies who are rushing to ship software on a too short schedule. They pay lip service to QA and rely on their early adopter users to find any lingering problems. In this case, they totally dicked over their customers by not doing their job. However, they're only partially to blame since I think the development of iTunes is still done by Casady and Greene under contract to Apple. I'd be surprised if they weren't ultimately responsible for creating everything, including the installer. Regardless, Apple should have tested this before sticking it on-line on a Saturday night.

Re:How the hell does this happen?

[Phil Wherry]

Hmmm. I'm just expecting Apple to issue a press release soon that says something to the effect of, "in retrospect, perhaps we shouldn't have subcontracted the installation script to RIAA after all."

Re:How the hell does this happen?

[benedict]

I know plenty of people who program "for the thrill of writing code" and *still* can't code their way out of a paper bag.

It takes more than a love of computing to make a good programmer. In my humble opinion, it takes a fair amount of education. A brilliant but naive programmer can screw things up pretty impressively. I'd prefer to use code written by someone who isn't such hot shit but takes the time to learn APIs, read documentation, and familiarize himself with the idiom of the language and/or operating system in question.

New Apple Slogan

[Anonymous Coward]

Rip. Mix. Burn. Format. Reinstall.

Already updated

[Anonymous Coward]

Apple has already put iTunes 2.0.1 that purportedly takes care of the problem:

http://www.apple.com/itunes/download/

Corrected version 2.01 already posted by Apple.

[sakusha]

Come on now, Apple jumped on this one, it was only reported by a couple of people, and they corrected the problem almost immediately. This problem only came to light today, and they have a fix out the same day. I downloaded the new 2.01 version, installed with no problems.

The bug

[hysterion]

Summarizing discussions on MacNN and the Apple Forum:

The problem appears to be in two portions of the installer script which could translate into rm -rf /your_drive, if certain paths $1 or $2 contain spaces:

#!/bin/sh

# if current iTunes pkg exists, delete it b/c of Installer bug
if [ -e $1Library/Receipts/iTunes.pkg ] ; then
rm -rf $1Library/Receipts/iTunes.pkg 2> /dev/null
fi

# if iTunes application currently exists, delete it
if [ -e $2Applications/iTunes.app ] ; then
rm -rf $2Applications/iTunes.app 2> /dev/null
fi

Though when I looked, nobody seemed to have found where exactly $1 and $2 are defined; also it might be that disaster only strikes with localized versions of the OS.

Re:The bug

[benedict]

Your post reminds me of a conversation I had with an Apple employee. He shared your opinion of the Unix tools. I challenged him to come up with a replacement syntax for the shell that wouldn't have any problems with spaces.

After I shot down his first half a dozen proposals, he started to gain an appreciation for the difficulty of the problem.

So how would *you* rewrite the shell to get rid of problems like this? Be specific. Remember, in 30 years of unix, no one has found a solution to this problem that doesn't break more things than it fixes. So if you do so, you'll be famous.

I wait with bated breath.

Nature of the bug

[HalfFlat]

From the discussion on the Apple discussion web site, the nature of the bug is as follows.

The original installer script has the lines

# if iTunes application currently exists, delete it
if [ -e $2Applications/iTunes.app ] ; then
rm -rf $2Applications/iTunes.app 2> /dev/null
fi

while the replacement (2.0.1) has

# if iTunes application currently exists, delete it
if [ -e "$2Applications/iTunes.app" ] ; then
rm -rf "$2Applications/iTunes.app" 2> /dev/null
fi

In these scripts, $2 corresponds to the volume on which iTunes is to be installed, and will be of the form /Volumes/name-of-volume.

For those unfamiliar with Bourne shell variable expansion, if $2 has spaces in it, the argument to the rm command in the first version of the script will expand to more than one word, and rm will try and delete both of these. The -rf tells rm to delete everything down recursively and not complain about it.

This is particularly a problem on the Mac, where filenames and volume names often have spaces in them., even at the beginning of the name. If one had multiple partitions mounted in /Volumes, and the one on which iTunes was to be installed was called, say, ' OS X', then the rm command would expand to

rm -rf /Volumes/ OS X/Applications/iTunes.app 2> /dev/null

and would then try and delete everything under /Volumes. This is clearly bad.

The second version, by including quotes around the argument, fixes the problem. The quotes force the argument to be treated as a single argument after variable expansion.

Traditionally, people have been super careful about destructive operations and shell expansions. I don't think I've ever seen something like this written in a 3rd party script before, in fact (let alone from the OS vendor!). This could well be an example of programmers new to a Unix-like platform still getting used to the Unix way of doing things, and getting bitten as a result.

quote

[Jotham]

the bug is apparently caused by the lack of quote marks in the install script.

Apparently it only strikes if you 1) havn't uninstalled iTunes first 2) have multiple partitions and 3) have spaces in the name of your partitions

This from MacSlash (posted by Graff as AC):

Well, there is a fixed installer up now. Looks like the following change was made to the "Preflight" file inside the "iTunes.pkg" package:

old version:

#!/bin/sh

# if iTunes application currently exists, delete it
if [ -e $2Applications/iTunes.app ] ; then
rm -rf $2Applications/iTunes.app 2> /dev/null
fi

exit 0

new version:

#!/bin/sh

# if iTunes application currently exists, delete it
if [ -e "$2Applications/iTunes.app" ] ; then
rm -rf "$2Applications/iTunes.app" 2> /dev/null
fi

exit 0

As you can see, they basically placed quotes around the file paths so that any characters such as spaces in path names would not mess up the rm command. So easy, and yet even the best of us forget to do it at times. That's one of the things about the command line - lots of power when used properly, but also many powerful ways to mess everything up.

- Graff

Re:quote

[ecampbel]

Quotes get evaluated first and only once, and then variable substitutions happen, so having quotes in your drive name will not cause problems.
For example (in csh):
% setenv foo \"
% echo $foo
"
% echo "some quoted text:$foo"
some quoted text:"

Everything within quotes is passed as one argument to the echo command, and the fact that $foo contains a quote character doesn't cause any problems. If the quotes were evaluated after variable substitutions, an unmatched quote error would have been reported in the last example.

Re:[PATCH] Re:How the hell does this happen?

[jesser]

There's a bug in you pseudo-code. Attached is a patch which fixes the problem.

It's pseudocode. You can't patch pseudocode. You have to pseudopatch it, like this:

- return startInstall(instalDrive);
+ return startInstall(installDrive);

or like this:

Replace instalDrive with installDrive.

Re:Users in OS X

[IronChef]

You can enable root login in the NetInfoManager. When you do, it asks you to put in a root password, and then you can log in or su to root if you want.

Re:The big mistake here...

[aberkvam]

It sounds like you aren't familiar with what iTunes does. iTunes isn't just a pretty version of WinAmp. It actually is a CD ripper and CD burner that handles audio and data CDs. It supports the CD-RWs that come with Macs as well as many third party CD-RWs In order to properly support CD burning it has to install drivers "outside its bundle". It has to change things outside of iTunes.app.

I will admit that most applications should just use the "drag and drop" installation and Apple agrees. But there are some cases where the use of an installer can not be avoided and this is one of those cases.

Re:Need to have a warranty!

[Macka]

Hmm .. I'm not going to waste a moderator point jacking this back up again, but this is NOT a Troll. The man has a point! This IS an Apple article, and MacOS X is supposed to be a click and point friendly face to Unix where you don't have to be a command line jockey to get things done.

Also, the 'read the code and fix it yourself' argument is starting to wear very thin, as both Linux and MacOS X can quite easily be found in the hands of non-techies these days.

To whoever mod'd this down as a troll .. shame on you, for not being more open minded!

Re:Need to have a warranty!

[Detritus]

You have an obligation to take reasonable precautions to protect the data on your computer. That means making backups of any valuable data. Are you going to sue Western Digital if your hard drive fails? What if it gets fried by a lightning strike? Even if Apple was found to be grossly negligent, they shouldn't be held responsible for data that was lost due to the negligence of the computer's owner.

bogus shell quoting rules

[mj6798]

This is really a problem in how the Bourne shell handles variables. Lots of UNIX scripts fail when file names contain spaces, which is why most people don't use spaces in file names.

The folks at Bell Labs seem to have realized that this was a mistake, which is why the "rc" shell (also available for Linux) now handles things differently: variable substitution does not result in re-tokenizing.

Re:bogus shell quoting rules

[dangermouse]

This is really a problem in how the Bourne shell handles variables.

Okay, there have been multiple posts to this effect, and they're all wrong.

Sorry, but this is not a "problem". The shell operates on lists of words. A variable is assumed to contain a list under most circumstances. List elements (words) are separated by an Internal Field Separater character, defined by $IFS.

Every single one of my Bourne-compatible shells' man pages explicitly state the types of expansion that are employed generally (including word splitting) and provide a short list of exceptions.

This is expected, normal, useful behavior. And it's easily handled, as you can simply wrap any variable in quotes if you don't want it to be tokenized as multiple parameters. If it's something you can't cope with, that's a shame, but don't act as though it's some flaw in the shell... it's there on purpose, and with good reason.

Re:bogus shell quoting rules

[dangermouse]

If variables really are lists (like you said), then $1foo means "concetenate foo to the last term in the list". How often do you want to do that? I'd bet that $1foo is more often typed by someone who didn't realize $1 could contain a space than by someone who actually wants to treat $1 as a list and concatenate foo to the last term. We just saw an example where $1foo not being a syntax error led to severe data loss. Can you give a reason why bash shouldn't issue an error or warning when it sees $1foo outside of quotes?

No, $1foo means "expand $1 and append foo". You, the intrepid Bourne scripter, should realize that $1 may very well contain IFS characters, recognize what that will mean if the string is retokenized, and quote appropriately.

Look at this:

VARIABLE="$VARIABLE $ELEMENT"

This is a common way of building lists in Bourne. The reason this is common is that it is understood that $VARIABLE will be tokenized when it is expanded, and word splitting on IFS characters will occur. It is also commonly understood that if you do not want a string to be word-split, you quote it so that its IFS characters are ignored. Like I said before, not hard.

I'm sorry, but it takes all of ten seconds to grasp the fact that variables will be expanded and their words split. Occasionally you take it for granted that a list will only have one element, and get away with stuff like $1foo. However, it's not something you do unless you have total control over the contents of $1 (as in, say, a function you've defined that only you will call), and even then it's bad practice.

That's not a justification any more than "Windows crashes because it dereferences dangling pointers" is a justification for Windows crashing.

I couldn't have said it better. The problem there is that someone unfamiliar with basics of pointer manipulation went and dereferenced a null one... the problem is not that the language let him have a null pointer to begin with.

Gotta be said

[Lord_Pall]

So i guess the Ipod/Itunes combo really IS a killer app.

Relative severity vs. a MS flaw

[jht]

Apple posted the initial update either late Friday or early Saturday (I'm not sure exactly when). It was pulled by late in the morning Saturday, they posted a warning shortly afterwards, and when I got up this morning there was a fixed installer online to use.

The Classic version (which most Mac owners are still running) was fine, and the bug seems to have only hit people who didn't follow Apple's instructions that said "remove the old one first" and/or had multi-partitioned drives (multiple partitions aren't nearly as common among Mac users as they are among Windows and Linux users).

So Apple made a gross mistake on one hand, but on the other hand they owned up to it quickly, pulled the offending installer, and fixed/reposted it less than 24 hours later. Most Linux vendors respond about as well, Microsoft usually doesn't (though they were very good about pulling, fixing, and notification with their recent RDP fix that knocked people's Terminal Server systems off the network entirely).

The other mitigating factor was that there aren't that many Mac users relative to the installed base who were affected by the bug - but unfortunately the people who were likeliest to be affected (users who are already running 10.1 as their base OS, have multiple partitions, and don't read the instructions thorougly because - after all - "it's a Mac, who needs instructions?") are exactly the kind of Mac "power users" who swarm Apple's servers constantly looking for new stuff and install it the second it's posted.

I run 10.1 on my TiBook 667, and I downloaded the update. But I deleted the old iTunes version beforehand and only have a single 30GB partition, hence the install went fine..

Re:Relative severity vs. a MS flaw

[nycdewd]

Tell it, bro. You have got it down, especially: "the people who were likeliest to be affected (users who are already running 10.1 as their base OS, have multiple partitions, and don't read the instructions thorougly because - after all - "it's a Mac, who needs instructions?") are exactly the kind of Mac "power users" who swarm Apple's servers constantly looking for new stuff and install it the second it's posted." Damn straight! And, fortunately for me, the only difference between said "power users" and myself is that I CAN READ and I DO READ. So, as instructed by the iTunes2 ReadMe , I deleted the previous version of iTunes2 (beta, got it from a Hotline server in Japan) from my OS X partition of my HD which has THREE partitions, BEFORE I installed the official release of iTunes2. Go figure, my partitions are intact. No loss of data. iTunes2 is freakin' great.

Re:Wide spread?

[moof1138]

For this to come up you need to have multiple partitions, one of which is named 'Applications.' This is not too common, but it is done.
I knew a guy who did graphic design who did this, I always though it was kind of dumb back then, since apps not running on the boot volume in Mac OS 8.1 - 9.x took a performance/VM hit. It doesn't have that impact on X, but I still don't see much benefit.

Why I hate the software industry

[Carnage4Life]

You have an obligation to take reasonable precautions to protect the data on your computer. That means making backups of any valuable data. Are you going to sue Western Digital if your hard drive fails?

People regularly sue if hardware is made faultily. Toshiba paid billions to settle a lawsuit with floppy disks that never showed up in the field and couldn't be reproduced. I personally have lost track of the number of class action lawsuits I've seen for faulty computer products.

What if it gets fried by a lightning strike?

Being struck by lightening is an act of nature which is completely different from human negligence. Please get your analogies right.

Even if Apple was found to be grossly negligent, they shouldn't be held responsible for data that was lost due to the negligence of the computer's owner.

Why shouldn't they be held responsible? If attaching your DVD player to your TV blows it up or your fax machine shreds your documents, are you also liable in such situations? Quite frankly I am disgusted with the attitudes of most people in the software industry that assumes that shoddy work is inevitable (all software has bugs? WTF?) and then blames customers when their shittily written software fails to behave as it should.

Programming is less difficult than building a bridge or an airplane and yet software companies have hoodwinked the public into making it seem that badly made software is a fact of life. One day people are going to realize that the software industry has been shamming them all this time and the lawsuits will start to pour in. This is probably when software companies will finally go back to using techniques developed decades ago to improve and measure software quality but by then the damage will be done.

Re:Why I hate the software industry

[Some Dumbass...]

Quite frankly I am disgusted with the attitudes of most people in the software industry that assumes that shoddy work is inevitable (all software has bugs? WTF?)

Not a programmer, obviously.

You need to understand that not all bugs are actual errors or "shoddy work". Sometimes it's the interaction between multiple pieces of code, each of which works perfectly well, which causes the problem. Sometimes it's people using the code in unexpected ways which causes the problem (the programmer is not omnipotent, remember). That's why programmers say "all software has bugs". It's not that all software has flaws, but rather that it's almost a sure thing that any given piece of software can be used in a way which causes a problem (or "bug") to arise.

Take this iTunes software bug, for example. Even here, it sounds like there would have been no problem if people followed the directions. Given the simplicity of the fix, it should probably still be called a programming bug. However, what happens if some user actually had the old sh shell on there instead of a link to bash? There could be some subtle errors, as bash does not behave 100% like sh. Bug? Probably not, unless you argue that the programmer should have seen it coming. All right then, suppose some user decides that they only want to use csh, so they link sh to csh instead of bash (note: a Bad Idea)? The installer wouldn't work at all. Bug? Hardly.

The point being, the programmer can't plan for every contingency, and this can lead to problems which people call "bugs". That's why "all software has bugs"... because even if the code is flawless, there's no guarantee that it won't fail somehow.

the REAL code, and how this did happen

[Anonymous Coward]

In the installer is a small shell script to remove any old copies of iTunes. It contained the following line of code:

rm -rf $2Applications/iTunes.app 2
where "$2" is the name of the drive iTunes is being installed on.

The problem is, since the pathname is not in quotes, if the drive name has a space, and there are other drives named similarly then the installer will delete the similarly named drive (for instance if your drives are: "Disk", "Disk 1", and Disk 2" and you install on "Disk 1" then the command will become "rm -rf Disk 1/Applications/iTunes.app 2

The new updated version of the installer replaced that line of code with:

rm -rf "$2Applications/iTunes.app" 2
so things should work fine now.

Worked for me.

[prwood]

I have a Pismo PowerBook with MacOS X 10.1, and I downloaded iTunes 2 immediately after it was released. My hard drive has two partitions, one for MacOS 9.2.1, and one for MacOS X 10.1. I also already had a previously installed copy of iTunes on both drives. I ran the iTunes installer, and everything worked fine. It didn't wipe out any data, and I am quite enjoying the new iTunes 2. I

Gee, I guess I was just lucky?

So old, it's a classic joke

[McD]

From the classic old joke list "How To Shoot Yourself In The Foot" in various programming languages and computing environments, here's the entry for Unix:

% ls
foot.c foot.h foot.o toe.c toe.o
% rm * .o
rm: .o: No such file or directory
% ls
%

Same bug. Welcome to the world, Apple. :-)

Peace,
-McD

Re:Oh, come on...

[GISboy]

Really, in the current economic climate, all the monkeys should have been thrown out of the high-tech jobs, leaving only clueful people.

Well, what you said is the working theory, anyway.

Having worked in the corporate world and the academic world this is the furthest from the truth. The people with a clue, ethics, responsability, talent, skills or value customers are usually the first on the chopping block.
After all, the managers making those 5 and 6 figure salaries have to remain employed so they can continue the (vicous) cycle.

Cynical? Oh, yeah, been there, been IT, seen it happen too many times.

Could apple be any different? That is a tough one to answer. I would have to say no, but to a lesser extent, perhaps.

Why to a lesser extent? For the simple reason that Steve Jobs and Lee Iacocoa (sp?) understood two things about running a company/taking over one:
First get everybody on board with a plan to succede/improve morale.
Second (and this is the kickass part) when you clean house *never, ever* get rid of your workers.
Clean up/fire your middle and upper management levels.

This solves 2 problems (imagine a pyramid):
1) when most layoffs happen they happen to the "base of the pyramid". What happens when you weaken the "foundation" of a company/structure.
Yeah, it falls down or does irrepairable damage.
2)Wiping out the middle section brings those "at the top" closer to the base. Most executive understand the "how and what" of a business, but understanding the "who and why" is what keeps thing "moving forward".

If I remember correctly, Lee I was first, and Jobs subscribed to the idea...it may have come from a /. link when Jobs returned to Apple.

Very good interview.

Of course I've always said a "Phd/manager saying 'in theory' is akin to a used car salesman saying 'trust me' ".

I guess in my snide cynicism I found humor in your altruistic logic

Re:Liability

[wrt]

The EULA doesn't come up until after you install itunes. You accept the license agreement when you run it for the first time.

I didn't lose any data, that would have sucked.

Re:Oh, come on...

[Greyfox]

Clueful people cost more. We recently interviewed a few people for an open position and I reccomended the guy who could actually have helped fix our project up. Our team-lead went for the second most competant person we interviewed on the basis that the guy I wanted would be bored and leave quickly. Our manager went for the wet-behind the ears college graduate. The money involved was the largest factor.

Ask a bunch of hiring managers right now and I think they'll tell you that they'd prefer someone adequate for a job over someone perfect for a job if it meant a salary difference of $10K - $20K. This recession isn't going to lead to a concentration of clueful people in our industry. It's going to lead to a concentration of monkeys.

Possible Fix...

[DragonPup]

Andrew Welch of Ambrosia Software posted a method that MIGHT work on recovering the files here. Basically sometimes the installer, according to Andrew, just messes with file permissions and visability, not actually deleting them.

I didn't test this because iTunes didn't mess up my 5 partitions, thankfully.

-Henry