100,000 iPhones Overwhelm Activation Server

dstates writes "What happens when Apple ships 100,000 iPhone 4S in a day? Answer, 100,000 users all try to activate their new phones. AT&T's activation servers are struggling under the load. Apparently Verizon and Sprint are doing a better job keeping up with the load." Adds an anonymous optimist: "The solution? Call AT&T by dialing 611 and talking to an operator to perform a manual activation with your IMEI and SIM card #, works every time!"

Comments

Good service is good service

[Pieroxy]

Surprise of the day: AT&T activation servers work roughly as well as their cell coverage in urban areas.

Note that if you can't activate your iPhone, you can't drop calls!!!

Something's coded stupidly methink

[Rosco P. Coltrane]

"Activating" a cellphone means little less than recovering a few personal details from the new customer, the phone's serial number or equivalent, stuffing everything in a database, working out some magic number based on some algorithm and send it back to the phone. Big deal... I can write an application like that without even being a specialist and not hose a small server with a million requests a day, let alone 100,000...

Re:Something's coded stupidly methink

[hawguy]

Then you're doing it very wrong. Firs and foremost, most of these customers were already identified ahead of time (because they preordered the phone), so you could have *easily* extracted their information beforehand and copied it to the "local mysql database". Additionally, writing back to the "central database" could be also easily offloaded to a background job that performed the work asynchronously. There's no need to have this information instantly available in the central.

While the customers may have been preidentified, their phones IMEI/SIM's weren't assigned until the phone was shipped. And until you link a customer account to a phone, you can't activate the phone. In any case, even if you prestaged the data somewhere, you still need to flip the switch at the appropriate time to make the new phone active, and that's probably the heavyweight transaction, not the act of entering the new data into the database. I imagine that a phone activation means replicating the data across many regional sites. Even though I called it a "database", it may not even be a database in the traditional sense, it may be a custom cell phone controller with a complicated API with high latency for updates.

Since in many (most? all?) of these cases, the old phone was replaced by the new phone, customers don't want to activate it online, then find at some random time in the future (minutes? Hours? Days?), their old phone stops working and they have to switch to the new phone - they want it activated immediately so they can turn off their old phone and turn on the new phone and have it up and running immediately.

I imagine that the transaction monitor on their transaction processing system allocated a limited number of transaction slots to the activation servers - they don't want to take down their entire network due to high activation demand.

Is it just me,

[Cryacin]

or does it look *exactly* like the Galaxy? *ducks*

Wow, 100,000 activations...

[nweaver]

100,000 AT&T activations, out of well over 1M sales!?!?

If so, most people have heeded the advice: Sprint is cheaper, and Verizon you can make phone calls on.

Re:conspicuous consumption

[iluvcapra]

Something cannot be "par" and a status symbol at the same time.

It seems to me that if you're making this argument, you're just as positionally-conscious as the iSheep (or whatever we'd like to call them), you just use different criteria, no doubt better criteria that is obviously more aligned with value than those other people you don't understand. /s

Re:conspicuous consumption

[iluvcapra]

Because of the level of technology it's become more obvious that it is a status symbol.

I think the iPhone's circumstance as a status symbol has always been rather patent.

The interesting fact is that a Nexus S or a Blackberry or a Droid Bionic are also status symbols -- just because your phone runs a different OS, or it has twice the RAM, that doesn't suddenly make your purchase decision perforce more rational or less status-conscious. The fact that Android and Blackberrys (not so much the second one lately) have defenders and people proudly stating their ownership on this forum clearly demonstrates that owning these phones confers status and attributes the owner with a particular set of values, independent of the actual rational decision to buy the thing.

I just don't think the "status symbol" argument is a useful one -- everybody buys status, and people who run around with Frodo t-shirts and Star Trek bumper stickers (that's me) and hiking boots that never see a dirt road should probably be careful about how they critique social signaling.

They forgot slowdown curves are hyperbolas

[davecb]

People assume slowdowns are always linear, so they get the wrong answers, and under-provision all the time (;-))

Assume a really fast activation in 1/10 second, on a machine that's always got 10 CPUs free for the activation jobs. Each CPU will activate 10 phones in 1 second, but if 11 people per CPU request activation, the 11th will wait a full tenth before they start, plus 1/10 second to do the work. The 12th will wait 2/10 plus 1/10 to do the work, and so on.

100,000 people / 10 CPUs = a load of 10,000 users. Plug that into the queuing equation from which I got the above, and the average time to activate will be 999.1 seconds, or 16 minutes. Not fun!

The actual case is probably a lot worse, with slow activations and overloaded servers, but any time when you can get a really large number of users trying to do something in a short period of time, the average time to do the work will be scary large. Unless they just happen to be within the first 10 callers, of course!

That means that you need to temporarily allocate a hugely larger number of resources than you'd expect on first glance. If you and your manager don't already know that the response time curve looks like a hockey stick, you can easily get into a career-limiting situation by under-planning for a predicted overload.

--dave (wearing his capacity planner hat) c-b

DONT FOLLOW THE ADVICE ON THE BOTTOM OF THE POST

[PowerMacG4]

DONT FOLLOW THIS ADVICE: Adds an anonymous optimist: "The solution? Call AT&T by dialing 611 and talking to an operator to perform a manual activation with your IMEI and SIM card #, works every time!" It will brick your phone (Apple's servers will reject your phone due to "mismatched SIM" and it will refuse to activate) and you will need to go to an Apple Store for a replacement. I spent all fucking day doing this.