Slashdot Mirror
100,000 iPhones Overwhelm Activation Server
dstates writes "What happens when Apple ships 100,000 iPhone 4S in a day? Answer, 100,000 users all try to activate their new phones. AT&T's activation servers are struggling under the load. Apparently Verizon and Sprint are doing a better job keeping up with the load." Adds an anonymous optimist: "The solution? Call AT&T by dialing 611 and talking to an operator to perform a manual activation with your IMEI and SIM card #, works every time!"
Surprise of the day: AT&T activation servers work roughly as well as their cell coverage in urban areas.
Note that if you can't activate your iPhone, you can't drop calls!!!
Write boring code, not shiny code!
100,000 AT&T activations, out of well over 1M sales!?!?
If so, most people have heeded the advice: Sprint is cheaper, and Verizon you can make phone calls on.
Test your net with Netalyzr
Something cannot be "par" and a status symbol at the same time.
It seems to me that if you're making this argument, you're just as positionally-conscious as the iSheep (or whatever we'd like to call them), you just use different criteria, no doubt better criteria that is obviously more aligned with value than those other people you don't understand. /s
Don't blame me, I voted for Baltar.
Then you're doing it very wrong. Firs and foremost, most of these customers were already identified ahead of time (because they preordered the phone), so you could have *easily* extracted their information beforehand and copied it to the "local mysql database". Additionally, writing back to the "central database" could be also easily offloaded to a background job that performed the work asynchronously. There's no need to have this information instantly available in the central.
While the customers may have been preidentified, their phones IMEI/SIM's weren't assigned until the phone was shipped. And until you link a customer account to a phone, you can't activate the phone. In any case, even if you prestaged the data somewhere, you still need to flip the switch at the appropriate time to make the new phone active, and that's probably the heavyweight transaction, not the act of entering the new data into the database. I imagine that a phone activation means replicating the data across many regional sites. Even though I called it a "database", it may not even be a database in the traditional sense, it may be a custom cell phone controller with a complicated API with high latency for updates.
Since in many (most? all?) of these cases, the old phone was replaced by the new phone, customers don't want to activate it online, then find at some random time in the future (minutes? Hours? Days?), their old phone stops working and they have to switch to the new phone - they want it activated immediately so they can turn off their old phone and turn on the new phone and have it up and running immediately.
I imagine that the transaction monitor on their transaction processing system allocated a limited number of transaction slots to the activation servers - they don't want to take down their entire network due to high activation demand.
People assume slowdowns are always linear, so they get the wrong answers, and under-provision all the time (;-))
Assume a really fast activation in 1/10 second, on a machine that's always got 10 CPUs free for the activation jobs. Each CPU will activate 10 phones in 1 second, but if 11 people per CPU request activation, the 11th will wait a full tenth before they start, plus 1/10 second to do the work. The 12th will wait 2/10 plus 1/10 to do the work, and so on.
100,000 people / 10 CPUs = a load of 10,000 users. Plug that into the queuing equation from which I got the above, and the average time to activate will be 999.1 seconds, or 16 minutes. Not fun!
The actual case is probably a lot worse, with slow activations and overloaded servers, but any time when you can get a really large number of users trying to do something in a short period of time, the average time to do the work will be scary large. Unless they just happen to be within the first 10 callers, of course!
That means that you need to temporarily allocate a hugely larger number of resources than you'd expect on first glance. If you and your manager don't already know that the response time curve looks like a hockey stick, you can easily get into a career-limiting situation by under-planning for a predicted overload.
--dave (wearing his capacity planner hat) c-b
davecb@spamcop.net
DONT FOLLOW THIS ADVICE: Adds an anonymous optimist: "The solution? Call AT&T by dialing 611 and talking to an operator to perform a manual activation with your IMEI and SIM card #, works every time!" It will brick your phone (Apple's servers will reject your phone due to "mismatched SIM" and it will refuse to activate) and you will need to go to an Apple Store for a replacement. I spent all fucking day doing this.