Slashdot Mirror
Why Is Less Than 99.9% Uptime Acceptable?
Ian Lamont writes "Telcos, ISPs, mobile phone companies and other communication service providers are known for their complex pricing plans and creative attempts to give less for more. But Larry Borsato asks why we as customers are willing to put up with anything less than 99.999% uptime? That's the gold standard, and one that we are used to thanks to regulated telephone service. When it comes to mobile phone service, cable TV, Internet access, service interruptions are the norm — and everyone seems willing to grin and bear it: 'We're so used cable and satellite television reception problems that we don't even notice them anymore. We know that many of our emails never reach their destination. Mobile phone companies compare who has the fewest dropped calls (after decades of mobile phones, why do we even still have dropped calls?) And the ubiquitous BlackBerry, which is a mission-critical device for millions, has experienced mass outages several times this month. All of these services are unregulated, which means there are no demands on reliability, other than what the marketplace demands.' So here's the question for you: Why does the marketplace demand so little when it comes to these services?"
The marketplace has been duped into believing that this is the best technology can provide. People don't have time to know, understand, or research history and find that technology really can be reliable.
I'll get modded troll, but I lay much of this at Microsoft's feet. I laughed them off when I first heard of them and their goal of taking over the industry. After all, I'd been working on systems that ran 24x7 with five-9 reliability for years, and DOS/Windows couldn't touch that.
One time I had an opportunity to visit Microsoft and have lunch with a friend there. I figured while there I'd take the opportunity. I asked them in hushed tones, "Just how do you configure Windows so that you don't have to reboot it all of the time?" They looked at me like I was crazy.
Technology can provide reliability. The general public is no longer even aware that it's possible.
Complacent consumerism. "Hey, it's always been this way so they [service providers] must not be able to have 99.9% uptime. If they had the capability, they sure would provide it to us, their customers."
We figured out a long time ago that it's easier to elect seven judges than to elect 132 legislators.
Basically, we don't rely so much on a single system that a brief outage can be tolerated because there are alternatives to choose from.
This is also the basis of Clayton Christensen's theories on disruptive innovation - that a consumer of something (technology, etc.) is willing to trade off some of these aspects, like reliability, for cost or performance benefits (however you wish to define those benefits...).
The last time I wrote code, it was Morse
... after decades of mobile phones, why do we even still have dropped calls? It's a little thing called physics. When you're traveling while using your phone, you may transit into dead zones. We could solve this by cutting down all the trees and flattening the landscape, but that might make some people angry...512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.
You can have one or the other.
We're not talking about software, we're talking about hardware and man-hours. Those will never be free.
Gone!
If offered cell plans that cost $50/month with rare outages or $150 a month with extremely rare outages, which would most people take?
99.999% (5 nines) of reliability is achievable, but it's very expensive and hard to do. Everything has to be redundant, with no single point of failure, everything has to support fail-over seamlessly, the software has to be tested with extreme rigor, and upgrade procedures need to function nearly instantly and support rollback without loss of service.
To put it simply, it's the money stupid. It requires a lot more equipment and manpower to offer a high availability service. This extra cost results in higher prices. It can cost 1000% more a month for less than 1% more reliability. Think of a $400 a month T1 with a SLA versus a $40/month cable line. Being sheep has nothing to do with it.
You don't make the poor richer by making the rich poorer. - Winston Churchill
Because 90% of stuff labeled 'mission critical' actually isn't. Think about it - for most of us, being able to receive or send cellphone calls or emails at any time seems super important, but the number of hours in any given month where it really *was* super important (the grant application was due in two hours; your mother was sick; your partner was about to go into labor; whatever) is generally pretty low - our real tolerance for occasional downtime is therefore quite high.
Don't forget that the support costs on a 5E dwarf even the cost of most, if not all, Smartnet contracts.
Simply said, because the equipment isn't/hasn't been able to support it, the only way to build 5 9's or better has been to add more equipment, which increases operations costs, capital costs, etc across the board in an almost linear fashion.
The market has for the most part established the level of service available by establishing the price point the customer is willing to pay for said service.
People love to point towards the big bad telcos and other companies as monopolies and only being concerned about profit margins. They forget that those same profit margins are what drive the company's stock price, in turn causing growth in people's portfolios. It's a vicious cycle and won't end until enough people decide they have enough.
Sig??? I don't need no stinkin Sig!
"The marketplace has been duped into believing that this is the best technology can provide. People don't have time to know, understand, or research history and find that technology really can be reliable."
No. They believe it is the best the technology can provide at a given price. Why do people "put up" with cars that only give them X amount of protection in a car crash even though there is technology out there that would make them safer? Because they aren't willing to pay the marginal cost for the extra protection. Arguing about what is possible with technology is pointless. What matters is what a piece of technology can do at a given price.
Everything is a trade-off. The sooner Slashdot learns this the less we will have these stupid "Why don't consumers use the latest, greatest, most expensive technology? We need to force them somehow!" articles.
Creative Demolition
When Comtrash Internet dropped my speed from 6 Mbps to 1 Mbps but kept the rate at 6 times DSL, I dropped Comtrash and went with the 1.5 Mbps DSL from my local telco. I got 50% more than Comtrash was delivering at 1/6th the cost. No problem.
When Microsoft decided that I didn't own the rights to my own media and stopped me from being able to copy my own DVDs, I decided to drop them for my media development system and I switched to Linux and Apple. Microsoft doesn't want my business so I went with the people who do. No problem.
When my Long Distance company decided to charge over $1.00 per minute for International calls, I switched to AT&T and their 17 cents a minute program. No problem.
When Frigidaire washers charged extra for the warm water cycle but only give you 5 seconds of hot water and thus, never any, it was no problem to return the unit and buy a different brand. Sure, the salesman wasn't happy but, that is now his problem and not mine. I bought a different brand that did give me what they advertised and promised. No problem.
The list is endless and across all businesses and domains.
The point being is that there are alternatives but, many (or most) people are either too lazy to do anything about it or, like this article, they are too apathetic to do anything about it.
The choice is up to the consumer and, if the consumer would take action, the industry would have to adapt because the market demands it. So far, the market is willing to accept this and thus, the industry sees no reason to change. The less the consumer will accept for their dollar the less they will receive. That, is the problem.
Banjo - The more I know about Windoze, the more I love *nix
It's all about cost vs. the cost of downtime. You'll find in business lines such as the financial sector, customers are willing to pay for extremely high availability because time is indeed money. Business lines that have lower costs for downtime have to weigh availability vs. ROI.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
Engineering has always been about compromise. Any idiot can design a structure that is X feet tall but it would prove more useful it if wasn't a giant block of concrete -- if it had room for offices and the materials used to build it had minimal cost without sacrificing structural integrity.
The same applies to computer engineering. We would easily build a cell phone network that had so many redundancies that it would virtually never go down and would support for thousands of times the expected average load, but we would pay for it in terms of cost. Customers demand reliability. Customers demand affordable cost. What the customer is "willing to accept" is a balance between the two.
Hint: Just because you live somewhere without such problems does not mean they don't exist. Ditto for lost e-mail.
[Fuck Beta]
o0t!
This has everything to do with cost and nothing to do with Microsoft. Consider VoIP... people are deliberately choosing telephony services that are less reliable and lower quality than POTS, because VoIP is cheaper. If you want 99.999% uptime, that's fine -- but you're going to pay for it. High availability services require better equipment, redundant equipment that doesn't come cheap and more, higher quality staff to operate it. So it costs more.
I've been in the technology services business for a long time, and with few exceptions, 80%+ customers want their services are delivered as cheaply as possible. Most hospital systems don't even have a 99.999% availability requirement. The 20% the want varying levels of higher than normal availability usually have a government regulation, SLA or other mandate requiring that they do so.
Conformity is the jailer of freedom and enemy of growth. -JFK
On the other hand, if google was unavailable for 9.863 seconds per day, every day (which is the equivalent of 1 hour per year), who would care? Just resubmit your query.
What's important about reliability is often not the total downtime but the duration of downtime.
This reminds me of why Bruce Schneier's dream of legislating liability for software defects is misguided. Sure, statutory liability would make software more reliable, but it would mean that the many who don't need the additional reliability (and currently aren't willing to pay for it) would be forced to subsidize the handful who do. It would also likely claim volunteer-developed software as a casualty.
http://outcampaign.org/
As a consumer, you're more than entitled to take Comcast to small claims court, which is most likely the mechanism that Comcast would use to extract unpaid bills from you. That Comcast is more likely to enact this mechanism than you are is not a fault of politicians.
It varies by state, but usually it costs 15 dollars to take a company to court, and no lawyers are required. It is generally quick and painless, and people at your local courthouse can fill you in on the details and help you through the process.
The ______ Agenda
If we wanted better uptime we could have it. We would just have to pay more for, and look at, a whole lot of redundant systems. Personally, I'm happier to keep paying less and only have one power line coming into my house, with the nearest plant many miles away. The same goes for cable and telephone service. And my cellular service does work about 99.9% of the time.
when my employers blackberries failed earlier this month they fell back to laptops with a bluetooth tethered phone and outlook/exchange. redundancy is built into the mindset. No messages were lost
I have been a user for about 10 years. This ends Feb 2014. The site's been ruined. I'm off. Dice, FU
I'm sure there's a lot of the attraction of Internet service in being you pay a single flat fee, no matter how "important" the packet is. Who wants to have a 2.99 extra surcharge per call if the caller is a job recruiter(presumably, because he is offering you a job)? How about a 5 dollars surcharge if the call comes from your doctor? vet? The Internet caught on so far with the "a packet is a packet" mantra. Now all the internet suppliers compete on price(because people want cheaper internet) and want to charge extra for things... people haven't considered when they signed up... so they can charge more. This is what this is about, period. I imagine similar efforts are underway, paid for by different cable companies, etc... Anything to not have 5mbps to the internet, unfettered, 24hrs per day, 7 days a week, always-on, for a flat fee.
Unfortunately for them, I'd be willing to downgrade to 1mbps, but not on the always on, nor the unfettered, and if they do downgrade, I will be readjusting my idea of how much it should cost.
As Joel Spolsky pointed out on his blog JoelOnSoftware, 99.999% is pretty much fictional.
99.999% over a year is 31.526 seconds.
No matter how good your staff, no matter how many people you have on site, no matter how robust your systems, no matter how many failsafes you have standing by, ready to be plugged in...
IF something does go down, even the fastest tech on earth is unlikely to identify, pull out, replace and have fired back up whatever the faulty item is in under 30 seconds.
99.999% uptime is essentially fictional. It's simply an impressive sounding number that says, "We'll do everything realistically possible to keep you up 100% of the time. In a typical year, you won't see anything bring you down. You can now tell your investors/clients this and make them feel warm and fuzzy."
It ignores the second part, "But, honestly, if it does go down, we won't have it back within 30 seconds, 100% of the time. Sorry, but welcome to reality. But, for what it's worth, our board's happy to pay you outage fees because it's a small enough risk and the amounts are capped enough, that we're happy to take the risk and costs in exchange for advertising a service we know no one can deliver."
Let's look at regulated phone service, the example in the original post. Can anyone point to a major carrier that hasn't had a major outage at some point? Be it an idiot in a switch room, a power outage affecting a whole side of the country, an anchor ripping up an undersea cable? And how many of them have actually been back within the mandated 30 seconds?
It doesn't happen. That two hour outage is going to take quarter of a millenium of absolutely no more faults to earn back at 30 seconds/year. With luck, it only hit one in 250 customers so you can pretend you're well within your 99.999% uptime but that 1 in 250 isn't really going to agree they got 99.999% after they were down for 1:59:30 more than their contract said they would be.
So, no, 99.999% doesn't exist. It's just a really cool story we tell ourselves whilst being willing to pay whatever the penalties are for missing it, on rare occasions, in exchange for great advertising.
Thanks for actually listing out the figures. It really puts things in perspective, and it made me realize something. My internet service probably gets somewhere between 99.9% and 99.99% uptime. My cell phone is probably in a similar range. My cable is better than 99.999% (maybe even 99.9999%).
Maybe they realize that the oil companies(and countries) can't do a whole lot about the price of oil.
I wonder how much Exxon and Shell make when we import a barrel of oil from Canada?
http://www.eia.doe.gov/pub/oil_gas/petroleum/data_publications/company_level_imports/current/import.html
Nerd rage is the funniest rage.
Bottom line, things that aren't supposed to happen, do sometimes happen.EXACTLY. That's the part people tend to gloss over. Seeing the latest Southpark episode is not a life or death situation. Likewise, your heart isn't going to explode because you cannot get to Yahoo! immediately.
Trusting one's life to a cell phone is a gamble. While they are a fairly stable technology, there are numerous troubling issues... Batteries don't last forever. Service isn't available everywhere. 911 calls aren't always routed to the most appropriate call center -- although it's much better than in years past. In an accident, your cell phone is just as likely to be damaged as you -- or worse, lost. etc.
If you have to choose your hardware around the OS, that hardly counts as simplicity.
Benford's Corollary to Clarke's Law: "Any technology distinguishable from magic is insufficiently advanced."
Thank you for bringing some sanity into this argument. Before you showed up it was dominated by idiotic hippies ranting about our mindless consumer-driven existence, the destruction of the environment, Microsoft, and just about everything else that has nothing to do with the issue at hand.
99.999% uptime is orders of magnitude more expensive than 99.99%, which in turn is orders of magnitude more expensive than 99.9% uptime, and so on.
The added cost is simply not worth it, in any sense of the word, to the general public.
I, for one, would prefer to deal with a day's worth of power loss in a major storm, than paying 10x as much for my electricity in order to make it bulletproof.
The savings would be better spent elsewhere.
Note that this is not an argument against proper planning and preventative maintenance to REDUCE downtime as much as possible, just an argument against designing everything in the world to survive a nuclear bomb when that level of reliability is simply not worth the cost.
Apple users would disagree.
We hope your rules and wisdom choke you / Now we are one in everlasting peace