Slashdot Mirror
Extreme Heat Knocks Out Internet In Australia
An anonymous reader writes with news that bad weather caused internet connectivity problems for users in Perth, Western Australia on Monday. But it wasn't raging storms or lightning that caused this outage — it was extreme heat. Monday was the 6th hottest day on record for Perth, peaking around 44.4 C (111.9 F).
Thousands of iiNet customers across Australia found themselves offline for about six and a half hours after the company shut down some of its systems at its Perth data center at about 4.30pm AEDST because of record breaking-temperatures. ... "[W]e shut down our servers as a precautionary measure," an iiNet spokesman said late Monday night. "Although redundancy plans ensured over 98 per cent of customers remained unaffected, some customers experienced issues reconnecting to the internet." ... Users in Western Australia, NSW, Victoria and South Australia took to Twitter, Facebook and broadband forum Whirlpool to post their frustrations to the country's second largest DSL internet service provider.
Yes, many people were affected, but iiNet is not 'the internet'. All the other big providers were still running just fine.
You could post the same headline every time someone's modem cable gets knocked out or their router crashes.
they had multiple failures. The primary air con failed, and so did the backups
http://blog.iinet.net.au/statement-chief-technology-officer-mark-dioguardi/
http://blog.iinet.net.au/statement-chief-technology-officer-mark-dioguardi/
Basically both main and backup aircon went down.
Someone call Al Gore--as he's an expert on both the Internet and Global Warming--he'll know what to do.
did the severs get shut down or did a they do a hard power off after tripping the overheat shutdown system in the box?
" Users in Western Australia, NSW, Victoria and South Australia took to Twitter, Facebook and broadband forum Whirlpool to post their frustrations to the country's second largest DSL internet service provider."
Obviously it wasn't the whole of Australias internet that was affected
How do they get on in the outback? It must get near 50C there
One thing about global warming though - when it gets hot enough the ocean will dry up and they should be able to spot MH370 easily
Last summer in Wisconsin, believe it or not, it got around 100F for several days and it knocked out our internet. It wasn't some morons with inadequate server cooling though. Apparently Time Warner equipment runs on 90V lines and our energy company's equipment that drops to 90V was overheating. Unbelievable! Our digital phones were down too.
You're a galah - the summary clearly says it wasn't a storm.
As i understand it, they shut some servers down because they were worried about overheating.
It's unbelievable that a data centre can't cope with an extra degree or two. What sort of idiot designs these places? Haven't they heard of tolerances?
They had air conditioners fail. They probably needed more redundancy, but they shut down some systems as a precaution when the AC failed.
They were really clearing a nest of drop-bears from the server room and had to turn reverse the air-con to drive them out so the servers shut themselves down to prevent overheating.
It's a ClimateChangeDoom sort of thing. The real story is environmental control and contingency planning at the Perth Data Center. If it were my data center we'd rent coolers before we shut anything down. It was 44.2C on 26-Dec-2007, not as if it could never happen again.
It's the humidity.
"The average reporter we talk to is 27 years old......They literally know nothing." - Ben Rhodes
I used to work for an ISP in very hot Texas, but we planned ahead. We kept all of our servers indoors so they were out of the heat. It's a really good idea. I think everyone should do it.
Yes, it's unbelievable that something, somewhere goes wrong.
Let me just fix that for you:
Extreme heat, as compared to the climate in a specific part of Australia, causes one ISP to panic and shut down internet for a small percentage of customers in said specific part of Australia.
Sigh... first world problems. Few services were shutdown for precaution as result of A/C failure (primary/backup).
Hi All,
Due to heat in Perth we have lost a number of services and precautionary shut others down.
Customer will notice impact to the following services:
- iiNet Toolbox and Westnet MyAccount [RESTORED 7:00PM WST]
- Westnet Email [RESTORED 7:45PM WST]
- Westnet Hosted Websites [RESTORED 9:00PM WST]
- iiNet hosted email [RESTORED 7:45PM WST]
- iiNet/Westnet/Adam/Netspace Webmail [RESTORED 7:00PM WST]
- Customers may be unable to re-authenticate after disconnecting from the internet [RESTORED 8:00PM WST]
A number of internal tools are also affected, which will impact our ability to respond to certain customer enquiries.
Update 6pm WST: Due to issues with staff access, some contact centre queues have been closed. Affected queues will be reopened once the incident has been resolved.
Update 8pm WST: Most services have been restored, Engineers are continuing to review all services impacted by the incident. Customers that were off-line are recommended to perform a modem power-cycle to get back on-line.
Thanks,
Basically few gen Y's screaming that they can't post their sweaty selfies for a few hours.
Depending on the refrigerant used it is possible that the condenser temperature (the bit exposed to the outside air) exceeded the critical point of the gas at which point it is impossible to tell the difference between liquid or gas. The trouble is phase change cooling works best (most efficient) the closer to the critical point you can go but not past it.
The second problem is the condenser pressure would increase with increasing ambient air temperature. In the past this was enough to stall the compressor motors on a hot day.
My guess is they went for a system with a high efficiency that should work for 99.9% of the time, that last 0.1% is the 8 hours of the year when the temperature is above 42'C (normally for Perth it is normally only an hours before the sea breeze kicks in and drops the temperature by at least 5'C). This time the temperature went up and stayed up for a period of time.
http://blog.iinet.net.au/statement-chief-technology-officer-mark-dioguardi/
Take the statements with a huge cup of salt. "Network redundancy plans ensured over 98% of our customers’ broadband services were unaffected - See more at: http://blog.iinet.net.au/statement-chief-technology-officer-mark-dioguardi/#sthash.FSMpB8sc.dpuf" is using some big weasel words.
They turned off the PPP auth servers. So any users that weren't already logged in, couldn't log in. And with the PPP servers offline, they have no way to know how many people were affected. The layer 2 broadband service was fine, it was just those pesky layer 3+ that were affected.
If you ever go the the weather observatory in New Delhi, its near Lodhi Gardens, and the instruments themselves are in an actual garden. That place is actually a couple of degrees cooler than the rest of Delhi. The hottest recorded may be 48.6 but it does stay over 45 for a week or two and over 40 for around a month easy.
On the downside though, its sewerage and sanitation systems aren't 'online' yet.
I remember when slashdot
- didn't treat their readers like mindless media starved zombies
- had content targeting intelligent geeks, not the retarded Inbred 7 year old crowd.
- had stories with more than a catchy but blatantly false headline.
Fuck you.
Perth is a state capital, so its not exactly remote.
It was more widespread than just a three states. I'm in the most populous state (NSW, where Sydney is) and many people at work today commended that they were down, as I was. iiNet is the #2 DSL provider and I suspect it was a lot more than this 2% they spout. Bottom line, a significant amount os Australian residential customers and business had no Internet. Someone is saying "inept isn't the Internet" but if the pipe from someones computer to the Internet isn't working, "the Internet is down" to that person.
Only big ligs use sigs.
Perth is very remote. sure it's the state capital, but once you're out of the metropolitan area it's a thousand kilometers to anywhere else.
SJW n. One who posts facts.
Average temperature seems to sit between about 32 and 37 degrees Celsius for Perth. I'm certain if the average were a full 10+ degrees over this, the DC would have been built to accommodate this significantly higher temperature. Not really a useful comparison.
[Sorry, this signature is unavailable in your country/region]
They should have been using fiber. Specs on a popular GPON chassis says maximum operating ambient air temp is 131f @ 95% relative humidity.
The problem seems to be routers. A regular router seems to be limited to about 121f and "core" routers are more around 104f.
It's one of the most remote cities in the world. That said it affected all of Australia not just Perth.
We don't start complaining about the heat until it hits 50C.
$.
Also, there are still modes of failure for ground loop systems. None of the links say what specifically happened, so it's hard to judge.
Sounds like the temps we get here each and every year... Las Vegas Nevada *lives* between mid June and late October with temps over 100F, and frequent transients to over 110F... Guess the Aussies aren't used to such temps...
THANK YOU, Edward Snowden!! Americans owe you a debt of gratitude (whether they know it or not..)
posting to suppress mis-moderation