Slashdot Mirror

← Back to Stories (view on slashdot.org)

Fire Takes Azerbaijan Offline (datacenterdynamics.com)

Posted by timothy on from the right-the-whole-place dept.

judgecorp writes: On Monday, 90 percent of Azerbaijan lost Internet access, due to a fire at one data center in Baku, the capital of the former Soviet Republic. Cables caught fire at the Delta Telecom facility, and international providers including NTT and Telecom Italia all lost service for nearly eight hours. Some interesting snippets: Azerbaijan is a former Soviet republic that has seen rapid development thanks to its rich oil and gas reserves. The country has been running several projects aimed at modernizing its communications infrastructure, including participation in Trans-Eurasian Information Highway (TASIM). ... At about 16:10 on Monday, consumers, businesses and government agencies across Azerbaijan suddenly lost their connections to the Internet. Banks couldn’t make domestic money transfers, and even Point-of-Sale terminals were not working. ... Interestingly, no international traffic flowing though Azerbaijan was affected by the outage. “Transmission channels to Georgia, Iran, and the Middle East were working at full capacity,” Iltimas Mammadov, the minister of communications, told AzerNews.

57 comments

  1. Shaming a country by michaelmalak · · Score: 1

    the former Soviet Republic

    Might as well just say pwned by Lenin

  2. Goes through one spot for a reason by rahvin112 · · Score: 4, Interesting

    The entire countries internet goes through a single building for a reason and it's not cost. They've not forgot their KGB roots.

    1. Re:Goes through one spot for a reason by Aaden42 · · Score: 4, Insightful

      Single point to take the entire country offline?

      CLOSED: Works as designed.

    2. Re:Goes through one spot for a reason by Anonymous Coward · · Score: 2, Interesting

      It is because of cost. The US has more sites and the monitoring works just fine. China is probably the same.
      Not that there's anything wrong with keeping costs down. I don't know Azerbaijan, but maybe 99,9999% uptime is not their top priority.

    3. Re:Goes through one spot for a reason by bobbied · · Score: 3, Informative

      Actually, it's most likely because they didn't think of physical location redundancy... I've seen "redundant" set ups which where strictly maintained within a building get totally undone the second the fiber runs get mounted on the same poles to cross the street before they split and when their separate ways. Nobody was paying attention until a dump truck accidentally snagged both fiber runs and yanked them down the street, disconnecting both redundant fibers in one accident.

      So, where it may have started as a KGB thing, nobody really thought though how their redundant design on paper, fell to a single physical event because they didn't maintain enough separation. Or, maybe it just was the NSA, providing low cost engineering services to keep them under their thumb?

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    4. Re:Goes through one spot for a reason by goombah99 · · Score: 1

      I though the internet was supposed to route around damage. Something is too centralized.

      --
      Some drink at the fountain of knowledge. Others just gargle.
    5. Re:Goes through one spot for a reason by Anonymous Coward · · Score: 0

      I though the internet was supposed to route around damage.

      No, the Internet is not /supposed/ to route around damage, the Internet /can/ route around damage.

      When you don't set up any redundancies to take advantage of, of course you will have no redundancies.
      When you specifically go out of your way by design to remove all points of redundancy, of course that will be the outcome.

      Do you have two Internet providers terminating at your home? If not, and you go cut the cable/fiber/whatever on your single Internet provider, how do you expect your packets to magically have another route to take?

      The Internet can't magically choose any old random unrelated route to send packets over, it can only choose between the given routes you provided for.
      If you don't provide for that, everything is still working perfectly as designed, even if not intended (although in this specific instance I'm pretty sure it was wholly intended too.)

    6. Re:Goes through one spot for a reason by schnell · · Score: 1

      I though the internet was supposed to route around damage. Something is too centralized.

      At an IP routing level, sure. But that doesn't do any good if there's only one physical cable or set of cables linking you to the rest of the world.

      --
      "95% of all Slashdot .sig quotes are incorrect or completely fabricated." -Benjamin Franklin
    7. Re:Goes through one spot for a reason by AK+Marc · · Score: 1

      The Internet doesn't exist within most countries. There are a few POPs to the Internet, one or two, but two people on different networks usually don't change carriers at the tiny POPs. NTT and TI might not peer at that POP, so two people would have their traffic leave the country to come back. I've seen that happen with Pacific islands with a single POP. That means that the Internet starts one hop out of the country.

      The Internet routes around the damage. Nobody outside Azerbaijan noticed the outage. But the people inside are on the Internet the same way as someone on Comcast is on the Internet when Comcast goes down.

      When I worked for a US company with an office there, we got training on how to bribe. The issue is that bribing a foreign government is illegal. And in Azerbaijan, so much is still owned by the government. The government workers are so poor that they use their power to work for tips. In my case, we had a microwave link in a government owned tower. The equipment was up on the top floor. If we wanted access, we had to pay an "elevator fee" bribe paid to a government worker for access to a government building. It was illegal, but the options were down to "don't do business in Baku, or do so in violation of US law".

      --
      Learn to love Alaska
    8. Re:Goes through one spot for a reason by AK+Marc · · Score: 1

      99% of redundancy I've seen comes down to a single line of config in a single box. Redundancy is great, but a delicate cluster of a single type of gear that requires that single vendor proprietary redundancy protocol work perfectly 100% of the time is redundant in theory only.

      That and I've seen great cost for redundancy for redundancy's sake where the complexity added to get redundancy made the whole network less stable. 1 box with a MTBF of 10,000 hours means one outage every year. 4 boxes with the same MTBF means 4 outage incidents a year, though hopefully without a network down incident. Replacing the 1 box with another 1 box with a 100,000 MTBF would see a better increase in MTBF-related downtime, and the 4-box solution has full outages on a regular basis from the delicacy of the clustering used.

      So I wouldn't put it down to malice and KGB. All the population is in Baku, and it is easier to have one place to go. They had lots of redundancy, just not physically separate. Every place I've seen with multiple power feeds all had a single point of failure. One went so far as to have 3 power feeds from 2 carriers at 3 building entry points, 2 generators, 3 UPSs, and all that, with the 3 UPSs in the same small room, and the power from them out to the facility through a single power panel. Though the feeds alternated A B and C, a single event at that one panel could bring down the entire facility, regardless of the state of the 3 UPSs and 2 generators.

      --
      Learn to love Alaska
    9. Re:Goes through one spot for a reason by bobbied · · Score: 1

      Yep, that reminds me of a "redundant" system we purchased once. It had two processors that ran in locked step so you could fail over seamlessly from the primary to the secondary at the slightest indication of a fault (say an ECC error, or some noise on some address line). Sounds great on paper... Problem was, when this happened, you where no longer redundant until you could reboot the system which took something like 20 min to cleanly shut down, reset the redundancy and restart everything. Starting the system in non-redundant mode took like 5 min because it didn't have to re-sync everything. The failure rate of the individual systems was such that it failed about once a quarter or so and we'd process on w/o a break, but it took us 20 min a quarter, 80 min a year to get it back into redundant mode. The thing was actually *better* availability when we didn't run it fully redundant because the amount of down time was much less to just reboot the one side... So we'd go fully redundant when we could afford the boot time, like in planned maintenance windows which where at the customer's request, but then once it failed over, we'd leave it in non-redundant mode and just boot the one side....

      Sometimes redundancy is just more complicated and depending on the MTBF's involved, not a good idea...

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    10. Re:Goes through one spot for a reason by Anonymous Coward · · Score: 0

      cute - you are using jargon you don't understand and laying claims to experience you do not have

      claiming that you got training to bribe (which would seriously get you in hot water with your state department) is a dead giveaway that you're full of shit

    11. Re:Goes through one spot for a reason by AK+Marc · · Score: 1

      I'm full of shit because why? You think I'd lie about it if I were given training in how to bribe? Or are you claiming the situation where government employees demand bribes to do their job is impossible?

      --
      Learn to love Alaska
  3. And the whole country says.. by the_skywise · · Score: 1

    Network's down... bagel run!

    1. Re:And the whole country says.. by Anonymous Coward · · Score: 0

      Are they particularly known for their bagel affinity?

  4. And this friends, is a real problem by Anonymous Coward · · Score: 1

    If terrorists were halfway competent, this is the kind of thing they'd do rather than trying to frighten the population with useless displays of violence.

    And it can happen by accident.

    Or was it?

    1. Re:And this friends, is a real problem by Anonymous Coward · · Score: 0

      useless displays of violence.

      It seems to me that the purpose was to add fire to a Holy War, providing propaganda to turn a generation of harmless religious sorts into violent extremists, and Western governments have - in under a week - taken the bait.

      We should have regarded them as mass murderers with no more skill than any other extended family with a pile of guns. We should have under no circumstances entertained the idea that we are at "war", or - even worse - responded by curtailing our own citizens' freedom (have you seen what France is like now on city streets? and they already have some of the most draconian Big Brother law in the West).

  5. Very Nice! by Anonymous Coward · · Score: 0

    Obviously they had fallen behind on their homosexuals chucked down wells quota, and were punished accordingly.

  6. Route around problems by Anonymous Coward · · Score: 0

    Why is it the Internal fails to router around fucked routers and switches (not the consumer crap). Back in the 80s, this ability was part of the hype; if there's a problem, it knows how to work around it. Never fucking happens, though, does it.

    1. Re:Route around problems by RavenLrD20k · · Score: 2

      Can't route around a horrible network design. 90% of the country's network was all routed to a single building with evidently no redundant links. Great for spying...not so great for disaster recovery and avoidance. But there's good news on multiple levels: The rest of the world's network didn't go down because of it!

    2. Re:Route around problems by AK+Marc · · Score: 1

      Can't route around a horrible network design. 90% of the country's network was all routed to a single building with evidently no redundant links.

      The Internet routed around Baku without issue. And there were redundant links. They just all went into the same building.

      --
      Learn to love Alaska
  7. In soviet russia we fire you! by Joe_Dragon · · Score: 0

    In soviet russia we fire you!

  8. Azerbaijan Land of Fire - Truth in Marketing :) by Anonymous Coward · · Score: 0

    My first thought when I saw this article was the Atletico Madrid soccer team's jersey which proudly reads "Azerbaijan Land of Fire".
    See https://www.facebook.com/Azerbaijan-Land-of-Fire-835277386487464/
    Maybe their next logo can be "Azerbaijan Land of Redundant Disaster Recovery Sites" or at least "Azerbaijan Land of Fire Extinguishers"

  9. no surprise... by Anonymous Coward · · Score: 0

    Georgia, Iran, and the Middle East

    that connections to/from "regions of interest" remained unaffected.. but do you thank the c i a for the funding that made that possible?

  10. Re: They should have used APPS! by ModernGeek · · Score: 4, Funny

    Yeah, why use these data centers when you can just put your data in the cloud?

    --
    Sig: I stole this sig.
  11. Re:why does this happen? by Anonymous Coward · · Score: 1

    I thought it was to send nekkid ascii pics back and forth

  12. Re: They should have used APPS! by Anonymous Coward · · Score: 0

    OK. That was funny right there.

  13. Re:why does this happen? by Anonymous Coward · · Score: 0

    I'm with you on this.. Redundancy wasn't the primary driver, porn was...

  14. putsch comes to shove by Anonymous Coward · · Score: 0

    IT'S HAPPENING!

  15. Shoulda used premium Monster cables... by Anonymous Coward · · Score: 0

    Tsk'ski.

  16. Re:why does this happen? by Anonymous Coward · · Score: 0

    Yes but WHOLE POINTS, even in uppercase, don't stop anyone from using non-redundant links.

  17. For Context by avandesande · · Score: 2

    Pennsylvania:
    46,055 sq mi
    12,787,209 population

    Azerbaijan:
    33,436 sq mi
    9,624,900 population

    --
    love is just extroverted narcissism
    1. Re:For Context by Anonymous Coward · · Score: 2

      Pennsylvania:
      12 letters
      Starts with a P

      Azerbaijan:
      10 letters
      Starts with an A

  18. Hmm ready this by Anonymous Coward · · Score: 0

    AzureBiajizan BITCH!!!!

  19. Cold War much? by Crimey+McBiggles · · Score: 1

    So what you're saying, is that Azerbaijan was a former soviet republic; seriously? Let's move on from the cold war and call it what it is: part of Europe. We don't commonly refer to the Ukraine as a former soviet republic, nor Belarus, nor Estonia. Get with the times and stop posting ignorant propaganda.

    --
    Crimey
    1. Re:Cold War much? by tsqr · · Score: 1

      So what you're saying, is that Azerbaijan was a former soviet republic; seriously? Let's move on from the cold war and call it what it is: part of Europe.

      Part of Europe? Oh, yeah -- the part of Europe that is called the Russia and Caucasus region of Asia, though some might consider it part of the Middle East region of Asia.

    2. Re:Cold War much? by thegarbz · · Score: 1

      At this rate it will only be a matter of years before Australia is considered part of Europe too.

    3. Re:Cold War much? by AK+Marc · · Score: 1

      Since all that's so complicated, let's just call it a former soviet republic.

      --
      Learn to love Alaska
  20. Part of WHERE? by Hognoxious · · Score: 1

    Let's move on from the cold war and call it what it is: part of Europe.

    Azerbaijan
    Ukraine
    Belarus
    Estonia.

    One of those is not like the others.

    (Hint: it borders Iran).

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    1. Re:Part of WHERE? by unixisc · · Score: 2

      The thing about Azerbaijan, Turkey and the stans is that the secular elements of those countries see themselves as a part of Europe, while the Islamic elements see them as a part of the medieval Timuride, Seljuk or Ottoman empires. (I'm not including Tajikistan in this analysis, since that country, despite being Sunni, is culturally more allied to Iran).

      Azerbaijan is Shi'ite like Iran, but Turkic, like 4 of the stans and Turkey. Kazakh, Kyrgyz, Turkmen and Uzbek are all Sunnite and Turkic. Turkey was a secular country since Kemal Ataturk, but has reverted to being an Islamic country under Erdogan, and is past the point where it considers itself a part of Europe. However, Azerbaijan and the other stans still have secular Soviet era quasi-democracies, and those parts still consider themselves a part of Europe, as opposed to the Middle East or Central Asia.

      Personally, I group Turkey along w/ Kazakhstan, Kyrgyzstan, Turkmenistan and Uzbekistan as the Turkic world, just like the Arabs have the Arab world - everything from Iraq to Morocco. Azerbaijan and Tajikistan are oddballs - Azerbaijan being Turkic but Shi'ite, while Tajikistan is Farsi but Sunnite. So these 2 could either be influenced by Iran or Turkey, but at this point, both those countries are Islamic, and the differences just theological

    2. Re:Part of WHERE? by AK+Marc · · Score: 1

      I always lumped Kazakhstan in with Mongolia. They don't even border, but they are similarly rural with nomadic roots.

      Part of the issue with the former soviet republics is that Russia shuffled the locals around. Russia liked the idea of a winter retreat Crimea, so they shipped in Russians, and move the others out.

      --
      Learn to love Alaska
    3. Re:Part of WHERE? by unixisc · · Score: 1

      But that's been a process that's been undone since the Cold War. Like Kazakhstan was 50% Kazakh and 45% Russian in 1991, when the Soviet Union came apart. Today, it's 63% Kazakh, and 23% Russian: since that time, most Soviet people have returned to their native lands - like Russians returning to Russia, Ukrainians returning to Ukraine, Uzbeks returning to Uzbekistan and so on. As for Crimea, Stalin saw Crimean Tatars as a potential threat, and shipped them out to Central Asia. Culturally, the Crimean Tatars have more in common w/ the people of the stans than w/ Cossacks, who are the people native to the Crimea.

      While Kazakhstan and Mongolia are similar, Turkic and Mongol are 2 related but distinct ethnicities. Kazakhs are Turkic, like their neighbors to the south. And there's religion as well - Mongols being predominantly Tibetan Buddhist (Shamanism having been stamped out during Chinese rule) while Turks/Kazakhs/Kyrgyz et al are predominantly Muslim.

  21. Re:95% muslim country can't keep basic infrastruct by unixisc · · Score: 4, Interesting

    I am the first person to criticize Muslims, but people of the ex Soviet Republics are just nominal Muslims - nothing like the Quran thumpers that you'll find in Saudi Arabia, Pakistan, Afghanistan, Palestinian Authority, Syria, Egypt, Iraq or Iran. In fact, most of them also recognize Israel and have full diplomatic relations w/ Israel - something that's unthinkable for most Muslim countries.

    Also, while I'm no fan of Communism either, that's the one thing that has kept the stans - particularly Uzbekistan - from becoming an Islamic regime like Afghanistan.

  22. Bummer by nospam007 · · Score: 2

    Seems like their data center didn't have a firewall.

  23. Someone took this article too seriously by Ukab+the+Great · · Score: 1

    http://gizmodo.com/5833267/why-the-internet-should-die-in-a-fire

  24. 90%? by smoot123 · · Score: 1

    That means they have more than 10 users?

  25. My country too by Anonymous Coward · · Score: 0

    Is like disaster in my country Latvia, goat released by imperialism powers ate half of country's strategic reserve potato bushel. Much sad. : (

  26. Azerbaijan - Land of Fire by Dr.+Spork · · Score: 1
    I always thought that it was weird for a country to advertise on the jerseys of Atletico Madrid, and I thought that "Azerbaijan - Land of Fire" was always a weird motto. I think they were trying to indicate passion, but really, who would want to live in, or even visit, the Land of Fire? Especially now that the fire took out their internet?

    If you think I'm kidding, click here.

  27. Re:95% muslim country can't keep basic infrastruct by Anonymous Coward · · Score: 0

    I am the first person to criticize Muslims, but people of the ex Soviet Republics are just nominal Muslims - nothing like the Quran thumpers that you'll find in Saudi Arabia, Pakistan, Afghanistan, Palestinian Authority, Syria, Egypt, Iraq or Iran

    Syria is secular and socialist. ISIL was formed in post-invasion Iraq (which also was secular and socialist until the removal of Saddam) with origins in Saudi Arabia. Other than that, sure.

  28. Re:95% muslim country can't keep basic infrastruct by unixisc · · Score: 1

    The Syrian regime is 'secular' and socialist. The Syrian people - the vast majority of them are Sunni fanatics, indistinguishable from their comrades in Saudi Arabia and Egypt. The reason ISIS has the support it has is that an overwhelming majority of Syrian Sunnis support it. They are 80% of the Syrian population

    As for the Baath regimes in both Syria and Iraq, they are/were 'secular' in the following aspect. Whenever a country is under pressure to be declared 'Islamic', what that means is that the majority sect would be the one whose practices would define Islam in that country. So it's Wahabi for Saudi Arabia, Shia for Iran, Iraq, Bahrein and Azerbaijan, Muslim Brotherhood in Syria or Egypt and so on. Given this standard, it was obvious that Sunnis in Iraq would be superficially opposed to an 'Islamic' state, as would Alawites in Syria, while Sunnis in Syria and Shias in Iraq would both be supportive of it.

    However, just b'cos Sunnis in Iraq didn't support the Islamic state of Iraq didn't mean that they are secular: they were happy to join ISIS, which is explicit about having an Islamic state that is Sunni. That they support. Conversely, in Syria, the Baathists, while in an Alawite coalition w/ Druze and Christians, would be perfectly happy w/ an Islamic state where the standard was Alawite. And thanks to their power, at least 1 seminary in Iran recognizes them as Shia Muslims.