Loud Sound From Fire Alarm System Shuts Down Nasdaq's Scandinavian Data Center (bleepingcomputer.com)

← Back to Stories (view on slashdot.org)

Loud Sound From Fire Alarm System Shuts Down Nasdaq's Scandinavian Data Center (bleepingcomputer.com)

Posted by msmash on Friday April 20, 2018 @02:43AM from the oh-damn dept.

Catalin Cimpanu, writing for BleepingComputer: A loud sound emitted by a fire alarm system has destroyed the hard drives of a Swedish data center, downing Nasdaq operations across Northern Europe. The incident took place in the early hours of Wednesday, April 19, and was caused by a gas-based fire alarm system that are typically deployed in data centers because of their ability to put out fires without destroying non-burnt equipment. These systems work by releasing inert gas at high speeds, a mechanism usually accompanied by a loud whistle-like sound. With non-calibrated systems, this sound can get very loud, a big no-no in data centers, where loud sounds are known to affect performance, shut down, or even destroy hard drives.

114 comments

Min score:

Reason:

Sort:

I had no idea this could happen by Anonymous Coward · 2018-04-20 02:49 · Score: 0

I guess I shouldn't burp near my NAS then
1. Re:I had no idea this could happen by darkain · 2018-04-20 04:04 · Score: 4, Informative
  
  From a decade ago: https://www.youtube.com/watch?...
2. Re:I had no idea this could happen by rickb928 · 2018-04-20 04:10 · Score: 1
  
  I guess I'll be wearing active hearing protection to the data center from now on, eh?
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
3. Re:I had no idea this could happen by Joce640k · 2018-04-20 07:30 · Score: 1
  
  I guess I shouldn't burp near my NAS then
  You jest, but, there's people who can burp at 100dB.
  https://encrypted.google.com/s...
  ie. This guy can burp at a heavy metal concert and the people around him will tell him to STFU because they can't hear the music for all the burping.
  
  --
  No sig today...
4. Re:I had no idea this could happen by Joce640k · 2018-04-20 07:36 · Score: 1
  
  I meant 110dB, dammit.
  
  --
  No sig today...
5. Re:I had no idea this could happen by Anonymous Coward · 2018-04-20 10:18 · Score: 0
  
  My every single drive I have is an SSD or flash, so I don't have to worry. Hard drives are obsolete tech.
Monosyllabic Much? by xxxJonBoyxxx · 2018-04-20 02:51 · Score: 1

>> this sound can get very loud, a big no-no

Jeezzez Crisco. What did we do now to get SlashDot editors talking down to us with one syllable at a time?
1. Re:Monosyllabic Much? by Nidi62 · 2018-04-20 02:57 · Score: 3
  
  >> this sound can get very loud, a big no-no Jeezzez Crisco. What did we do now to get SlashDot editors talking down to us with one syllable at a time?
  Would you prefer "the aforementioned audible emanation attained a decibel level detrimental to the proper operation of the installed data storage mechanisms"?
  
  --
  The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
2. Re:Monosyllabic Much? by frank_adrian314159 · 2018-04-20 03:06 · Score: 1
  
  Quite.
  
  --
  That is all.
3. Re:Monosyllabic Much? by Anonymous Coward · 2018-04-20 03:13 · Score: 1
  
  Yes actually. We're not in kindergarten. Ideally the writing level would at least be appropriate for a 12th grade audience.
4. Re: Monosyllabic Much? by haonlladdis · 2018-04-20 03:24 · Score: 1
  
  I was told once that most news sources are actually written for a seventh grade reading level.
5. Re:Monosyllabic Much? by forkfail · 2018-04-20 03:26 · Score: 1
  
  Whelp, there go your mod privileges for life.
  Pointing out the... ah... youthful and inexperienced nature of our current crop of editors, let alone their lack of technical prowess, blatant political agenda, or patronizing manner of address to the folks who have been here just a wee bit longer than they have has consequences beyond being called an insensitive clod.
  How those who themselves are in need of a serious LARTing are now running this place is beyond me.
  
  --
  Check your premises.
6. Re:Monosyllabic Much? by Nidi62 · 2018-04-20 03:33 · Score: 1
  
  Yes actually. We're not in kindergarten. Ideally the writing level would at least be appropriate for a 12th grade audience.
  Anyone can usually memorize more complicated synonyms of everyday words in order to try and sound smarter. However, a much more effective demonstration of intelligence would be showing the ability to understand what level of communication is appropriate and necessary for both the subject and target of the communication. In effect: Keep It (as)Simple (as possible) Stupid.
  
  --
  The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
7. Re:Monosyllabic Much? by suutar · 2018-04-20 03:42 · Score: 1
  
  that's a quote from TFA
8. Re:Monosyllabic Much? by Richard+Stalin · 2018-04-20 04:08 · Score: 3, Interesting
  
  “It's a beautiful thing, the destruction of words. Of course the great wastage is in the verbs and adjectives, but there are hundreds of nouns that can be got rid of as well. It isn't only the synonyms; there are also the antonyms. After all, what justification is there for a word which is simply the opposite of some other words? A word contains its opposite in itself. Take 'good,' for instance. If you have a word like 'good,' what need is there for a word like 'bad'? 'Ungood' will do just as well--better, because it's an exact opposite, which the other is not. Or again, if you want a stronger version of 'good,' what sense is there in having a whole string of vague useless words like 'excellent' and 'splendid' and all the rest of them? 'Plusgood' covers the meaning, or 'doubleplusgood' if you want something stronger still...In the end the whole notion of goodness and badness will be covered by only six words--in reality, only one word. Don't you see the beauty of that, Winston?”
9. Re: Monosyllabic Much? by rickb928 · 2018-04-20 04:16 · Score: 1
  
  I wrote tech bulletins and advisories at the 8th grade level. on advice of the vendors.
  Troubleshooting and diagnosis guides and general documentation I wrote at the 5th grade level, by request, again by those with long experience writing to technical audiences. Not because they are deficient, or marginally illiterate, but because they pay too little attention to detail, are hurried, and don't really care enough. And to limit the vocabulary to the most commonly shared set. Being clever doesn't serve your audience in these situations, even if they are as clever as you.
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
10. Re:Monosyllabic Much? by rickb928 · 2018-04-20 04:20 · Score: 1
  
  This is wrong, by the way, demonstrably wrong.
  'Ungood' isn't the exact opposite of 'good'. It may be better defined as the absence of good, which could be neutral or 'bad'.
  You started it, not including the sarcasm flag is your fault.
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
11. Re:Monosyllabic Much? by forkfail · 2018-04-20 04:30 · Score: 1
  
  Please tell me that you yourself left off the /sarcasm flag in your response here, that you understood the reference being made.
  
  --
  Check your premises.
12. Re:Monosyllabic Much? by Anonymous Coward · 2018-04-20 04:56 · Score: 0
  
  That whoosh is not the air being sucked out a 737 window.
13. Re:Monosyllabic Much? by Anonymous Coward · 2018-04-20 05:37 · Score: 0
  
  really?
14. Re:Monosyllabic Much? by rickb928 · 2018-04-20 06:01 · Score: 1
  
  How would you know?
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
15. Re:Monosyllabic Much? by sjames · 2018-04-20 06:29 · Score: 1
  
  Nobody wants subtle nuance in a tech manual, save it for poetry and novels. Otherwise you just read like an 8th grader trying to sound sophisticated.
16. Re:Monosyllabic Much? by Pascoea · 2018-04-20 06:36 · Score: 1
  
  Exactly. That's the key to technical writing, use exactly as many words as are necessary to accurately get a point across, and no more.
17. Re: Monosyllabic Much? by Anonymous Coward · 2018-04-20 06:56 · Score: 0
  
  Interesting. I understood exactly what you wrote, but, because of the vocabulary used, run on sentences, and thoughtless construction - it was difficult to read. And even more difficult to remember. I do remember what the big no-no is.
18. Re: Monosyllabic Much? by Anonymous Coward · 2018-04-20 12:58 · Score: 0
  
  See Mac run. Run, Mac, run!
Total failure by Daetrin · 2018-04-20 02:51 · Score: 2

"caused by a gas-based fire alarm system that are typically deployed in data centers because of their ability to put out fires without destroying non-burnt equipment. [...] With non-calibrated systems, this sound can get very loud [and destroy-non burnt-equipment]"

You had one job! (Directed at both the system itself and whoever was supposed to be in charge of calibrating it.)

...actually, that person may have zero job now.

--
This Space Intentionally Left Blank
1. Re:Total failure by painandgreed · 2018-04-20 09:49 · Score: 1
  
  You had one job! (Directed at both the system itself and whoever was supposed to be in charge of calibrating it.) ...actually, that person may have zero job now.
  They were probably a contractor that offered to calibrate the system but some manager decided he didn't want it coming out of their budget.
2. Re:Total failure by aaarrrgggh · 2018-04-21 12:36 · Score: 1
  
  This is actually a common failure mode for FM200 systems (fire PROTECTION, not fire ALARM). The last I had heard about it, research was suggesting that it was the sudden pressure change, but I hadn't been following it closely since.
  A couple bank data centers were hit by it (HSBC was one) in 2016.
Comment removed by account_deleted · 2018-04-20 02:52 · Score: 1

Comment removed based on user account deletion
SSDs by Anonymous Coward · 2018-04-20 02:57 · Score: 0

I herd SSDs are silent and mostly immune to vibration.
1. Re:SSDs by Anonymous Coward · 2018-04-20 02:59 · Score: 3, Funny
  
  I herd SSDs are silent and mostly immune to vibration.
  How can you herd them if they are silent?
2. Re:SSDs by Anonymous Coward · 2018-04-20 03:16 · Score: 0
  
  All power electronics make noise. :P So it depends on your definition of silent.
3. Re:SSDs by Anonymous Coward · 2018-04-20 03:17 · Score: 0
  
  Everything has a harmonic frequency. But yeah, I expect that your target being stationary silicon rather than a metal disk spinning at 10K RPMs might make this a tad more difficult to accomplish. Or, at the very least, should it happen, you'll have a lot more to worry about than hard drives.
4. Re:SSDs by Anonymous Coward · 2018-04-20 03:22 · Score: 0
  
  Everything everywhere makes noise as a result of molecular vibration.
5. Re:SSDs by mschaffer · 2018-04-20 03:54 · Score: 4, Funny
  
  You herd them the usual way---with dogs. The dogs can see them and smell them, so they don't need to hear them.
6. Re:SSDs by Anonymous Coward · 2018-04-20 04:02 · Score: 0
  
  All power electronics make noise. :P So it depends on your definition of silent.
  Yeah well the amount of noise a linear regulator makes is essentially not measurable, so not entire true.
7. Re:SSDs by Anonymous Coward · 2018-04-20 04:03 · Score: 0
  
  I was hoping to use dirt bikes to herd them. More fun, but less adorable.
8. Re:SSDs by rickb928 · 2018-04-20 04:22 · Score: 1
  
  "How can you herd them if they are silent?"
  You're doing it wrong. You're supposed to cluster
  Oh, wait...
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
9. Re: SSDs by BKX · 2018-04-20 04:27 · Score: 1
  
  Most electronics use switching supplies for the base power rails. Switching supplies have capacitors, some of which may be charged and discharged at a frequency in the audible range. Those capacitors can be quite loud. You can even use them as speakers. Nowadays most switching supplies use higher frequencies to cut down on audible noise, but noisy caps are still a thing. Furthermore, there are almost always caps that smooth out the AC input. 50/60hz is audible, so those caps will always hum.
10. Re:SSDs by webnut77 · 2018-04-20 04:41 · Score: 1
  
  I herd SSDs are silent and mostly immune to vibration.
  How can you herd them if they are silent?
  Just gather them up in a coral. A sheep dog can help with that.
  A keep those doggies rolling. Rawhide!
11. Re:SSDs by cellocgw · 2018-04-20 04:49 · Score: 3, Funny
  
  I herd SSDs are silent and mostly immune to vibration.
  How can you herd them if they are silent?
  Just gather them up in a coral. A sheep dog can help with that.
  That won't work either. All the coral is dying.
  
  --
  https://app.box.com/WitthoftResume Code: https://github.com/cellocgw
12. Re: SSDs by Anonymous Coward · 2018-04-20 05:11 · Score: 0
  
  Inductors hum, caps don't. Smoothing caps are on the rectified side and operate at 120 Hz, not 60 Hz.
13. Re:SSDs by UnknownSoldier · 2018-04-20 06:07 · Score: 1
  
  Allegedly (*) the same way you herd cats -- with great difficulty. =P
  (*) I've never met anyone who has actually done (**) this yet everyone is magically an "armchar expert".
  (**) I don't understand what the big deal is -- you just wait till everyone is hung and then string them along with food. How hard (***) could it be? :-)
  (***) Famous last words.
14. Re:SSDs by UnknownSoldier · 2018-04-20 06:09 · Score: 1
  
  Whoops, Freudian (?) slip.
  s/hung/hungry
15. Re: SSDs by Anonymous Coward · 2018-04-20 06:18 · Score: 0
  
  I've never seen an electrolytic cap hum at line frequency.
  Well of course not, things would need to vibrate at much higher frequencies than 50/60hz to get into the visible light range ;P
16. Re:SSDs by webnut77 · 2018-04-20 06:21 · Score: 1
  
  I herd SSDs are silent and mostly immune to vibration.
  How can you herd them if they are silent?
  Just gather them up in a coral. A sheep dog can help with that.
  That won't work either. All the coral is dying.
  Really? What color? Blue, I hope. I have a bolt of cotton cloth I'd like dyed blue.
  How do I contact the coral?
Okay, so... by Anonymous Coward · 2018-04-20 02:57 · Score: 0

...no more Taco Bell for lunch when I need to visit the data center that afternoon.
Re:Happy Friday From A Crashing Market! by Richard+Stalin · 2018-04-20 02:58 · Score: 1

There's a kernel of truth to this. I know a few guys who work for the NYSE, and it's the same there. One of them told me to watch a few particular derivatives before they take down the system and watch them after they bring the system back online. Three out of four times this is to prevent a catastrophe and had nothing to do with a true hardware/software failure. But it does keep a '29 stock market crash from happening.
Could this be used maliciously? by Anonymous Coward · 2018-04-20 03:00 · Score: 0

I expect vandals and rogue employees with now be releasing loud noises to destroy data centers. I hope hard drives in the future get soundproofing to stop this.
Fire SUPPRESSION system not fire ALARM system by jtara · 2018-04-20 03:08 · Score: 4, Informative

It's a fire SUPPRESSION system, not a fire ALARM system.
At least this time, it's the article (on Bleeping Computer) that is wrong, not the summary on /.
It's apparent the author of the article didn't bother to read the article (on Motherboard) that she cites about a similar incident in Romania at ING Bank. It clearly states that incident resulted from a "fire extinguishing test".
The sound BTW comes from the release of the gas, not some alarm. In both cases, the pressure was set too high. It was basically a - very loud - over 130db - hissing sound!
The second article cites a study about the effect of this sound done by Siemens. Siemens has a vested interest. I guess neither of these data centers were equipped with the Sinorix Silent Nozzle.
1. Re:Fire SUPPRESSION system not fire ALARM system by Critical+Facilities · 2018-04-20 03:23 · Score: 1
  
  Thank you. Moreover, I would bet that it wasn't the sound, but the particulates that did it. Whether this was Halon, or FM-200, or some other "clean agent", if you've ever seen a Data Center after one of those goes off, it's covered in powder and dust. The idea that they're truly "clean" is complete crap. Sure, they won't soak your gear in water, but it's not this lovely, clean, unobtrusive gas like the sales folks tell you it is.
2. Re:Fire SUPPRESSION system not fire ALARM system by snugge · 2018-04-20 03:50 · Score: 1
  
  Hard disks don't die from dust. They *do* die from too loud noises, though.
  Seen that first hand after an incident just like this one.
3. Re:Fire SUPPRESSION system not fire ALARM system by bobbied · 2018-04-20 03:52 · Score: 1
  
  Oh sure the GAS is clean and unobtrusive... The problem is the speed it is released and where it blows.... I've seen what ends up under the raised floors or in the rafters in a data center after a few years in operation and it's not pretty. Imagine all this dirt, dust and trash suddenly flying around.
  House keeping! Clean up in the server room!
  Then there is the whole, fire alarm pulled, kill the power to limit the damage! Use case. Modern computers do NOT take kindly to having the power yanked out from under them and few take the time to do an orderly shutdown when a fire was reported.
  
  --
  "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
4. Re:Fire SUPPRESSION system not fire ALARM system by Khyber · 2018-04-20 03:53 · Score: 1
  
  "Moreover, I would bet that it wasn't the sound, but the particulates that did it."
  All spinning rust hard drives are fully-sealed against particulates. SSDs don't need that, and the stuff used in electronics fire control tends to be non-conductive.
  
  --
  Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
5. Re:Fire SUPPRESSION system not fire ALARM system by Megol · 2018-04-20 03:57 · Score: 1
  
  Do they still use halon? Thought it was reserved for military and critical civilian uses only.
6. Re:Fire SUPPRESSION system not fire ALARM system by Critical+Facilities · 2018-04-20 04:00 · Score: 1
  
  Perhaps the drives themselves don't, but the servers or DASD boxes they're sitting in are very susceptible to dust. I realize the article says the hard drives failed, but further down they talk about needing to replace the servers. I'm asserting that it was more likely the dirt and debris that ultimately caused the problem, not the sound.
7. Re:Fire SUPPRESSION system not fire ALARM system by Szeraax · 2018-04-20 04:13 · Score: 2
  
  Obligatory story about new "clean agent" fire suppression systems (hint: nitrogen + water):
  Tuesday was a regular day for me. I am working remotely and live almost 2 hours away from work. At 11:02am, this chat began between my boss and me:
  Boss: Please monitor all systems. Alarm system fired in server room
  Me: I can't get on firewall
  Me: I'm off vpn
  Me: You still have internet?
  Me: Do you mean the fire alarm went off?
  Me: Or other alarm
  Me: Still can't get on vpn
  Me: I was hoping to be able to get on via X2 or X3, but I still can't get on VPN via any WAN interfaces.
  Me: $TaskServer seems to be working; just got an email generated by it. so $SMTPServer must also working
  Boss (8 agonizing minutes later): Fire
  Turns out... our data center fire alarm was getting tested today! And the company that installed it had one of their techs come out to do the test.
  My boss was in the server room escorting him when it happened. Normally they disengage the solenoid from the suppression system when testing, but he must not have done it right. They heard a weird hissing noise... like something was charging.
  Now, since our server room is NOT sealed, we can't use systems that remove oxygen from the air. We bought a new fancy suppression system that will basically atomize a gallon of water for a 12'x12' space distributed by a tank of pressurized nitrogen. This distributed water is safe to breathe, safe to suck through fans and all equipment, and totally kills fire.
  Well guess what? the Cisco ASR 1000 router that provides our primary internet (that we all go through to get to firewall for VPN) didn't like all the moisture and rebooted itself (maybe it shorted? dunno). Hence, it took like 15 minutes for me to be able to get back onto VPN. It also was making weird fan noises.
  $Boss described it as a very thick fog that cleared after a minute or two. Like... can't see 6" from your face initially. Needless to say, he isn't very happy with the alarm company at the moment. Hopefully none of our equipment will suffer from the test.
  In the aftermath, I noticed that some yahoo (hopefully not me) had disabled incoming VPN on the X2 and X3 backup ISPs. So that's why I couldn't get in on those other interfaces...
8. Re:Fire SUPPRESSION system not fire ALARM system by rickb928 · 2018-04-20 04:30 · Score: 1
  
  There are so many reasons to not use water.
  Unless it's very pure, you will deliver minerals, salts, with predictable consequences, not all immediate.
  Water will also dissolve and/or deliver contaminants throughout equipment. Again, not good, and the impacts not necessarily immediate.
  As a solvent, water is different than many used in manufacturing electronic equipment, so what wasn't a problem before exposure may become later. Even high humidity can cause this.
  Pure water should be safe, but it's all the other contaminants that it dissolves and delivers that are the problem. And datacenters are often not very clean, despite looking so.
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
9. Re:Fire SUPPRESSION system not fire ALARM system by Anonymous Coward · 2018-04-20 04:37 · Score: 0
  
  Exposed circuit traces and wires gain capacitance when moist. In some cases, this is enough to upset a delicate circuit, leading to temporary or sometimes permanent failure.
10. Re:Fire SUPPRESSION system not fire ALARM system by Anonymous Coward · 2018-04-20 04:39 · Score: 0
  
  Does anyone have the physics on this? How does a sound above 130db, end up destroying HD's in this case?
  For spinning disk, I could see it, even if buried in a rack. But for SSD's? Not sure I follow.
11. Re:Fire SUPPRESSION system not fire ALARM system by Szeraax · 2018-04-20 04:56 · Score: 1
  
  It is just regular filtered, distilled water. The high humidity is what caused our WAN router to reboot and freak out. We still haven't had any extra issues due to it, but that's not to say that it WONT happen in the future from this one event. We were not pleased with the alarm company.
12. Re:Fire SUPPRESSION system not fire ALARM system by painandgreed · 2018-04-20 09:52 · Score: 1
  
  The sound BTW comes from the release of the gas, not some alarm. In both cases, the pressure was set too high. It was basically a - very loud - over 130db - hissing sound!
  Ya, but if you don't pay for calibration and extra nozzles, you get a fire extinguishing system AND an alarm for less cost!
13. Re: Fire SUPPRESSION system not fire ALARM system by Anonymous Coward · 2018-04-21 23:43 · Score: 0
  
  You sir are uninformed, inert and halocarbon based media are indeed "clean agents", they leave nothing behind and they do not under any circumstances damage equipment.
  No contractor in the business would not present a solution to the customer on how to avoid the potential damaging of modern high speed hard drives upon the release of the gas. This is of course a well known issue with certain types of systems (not an issue with low pressure systems) if you don't take measure. The counter measure is not cheap to install and that means managers sometimes opt not to buy it. The problem was non existent before when the discs were slower and less sensitive.
Odd by Virtucon · 2018-04-20 03:13 · Score: 3, Informative

When my Ex-Wife let out loud, obnoxious sounds my hard drives didn't self-destruct. My mind did, but not my hard drives.

--
Harrison's Postulate - "For every action there is an equal and opposite criticism"
1. Re:Odd by Anonymous Coward · 2018-04-20 03:55 · Score: 0
  
  Dude, your mind was obviously blown before she became your wife....
2. Re:Odd by hraponssi · 2018-04-20 05:59 · Score: 1
  
  Sounds like you got away, and I like the sound of that.
3. Re:Odd by antdude · 2018-04-20 14:02 · Score: 1
  
  What about your dongle? Not the computer hardwares! Your body's. :P
  
  --
  Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
Well known for ages now by green1 · 2018-04-20 03:22 · Score: 1

These sorts of systems, have been well known for ages now to be a high risk for damaging equipment, most of them release with such force that they can do a lot of harm. As a result, many data centers, and other similar businesses have moved away from them to avoid exactly this sort of incident. A popular alternative right now is water mist, despite what people immediately think, these have actually been proven to be safe around electronics when properly designed and deployed.
1. Re:Well known for ages now by Anonymous Coward · 2018-04-20 04:00 · Score: 1
  
  These type of systems work by displacing of reacting or otherwise removing the oxygen from the room - starving the fire. I am pretty sure most businesses have moved away from them because of the liability of killing all the humans that don't make it out of the room before the gas releases.
2. Re:Well known for ages now by Strider- · 2018-04-20 05:33 · Score: 3, Insightful
  
  For most common materials to burn, you need an oxygen content of 12% or higher. On the other hand, a human can maintain consciousness down to something like 5%. As such, inerting systems are designed to drop the oxygen content to something like 8%. Too low for combustion to take place, but high enough not to kill the occupants. Nitrogen type inerting systems will actually often include a small amount of CO2 in the gas mix; this causes any remaining occupants to breath harder, thus allowing them to work better in a low oxygen atmosphere.
  reference: I worked on a small power plant with a two compartment FM-200 fire suppression system. One of the things that the design Engineers needed was an accurate measurement of the room volume so that they could calibrate the amount of agent in the tanks appropriately. However, just in case, before an agent dump there is a 30 second siren and several large "Cancel dump" mushroom switches.
  
  --
  ...si hoc legere nimium eruditionis habes...
3. Re: Well known for ages now by Anonymous Coward · 2018-04-21 23:51 · Score: 0
  
  You are just trying to discredit gas suppression systems, you obviously work with water mist. There is no moving away from gas since there is no problem, you can easily build in counter measures in the systems.
Poor Planning? by bestweasel · 2018-04-20 03:22 · Score: 3, Informative

Another site without replication, redundancy and automatic failover, similar to when British Airways' entire computer system collapsed because of a power surge when someone turned it off and on again.
It's not possible to test all scenarios ("Right, now we're going to see how our network would cope if someone took an axe to that cabinet. Derek, when you're ready") but someone should have planned for a data centre failure. Is this lack of resilience common in big networks?
1. Re:Poor Planning? by bobbied · 2018-04-20 04:02 · Score: 1
  
  Is this lack of resilience common in big networks?
  Yes, it is. Resilience is conceptually easy to understand, but historically exceptionally hard to implement because it requires rigorous controls on such mundane things as where you plugged that server and network switch in. Because it's so hard to maintain, supposedly resilient systems often are not really. Somebody moved some power cable, switched a network port to another switch or forgot to update the standby system and their data backup scripts... Then the unthinkable happens and you discover some simple mistake that renders all that effort and money spent pointless.
  
  --
  "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
2. Re:Poor Planning? by rickb928 · 2018-04-20 04:33 · Score: 1
  
  Yup, as simple as unplugging a power cord because 'everything's redundant, we can cover this'.
  You can do that once only.
  
  --
  deleting the extra space after periods so i can stay relevant, yeah.
3. Re:Poor Planning? by F.Ultra · 2018-04-20 05:15 · Score: 1
  
  Well they have a secondary site at which they opened trading hours later, the question that is left for Nasdaq to answer is why it took them so long to open trading on the secondary site when they and us the members take part in regularly disaster exercises where everyone switches to use the secondary site within seconds.
4. Re:Poor Planning? by F.Ultra · 2018-04-20 05:18 · Score: 1
  
  And it's somewhat more complicated here since also every member of the exchange have to switch over their systems to use the secondary site, and due to their rules they cannot open unless all members have switched. Also there had already been trades between members right before and right at the moment the servers was destroyed so there where some cleaning up to do as well (all the trades had to be cancelled).
5. Re:Poor Planning? by sjames · 2018-04-20 08:10 · Score: 1
  
  One of my favorites was a diesel generator with a small local tank and a larger tank installed well behind the building. Never a problem during the monthly tests where they actually failed over to the generator.
  Actual power failure happens, generator picks up the load just fine. An hour later, the generator shuts down. Turns out the transfer pump that fed the small local tank from the large tank was connected to grid power only.
6. Re: Poor Planning? by Anonymous Coward · 2018-04-20 22:08 · Score: 0
  
  An electric substation flooded and blew up leaving the office without power so we had this huge generator sitting in the car park for a few weeks. We powered up the essential systems and came in the next day to find everything off again. Nobody had checked how much fuel was in the generator.
Didn't we learn not to yell at hard drives? by Anonymous Coward · 2018-04-20 03:28 · Score: 2, Insightful

No seriously, I can't find the video atm but many years ago there was a guy from (Sun?) showing how susceptible they are to sound vibrations. His test literally was yelling at them while watching dtrace.
Update: Found it. Was Brendan Gregg - https://www.youtube.com/watch?v=tDacjrSCeq4
* (headphone warning)
1. Re:Didn't we learn not to yell at hard drives? by nanoflower · 2018-04-20 04:14 · Score: 1
  
  Ah, yes. I remember that video. I was quite surprised that drives could be impacted that easily just by the human voice.
Reminds me of an old Usenet legend... by Anonymous Coward · 2018-04-20 03:28 · Score: 1

Dunno if it's true or not, but I've always found it amusing nonetheless:
https://www.hactrn.net/sra/vaxen.html
Not fire alarm by makerfixer · 2018-04-20 03:35 · Score: 1

Pro tip: fire alarms make noise, fire protection systems put out fires.... this was the latter. A clean agent release system.
1. Re:Not fire alarm by omnichad · 2018-04-20 04:14 · Score: 1
  
  But it was set up wrong, so it made noise. So it's also an alarm by your definition.
2. Re: Not fire alarm by makerfixer · 2018-04-20 18:31 · Score: 1
  
  My definition of an alarm is a temporal pattern set up to alert occupants. Not the sound of rushing gas.
screaming by supernova87a · 2018-04-20 03:45 · Score: 0

There's an interesting Youtube video where a server admin screams directly into a rack of spinning drives, and shows the latency / error rate spiking on the screen as he does that. I forget where it is but it's interesting.
1. Re:screaming by mschaffer · 2018-04-20 04:09 · Score: 2
  
  It's here: https://www.youtube.com/watch?...
Actually, it's the opposite by Anonymous Coward · 2018-04-20 03:51 · Score: 1

Reporters and bloggers cannot even get the facts right let alone write at a higher level.
Fire alarms make noise to warn people.
Fire suppression systems extinguish fires.
Fire protection systems often incorporate suppression and alarm systems.
If you've ever wondered what it sounded like.. by Alpha232 · 2018-04-20 04:22 · Score: 2

This sounds like an Inergen type system... the two failings are the particulate matter it stirs up and the vibration caused by the sounds of a freight train rolling through...
https://youtu.be/yM80eBR_b2w
1. Re:If you've ever wondered what it sounded like.. by Anonymous Coward · 2018-04-20 04:33 · Score: 0
  
  The old tech was Halon, which was taken out of service because it harmed the ozone layer. In the last few years our data centre had a fire suppression system installed based upon Sapphire, which is sold by Tyco.
  Sapphire is crazy stuff; it's a non-conducting fluid. It's not cheap though.
Doh! by Anonymous Coward · 2018-04-20 04:27 · Score: 0

Why the fuck do hard drive manufacturers pay top dollar for some of the cleanest assembly rooms on the planet then? Oh, yes, it's because a speck of dust on the platter can cause a head crash. Disassemble a drive or two - you will find labyrinth "seals" designed to prevent dust ingress during pressure fluctuations.
tldr; Dust will indeed kill hard disks, but manufacturers try to prevent it.
1. Re:Doh! by Anonymous Coward · 2018-04-20 06:25 · Score: 0
  
  Why do people on here have to go 'full-on pedantic commentard' ?
  You know damn well what the previous commenter meant about noise vs particulates killing the drives in question.
  Do data centers run their drives with exposed platters ?
Comment removed by account_deleted · 2018-04-20 04:33 · Score: 1

Comment removed based on user account deletion
Wednesday, April 19? by GungaDan · 2018-04-20 04:47 · Score: 1

Something's not quite right about that. For the year 2018, at least.

--
Eloi are stupid, throw morlocks at them!
Sigh by Anonymous Coward · 2018-04-20 05:06 · Score: 0

This is a fire SUPPRESSION system, not a fire ALARM system.
3M Novec 1230 by Anonymous Coward · 2018-04-20 05:15 · Score: 0

"Sapphire" is just a rebranding of 3M's Novec 1230 fire suppression liquid, which is a perfluorocarbon.
https://en.wikipedia.org/wiki/Novec_1230
https://www.3m.com/3M/en_US/novec-us/applications/fire-suppression/
Date by Anonymous Coward · 2018-04-20 05:20 · Score: 0

Wednesday was the 18th
"Suppression" is the word you were looking for by Pascoea · 2018-04-20 06:41 · Score: 1

A fire alarm system isn't designed to extinguish a fire, a suppression system is. I hate to be pedantic, but there is a difference. The device in your home that disturbs fajita night is an alarm system...
Re:Happy Friday From A Crashing Market! by Anonymous Coward · 2018-04-20 08:10 · Score: 0

So why does an american exchange have a scandinavian data center?
Re: Happy Friday From A Crashing Market! by Anonymous Coward · 2018-04-20 08:23 · Score: 0

Why does that matter?
Sometimes it happens on purpose by magarity · 2018-04-20 09:11 · Score: 1

Did anyone bother to check if there was an actual fire or are they too busy kvetching about the noise?
Re: Happy Friday From A Crashing Market! by Anonymous Coward · 2018-04-20 09:23 · Score: 0

Does it really matter? What reason would a company want for a data center in a stable place away from their primary one?
I don't know if they do use it as a backup for their NA operation but they do run exchanges in the Nordic and Baltic regions as well.
Can confirm by RabidMonkey · 2018-04-20 10:28 · Score: 2

At a previous employer, we lost an entire row of servers in a DC after a water leak (somehow) triggered the suppression system. The 'explosion' was strong enough to knock the doors off cabinets, bend 2 cabinets, and cause a couple hundreds drives to be dead. Thankfully our service was spread out far enough to survive the loss of a row for a few week while we waited for all new disks to arrive from IBM.
The pictures were crazy, it looks like a bomb went off.

--
We emerge from our mother's womb an unformatted diskette; our culture formats us. - Douglas Coupland
Muad'Dib! by Anonymous Coward · 2018-04-20 12:10 · Score: 0

Destroyed by sound: Muad'Dib!
Re:3rd time impersonating + libeling me? by Anonymous Coward · 2018-04-20 12:27 · Score: 0

Awww, fat gay virgin APK is all triggered. Sad.
3rd time impersonating + libeling me? by Anonymous Coward · 2018-04-22 02:46 · Score: 0

See subject you UNIDENTIFIABLE ac & how DUMB you are on hosts vs. addons https://it.slashdot.org/commen... lol!
It's why hosts files = faster & start LONG BEFORE redundant, crippled by default, LIMITED IN ABILITY vs. hosts & overhead ridden inefficient addons!
* "Butthurt" over that MASSIVE FUCKUP of yours? Yes.
(No wonder you HIDE behind UNIDENTIFIABLE ac posts - I've BLOWN YOU AWAY so many times you HAVE to hide vs. using your "registered 'luser'" FAKE NAME for your FAKE LIFE acct!)
3rd time today you pulled this https://news.slashdot.org/comm... & https://science.slashdot.org/c...
Your other lie about me = disproved too https://it.slashdot.org/commen... - Don't "F" w/ your betters (me)
* I am SURE it disappoints you projecting as you are? I'm not gay.
APK
P.S.=> Enjoy a downmod... apk
Re:Happy Friday From A Crashing Market! by Anonymous Coward · 2018-04-22 23:35 · Score: 0

So why does an american exchange have a scandinavian data center?
Ha, ha, ha, you're going to fucking shit yourself when you finally find out the NYSE is owned by Euronext!
Don't worry though - they're all fucking retard right-wing scum too, so the difference between French and American ownership should be virtually undetectable for most purposes.