The NHS's 1.2 Million Employees Are Trapped in a 'Reply-All' Email Thread (businessinsider.com)

An anonymous reader shares a Business Insider report:The NHS's 1.2 million employees are currently trapped in a "reply-all" email hell. A "test" email was accidentally sent to everyone who works at the UK health service - prompting a series of reply-all responses from annoyed recipients going out to all 1-million-plus employees of the organisations. An NHS employee told Business Insider that there have been at least 120 replies so far -- meaning that more than 140 million needless emails have been sent across the NHS's network today. As a result, they said, its email systems are running "very slow today." The NHS Pensions department is currently warning people on Twitter that "if contacting us by email please be aware that there may be delays in responding due to an issue currently affecting all NHS mail."

Comments

Please remove

[Mycroft-X]

Please remove me from this distribution, I don't know how I got on it.

Re:Please remove

[aicrules]

Sorry I don't own this distribution, please contact someone else.

Re:Please remove

[Jason Levine]

*replies all*

Hey everyone, stop replying all! This is very annoying.

Re:Please remove

[Jose]

Me too!

Re:Please remove

[Anonymous Coward]

Can people stop replying to this. Thanks.

Re:Please remove

[Joce640k]

It's almost as if there's no way for the mail server to filter messages with more than a million recipients.

Re: Please remove

[tripleevenfall]

Thank you for your message. I am out of the office until 23/11/2016. For immediate service, please contact the Help Desk.

Re:Please remove

[dgatwood]

The problem is that mail clients don't know whether the "all@" mailing list email expands to five users or five million. As far as it is concerned, you're sending it to one address.

The correct solution is server-side, not client-side. Specify a policy that any mailing list with more than... say thirty people must have an authorized senders list, and must reject emails from anyone not on that list. That way, when someone responds to the "all@" list without a "Resent-From" header from someone on the authorized senders list, it will get dropped.

I survived

[Anonymous Coward]

I survived Bedlam DL3

The story of when Microsoft themselves fell victim to the same issue, and how it was resolved.

Re:I survived

[Anonymous Coward]

I lived through one at a large, global financial company. It was kind of humorous.

Some lady in a small European office of the company attended a company sponsored Halloween party dressed as a policewoman. Part of her costume included mirror sunglasses. She happened to misplace them at the party.

So the Monday after the party she sent an email to her group explaining the situation and asking for everyone to look for them. We were on Lotus Notes and she accidentally chose * (iirc) instead of her groups distribution list. So 30,000 people across the world got the email.

The funny part is that every smart ass in the company decided to have fun with it. Reply alls started flooding the network with messages such as "Looked in my office in Germany and can't find them", "Not in Wisconsin office", and "Didn't find them but did find handcuffs, you want those?". Went on for hours and basically took the email system down for 2 days.

The worst part is that they had a few of the servers mis-configured or something because anyone who had an auto out of office reply would respond to all with that message, which would then trigger the other out of office replies again.

Everyone spent a really long time removing tens of thousands of emails from their mailboxes.

I'd say

[argStyopa]

...email was God's gift to business. Transformative, empowering, a paradigm-shift.

It's Satan that added Reply-all, and then BCC just to continue the general fuckery.

Re:I'd say

[Anonymous Coward]

But bcc prevents this entire problem. If you send a message to a large distribution you should always use bcc. Then when some retard hits reply-to-all the reply is only directed to the original sender rather than the entire distribution list.

So common...

[aicrules]

I don't work for a company with over a million employees but I see this happen frequently. People reply all out of rote habit, not even consciously. It's so annoying. I know one company who customized their distribution of Outlook to not have a reply all button. Short of that, my recommendation is to either protect the large distribution group so that only a select few can email it and/or making use of BCC for that group when original emails are sent so that even if people do reply to all there is no further waste of time. I tried individually shaming people when they did it for a while, sending them dumbass instructions on the location of the Reply button versus Reply to All, but that had a limited impact other than for more people to know I'm an asshole. Oh well...

Re:So common...

[religionofpeas]

On the other hand, if you send an e-mail to a small group of people, e.g. trying to solve a problem together, it's very annoying if one or more people to use Reply instead of Reply-All, and the rest of the group misses part of the conversation.

It is useful heuristic

[abies]

I have seen it few times in big corporations I worked in. Somebody sends email to wrong group by accident and then we have 3 waves of attack:

1) Clueless people hitting 'reply all' asking for removal from mailing group
2) Even more clueless people hitting 'reply all' asking people to not 'reply all'
3) "Champions" trying to save a day by putting all in BCC and telling people to not reply all, unless you put it in BCC [1]

And then, few hours later, next timezone wakes up and things start again.

Why is it useful? After it is obvious what is happening, you create folder called 'idiots' and redirect all these emails into that group by outlook/whatever rule. After that, if you need to deal with somebody in your organization, first check if he/she is in idiots folder and approach accordingly.

BTW, 120 replies seems very low. I have seen mailstorms with group of 10k recipients (it was not 'all' group, just some subset of company) generate over 600 replies total in these 3 waves. 120 replies from 1.2 million looks to be technical limitation (or, maybe, there was some hero in IT department who pulled the plug fast enough...)

[1] - My favorite is self correcting champion, which first sends 'reply all' and then does reply to that with everybody in BCC saying he should have put everybody in BCC in first place...

Re:Good ol' fun

[Archangel Michael]

Emailing a group like that should have been limited to a very small number of people. Everyone else should have got a message that they were not authorized to use that email message.

This is email server admin 101 level knowledge.

This gets expensive...

[__aaclcg7560]

A bean counter at a local hospital sent out an email to everyone that explained the cost in lost productivity for each "reply all" email was $0.08 per person. Not surprisingly, someone hit the "reply all" button to respond that the bean counter's email cost the hospital $800 in lost productivity. It went downhill from there. Not sure if the tab was $80,000 or $800,000 in lost productivity when everyone stopped hitting "reply all" button. Executive management wasn't amused.

Re:Really?

[Nutria]

Which requires an Emergency Change Request (which takes hours to get approved for a Sev 2).

this has happened many times at Northrop Grumman

[pezpunk]

i worked at NG for 7 years, and there were several instances (the last one being in 2010) where email system company-wide was crippled or knocked offline by an email that was sent to the wrong mailing list. apparently, there was one available that included literally every single person in the system (probably about 100,000 people).

i remember one morning in about 2008 or so, getting an email addressed to some team i wasn't a part of, seeing the "CC" list was several miles long, and i knew instantly what was going to happen. i guess the "first post" instinct in me acted up for the first time ever. i knew we were all already doomed, so i hit reply-all and simply posted: "oh no, not again." i did manage to be first, but before i could blink, i had over a thousand new emails all saying some variation of "WHAT IS THIS?" "REMOVE ME FROM THIS LIST" and "STOP REPLYING FOR GODS SAKE". my new emails hit 30,000 in a few minutes.

the entire NG email system was down for more than a day.

two days later i got called into my boss's office and he explained that top-tier management at NG had demanded that i be fired. my "oh no not again" was the last email most people saw before the system exploded. a very heated conversation between my supervisors and NG executives apparently just barely saved my job, but my supervisors were not pleased either and mentioned this would go on my permanent record (i thought that was just a high school thing). it didn't matter that i didn't actually do anything to cause the crash. i had merely made myself visible at the wrong time, and NG wanted someone's head.

so glad i don't work there now.

It could be worse

[rickb928]

I get an email for a division-wide thing.

I get a copy from my VP's admin, specifically targeting my team in case we weren't on the original distribution list.

I get a copy of this from my manager, since he doesn't want me to miss this.

I also get a copy of the original from a corporate level special interest group I'm part of.

Then I get a copy from a former team member. Just in case I was left of their distribution, since they left our team but believe they may be getting team emails that current team members are not.

And a copy from their manager, with a note to be sure the distribution list I cannot administer is properly updated to get these mails from the list they should not be and indeed are not part of. Just in case.

Then I get a copy from an interested team leader who wants to make sure we are in the loop.

And another from their #1 team member, who looks out for us.

And finally my cubicle mate leans over and tell me 'hey, did you get the email from......'

And so I have a 14GB .OST that i cannot backup locally due to GPO. and i get warnings occasionally that my file will be groomed back to an unspecified maximum size. Some day, real soon now. Right after they encrypt my files for no apparent reason, in accordance with some policy I cannot get a copy of.

I'm not bitter, really. I feel for the corporate security and cloud services guys. They can't fix stupid.