Slashdot Mirror


Microsoft's Multi-Factor Authentication Service Goes Down For Second Week in a Row (zdnet.com)

Just over a week after a global problem with its multi-factor authentication (MFA) service plagued a number of users, another Microsoft MFA outage is impacting a number of customers. Many, but not all, of the customers reporting problems today seem to be U.S.-based. From a report: Starting around 9:15 a.m. ET, a number of Office 365 customers began reporting on Twitter that they were unable to sign into that service because of an MFA issue. Office 365 is one of a number of Microsoft services that uses Azure Active Directory MFA to authenticate. Around 10:15 a.m. ET, Microsoft's Azure status dashboard was updated to reflect the possibility of a cross-region potential outage impacting MFA. "Impacted customers may experience failures when attempting to authenticate into Azure resources where MFA is required by policy. Engineers are investigating the issue and the next update will be provided in 60 minutes or as events warrant," the dashboard status said.

54 of 101 comments (clear)

  1. You don't own your software by Geoffrey.landis · · Score: 2

    Yes: this is what happens when you don't own your software, you just "license" the use of it.

    --
    http://www.geoffreylandis.com
    1. Re:You don't own your software by Anonymous Coward · · Score: 5, Insightful

      Cloud is just a server run by someone else.

    2. Re:You don't own your software by phantomfive · · Score: 4, Informative

      It also seems to be something Microsoft does fairly regularly. They have a history of catastrophic failure of services. If they manage to get this one back up, it won't be their worst disaster.

      Remember not to trust the cloud: have backups because your stuff might be lost.

      --
      "First they came for the slanderers and i said nothing."
    3. Re:You don't own your software by eneville · · Score: 1

      Remember not to trust the cloud: have backups because your stuff might be lost.

      Are you sure? I thought the onedrive EULA made the content MS's property, so if it's lost, it wasn't yours to loose..

    4. Re: You don't own your software by Chris+Mattern · · Score: 1

      Why donÃ(TM)t they just do a pass through to the native multi factor authentication? There are a lot of tutorials on this.

      What native multi-factor authentication? MSA provides your one time code in their setup. No MSA, no one-time code, no two factor authentication. To continue on, you'd have to be distributing something else and be set up to use it as fall-back (or else be prepared to turn off 2FA entirely), in which case you have to ask why you're bothering with MSA at all.

    5. Re: You don't own your software by Chris+Mattern · · Score: 1

      Auto shops are just car repair done by someone else.

      Exactly. Which is why companies that own fleets of vehicles as part of their business tend to have in-house maintenance for those vehicles.

    6. Re:You don't own your software by im_thatoneguy · · Score: 2

      You plan on running your own 2 factor authentication token system? Good luck keeping it up 100% of the time for 100% of your users across the globe.

      Maintaining user authentication systems is pretty challenging. Keeping credentials maintained across phones, tablets, PCs, back end services and internal servers is not a simple service to maintain in house.

      How do you propose you "Own" a service which gives you single sign on authentication across your internal network, remote web services and offers 2 factor authentication? That's a lot of fragile infrastructure to maintain.

    7. Re:You don't own your software by BlackOverflow · · Score: 1

      What do you consider their worst disaster to be?

    8. Re:You don't own your software by im_thatoneguy · · Score: 1

      An airplane is just a car that has wings.

      Cloud is much more than a "server run by someone else", it's also "a server that you can lease by the second" and that's a huge shift in how you can look at your infrastructure. Need more database capacity? You don't need to plan ahead days or months and do a cost/benefit projection on whether or not you'll still need that capacity in 6 months, you can add another shard in a couple minutes and stop paying for it just as quickly when demand drops.

      You could run a 'private cloud' on your own servers run by you but "Cloud" will still be the distinguishing feature of your cluster. It describes the abstraction of hardware into compute and storage services to your developers and users.

      I can rent a server run by someone else and then I can rent 10,000 servers for 1 minute run by someone else and then stop renting 10,000 servers. If you run 10,000 servers you need to justify them taking up space 24/7 for 365 day and pay for them in perpetuity not just when you need them.

      I can buy storage by the GB and only pay for the GB that I need when I need it. Delete the file... stop paying. I can utilize a Petabyte for an hour and then delete it and stop paying. If I need a petabyte of storage for an hour I have to build out infrastructure and then find a buyer to sell it to when I'm done.

      I can rent server-less processor time by the millisecond. I can instantly scale a back-end web process that charges me $0.0000000000000001 to run with one user and then if I suddenly see 1,000,000 users slashdot my server instantaneously scale up for each additional user with cloud functions.

      The future of cloud computing is also server-less. You don't even manage shards or clusters or auto-scale virtual machines. The cloud is just a large shared mainframe that you lease utilization on. In the not too distant future you'll just pay by the query-millisecond and the rows and fields count you're using. You won't spin up virtual machines to process data you'll execute cloud functions which charge you by the millisecond of processor time. There'll be very few dedicated Virtual Machines and containers will contain smaller and smaller services that only run when called.

       

    9. Re: You don't own your software by phantomfive · · Score: 1

      The Danger incident was pretty bad....

      --
      "First they came for the slanderers and i said nothing."
    10. Re: You don't own your software by kaatochacha · · Score: 2

      Cool, saying " I had to take my car to the shop" makes me sound incompetent and incapable of fixing anything.
      Now, I can say "My cars going into the auto cloud",and I'm cool again!

    11. Re:You don't own your software by ilsaloving · · Score: 3, Insightful

      Rolling your own MFA would be a nightmare, considering how tightly the security needs to be controlled, so while what the parent says is true, sometimes it's just not practical.

      That means if you need to outsource to a vendor, that vendor has to be rock solid. Microsoft has a demonstrable track record of *not* being able to keep their infrastructure up, so I'm honestly dumbfounded that anybody would use their software willingly. Office365 is one thing because you really don't have a choice, and you can at least run the local version (unless Microsoft breaks the big brother functionality) but I would *never* trust mission-critical infrastructure to be managed by Microsoft.

    12. Re:You don't own your software by Geoffrey.landis · · Score: 1

      You plan on running your own 2 factor authentication token system? Good luck keeping it up 100% of the time for 100% of your users across the globe.

      I'm not sure why I should need two factor authentication to run my word processor.

      You do know that this is what we're talking about, right? Office 365. Which most people use as a word processor.

      --
      http://www.geoffreylandis.com
    13. Re:You don't own your software by antdude · · Score: 1

      "Trust no one." --The X-Files

      --
      Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
    14. Re:You don't own your software by khchung · · Score: 1

      And the grid is just a generator run by someone else.

      Do you also have generator in your house and office?

      --
      Oliver.
    15. Re:You don't own your software by dromgodis · · Score: 1

      No, it is Azure. Office 365 is just one of the services (presumably) deployed there. The outage affected systems from other organizations and people as well.

    16. Re:You don't own your software by thegarbz · · Score: 1

      Cloud is just a server run by someone else.

      Someone who is usually much better at running that server than most people.

    17. Re:You don't own your software by Geoffrey.landis · · Score: 1

      No, it is Azure. Office 365 is just one of the services (presumably) deployed there.

      Yes, "presumably". Specifically, the summary we are talking about starts "Starting around 9:15 a.m. ET, a number of Office 365 customers began reporting on Twitter that they were unable to sign into that service "

      The outage affected systems from other organizations and people as well.

      Sure.

      --
      http://www.geoffreylandis.com
    18. Re:You don't own your software by DeVilla · · Score: 1

      Nope. But I can still open the door to my house when the grid goes down. Sounds like people are having trouble running software locally (on their computers in their homes) now when there's a problem somewhere else in the world.

    19. Re:You don't own your software by ilsaloving · · Score: 1

      That's what momentum gets you, and why Microsoft can charge whatever they want and people have to pay (or pirate). I'm sure there are plenty of IT departments that would *love* to get away from Office. But even if they want to, they can't because staff insist on using it, and supporting multiple suites of tools is just not realistic when you have a large userbase.

  2. A particular movie quote comes to mind. by mcmonkey · · Score: 2

    Locally installed applications are not exposed to this mode of failure. This story is about as interesting as people who complain about breakfast hours at restaurants. Cook your own breakfast any time of day.

    Cue Airplane "They bought their tickets. They knew what they were getting in to. I say, let 'em crash."

    1. Re:A particular movie quote comes to mind. by nuckfuts · · Score: 2

      Locally installed applications are not exposed to this mode of failure. This story is about as interesting as people who complain about breakfast hours at restaurants. Cook your own breakfast any time of day.

      Show me the locally installed Multi-Factor Authentication solution that doesn't have any cloud component.

    2. Re:A particular movie quote comes to mind. by thegarbz · · Score: 1

      This story is about as interesting as people who complain about breakfast hours at restaurants. Cook your own breakfast any time of day.

      To take my very real life into your analogy I can't. I live in a Hotel. I am at the mercy of the breakfast hours of restaurants. I actively tell people at work not to book meetings at 7am with me as a result.

      Likewise MFA isn't just about accessing Word or Outlook. MFA from Microsoft can be deployed as the SSO option for an entire corporate infrastructure. If MFA is down and I type my domain password in incorrectly, I'm shitouttaluck as I need to pass the MFA to use our password reset facilities at work. Likewise I can't book flights, claim expenses, access Onedrive, my own Payslips, just to name a few of the many things that Microsoft's MFA Authentication has been baked into our system.

      What if you lock yourself out of your own PC from a Microsoft account? Sounds bizarre but remember this is precisely the option that Windows 10 forced down user's throats by linking Microsoft and Physical accounts. You may know better, but my dad is dependent on going to that breakfast restaurant if he has a problem with logging into his own computer.

  3. MFA by MobyDisk · · Score: 1

    Oh... THAT is what the "MF" in "MFA" stands for! I thought it was something else!

    1. Re:MFA by Catiline · · Score: 1

      Right?!? If they didn't spell it out in the summary, I'd have assumed it meant "Massive Failure Architecture".

  4. Looks like they tried it by Pikoro · · Score: 4, Funny

    "Engineers are currently in the process of cycling backend services responsible for processing MFA requests."

    So, they're turning it off and back on again.

    --
    "Freedom in the USA is not the ability to do what you want. It is the ability to stop others from doing what THEY want"
    1. Re:Looks like they tried it by DickBreath · · Score: 1

      The Windows support guy said I really should power cycle twice a day, or at least once per day.

      But I told him I don't even own a bike.

      --

      I'll see your senator, and I'll raise you two judges.
    2. Re:Looks like they tried it by Chris+Mattern · · Score: 2

      "In some cases, service restarts appear provide only temporary relief, so we're continuing to explore alternate remediation options."

      So, they're going to re-install it?

    3. Re:Looks like they tried it by Tablizer · · Score: 1

      So, they're turning it off and back on again.

      They need equipment to simplify the process.

    4. Re:Looks like they tried it by nuckfuts · · Score: 2

      "Engineers are currently in the process of cycling backend services responsible for processing MFA requests."

      So, they're turning it off and back on again.

      Exactly. And they're been staring at this for the last 90 minutes:
      "Windows is installing updates. Please do not power off or unplug your machine".

  5. Live Like Lemmings by Crashmarik · · Score: 2

    Die Like Lemmings

    You have critical applications they have no business being in the cloud. Especially not someone else's cloud.

    1. Re:Live Like Lemmings by thegarbz · · Score: 1

      Microsoft's MFA isn't just about accessing Microsoft's Cloud. They also form the basis of SSO solutions that can be deployed in corporate and personal infrastructure.

  6. Security improvement! by WoodstockJeff · · Score: 1

    If no one can log in with MFA, no one can be hacked, can they?

    1. Re:Security improvement! by Crash+Dummy+Redux · · Score: 1, Troll

      My worksite switched back to passwords for the time being. Passwords are still hackable.

    2. Re: Security improvement! by Zero__Kelvin · · Score: 1

      Cracking generally involves bypassing normal auth mechanisms, so no, this does not suggest immunity from being cracked.

      --
      Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
  7. Broken either way by SirAstral · · Score: 4, Insightful

    Choose how you want to run IT.

    If you think you can run to the cloud and get better service you are mistaken. Like playing musical chairs you only move the problems and goal posts around.

    There is no end to Management willing to pay through the nose for the promise of "Cloud" and following the advice of the providers along the way with little question, but when you have to build it on-prem you have to justify every blithering dollar you ask to spend and then have to face them trying to screw up your project plans with scope creep and "know-it-all" management interference and second guessing junior idiots.

    In short, your shit is going offline... you want that reduced? Find quality IT pros and fucking pay them what they are worth and stop promoting high quality pro's to justify giving them a higher salary. If you need too... pay a helpdesk worker that gets their fucking shit done twice what you pay the others. It's that simple and stay the fuck out of their way... they are the professionals... not the fucking management. Managements ONLY job should be to make sure that money is wisely spent by make sure the teams are aware of talent and licenses product are not unnecessarily duplicated and that the nerds or silo managers are not busy fighting like children over stupid shit between themselves or other teams. Those are two huge problems but get very little attention in many businesses.

    1. Re:Broken either way by im_thatoneguy · · Score: 3, Interesting

      You're going to provide me a nice storage service on-prem that I can access on an iPhone or Android device with conflict resolution and live cooperative editing between say 10 collaborators? And this service is going to manage sync conflicts? And this service is going to scale instantly? And it'll have a single sign on portal so that I can access said collaborative data share? And when I need to share that data with someone outside of the organization you're going to maintain the registration and securities permission of sharing said document? Also is your data service going to OCR and scan all photos in a project folder? Are you going to let me have federated search on my phone to search the contents of documents on my phone quickly while on a public wifi?

    2. Re:Broken either way by Tablizer · · Score: 1

      If you think you can run to the cloud and get better service you are mistaken.

      I believe on average it will be better. Local installations are often duck wire and chicken tape in my observation. Cloud problems just get more press similar to how jet crashes get more coverage than car crashes despite cars being more dangerous per mile traveled.

    3. Re:Broken either way by Billly+Gates · · Score: 1

      The problem is the people who decide to move to cloud aren't in IT. They are in accounting.

      They tell IT to lay people off so the CFO can get his bonus for being smart. Then blame the IT department when MFA breaks and not hold the CFO and his accountants responsible as they saved the organization money so fsck off etc.

    4. Re:Broken either way by Anonymous Coward · · Score: 1

      You share a file with email. It does the same thing (collaborating) as all that fancy crap and you can still search through emails

    5. Re:Broken either way by thegarbz · · Score: 1

      This! I'm surprised here on a nerd news site that many commentators don't know that "Cloud" is more than "opening Excel in a browser and storing files".

    6. Re:Broken either way by pacman+on+prozac · · Score: 1

      Emailing is a very limited way of doing collaboration, it spreads more copies of the same data around in multiple different versions. People can't work on the same specific document at the same time, if you edit it and I edit it then we have to manually recombine our changes. How do you then make sure everyone else is looking at the most recent version, email it again? Manual version control? All technically possible to achieve, but not in a way that's particularly efficient.

      Ideally you want one version of the truth which has shared access to anyone who needs it, where updates are visible to all other users immediately and it's clear what the latest version of that truth is. Then add in other features like the ability to audit who accessed that document and when, simple backup, one-click DR, ability to revoke access to certain users, if you really want then ability to constraint printing/emailing of documents (within limits).

    7. Re:Broken either way by strikethree · · Score: 1

      OOoooo. That sounds so technically difficult. Scarrrrrrrrrryyyyyyy.

      LOL None of those things are "hard". It could be difficult to be put them all together, but I doubt it. It also depends on how far you need to scale. For millions, it would take much more time. For thousands, this could be architected within a month and rolled out with full QA checks in 4 months.

      The only issue is that there are not enough knowledgeable people to architect this solution individually at hundreds of thousands of disparate locations. But trusting Microsoft to do it is like trusting a known pedophile in a day care center not to diddle children.

      --
      "Someone needs to talk to the tree of liberty about its ghoulish drinking problem." by ohnocitizen
  8. MS's predictability by OneHundredAndTen · · Score: 1

    Which is not a bad thing, in a world of constant change. In addition, if you get Microsoft products, you know what to expect.

    1. Re:MS's predictability by SirAstral · · Score: 1

      This is well said... I love Microsoft because they are guaranteed to break... this is job security for me. I wish I had a dime for every time I had to say... "told you so". And despite that they never listen. They always think my requests to add backups or redundancy are too expensive and pie in the sky jackassery, but boy do they pucker the fuck up when they are losing phat stacks of cash during down time and spitting in my face every 5 minutes asking for updates while I am in the middle of analyzing logs and error events to track own what happened in the middle of working on a POC they are too cheap to dedicate a PM for. It can't be a coincidence that analyzing begins with "anal".

      I wonder if management was supposed to be spelled "moronagement"?

    2. Re:MS's predictability by Chris+Mattern · · Score: 1

      if you get Microsoft products, you know what to expect.

      We have a massive service unavailability, but hey, we expected it!

  9. To the cloud by dkone · · Score: 1

    About every 12 to 18 months, the owner of the company I work for will come to me about moving 'everything' to the cloud. I always say the same thing, "Maybe we could move {a few non-essential things} and see how that goes, but I wouldn't trust moving {anything we rely on}".

    This article and many others like it are the reason I will keep saying this.

    1. Re:To the cloud by Billly+Gates · · Score: 1

      About every 12 to 18 months, the owner of the company I work for will come to me about moving 'everything' to the cloud. I always say the same thing, "Maybe we could move {a few non-essential things} and see how that goes, but I wouldn't trust moving {anything we rely on}".

      This article and many others like it are the reason I will keep saying this.

      I would keep my resume updated. When an owner looks to moving to the cloud it means they want to outsource and eliminate most if not all of IT to save money.

      The cloud really is about cutting costs. Not providing benefits and having MBA types circumvent IT by administering it themselves are the reason. Slashdot had an older article when cloud was new was the majority of organizations wanted the cloud to circumvent IT and do shadow IT stuff with an outousrced cloud partner.

      Even if your job is secure you are ultimately responsible for Microsoft's or Amazon's uptime as the people are not technical enough to understand and will demand numbers when it will be back up etc.

  10. Getting a little tired of this by Chris+Mattern · · Score: 1

    My workplace uses MSA for our VPN (which you have to be on for admin access to the servers). I'm starting to miss the RSA SecurID fobs we used to have.

  11. CROSS REGION!?! by the_skywise · · Score: 2

    cross-region potential outage impacting MFA

    The whole point of being in the cloud is so if one region goes down you can switch over/fallback to the other region's servers to maintain uptime!!!

  12. Please stop your chirping by Spinlock_1977 · · Score: 1

    To all you ops guys who think no one can run infra as well as you:

    Please stop the I told you so crap. For every one of you power-wizards, there are 100 fallible ops guys sitting in other chairs. Trust me, I've worked with a bunch of them over the last 40 years. Cloud platforms have outages a lot less than all the custom shops I've worked in, and I've worked in both big and small. Sure, Microsoft's outages are bigger and affect more people, but any particular company has only so much stuff that gets impacted.

    Give it a rest - make the world a slightly better place.

    --
    - The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
  13. This would have prevented employees... by rnturn · · Score: 1

    ... from accessing a host of internal applications at the company I was contracting with last Spring. And the internally-written authentication application was being slowly phased out and more internal applications were being migrated over to use the Microsoft application. By now, I expect that most, if not all, of those employee services were nicely locked down by Microsoft. One of these days, managers (and bean counters) will learn what is meant by "single point of failure".

    --
    CUR ALLOC 20195.....5804M
  14. Safety ? you don't need THAT feature ! by stooo · · Score: 1

    Safety ? you don't need THAT feature !

    --
    aaaaaaa
  15. The whole point of the cloud is data access by stooo · · Score: 1

    The whole point of the cloud is not to have good service.
    The whole point of the cloud is to hand over your data to a third party, and to NSA, not to let your users acess it.

    --
    aaaaaaa