Slashdot Mirror


Atlanta Still Struggles To Recover From Ransomware Attack (reuters.com)

An anonymous reader quotes Reuters: Atlanta's top officials holed up in their offices on Saturday as they worked to restore critical systems knocked out by a nine-day-old cyber attack that plunged the Southeastern U.S. metropolis into technological chaos and forced some city workers to revert to paper... Police and other public servants have spent the past week trying to piece together their digital work lives, recreating audit spreadsheets and conducting business on mobile phones in response to one of the most devastating "ransomware" virus attacks to hit an American city. Three city council staffers have been sharing a single clunky personal laptop brought in after cyber extortionists attacked Atlanta's computer network with a virus that scrambled data and still prevents access to critical systems. "It's extraordinarily frustrating," said Councilman Howard Shook, whose office lost 16 years of digital records...

City officials have declined to discuss the extent of damage beyond disclosed outages that have shut down some services at municipal offices, including courts and the water department. Nearly 6 million people live in the Atlanta metropolitan area... Atlanta police returned to taking written case notes and have lost access to some investigative databases, department spokesman Carlos Campos told Reuters... Meanwhile, some city employees complained they have been left in the dark, unsure when it is safe to turn on their computers. "We don't know anything," said one frustrated employee as she left for a lunch break on Friday.

"Our data management teams are working diligently to restore normal operations and functionalities to these systems," said a spokesperson for the police department, adding that they "hope to be back online in the very near future."

19 of 91 comments (clear)

  1. They should all be sacked. by Anonymous Coward · · Score: 5, Insightful

    They should all be sacked.
    Backups. Backups. Backups.
    Simple. Known process.
    Not done = sacked.

    1. Re:They should all be sacked. by Z00L00K · · Score: 2

      Never underestimate the power of human stupidity.

      Anyway - this also highlights the need to really segment your data nets so that an intrusion don't propagate easily.

      And backups are also important of course. CD-ROMs are decent for short term archiving, but for long term archiving we need something better. SD cards also have a little "lock" switch, but it's in reality telling the computer that the device is read only so it's not proof against extreme hacks.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    2. Re:They should all be sacked. by Narcocide · · Score: 2

      The part that bugs me is that the effect is completely indistinguishable from the equally likely probability of completely accidental cross-contamination from an employee's personal USB device.

    3. Re:They should all be sacked. by hey! · · Score: 3, Informative

      What I can't understand is why these high profile ransomware attacks haven't prompted a rush to adopt copy-on-write filesystems. It's not like ZFS is exactly new.

      I understand that because of cost places like Atlanta try to run their networks with the least expertise they can get away with, but projects like FreeNAS make it really easy. I have a cheap server running at home and have background tasks scheduled to rsync changes to it. It's like it's not even there, but if I need to I can mount the NAS box and right click on a file in Windows and access the all the previous versions.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    4. Re:They should all be sacked. by sacrilicious · · Score: 2

      Never underestimate the power of human stupidity.

      To fail to do so would be stupid.

      --
      - First they ignore you, then they laugh at you, then ???, then profit.
    5. Re:They should all be sacked. by jellomizer · · Score: 4, Insightful

      Unless you are working for a government agency with bosses who don’t want to fund your department.

      Backups cost money. Redundant off site hot failover systems cost more.
      Please explain to the general public on why the city should have computers running in hope you don’t need to use them. When they can use that money to feed the poor.

      I have done years of consulting and working across many agencies. And for nearly every agency the tech workers are not incompetent, I may disagree with their methods, but they know what they are talking about. The bosses on the other hand especially ones without technical background, see the IT departments as a cost center. So will invest the minimum necessary to keep it running. They don’t realize that their equipment is being attacked constantly and it is only matter of time until something gets across.

      Current I work in healthcare and luckily management invest a lot into IT. So when spyware hit we only suffered minor damage and had it restored running with 15 minuets of missing data. After that incident we in the IT area was livid, and doubled our efforts to stop it again.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    6. Re:They should all be sacked. by hey! · · Score: 3, Insightful

      If you think of security exclusively in terms of prevention you are in deep trouble.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    7. Re: They should all be sacked. by Type44Q · · Score: 2

      Patient zero is the mouthbreather who specified the requirement for Microsoft products... lots of 'patient zeroes' in the corporate world... the 'B Ark' comes to mind; so does forced sterilization...

    8. Re: They should all be sacked. by hey! · · Score: 2

      You can have rsync only send updated blocks. Or you could simply use a COW file system mounted by iSCSI on your windows servers, if you prefer simplicity to features.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
  2. Danny Droptables hits Atlanta? by deviated_prevert · · Score: 3, Insightful
    WTF? From the brief description what happened sounds like the "virus" spread instantly with a DB injection attack. A simple thing to do if vulnerable old VB6 scripted front end from 20 years ago is still shoe horned into an internet exposed db. Hell there are banks running VB6 coded garbage from 20 years ago and one wonders why we are still getting hosed. There are even a few banks here like the Bank of Nova Scotia that run backend XP desktops up until just a year ago because all of their key db software would only work with a really old activeX front end.

    We complain bitterly about problems with industrial espionage and yet we still cheap out and use crapware swiss cheese .Net garbage that hackers in China and Russian can drive a truck through.

    --
    This message was not sent from an iPhone because Peter Sellers really was a deviated prevert without a dime for the call
  3. Inside job (allegedly) by Max_W · · Score: 4, Interesting

    It could be very convenient. No further audits are possible, since all documents are gone. All is to start from zero.

  4. Yes and no. by Anonymous Coward · · Score: 3, Insightful

    Yes, they should all be sacked.

    No, not the IT guys. The beancounters and managers who ignored their advice and failed to foresee the need for a proper backup management strategy for the city. IT knows this crap can happen, and IT tells Management about the need for proper backups, daily, weekly, monthly, on-site, off-site, and tape. We tell them RAID is not a backup strategy. WE tell them without backups their necks are in the noose when, not if, the shit hits the fan.

    Well, 9 days ago, the fan got crushed under 16 tons of Grade-A manure. And a LOT of necks are about to get wrung. Sure, IT will get fired, they always do. But this time, everyone who was against backups is gonna go down with them. Cause its not IT's fault the city chose not to have solid backup strategies in place with the vulnerabilities of today, that fault lies solely with everyone who said it was too expensive for no return, too much time for something that didn't make money, or that "education" would be enough protection so we don't need other solutions.

    1. Re:Yes and no. by Anonymous Coward · · Score: 2, Insightful

      Audits are completely useless and meaningless because:

      (a) The auditors are just as stupid and incompetent as the people they are auditing
      (2) If the auditors start flunking people and telling their clients that they are going to have to fix their shitty broken systems --- i.e., spend a lot of money -- they will quickly lose all their business. So, everyone passes their audits with flying colors.

      This, this, THIS. I briefly worked for a company that provided lockbox services for a number of banks, running them on a "cloud infrastructure" that was actually a number of ancient servers sharing an even more ancient SAN with a failing RAID card. The "data center" was located in an outside-facing suite in an office building next door connected via a single WiFi antenna, with a glass entry door with a single cylinder lock, no cameras/monitoring, no environmental monitoring, no fire suppression beyond the existing sprinkler system, no mantrap (you could clearly see the racked servers and UPSs from outside), and a way underpowered generator that hadn't been started at all in at least a year. Yet, they passed their SSAE 16 Type 2 audit with flying colors.

    2. Re:Yes and no. by dcw3 · · Score: 2

      It's been quite a few years since I went through an ISO 9001 audit, but I recall thinking what a load of crap it was at the time. I'm no expert on it, but what I was told was that it's primary function was to verify that you followed your processes. But, it did nothing to ensure that those processes were worth a damn. Then, what the fuck good is it? Maybe I was mislead...I've never bothered to look it up.

      --
      Just another day in Paradise
  5. kill the windows servers, and do backups by Anonymous Coward · · Score: 2, Insightful

    throw the windows servers in the trash.

    and do backups.

  6. OK, so it's April 1 & all, but still.... by t2callingt3 · · Score: 3

    Who can expect anyone to believe they "lost 16 years" of data? 192 consecutive months without backups? Zero offline storage? Pull the other one: it's got bells on it!

  7. Re:From TFA by thegarbz · · Score: 2

    Why not? The first thing every Linux installation does is enable interoperability with Windows networking. Wanacry very quickly spreads to SMB shares. If they are writable then a remote client can happily encrypt your shit. Or if you want, https://www.samba.org/samba/se... gives you your own Linux special flavour of Wanacry.

    Now yes the GP is a troll, and it most likely wasn't the case. But security is about dealing with the possible, and just running Linux doesn't make you immune from anything, especially not user stupidity.

  8. Re:"Unknown User" by CAOgdin · · Score: 5, Informative

    Nonsense! 100% daily backups of systems, using a suite of tools kept offline except during backups activity is ALWAYS a solution....simply because an attack starts at a particular time; anything you've kept offline prior to that time is a resource to be used to recover. Yes, there is the problem of recapturing the lost data in that time interval, but it's a LOT better than having to start redesigning software from scratch AFTER the attack has occurred!

    100% daily backups, with recycling of media over a period of a few weeks is a MANDATORY requirement for every computer under my management. Since I started doing that in 2001, I have never had (nor has any client had) an unrecoverable loss of data.

    The other trick is keeping data separated from executables. My mantra is "C: is for Code, D: is for Data". The idea that everything should be on the same logical drive is simply WRONG.

    There are no perfectly secure systems, and perfection is a fools game. But, simple strategies, unerringly repeated over time, can make recovery from assaults (or hard-disk failure) a straight-forward solution.

  9. Re:Question: by dcw3 · · Score: 3, Funny

    So has Georgia actually passed a law that will effectively make the investigation of this ransomware attack illegal? That would be both stupid and highly amusing.

    They don't know. All the laws were on the servers.

    --
    Just another day in Paradise