Slashdot Mirror

← Back to Stories (view on slashdot.org)

Hackers Wipe US Servers of Email Provider VFEmail (zdnet.com)

Posted by msmash on from the attack-and-destroy dept.

Hackers have breached the severs of email provider VFEmail.net and wiped the data from all its US servers, destroying all US customers' data in the process. From a report: The attack took place yesterday, February 11, and was detected after the company's site and webmail client went down without notice. "At this time, the attacker has formatted all the disks on every server," the company said yesterday. "Every VM is lost. Every file server is lost, every backup server is lost. This was more than a multi-password via SSH exploit, and there was no ransom. Just attack and destroy," VFEmail said. The company's staff is now working to recover user emails, but as things stand right now, all data for US customers appears to have been deleted for good and gone into /dev/null.

31 of 157 comments (clear)

  1. There were NO offsite backups????? by sconeu · · Score: 4, Insightful

    No offsite backups? No tapes????

    Who designed the disaster plan for these guys?

    --
    General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
    1. Re:There were NO offsite backups????? by TigerPlish · · Score: 2

      No offsite backups? No tapes????

      Who designed the disaster plan for these guys?

      Same geniuses as Wells Fargo?

      --
      The "Civilized World" jumped the shark ca. 1973.
    2. Re:There were NO offsite backups????? by spudnic · · Score: 2

      It's all in their private cloud, of course!

      --
      load "linux",8,1
    3. Re:There were NO offsite backups????? by leehwtsohg · · Score: 2

      You mean offline.

      Nothing happened to any particular location.

    4. Re:There were NO offsite backups????? by lgw · · Score: 2

      No offsite backups? No tapes????

      Who designed the disaster plan for these guys?

      The plan was a disaster - mission complete!

      An online copy is not a backup, guys. It can be a great cache of a backup, but it's not a backup. Who still doesn't know this?

      --
      Socialism: a lie told by totalitarians and believed by fools.
    5. Re:There were NO offsite backups????? by Anonymous Coward · · Score: 3, Interesting

      The business plan probably.

      If you do make backups, you are too expensive, certainly cannot compete, and will go out of business. No income for you.

      If you do not make backups, you may make a nice buck for a while before the thing explodes in your face. Hell, maybe you are lucky and it never explodes at all.
      Regardless, at least you will make money for a while. So this scenario is clearly the winner. Screw the damage to your future ex customers, that is not your problem.

    6. Re:There were NO offsite backups????? by Anonymous Coward · · Score: 4, Funny

      From a cannon. Into the sun.

    7. Re:There were NO offsite backups????? by Anonymous Coward · · Score: 2, Insightful

      What has a higher chance of getting owned? A network accessible box wide open to the web, or a backup server that can only be accessed by SSH via a specific management VLAN?

    8. Re:There were NO offsite backups????? by Anonymous Coward · · Score: 2, Funny

      This is why DevOps is a bad idea.

    9. Re:There were NO offsite backups????? by bobbied · · Score: 2

      No offsite backups? No tapes????

      Who designed the disaster plan for these guys?

      No, no.. The Admin E-mailed the backups to himself every night.... They are all in his inbox... Don't worry, he encrypted them.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    10. Re:There were NO offsite backups????? by rickb928 · · Score: 3, Insightful

      It *is* a PITA to put a tape in your bag, open up the fireproof safe at home, throw it in, get the *correct* one out, put it in your bag, and remember the next day to put that where it needs to be. And repeat. /s

      I did that for years. And I slept a little better.

      --
      deleting the extra space after periods so i can stay relevant, yeah.
    11. Re:There were NO offsite backups????? by rickb928 · · Score: 3, Insightful

      Once you're in the front door, you're going through the system. Only offline backups can be trusted to 'be there'.

      And no offline copies of the VM environment? I think of those as especially precious. DO I want to rebuild those from scratch? Nope.

      --
      deleting the extra space after periods so i can stay relevant, yeah.
    12. Re:There were NO offsite backups????? by pnutjam · · Score: 2

      At the very least, you should be using a 2nd cloud service for backup, like rsync.net or those guys that are always releasing hard drive stats, backblaze.

      --
      Cheap storage VM.
  2. Backups? by byteherder · · Score: 2

    Time to pull yesterday's backup tapes. You do have the tapes from yesterday, don't you?

    1. Re:Backups? by bobbied · · Score: 2

      Plot twist: the last remaining copy of the encryption key is backed up on the encrypted backup tapes.

      Yea, but it's "12345".... What idiot uses THAT as a combination?

      Remind me to change the combination on my luggage..

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  3. nice!! by zlives · · Score: 2

    offsite tape backup is sounding good right about now

  4. You mean just the online backup servers... by SuperKendall · · Score: 4, Interesting

    Every file server is lost, every backup server is lost.

    So, that's the online backup servers, but what about the offline backups... there were offline backups, right? RIGHT???

    I am starting to wonder if I don't need to ask every single electronic service I interact with to put in writing what tighter backup policies are. I imagine my stuff on gmail servers is safe... but that is truly only my imagination, who can say for sure even they have offline backups (that can be restored from)??

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:You mean just the online backup servers... by jythie · · Score: 2

      Also, depending on how nasty they were being, they might have lurked long enough to poison the offline backups too. People tend to not actually check them till something goes wrong.

    2. Re:You mean just the online backup servers... by b0s0z0ku · · Score: 2

      Also, they may only keep backups for a few days for security reasons -- i.e. they want their users to be able to "truly delete" data.

    3. Re:You mean just the online backup servers... by bobbied · · Score: 4, Insightful

      Also, depending on how nasty they were being, they might have lurked long enough to poison the offline backups too. People tend to not actually check them till something goes wrong.

      AND, when they check, some 70% turn out to be insufficient or not restorable. Most turn out to be nearly useless for anything but giving you a warm fuzzy feeling as you trot them off to offsite storage.

      Having a backup plan is one thing, TESTING your backup plan is the next level.... However, revising your backup plan and TESTING your backups are restorable on a regular basis is the only way to know it will work when the chips are down. IF you don't do all this work, it's NOT really backed up, regardless of how many tapes you put into storage.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    4. Re:You mean just the online backup servers... by JustAnotherOldGuy · · Score: 2

      The only thing I would realistically worry about with Google is, what happens with a really big natural disaster that destroys one (or more) entire data centers?

      AWS.

      AWS makes lots of copies of every damn file and scatters them all over the world in geographically different Availability Zones.

      So even if the entire us-east-2 (Ohio) AZ is blown off the map by a nuke, AND eu-north-1 (Stockholm) is also blown off the map, along with Tokyo, Sydney, and Frankfurt, your file is still floating around in us-east-1 or eu-west-3 or ap-northeast-3, etc etc, about a dozen other AZs.

      You'd need a genuine global disaster to lose files from AWS, and at that point I probably wouldn't be giving a shit about files, I'd be concentrating on food and ammo.

      --
      Just cruising through this digital world at 33 1/3 rpm...
  5. IMAP/POP3 provider... by b0s0z0ku · · Score: 4, Interesting

    Thankfully, VFEmail was primarily an IMAP/POP3 provider. I suspect that the majority of its users had a local backup in the form of an email client with a local store...

    1. Re:IMAP/POP3 provider... by chiefcrash · · Score: 3, Informative

      Which, hopefully they've been paying attention: the current state of recovery means if you reconnect your client to your new mailbox, all your local mail will be lost (according to an update on their website)

      --
      Show me on the 1st Amendment bobblehead where the moderator touched you...
  6. No backup can be a feature by b0s0z0ku · · Score: 4, Interesting

    That can be both a bug and a feature. No backups mean that there's no cache of deleted emails. Some users may want the ability to truly delete data, not have it able to "appear" due to legal proceedings 5 years from now.

    I'd say it's on the users to back up their email using a client that locally caches IMAP folders or downloads via POP3.

    1. Re:No backup can be a feature by Aighearach · · Score: 3, Insightful

      It would seem more practical to just limit the stored backups to the last n copies, like you do with rotated log files.

      If it can only come back for two weeks or something, that is sufficient for most use cases.

  7. Sounds like a cleanup operation by misnohmer · · Score: 4, Interesting

    Maybe someone needed an email to disappear to avoid public embarrassment or legal trouble.

    1. Re:Sounds like a cleanup operation by Anonymous Coward · · Score: 2, Insightful

      If you're in a tight enough spot that you need to contact some hackers to annihilate an email company then you also probably don't have the time to wait around while they figure out if they can even get into that email company to do the job.

      So:
      1. They were already in and held the sword of Damocles over this company's head for a long time without them even knowing it just waiting for someone to fork over enough money to make it worth their while to let the sword fall
      or
      2. They had help from an insider employee-- which would have had to be tunneled in way beforehand (which costs much more than simply leaving the backdoor of Damocles in place)
      or
      3. There were no hackers, it was done by people at the company itself (i.e. CEO trying to evade indictment for insider trading or something)
      or
      4. Some kind of psychopathic rehearsal for a real cyber war? (no skin off, say, the Chinese Communist Party's nose if some Western email company gets splattered)

      The real news here isn't that the company got whacked, but the whacking itself.
      What purpose?
      No ransom demand?
      Why wipe all the servers instead of simply continuing to harvest data from them?
      It's like a drug cartel nuking a city without warning. Cartels are in the business of making money. A glassed crater doesn't yield an income.
      So as the parent post points out: there's something going on here besides some mean ole hacker-dashery.

  8. Pull not push for backups. by unkmar · · Score: 2, Insightful

    First onsite backup
    Second offsite backup that pulls, not pushes.
    - A push backup leaves a trace that there is a backup and to where it is being pushed.
    - - Just track the push and wipeout the backup as well.
    - A pull backup is only visible from the pulling location and, anyone inside that knows it exists.
    - - No trail to trace and wipeout. If it is wiped out, Then it is clearly an inside job.
    - - A pulling backup does mean the pulling system has access to the onsite backups.
    - - - But the onsite backup can be isolated from the onsite system and data.
    Conclusion:
    - Onsite hack can wipeout onsite system and data and onsite backup. but not offsite backup.
    - Offsite hack can wipeout onsite backup and offsite backup, but not onsite system and data.
    - Internal knowledge required to hit both targets.

  9. Replication != Backup by bodog · · Score: 3, Insightful

    Looks like ZFS replication may have been their backup plan? https://www.vfemail.net/design...

  10. Just recover it? by DontBeAMoran · · Score: 2

    ...as things stand right now, all data for US customers appears to have been deleted for good...

    Damn, talk about annoying.

    ...and gone into /dev/null.

    Oh! So they do know where the data ended up. Just restore it! You know, like in the movies?

    --
    #DeleteFacebook
  11. Just do a restore from Wikileaks. by jfdavis668 · · Score: 2

    I'm sure they have a recent copy.