Slashdot Mirror


Cloud DDoS Mitigation Services Can Be Easily Bypassed (softpedia.com)

An anonymous reader writes: A recent research paper shows that most Cloud-Based Security Providers are ineffective in protecting websites from DDoS attacks, mainly because they cannot entirely hide the origin website's IP address from attackers. As five security researchers from Belgium and the U.S. are claiming, there are eight methods through which these mitigation services can be bypassed. The techniques of obtaining a website's origin IP address rely on hackers searching through historical Web traffic databases, in DNS records, subdomains that resolve to the main domain directly, the site's own source code, when the main website triggers outbound connections, via SSL certificates, via sensitive files hosted on the website's server, and during migration or maintenance operations on the mitigation service itself, which leaves the target website temporarily exposed.

40 comments

  1. Duh by Anonymous Coward · · Score: 0, Insightful

    I've been telling people for years not to trust the cloud. And here's yet another flaw with the all wonderful cloud.

    Hard drives are damn cheap now. There's no reason to use the cloud.

    1. Re:Duh by fisted · · Score: 4, Insightful

      I'm not so sure if hard drives help mitigate DDoS. But hey, feel free to give it a try!

    2. Re:Duh by davester666 · · Score: 1

      sure. store all those packets on the drives, then respond to them later when you have time!

      --
      Sleep your way to a whiter smile...date a dentist!
    3. Re:Duh by fisted · · Score: 1

      Your ideas are intriguing to me and I wish to subscribe to your newsletter.

  2. Glad this is becoming more known. by Anonymous Coward · · Score: 0

    Been known for a while in the "technically literate" sector, but the average webmaster typically has no clue about these sorts of things.

    The only way to get around a bunch of these major attacks is by requesting new IPs for your servers and trying to mask those from being leaked.
    But there are still issues with that as well.

    The only real alternative is a hosted solution if you don't have the money to protect or maintain the server during it.
    Most of those services will typically monitor traffic and temp or permanent block IPs for unusual or large traffic, respectively. (sometimes which also leads to accidental blocks!)

    It isn't perfect, but that is the nature of the internet beast.

  3. have your origin accessible to only your provider by Anonymous Coward · · Score: 3, Interesting

    Akamai sells as an add-on for "origin cloaking", called "Site Shield", inwhich the origin to limits access to only a subset of akamai systems (which then distribute to the rest of akamai), and drops the rest of the internet. I wonder if that is effective against these attacks?

  4. Paper finds most webmasters don't have a clue by neorush · · Score: 2, Insightful

    Wow, revelations here. I guess the point of the paper is to really show most webmasters don't know what they're doing. All of these things can totally be avoided if you do your job carefully and methodically. e.g. maybe change the IP address of the server after launching your DDoS mitigation service, oh look, now half that list is moot.

    --
    neorush
    1. Re:Paper finds most webmasters don't have a clue by JustAnotherOldGuy · · Score: 1

      All of these things can totally be avoided if you do your job carefully and methodically. e.g. maybe change the IP address of the server after launching your DDoS mitigation service, oh look, now half that list is moot.

      One "hole" that they missed is that a lot of sites send confirmation emails when creating an account, and that can reveal the IP in the email headers.

      --
      Just cruising through this digital world at 33 1/3 rpm...
    2. Re:Paper finds most webmasters don't have a clue by Bengie · · Score: 1

      Some of the higher end DDOS services involve not announcing your blocks on the Internet, but making sure the routes between your and your DDOS service has your blocks announced. This way only your DDOS service can talk directly to you. They act as your gateway to the Internet for your hosted services.

      P.S. I love the point you made. I did not think of that hole in the armor.

    3. Re:Paper finds most webmasters don't have a clue by JustAnotherOldGuy · · Score: 1

      What I did at first to fix this potential hole was to catch the incoming email and then do a manual activation and reply from a Yahoo account. Later I had the email trigger a script that sent a signal to a different server and that server sent the email.*

      Of course, if they were really determined they'd just start DDOSing that server which would crash other sites I own, but I figured they'd be watching the original site to see if it was being affected, and when it wasn't (because they weren't hitting the right server) then after getting no satisfaction they'd give up after a while.

      .

      *this was pretty much a shameful, miserable hack, but it worked. :)

      --
      Just cruising through this digital world at 33 1/3 rpm...
  5. Easily? by IamTheRealMike · · Score: 4, Insightful

    Let me summarise the key findings of the paper. The headline figure is stunning: over 70% of all sites they tested leaked their origin IP in some way.

    But. It's not quite as simple as that. Virtually all websites that are DDoS protected are using CloudFlare, probably because it's a free service. The vast majority of the times they were able to find the origin IP address, it was due to basic oversights by the website admin, typically, having subdomains that resolve to the origin IP or simply never moving the server after signing up for CloudFlare at all. The most common subdomain that leaked the IP was called "ftp".

    Who the heck actually still runs an FTP server as part of their website, in this day and age? No big websites do that's for sure.

    And sure enough the paper concludes, not surprisingly, that bigger more important websites are much less likely to leak their origin IPs than smaller ones.

    I think all this paper really says is that CloudFlare have a lot of small non-paying customers who aren't really playing in the big leagues and aren't being attacked by sophisticated attackers ... or possibly aren't being attacked at all .... and as a result are more likely to have made simple errors.

    So when the headline says these protections are "easily" bypassed, all it's really saying is that if someone using a defensive system makes mistakes, they can still be attacked. That's not really news and doesn't tell us anything about the efficiency of these services when the people using them have done their homework.

    1. Re:Easily? by ThatsMyNick · · Score: 4, Insightful

      I think all this paper really says is that CloudFlare have a lot of small non-paying customers who aren't really playing in the big leagues and aren't being attacked by sophisticated attackers ... or possibly aren't being attacked at all .... and as a result are more likely to have made simple errors.

      Or they are using it as a free caching CDN like me, and dont care about IP being exposed.

    2. Re:Easily? by steveo777 · · Score: 1

      Who the heck actually still runs an FTP server as part of their website, in this day and age?

      More than I care to admit or remember... I've seen a lot of advertising firms using FTP for transferring material to/from clients all over the place. They figure user/pass and origin IP are secure enough. Well, maybe their data isn't important enough to transfer with any level of encryption.

      So when the headline says these protections are "easily" bypassed, all it's really saying is that if someone using a defensive system makes mistakes

      Very true, but many smaller websites may not have the luxury of moving their IPs about.

      --
      This sig isn't original enough, it's time to come up with something witty...
    3. Re:Easily? by Anonymous Coward · · Score: 0

      Who the heck actually still runs an FTP server as part of their website, in this day and age? No big websites do that's for sure.

      Every single hosting site on the planet does so by default. See Godaddy, Rackspace, SquareSpace, Google...

      AnonymousCoward@BigDick.local:~> host ftp.slashdot.org
      ftp.slashdot.org has address 216.34.181.48

      That bigger site admins have suffered enough breaches or problems to have learned to disable or firewall FTP is another matter.

    4. Re:Easily? by Khyber · · Score: 2

      "Who the heck actually still runs an FTP server as part of their website, in this day and age? No big websites do that's for sure."

      Every site that provides downloadable drivers for your hardware almost certainly has an FTP mirror.

      --
      Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
    5. Re:Easily? by Anonymous Coward · · Score: 0

      it doesn't matter if the origin ip address is exposed through poorly configured cdn / mitigation service.. throw enough at it and just take the whole fucking thing down. big deal, compromised systems are cheap.

    6. Re:Easily? by Anonymous Coward · · Score: 0

      And how the fuck are my clients supposed to upload their files to their domain/subdomain if they don't have FTP access? Should I give them ssh access and tell them to use RSync? Maybe use the clunky Web FTP that doesn't support recursive uploading and have them upload 5000 files one by one?

    7. Re:Easily? by fisted · · Score: 1

      $ ftp ftp.slashdot.org
      ftp: Can't connect to `216.34.181.48:21': Connection refused

      It's just a stale DNS record.

    8. Re: Easily? by Anonymous Coward · · Score: 0

      Wow. SFTP is perfectly adequate. If they're windows users a decent client is winSCP which to them will look just like an FTP client. It scares me that you have clients.

    9. Re:Easily? by theArtificial · · Score: 1

      To expand upon this a little, for those not in the know you can sign up for AWS free tier which is valid for one year and try most of their services. They've got over 40 different products. Pretty neat infrastructure they've got setup with excellent documentation to boot.

      --
      Man blir trött av att gå och göra ingenting.
  6. Re:have your origin accessible to only your provid by KClaisse · · Score: 1

    As long as the servers dropping the traffic can keep up with both the legit traffic and the bad traffic. If an attacker can overload the server which is dropping non-akamai traffic then even legit traffic wont be able to get through. Since the akamai server has to read a packet to know whether or not to drop it it is still possible to overload that part of the system.

  7. DDoS protection requires real services by Anonymous Coward · · Score: 0

    Finding origin servers is easy and CDNs like Akamai are not effective for DDoS protection for that reason. It's not because they say it's good that it is. It is good at putting content closer to the user, period. For real DDoS protection you need real scrubbing services relying on BGP redirection. This way all the traffic good or bad, goes thru the scrubber.

  8. No Way! by Anonymous Coward · · Score: 0

    No way! Cloud Flare assured me that they could hand 520 Unknown error

    1. Re:No Way! by myowntrueself · · Score: 1

      No way! Cloud Flare assured me that they could hand 520 Unknown error

      I don't think that means they can handle 520 different unknown errors...

      --
      In the free world the media isn't government run; the government is media run.
  9. meh... by Anonymous Coward · · Score: 0

    cloudflare is plenty unless you piss off someone with actual skills and resources to throw at you. which covers a good 95% of sites.

  10. Email Headers by XXeR · · Score: 4, Insightful

    One other detection method not specifically called out is via email headers. Often times automated emails are sent from the same origin IP (not always, of course). Even if the email is routed through an email service before delivery, you can still see the origin in the full header.

    1. Re:Email Headers by JustAnotherOldGuy · · Score: 1

      One other detection method not specifically called out is via email headers. Often times automated emails are sent from the same origin IP (not always, of course). Even if the email is routed through an email service before delivery, you can still see the origin in the full header.

      Dang, you beat me to it. :)

      I posted a slightly longer explanation, but yes, you are exactly right. Email confirmation messages can reveal the IP.

      --
      Just cruising through this digital world at 33 1/3 rpm...
  11. This is true, plus on they missed by JustAnotherOldGuy · · Score: 1

    All of what they described is true, plus one that they didn't cover explicitly.

    Even if you have your site behind something like cloudflare, if you allow people to sign up for an account and your site sends a confirmation email, that email can reveal the source IP.

    I've experienced this with one of my sites that had became somewhat popular. The owner of a competing site got his panties in a twist over the fact that my site was doing better and he started to DDOS my site. I changed the IP of my site and put it behind cloudflare, which worked fine- it totally mitigated the attack.

    But...when you sign up to the site it sends a confirmation email to the user...and the headers in that email contain the IP that the site is currently sitting on. For a few days I worried that this scum-sucking asswipe would figure that out and begin his attack again but he apparently he lacked the brain power to realize he could find the IP that way.

    The only way to get around this would be to have the target site route the outgoing mail through another IP or domain to mask its actual origin. I don't know if cloudflare has added some mechanism to do this or not, but confirmation emails are potentially a big hole in the service, and I'd guess that it's true of all such services similar to cloudflare.

    FWIW, I recommend cloudflare for this kind of thing as well as general traffic management and mitigation of malicious probers, bots, and similar problems. Using cloudflare in conjunction with a bot-screening service like StopForumSpam or BotScout can stop a lot of this kind of shit dead in its tracks.

    --
    Just cruising through this digital world at 33 1/3 rpm...
  12. Only applies to 'Proxied' Cloud DDOS services by cdogg4ya · · Score: 4, Informative

    This only applies if you are using a proxied service instead of a routed or tunneled service where you can't route around the proxy scrubbers. Most carrier DDOS service offerings allow you to route the traffic either through BGP steering or GRE tunneling such that your traffic must pass through the Cloud DDOS scrubbing center because the 'real' ip is routed that way.

  13. No Goober by Anonymous Coward · · Score: 0

    $ ftp ftp.slashdot.org
    ftp: Can't connect to `216.34.181.48:21': Connection refused

    It's just a stale DNS record.

    No Goober, it's a firewalled service. Just because you're banned doesn't mean that it doesn't exist or get used.

    Like the GP stated, the use of FTP in websites is indeed the norm. It more uncommon for a website to not utilize FTP.

    1. Re:No Goober by fisted · · Score: 1

      $ ftp ftp.slashdot.org
      ftp: Can't connect to `216.34.181.48:21': Connection refused

      It's just a stale DNS record.

      it's a firewalled service

      Yeah, because our technically literate Dive overlords would for once do everything right? (Must be a whitelisting firewall then which ALSO has the courtesy of not being a blackhole (since it returned a RST), then).

      Somehow, I doubt it. But in all cases, whether there's no ftpd running at all, or a polite, whitelisting firewall, the outcome is the same, i.e. good.

  14. G2O to the rescue by mi · · Score: 1

    The only way to get around a bunch of these major attacks is by requesting new IPs for your servers and trying to mask those from being leaked.

    If you use Akamai, you can turn on the G2O feature and configure your servers to check for it. Apache, Nginx, F5 load-balancers, IIS, and Varnish all have extensions to support it (though the last one is not, unfortunately, open-sourced — for purely bureaucratic reasons, I might add).

    Then, even if the enemies find your origin, all their hits will cost you is computing a digest of the requested URI and issuing a 403 or whatever — no file-lookups, no database-lookups, very little bandwidth. I suppose, your server can still be punished, but it certainly raises the bar quite a bit for any attacker.

    --
    In Soviet Washington the swamp drains you.
    1. Re:G2O to the rescue by Cramer · · Score: 1

      Do you understand how a DDoS works? If they can find your server IP, they can flood it with more traffic than you can imagine. Very few bother with "request spam" type resource attacks -- because they are "trivial" to deal with. (spin up more VMs, offload to CDNs, etc.)

    2. Re:G2O to the rescue by mi · · Score: 1

      spin up more VMs, offload to CDNs, etc.

      The "offload to CDN" method is exactly, what you need something like G2O for...

      --
      In Soviet Washington the swamp drains you.
  15. *facepalm* by Anonymous Coward · · Score: 0

    You should allow requests only from mitigation services IP's. Nothing can break it then.

    1. Re:*facepalm* by driblio · · Score: 1

      Do you know what DDOS is? Hint: they don't need to actually break it...

    2. Re:*facepalm* by Igal+Zeifman · · Score: 1

      You should allow requests only from mitigation services IP's. Nothing can break it then.

      That's actually a very good practice that will counter many of the origin-exposing vectors.

  16. Re:have your origin accessible to only your provid by Anonymous Coward · · Score: 0

    Major CDNs with security offerings like Akamai offer options like:
    * Not leaking your
    Its not enough to change your origin IP address if a volumetric attack on the old IP address still floods the the same router as your new origin.

  17. Re:have your origin accessible to only your provid by the+frizz · · Score: 2
    Levels of increasing protection:
    1. 1. Use a CDN and hope no one finds the origin domain or ips the CDN uses.
      Which as we can see from the article doesn't work due to the many ways they can be leaked.
      E.g., for www.example.com, try origin.www.example.com, ftp.example.com or IPs used in the past for www.example.com.
    2. 2. Have the origin servers only respond to white-listed IPs. That white-list needs to include those of the CDN.
      Still suspectible to a volumetric bandwidth attack. I.e., attacks with enough packets to overwhelm the origin server(s) or the ISP link to those servers.
    3. 3. Change your origin IPs periodically.
      Useless against a volumetric attack if they are just different IPs connected to the same uplink/router. Difficult to keep switching to use different ISP and each new provider brings its own problems.
    4. 4. Have origin(s) capable of withstanding a volumetric attack.
      Not cheap. The XOR DDoS botnet has recently produced DDoS attacks up to 150+ Gbps.
    5. 5. Use a BGP redirection service that routes all public internet packets whose destination IP address is the origin's through geo-graphically distributed scrubbing centers.
      Attackers sending traffic through the public internet to your origin are sending them to one of many scrubbing centers. The combined capacity on all these scrubbing centers can cope with volumetric attacks. The scrubbing centers will only forward desireable packets to the real origin using GRE tunneling.

    Akamai's BGP redirection service has some restrictions typical of other services. E.g.,

    • * A /24 prefix (Class C subnet) at a minimum. It needs to be is registered and belong to customer, as some ISP given not allow re-advertise.
    • * A BGP (Border Gateway Protocol) and GRE (Generic Routing Encapsulation) capable router.
    • * IP address space to terminate GRE tunnels located outside the prefixes you need to defend.
  18. Good for them for raising awareness by Igal+Zeifman · · Score: 1

    I actually work for one of the DDoS mitigation providers mentioned in this research paper. (Incapsula)

    Speaking as an "insider" I can tell you that, while the statistical study is very interesting, none of the origin-exposing vectors it mentions are particularly new.

    In fact all of these could be countered by few well-known best practices, which we are suggesting for years.

    I've put up a list of things you can do to immunize your website from origin-exposing attacks. https://www.incapsula.com/blog...

    I hope that now, with the subject getting some long overdue recognition, more people will get acquainted with these and pay more attention to their deployment configuration.

    PS: IP masking is really not the best way to protect your origin. Today, almost all cloud-based vendors offer BGP enabled DDoS protection for direct-to-origin attacks.