Slashdot Mirror


Dyn Executive Responds To Friday's DDOS Attack (dyn.com)

"It is said that eternal vigilance is the price of liberty...We must continue to work together to make the internet a more resilient place to work, play and communicate," wrote Dyn's Chief Strategy Officer in a Saturday blog post. An anonymous reader reports: Dyn CSO Kyle York says they're still investigating Friday's attack, "conducting a thorough root cause and forensic analysis" while "carefully monitoring" for any additional attacks. In a section titled "What We Know," he describes "a sophisticated attack across multiple attack vectors and internet locations...one source of the traffic for the attacks were devices infected by the Mirai botnet. We observed 10s of millions of discrete IP addresses associated with the Mirai botnet that were part of the attack." But he warns that "we are unlikely to share all details of the attack and our mitigation efforts to preserve future defenses."

He posted a timeline of the attacks (7:00 EST and 12:00 EST), adding "While there was a third attack attempted, we were able to successfully mitigate it without customer impact... We practice and prepare for scenarios like this on a regular basis, and we run constantly evolving playbooks and work with mitigation partners to address scenarios like these." He predicts Friday's attack will be seen as "historic," and acknowledges his staff's efforts to fight the attack as well as the support received from "the technology community, from the operations teams of the world's top internet companies, to law enforcement and the standards community, to our competition and vendors... On behalf of Dyn, I'd like to extend our sincere thanks and appreciation to the entire internet infrastructure community for their ongoing show of support."

Online businesses may have lost up to $110 million in sales and revenue, according to the CEO of Dynatrace, who tells CNN more than half of the 150 websites they monitor were affected.

36 of 77 comments (clear)

  1. We Were Attacked! by Anonymous Coward · · Score: 1

    And they made us look the foo!

    1. Re:We Were Attacked! by smallfries · · Score: 1

      1. System is designed with a decentralised resource to prevent single point of failure / target for attack.
      2. Company wants to monopolise resource.
      3. Spreads fear of attacks for reason to buy hardened service.
      4. Gets rekt by a bunch of kids who have hacked cctvs.
      5. Tries to use it spread more fear / downpkay own incompetence.
      6. ...
      7. People realise that running their own DNS is more resilient?

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    2. Re:We Were Attacked! by sithlord2 · · Score: 5, Informative

      >> 7. People realise that running their own DNS is more resilient?

      LOL! You think so? Let's say your own DNS infrastructure is a victim of this attack with the same magnitude. Are you able to handle this?

      There is a easy solution: Don't make your DNS a single point of failure. Make sure your DNS records are mirrored on two different DNS providers, and make sure you list all IP addresses of both providers' DNS servers in your registrar's settings.

      That's what we did. We have our DNS records on Dyn and another provider. We barely were impacted.

      --
      ...You are over-qualified and under-paid. If we give you a raise, we will break the cosmic balance of the universe.
    3. Re: We Were Attacked! by chipperdog · · Score: 2

      When you run TTLs less than 150 (like many of Dyn's customers), your DNS is no longer decentralized and fault tolerant....if you don't change your records often, use a longer TTL. Much of the effect of this attach could have been mitigated by using a 1800 or longer TTL...as long as a few isp and other common caches can get one response for each record every half hour things keep working

    4. Re:We Were Attacked! by Junta · · Score: 2

      The problem is this philosophy tends to create targets of great value by putting so much infrastructure into so few places.

      It's been a curious development in the internet. In the 90s, there was a trend from walled gardens and centralized resources to more federated approaches. In the last decade, the trend has reversed.

      We have increasingly powerful endpoint devices, even as their form factors have shrunk. This *should* have led to the reduction of the importance of 'datacenters', but now they are more important than ever *and* so much function has been consolidated into 3 or so companies, and a handful of physical locations.

      Now it's not as bad if everyone at least had their infrastructure to bank on a couple of providers as you do (so long as they all don't bank on the *same* two, but generally there's only a couple of companies people go to.for services)..

      In a decentralized case, a random entity is doubtlessly unlikely to withstand such an attack, but also they are far less likely to be the target of such an attack (being a bonus effect of taking down a target versus *being* the target).

      --
      XML is like violence. If it doesn't solve the problem, use more.
    5. Re:We Were Attacked! by Khyber · · Score: 1

      "LOL! You think so? Let's say your own DNS infrastructure is a victim of this attack with the same magnitude. Are you able to handle this?"

      Yep, all fucking day without even looking, and IPv6 will make it even easier. It's called a static IP address and not having more fucking domain names than you can handle.

      While everyone else was fucked, my sites ran without a problem, and they all use DynDNS.

      --
      Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
    6. Re:We Were Attacked! by smallfries · · Score: 2

      Sure do.

      It spreads out the attack value over multiple targets. It is not about whether a set of smaller replacements for Dyn could withstand 1tb/s, it is about whether an attacker could muster n tb/s to attack a whole set of smaller providers at once in order to create the same amount of widespread damage. Do you think it makes sense to put all the eggs in one basket?

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    7. Re: We Were Attacked! by Khyber · · Score: 1

      Son, I was playing with hardware load balancers on remote systems before you likely came to troll this site.

      --
      Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
  2. Lost business? by Z00L00K · · Score: 4, Insightful

    Is that really lost business or was it just a delay in the interaction for the customers?

    If shop's not available one day I'll wait a day or two to place my order. It's only if stuff is offline for a long period that it's really lost business because then I probably have gone elsewhere.

    --
    If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    1. Re:Lost business? by bookworm13 · · Score: 2

      You are forgetting advertising. You may not like it, but its still a major source of revenue for some businesses.

    2. Re:Lost business? by Anonymous Coward · · Score: 1
      Advertising is not a loss of business if you accumulate income across all companies. Advertising only moves funds between companies. The only loss is the one of customers who would buy on the given day but not later. If that is the case then they probably did not really need the thing anyway. Therefore it is good they did not waste money on "crap" and saved environment.

      Though it may be a loss of business in a sense that a part of income is moved from companies affected by the attack to the companies which were working/available well at that time.

    3. Re:Lost business? by grep+-v+'.*'+* · · Score: 2

      Is that really lost business or ... If shop's not available one day I'll wait ...

      You're ignoring the "instant gratification" bit. Wait a day -- a DAY? You must be joking, I don't want to wait 2 seconds while the page loads. The only reason i can even stand to wait for it to be delivered is because I can track it in motion.

      --
      If the universe is someone's simulation -- does that mean the stars are just stuck pixels?
    4. Re:Lost business? by thegarbz · · Score: 1

      Is that really lost business or was it just a delay in the interaction for the customers?

      If an item is perishable then it's actually lost business even if you intended to buy it the day after.

      If shop's not available one day I'll wait a day or two to place my order.

      Market research shows that a significant source of income is the result of impulse buys as the result of either short term discounts or advertising. Giving people time to think about it by preventing checkout causes them to (correctly) second guess their decision, whereas if there's no barrier to purchase you simply end up with post shopping cognitive dissonance (why the **** did I just buy that!)

      This is lost business.

    5. Re:Lost business? by Stewie241 · · Score: 1

      It depends... sometimes customers would probably come back and order later. Sometimes they might order somewhere else that is available. Or sometimes they might use the extra time to ponder whether they really need the item and ultimately might decide not to buy it.

    6. Re:Lost business? by Anonymous Coward · · Score: 1

      But it is better for the planet... And individual. And thereby, society.
      If it is a loss for someone, perhaps they are then better employed in something more productive.

      Like picking up trash.

    7. Re:Lost business? by hibiki_r · · Score: 2

      Of course it is: We see this pretty easily in the physical space when there's really bad weather, like a blizzard that makes travel difficult for 3 days. Businesses see a bit of a pickup afterwards, as some purchases just get delayed, but there's A LOT of economic activity that disappears.

      Imagine, for instance, that whoever processes credit cards for the Hillary campaign happened to have a catastrophic 4 hour outage around the last debate. Do you really think that the people that would have donated during the debate, or right after, are going to remember and donate just afterwards? I'd be very surprised of many didn't just give up at the time, and not remember to do the same, the day after.

    8. Re:Lost business? by Z00L00K · · Score: 1

      What advertising? I'm running an adblocker - and so do most people with sense these days.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    9. Re:Lost business? by Aqualung812 · · Score: 1

      What advertising? I'm running an adblocker - and so do most people with sense these days.

      I don't, on purpose. I think still I have some sense.

      -I like to pay for the services I use. Many companies will provide me service by showing me ads. That is fair to me.
      -I don't want ads for NFL games and tampons. I have no use for either of those. However, if a new 2m radio is on sale, I actually want to know that.
      -I mind giving small a small amount of information to advertisers (through cookies and fingerprinting) a LOT less than I mind giving any Ad blocking app FULL web browsing history.

      How are you paying for the services you use?

      --
      Grammer Nazis - I mod you "troll" unless you actually add something on-topic. Yes, I know I have mispellings in my sig.
  3. slashdot IoT sales banner by vocatan · · Score: 2

    Does anybody find it ironic to see the slashdot sales as for IoT cameras immediately above this sorry? Until we can somehow force vendors to responsibly patch, these devices have NO BUSINESS being on the web and we should boycott them. (Looking at you, AVTECH)

    1. Re:slashdot IoT sales banner by Hognoxious · · Score: 1

      Just make sure it's totally unspoofable like a Mac address.

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    2. Re:slashdot IoT sales banner by guruevi · · Score: 1

      Every device already has that. Luckily for us this ID isn't routable. Your proposal still wouldn't work, we know the IP's these attacks are coming from after all, the problem is getting providers like Verizon or ChinaNet to cooperate.

      --
      Custom electronics and digital signage for your business: www.evcircuits.com
    3. Re: slashdot IoT sales banner by buchanmilne · · Score: 1

      I didn't read the details of the attack, but if it was using UDP DNS requests the source IPs could have been spoofed (if they originate from networks that don't have uRPF enabled).

      In that case, their transit providers would only be able to identify them by traffic patterns on their circuits, or by more in-depth analysis if the provider can afford to run IPFIX/Netflow analysis on all their traffic.

  4. Re:how big was it? by AHuxley · · Score: 1

    "Attacks on the Internet keep getting bigger and nastier" (22 Oct 2016)
    http://www.latimes.com/busines...
    "1.2 trillion bits of data every second" or
    "How Hackers Make Money from DDoS Attacks"
    http://fortune.com/2016/10/22/...
    "1.2 terabits of data per second"

    --
    Domestic spying is now "Benign Information Gathering"
  5. Searchable database of attackers? by dattaway · · Score: 1

    Would be nice to check my addresses in case my network was an offender so I can fix something I may have missed.

    1. Re:Searchable database of attackers? by gregraven · · Score: 1

      I would welcome the ability to see if any of our devices is involved. We got the Cujo appliance for this a couple months back, and it hasn't alerted us to anything, but our DVR was running like mad (you could hear the hard drive from across the room) roughly during the time of the attack.

      --
      Greg Raven
      As long as there's any left, I'll take mine first.
    2. Re:Searchable database of attackers? by bromoseltzer · · Score: 1

      Would be nice to check my addresses in case my network was an offender so I can fix something I may have missed.

      Your router needs a better firewall to prohibit or at least rate limit outgoing traffic to unusual places. Monitor firewall hits. It's "easy", but typical routers don't offer much help.

      --
      Fiat Lux.
    3. Re:Searchable database of attackers? by toonces33 · · Score: 1

      Look at your router config, and look for UPNP and/or port forwards and see whether any firewall ports have been opened up for these devices.

      I would actually advocate disabling UPNP on the router, but I have no doubt that doing so would break some sort of lame device or application, and people would howl about how they just can't possibly do that.

  6. Re:Wikileaks = terrorism by MoarSauce123 · · Score: 1

    I'm more concerned about people writing comments like yours. What the heck do they play the national anthem before every friggin sports game? No other country does that as far as I know unless it is international competition. This hard core nationalism is what is causing many problems. As far as people shooting others, get rid of guns as much as possible and the number of shootings and deaths as well as crime in general will go down drastically. I have no idea what Jesus could do here, if he'd did what the Bible claims he did under today's laws, he'd be thrown into jail on terrorism charges.

  7. Re:how big was it? by guruevi · · Score: 2

    Dyn seems very quiet about a lot. They and their customers got their ass handed to them. This was pure incompetence on the hands of Dyn and many sites and services.

    DNS TTL 3600s or even 86400 (the gold standard back in the day) - because the cloud prides itself on individual machine uptime of 80% or less
    Single DNS provider - because the cloud prides itself on a single vendor being world-scale just by spreading out

    Twitter and co (still) has a TTL of 130s, way lower than RFC 6781 suggests and still has all their name servers at Dyn meaning they haven't learned anything yet.

    --
    Custom electronics and digital signage for your business: www.evcircuits.com
  8. how many bitcoins by golgotha007 · · Score: 1

    did the attackers ask for to stop the attack?

    Here's an actual letter sent to my company when we we're attacked earlier this year. By the way, they didn't breach us in any way, shape or form. They just hit us with traffic. The letter makes it sound like they had more, but nope, they didn't have shit.

    Hello Support,

    We are a team of highly skilled independent security consultants. One of your competitors hired us to take your site offline for an entire month (which we have the resources to do but don't like the contact and might be able to work together instead) and I must say that we have seen ALOT of miss-configured sites with security issues but it took our DB expert less then 30 minutes to dump your sql database without setting off your IDS system.

    We want to disclose some of the flaws we found with you and have already put a significant amount of time in researching, exploiting and then documenting the vulnerabilities we found. Unfortunately, most site owners don't give a shit and would rather wait for more malicious hackers to come along. We are going to stop that from happening.

    We are taking your site offline until we here from you. Our initial consultation will cost 1 BTC. That price will go up half a btc for every 12 hours we have to keep your site offline. I want to personally assure you that we have the power to keep your site down for an indefinite amount of time. We are the ones who took down xbox live all week (testing ONE of our new servers). In addition to letting your site up and giving you a report of what we found and how to fix it we will also let you know the ONLY way to stop a DDos attack the size we are capable of launching. We will also add you to a blacklist so no one else fucks with you.

    The BTC can be sent to the following address :

    I know that you are going to try to mitigate but in the end that is only going to cost you a lot more money. You make enough from betting and advertising alone that just an hour of downtime wont justify the cost. Our team also understands that you will try to mitigate but nothing will stop the attack except my command. Your hosting provider will not be able to help, the authorities wont be able to help you, your firewall is easily bypassed and any ddos service you try to bring in we can bring down (we have done this for a long time). believe it or not we are not the masked assholes stealing credit card numbers. Most of us have families and can't find legitimate jobs in our fields right now and have families to feed.

    Regards,

    GETDD0sed

  9. DNS blockchain by golgotha007 · · Score: 1

    The issue with DNS is that it's a centralizing service. As the world moves more towards a decentralized, distributed Internet, the first piece that moves in that direction should be DNS services.

    It could be done right now using a similar blockchain to the one bitcoin uses. In fact, you could also tie in SSL into the platform, to prevent centralizing services like Verasign from being a weak point. The design is already in my head - just need to build it. Anyone have some free time?

  10. Re:Solve the IoT security conundrum by toonces33 · · Score: 1

    And who then is responsible?

    The manufacturer? They are undoubtedly under pressure to keep the costs as low as possible, and keep the configuration as simple as possible. Make the config too hard, and people return the items to the store.

    The retailer? What's their responsibility here? Some like eBay/Amazon are just flea markets selling any crap that the associated merchant wants to sell. There is no "Underwriters Lab" to test some of the basic configuration stuff.

    The consumer? They don't care - it doesn't affect them unless they want to get to Twitter or whatever other site is under attack. The consumer's main interest is in low-prices for whatever device they are adding.

    The ISP? It isn't their device that is directly causing the problem. And yet they added support for UPNP to their firewall/router to make configuration easy without thinking about what the possible downsides might be.

    I see some here and other places argue that the problem is that we just need fatter pipes or more and/or better infrastructure. And while some improvements might be made, this is a cop-out basically because nobody else takes ownership of the problem, and it can potentially cost them lots of money.

    All I expect to see is more finger pointing, and ever more attacks. Eventually government is going to step in - maybe they try and force product recalls on the IoT devices? If we are lucky that's all they do.

  11. Need to work with IoT developers and/or shame them by TheDarkener · · Score: 1

    The fact that so many publicly facing, completely insecure devices ripe for hacking were able to be assembled in the first place is one of the biggest things we should be looking at moving forward.

    I think there should be a common, open-source framework for building secure IoT device firmware. Obviously people are going to be buying these things more and more as time progresses. Why not make it simple for them to implement something secure instead of leaving them to reinvent the wheel? Obviously they're concerned mostly about convenience otherwise they would have built better security in. Maybe they don't have the skill to do it correctly. Maybe they're not being paid enough. Point is that I think there should be efforts in the open-source ecosystem to help make these things easy and plug-n-play when developing a new baby monitor/security camera/etc.

    And, we should publicly shame the companies most to blame for these insecure devices being used in an attack of this magnitude. Paste their names all over and make sure people KNOW that they're buying utter insecure shit that may be used to attack others. Not to mention your own privacy and security regarding cameras/microphones/security devices, anything even semi-critical really.

    --
    It is pitch black. You are likely to be eaten by a grue.
  12. Re:Need to work with IoT developers and/or shame t by AHuxley · · Score: 1

    One way to do that is at the app level. Get the two big US brands to stop the smartphone integration of device apps that have a free flow onto the internet.
    The app gets delisted until the device is fixed or upgrades.
    It can be fixed by the IoT builders as they want cheap or use long supply chains.
    The consumer want easy, powered on, integrated, working devices. No entering long unique passwords deep in the packaging.
    Get AV firms to scan local networks and tell users their entire network and all their devices are unsafe on the internet?

    --
    Domestic spying is now "Benign Information Gathering"
  13. OH Dyn by eWarz · · Score: 1

    I remember when they were dyndns.org. I remember when they provided great dynamic DNS services for free or super cheap. Over time they became more and more commercialized, then more and more enterprised focused. They lost my business a long time ago. Nowadays you can have low TTLs with most service providers, and even if you can't, many ISPs have semi-static IPs for their residential and commercial accounts. There are also more rapid ttl based dyndns offerings including Google Domains. From a company perspective, nobody I work with uses Dyn...though apparently some pretty big names do. Many businesses I work with use Google domains, or some other large provider. One site I managed used GoDaddy, and oddly enough they were down. I wonder if GoDaddy uses Dyn on the backend?

  14. Re:Wikileaks = terrorism by ncc74656 · · Score: 1

    As far as people shooting others, get rid of guns as much as possible and the number of shootings and deaths as well as crime in general will go down drastically.

    Like it did in Australia? Oh, wait.

    GFY, you gun-grabbing fascist.

    --
    20 January 2017: the End of an Error.