Slashdot Mirror


New IP Address Blacklist Based On Web Chatter

itwbennett writes: A new approach to assembling blacklists analyzes chatter on the dark and open Web and can find malicious IP addresses that would have been missed using honeypots and intrusion detection systems, according to a report by security startup Recorded Future. On traditional blacklists, 99 percent of the addresses are for inbound activity, 'when someone is attacking your system from an external address,' said Staffan Truvé, chief scientist and co-founder at Recorded Future. On Recorded Future's new list, half of the addresses are for outbound activity, 'when an intruder is already in your systems, and is trying to connect to the outside world to exfiltrate data,' said Truvé. For example, Recorded Future identified 476 IP addresses associated with both the Dyreza and the Upatre malware families — only 41 of which were known to existing blacklists.

31 comments

  1. Does this mean victims are being blacklisted? by ebyrob · · Score: 1

    Seems like IPs sending out their sensitive data to attackers would normally be termed "victims"?

    1. Re:Does this mean victims are being blacklisted? by Penguinisto · · Score: 2

      Oatensibly, this would blacklist bots...

      Then again, if someone popped onto a random IRC server in the undernet, and started chatting about every IP address for windowsupdate.com...

      I am also curious as to how they handle DHCP, and if there's a timeout for the IPs listed?

      --
      Quo usque tandem abutere, Nimbus, patientia nostra?
    2. Re:Does this mean victims are being blacklisted? by Archangel+Michael · · Score: 1

      But, with the new features built into Windows10, Windows updates can come from anywhere!

      What could possibly go wrong with that??

      --
      Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
    3. Re:Does this mean victims are being blacklisted? by Anonymous Coward · · Score: 4, Interesting

      The article doesn't come out clearly to state this, but I can't see them adding end users IPs to a black list, I suspect that are referring to the IP the infected machine is trying to send data TO, as opposed to the IPs that the attacks are originating from.

      Think command an control network as inbound, it sends package updates and commands to the infected machine.
      The infected machine then attempts to send data off to another server, likely not connected in any way to the C&C system. This outbound IP would be blockable.

      But you can't block the users ip as it's likely a dynamic IP assigned by their ISP.

      Then again you can argue that once you are infected, you should be blacklisted and that could be something to look into.

      I read the article (not the full report) and they are talking about scanning tweets, chats, pastebins and other stuff looking for IPs / domains with at least 2 mentions of malware.

      I find it hard to believe these IPs are end users machines.

    4. Re:Does this mean victims are being blacklisted? by BronsCon · · Score: 2

      It sounds to me like it's blacklisting the IPs being connected to. Easy to spoof, though, just have your malware connect to dozens of random IPs along with the few actual IPs you're using, then the list becomes so full of false positives that it is rendered useless.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    5. Re:Does this mean victims are being blacklisted? by drinkypoo · · Score: 1

      The article doesn't come out clearly to state this, but I can't see them adding end users IPs to a black list,

      Why not? You might not blackhole the IP, but you could certainly ignore whole classes of traffic from such a host, and you could redirect them to a page telling them to get their act together.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  2. False positives by Anonymous Coward · · Score: 0

    If you're scanning for chatter and using that to blacklist IPs, surely there's a high rate of false positives. Is any screening done to prevent IPs from being improperly listed and is there any way to get removed from such a blacklist?

    1. Re:False positives by Anonymous Coward · · Score: 0

      That is acceptable collateral damage, the big guys will be protected however.
      This will become like the "no fly lists" that you cannot find out if you are on it, why you are on it, who to talk with to get off of it, how to not have it affect your search rankings for years to come if you manage to get off of it, etc. A great way to squash your competition.
      Just like the auto DMCA youtube takedowns are never wrong.

  3. Quick! by BronsCon · · Score: 2

    Somebody create a piece of malware that connects to random IP addresses!

    --
    APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
  4. Compiling blacklist from stale data... Morons! by Anonymous Coward · · Score: 0

    In their report they suggest blacklisting IP addresses that were infected 3-6 months ago and have been clean since then. In marketing speak, that's called "new approach". In real world, that's called stupidity.

  5. Really? In this age - Blacklists? by Anonymous Coward · · Score: 1, Funny

    Come on submitters and editors. Can't you understand that whitelists and blacklists have a racist history? The accepted terms are "allow list" and "block list". This isn't that hard.

    1. Re:Really? In this age - Blacklists? by Darinbob · · Score: 1

      They're not blueprints either, they're prints of color.

  6. Done by Anonymous Coward · · Score: 0

    It sounds to me like it's blacklisting the IPs being connected to. Easy to spoof, though, just have your malware connect to dozens of random IPs along with the few actual IPs you're using, then the list becomes so full of false positives that it is rendered useless.

    Already been done, I suspect: Pawn Storm Group Makes Trend Micro IP Address a C&C Server

    1. Re:Done by BronsCon · · Score: 1

      That seems more targeted than random to me. Also easy to combat with a whitelist overlaid on top of the blacklist. Truly hitting a few dozen, or a few hundred random IPs with every phone home to the actual C&C or dump server would render any blacklist based on those IPs useless. Think about it, if each machine hits 24 random IPs and 1 legit IP every time it phones home, only rotates out half of those so you can't easily pick out the one that's always the same, and phones home hourly, that's 12 new IPs per hour in the blacklist, 288 per day, 8,064 per month, 105,120 per year. From one machine.

      Now, here's where I start to get really wordy. I'm going somewhere with this, though, and I'm interested in actual workable mitigation techniques, as none come to mind for me, so please read through.

      There are 3,706,452,992 public ipv4 addresses. If 1 infected machine can blacklist 100k per year, that means less than 40k infected machines can blacklist the entirety of the ipv4 internet in under a year. Yes, of course, with randomness there will be much overlap and repetition, so it will take more machines, more time, or both, but it will happen eventually. And that's with a minimal number of fake IPs being pinged and a minimal effort to mask what is being done.

      And even with that minimal attempt, the best mitigation that can be done is to only blacklist the 13 (12 fake + 1 real) IPs that are not being rotated. That means our 40k machines can now only blacklist 480,000 false IPs (plus the 1 legit one). However, it also means that, by rotating between multiple C&C and data dump IPs, you can keep those IPs off the blacklist so the blacklist no contains only false positives, at least for your specific piece of malware. So, that mitigation technique actually harms the list more than it helps, by removing any possibly valid data along with only a portion of the invalid data.

      Going in the other direction, let's suppose that our malware has 100 C&C IPs and 100 data dump IPs, and rotates through them with each phone home. Let's also assume that, alongside the randomly-selected-from-our-known-pool IP address we're going to connect to, we also connect to 99 additional addresses on our first connection. On our second connection, we use a different C&C/dump IP, drop 49 of the random addresses used in the previous attempt (so we now have 50 different IPs and 50 repeated IPs, we've dropped half of the set), and connect to another 100 random IP addresses, for a total of 150 IP addresses. 3rd iteration, we use a different pool IP and drop 124 of the previous addresses, halving our connection pool again, then add 100 more randoms for a total of 225 IP addresses. The 4th run will be a little different because we have an odd number of addresses already. We swap out out pool address, drop 112 (half) of the old addresses, add 100 new ones, and we've now got 213 IP addresses to connect to. 5th run we swap the pool IP, drop 106, add 100, to get 207 IP addresses to connect to. 6th: swap, drop 103, add 100: 204. 7th: swap, drop 102, add 100: 202. 8th: swap, drop 101, add 100: 201. 8th: swap, drop 101, add 100: 200. 9th and on, swap, drop 99, add 99: 200.

      By the time we reach the 9th iteration, at which point each new iteration adds 100 new addresses to the list (99 false and 1 from the pool), we've already seen 808 unique IP addresses, at least 1/4 of which have been used repeatedly, and only 8 of which are legit C&C or data dump IPs. At iteration 9, until we've exhausted out C&C/dump pool, we're adding 100 new IPs to the list with each iteration; after the pool is exhausted, we're adding 99 with each iteration. Let's assume, for simplicity, that we exhaust the pool with our last iteration one day, so the next day starts by adding 99 new addresses to the list (since we're not considering past addresses added, again for simplicity, we don't have to concern with overlap; and we can also assume the malware tracks which addresses it has used and does not reuse ones it has dropped until

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    2. Re:Done by Anonymous Coward · · Score: 1

      You are assuming they are analyzing traffic reports. They aren't. they are scraping chat logs, twitter, pastebins and other resources to gather domains/ips with more than 1 mention of malware, it's right in the article.

      None of what they are talking about in this article relates to traffic monitoring, at all, by any stretch. It has nothing to do with coding your malware to connect to random IPs or domains. It's literally them sending a bot into rooms and recording the chats. And depending on which IRC networks we're talking about, it's quite likely they are getting the entire chat logs/dcc logs from all users including private chats, handed to them by the site operator. This has been going on since the early days of the warez scene and likely hasn't changed much.

      My thoughts? If this was working as your very detailed comment thinks, you are correct, it would only take 1 piece of malware and one guy like me to code it to run through every iteration of public ips on every session. Then what, they entire pool of ipv4 addresses are now blacklisted and the whole internet is dead?

      Not going to happen.

      Beyond that, go ahead and read the agreement you likely signed, but failed to read, with your ISP. Or, better yet, go ahead and write a ip/port scan script and run it. You'll have a phonecall or email from your ISP pretty quickly, they frown on that stuff. I know, I'm one of the guys who had to sit on the phone with my provider explaining it was a legitimate script that broke, desperately trying to get my 'net turned back on (they blacklisted my ips modems mac address!)

    3. Re: Done by Anonymous Coward · · Score: 0

      If we were looking at traffic, unlike the other commenter seems to think, it'd be many machines. So it's one static ip address out of 24 random, except there's a thousand sets with that one static ip address. Even if every instance chooses the same ip there's still only 25 to check

    4. Re:Done by AHuxley · · Score: 1

      A group of people need huge data moving pipes, distant command and control and some link to their safe location to make a long term project work without discovery.
      Days, weeks, months to chat, find, forum reading, ability to test the very latest and more expensive tools.
      The hope is the same type of ip networks will be used for the final testing and chat as for the "project"
      That can be detected.
      The feel of anonymity and having privacy can be strong once a small group of skilled people are ready for their "project". Projects that have always been a total success...
      Who has the look down power to see or trace the multiple C&C and data dump IPs?
      NSA, GCHQ, CIA (outside the USA) with their own networks. Australia, Canada, New Zealand, the United Kingdom would have their own domestic version of Tempora https://en.wikipedia.org/wiki/...
      All that multiple C&C is very easy to detect per nation in a domestic setting. Parallel construction can then set up a "chance" meeting on IRC, a forum with a "turned" or undercover individual with matching skills who becomes vey trusted over years. Local police using advanced social engineering in a random IRC room of 100 people is what is presented in open court.
      A lot of other nations get the same shared tips on their own citizens.
      Think of it as a global version of https://en.wikipedia.org/wiki/... with the numbers of real people per nation been not that huge.
      The pool of ip's used have might be huge on a given day or hour but the per person skill sets makes for a short list per city.
      Advanced air gapped networks take care of the really vital data in most nations.
      Shared lists slow down the flood of ip's per event but most of the work is done via trusted people in chat and getting a short list of interesting people to consider "travel" to a lovely safe fun holiday location ie the reality of extradition or rendition.

      --
      Domestic spying is now "Benign Information Gathering"
    5. Re:Done by BronsCon · · Score: 1

      Yes, when you're a government or quasi-government organization and have taps on every internet backbone to collect every single bit of traffic, you can do this. The company compiling this list? Not so much.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    6. Re:Done by BronsCon · · Score: 1

      My ISP knows I pen-test for a living. I pay a premium for my bandwidth and they leave me alone as a result. I do get an email from them once in a while if I've been testing a new exploit, making sure I'm actually the one doing it and not an infected system. A portscan? I doubt theyd even blink. Connecting to a couple hundred or so IPs in the frame of an hour? No ISP would think twice about it, especially if you're connecting on 443. Normal browsing habits for most households.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    7. Re:Done by BronsCon · · Score: 1

      Oh, and for the record, one email I got from them was about my Time Capsule being used in a DDOS. Fucking Apple ships the things with SNMP on by default, with default communities and no security; and no version of AirPort utility that runs on an Intel CPU can change the setting. I had to boot up an old Windows machine to fix that.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    8. Re:Done by AHuxley · · Score: 1

      With the private sector been herded into helping or staff helping or collaborating or been called on to "collaborate and cooperate"...
      Not much of the US private sector is really "free" anymore not to help make lists or have gov networks installed to track traffic in real time.
      Cyber Information Sharing Act is just the start with its FOIA issues and domestic access beyond NSL (national security letter) or FISC (FISA Court) access.
      Over time the entire US tech sectors private net logging is been guided into a legal, US court friendly government "collaborate and cooperate" database.
      Every aspect of 'Foreign" logging is now going to be used "legally" domestically.
      A collect it all list.

      --
      Domestic spying is now "Benign Information Gathering"
    9. Re:Done by BronsCon · · Score: 1

      Meds. You're off yours. You act like this is all some conspiracy we don't all already know about. Anyone in tech already knows this stuff, it's not relavent to this discussion, though.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
  7. list? by Anonymous Coward · · Score: 0

    does anyone have the list of 476 IPs?

  8. IPs often assoc with multi-homed hosts by laughingskeptic · · Score: 1

    The problem with IPs found this way is that they are often associated with hundreds to thousands of web sites, and the bad actors shift between these backends rapidly. For instance I have seen cases where there are a few Wordpress generated sites out of thousands being used to host malware configs and updates at a single IP of a low-end hosting provider. I have seen many similar instances where the IP was associated with AWS. The most precise way to blacklist sites like this is by hostname and not by IP.

  9. Useless w/IPV6 by JustAnotherOldGuy · · Score: 1

    Once IPV6 is widely adopted, the idea of having any meaningful data associated with an IP address is DEAD.

    The bad guys will have a nearly limitless pool of IPs to spoof and choose from, and they'll just discard them every few seconds or minutes and a get a new batch to use. That's because IPV6 has a mind-bogglingly immense address space. How much? Well....

    Let's assume every single one of the 100 billion stars in the galaxy is inhabited, and each star has a population of 10 trillion humans in orbit around it, and each human has 1 billion devices that need IP addresses.

    In that case, only 1/340,282nd of the possible 128-bit IPv6 addresses would need to be assigned.

    Put another way, IPv6 would (will) provide roughly 5,000 assignable IP addresses for every square micrometer of the Earth's surface.

    Good luck banning that suspect IP...it will have been used for a few seconds and then "thrown away", never to be used by the spammer again.

    --
    Just cruising through this digital world at 33 1/3 rpm...
    1. Re:Useless w/IPV6 by Anonymous Coward · · Score: 0

      You seem to miss one important point, which is that most of those enormous quantity IP addresses will start with the same bits. It's called a prefix.

      Most people are only going to be given a 48 or 64 bit prefix from the ISP, and you can disregard the rest of the bits for the purpose of blacklisting. And ISPs are going to group all of their customer IPv6 addresses in a few address ranges with an even shorter prefix. Since it's going to be easier to get a large address space, ISPs will be using fewer, larger blocks, resulting in fewer address ranges to block for incoming crap like e-mail spam and SSH abuse. Similar things apply to hosted address space, with fixed IP address ranges being used, just as with IPv4.

  10. Re:TAILS Linux 1.5 is out (Aug 11, 2015) by Anonymous Coward · · Score: 0

    # Tails is a spam system that aims to preserve shitposting and in your face advertising. It helps you to have to sort through more crap on the internet and make you wish for censorship so that you don't have to see idiots making off-topic posts about it almost anywhere, which they won't stop even if you ask them to explicitly.

  11. The new black? by Sebo · · Score: 0

    Has information security reached the point at which old is new? InfoSec and network researchers have been analyzing backscatter since the late 90s. If it's useful that's great but, calling it new and innovative is inaccurate.

  12. Unusual outbound activity by edtice1559 · · Score: 1

    For whatever reason, the most negative people on /. always manage to get first posts. Some posters have already pointed out limitations but let's talk about the benefits of this. If a bunch of hosts on my network start communicating in a way that they never have before, that me the sign that an infiltration has occurred. Inbound scanning looks for things trying to get through your firewall from the outside. But as we also point out on /. all the time, does almost nothing against social engineering attacks. It's not hard to get people to plug in USB thumb drives. (I've seen them given out by vendors at security conferences!) Nor can they resist attaching their personal devices to the corporate network. Now you have a bunch of hosts making outbound connections to new places. Encrypted outbound traffic to IP addresses where FCrDNS fails. Worth investigating. Of course this isn't a perfect solution, but it has at least some value. There is no claim that this is a replacement for other technologies only a supplement.