Slashdot Mirror
New IP Address Blacklist Based On Web Chatter
itwbennett writes: A new approach to assembling blacklists analyzes chatter on the dark and open Web and can find malicious IP addresses that would have been missed using honeypots and intrusion detection systems, according to a report by security startup Recorded Future. On traditional blacklists, 99 percent of the addresses are for inbound activity, 'when someone is attacking your system from an external address,' said Staffan Truvé, chief scientist and co-founder at Recorded Future. On Recorded Future's new list, half of the addresses are for outbound activity, 'when an intruder is already in your systems, and is trying to connect to the outside world to exfiltrate data,' said Truvé. For example, Recorded Future identified 476 IP addresses associated with both the Dyreza and the Upatre malware families — only 41 of which were known to existing blacklists.
Seems like IPs sending out their sensitive data to attackers would normally be termed "victims"?
Somebody create a piece of malware that connects to random IP addresses!
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
Come on submitters and editors. Can't you understand that whitelists and blacklists have a racist history? The accepted terms are "allow list" and "block list". This isn't that hard.
That seems more targeted than random to me. Also easy to combat with a whitelist overlaid on top of the blacklist. Truly hitting a few dozen, or a few hundred random IPs with every phone home to the actual C&C or dump server would render any blacklist based on those IPs useless. Think about it, if each machine hits 24 random IPs and 1 legit IP every time it phones home, only rotates out half of those so you can't easily pick out the one that's always the same, and phones home hourly, that's 12 new IPs per hour in the blacklist, 288 per day, 8,064 per month, 105,120 per year. From one machine.
Now, here's where I start to get really wordy. I'm going somewhere with this, though, and I'm interested in actual workable mitigation techniques, as none come to mind for me, so please read through.
There are 3,706,452,992 public ipv4 addresses. If 1 infected machine can blacklist 100k per year, that means less than 40k infected machines can blacklist the entirety of the ipv4 internet in under a year. Yes, of course, with randomness there will be much overlap and repetition, so it will take more machines, more time, or both, but it will happen eventually. And that's with a minimal number of fake IPs being pinged and a minimal effort to mask what is being done.
And even with that minimal attempt, the best mitigation that can be done is to only blacklist the 13 (12 fake + 1 real) IPs that are not being rotated. That means our 40k machines can now only blacklist 480,000 false IPs (plus the 1 legit one). However, it also means that, by rotating between multiple C&C and data dump IPs, you can keep those IPs off the blacklist so the blacklist no contains only false positives, at least for your specific piece of malware. So, that mitigation technique actually harms the list more than it helps, by removing any possibly valid data along with only a portion of the invalid data.
Going in the other direction, let's suppose that our malware has 100 C&C IPs and 100 data dump IPs, and rotates through them with each phone home. Let's also assume that, alongside the randomly-selected-from-our-known-pool IP address we're going to connect to, we also connect to 99 additional addresses on our first connection. On our second connection, we use a different C&C/dump IP, drop 49 of the random addresses used in the previous attempt (so we now have 50 different IPs and 50 repeated IPs, we've dropped half of the set), and connect to another 100 random IP addresses, for a total of 150 IP addresses. 3rd iteration, we use a different pool IP and drop 124 of the previous addresses, halving our connection pool again, then add 100 more randoms for a total of 225 IP addresses. The 4th run will be a little different because we have an odd number of addresses already. We swap out out pool address, drop 112 (half) of the old addresses, add 100 new ones, and we've now got 213 IP addresses to connect to. 5th run we swap the pool IP, drop 106, add 100, to get 207 IP addresses to connect to. 6th: swap, drop 103, add 100: 204. 7th: swap, drop 102, add 100: 202. 8th: swap, drop 101, add 100: 201. 8th: swap, drop 101, add 100: 200. 9th and on, swap, drop 99, add 99: 200.
By the time we reach the 9th iteration, at which point each new iteration adds 100 new addresses to the list (99 false and 1 from the pool), we've already seen 808 unique IP addresses, at least 1/4 of which have been used repeatedly, and only 8 of which are legit C&C or data dump IPs. At iteration 9, until we've exhausted out C&C/dump pool, we're adding 100 new IPs to the list with each iteration; after the pool is exhausted, we're adding 99 with each iteration. Let's assume, for simplicity, that we exhaust the pool with our last iteration one day, so the next day starts by adding 99 new addresses to the list (since we're not considering past addresses added, again for simplicity, we don't have to concern with overlap; and we can also assume the malware tracks which addresses it has used and does not reuse ones it has dropped until
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
You are assuming they are analyzing traffic reports. They aren't. they are scraping chat logs, twitter, pastebins and other resources to gather domains/ips with more than 1 mention of malware, it's right in the article.
None of what they are talking about in this article relates to traffic monitoring, at all, by any stretch. It has nothing to do with coding your malware to connect to random IPs or domains. It's literally them sending a bot into rooms and recording the chats. And depending on which IRC networks we're talking about, it's quite likely they are getting the entire chat logs/dcc logs from all users including private chats, handed to them by the site operator. This has been going on since the early days of the warez scene and likely hasn't changed much.
My thoughts? If this was working as your very detailed comment thinks, you are correct, it would only take 1 piece of malware and one guy like me to code it to run through every iteration of public ips on every session. Then what, they entire pool of ipv4 addresses are now blacklisted and the whole internet is dead?
Not going to happen.
Beyond that, go ahead and read the agreement you likely signed, but failed to read, with your ISP. Or, better yet, go ahead and write a ip/port scan script and run it. You'll have a phonecall or email from your ISP pretty quickly, they frown on that stuff. I know, I'm one of the guys who had to sit on the phone with my provider explaining it was a legitimate script that broke, desperately trying to get my 'net turned back on (they blacklisted my ips modems mac address!)
The problem with IPs found this way is that they are often associated with hundreds to thousands of web sites, and the bad actors shift between these backends rapidly. For instance I have seen cases where there are a few Wordpress generated sites out of thousands being used to host malware configs and updates at a single IP of a low-end hosting provider. I have seen many similar instances where the IP was associated with AWS. The most precise way to blacklist sites like this is by hostname and not by IP.
Once IPV6 is widely adopted, the idea of having any meaningful data associated with an IP address is DEAD.
The bad guys will have a nearly limitless pool of IPs to spoof and choose from, and they'll just discard them every few seconds or minutes and a get a new batch to use. That's because IPV6 has a mind-bogglingly immense address space. How much? Well....
Let's assume every single one of the 100 billion stars in the galaxy is inhabited, and each star has a population of 10 trillion humans in orbit around it, and each human has 1 billion devices that need IP addresses.
In that case, only 1/340,282nd of the possible 128-bit IPv6 addresses would need to be assigned.
Put another way, IPv6 would (will) provide roughly 5,000 assignable IP addresses for every square micrometer of the Earth's surface.
Good luck banning that suspect IP...it will have been used for a few seconds and then "thrown away", never to be used by the spammer again.
Just cruising through this digital world at 33 1/3 rpm...
A group of people need huge data moving pipes, distant command and control and some link to their safe location to make a long term project work without discovery.
Days, weeks, months to chat, find, forum reading, ability to test the very latest and more expensive tools.
The hope is the same type of ip networks will be used for the final testing and chat as for the "project"
That can be detected.
The feel of anonymity and having privacy can be strong once a small group of skilled people are ready for their "project". Projects that have always been a total success...
Who has the look down power to see or trace the multiple C&C and data dump IPs?
NSA, GCHQ, CIA (outside the USA) with their own networks. Australia, Canada, New Zealand, the United Kingdom would have their own domestic version of Tempora https://en.wikipedia.org/wiki/...
All that multiple C&C is very easy to detect per nation in a domestic setting. Parallel construction can then set up a "chance" meeting on IRC, a forum with a "turned" or undercover individual with matching skills who becomes vey trusted over years. Local police using advanced social engineering in a random IRC room of 100 people is what is presented in open court.
A lot of other nations get the same shared tips on their own citizens.
Think of it as a global version of https://en.wikipedia.org/wiki/... with the numbers of real people per nation been not that huge.
The pool of ip's used have might be huge on a given day or hour but the per person skill sets makes for a short list per city.
Advanced air gapped networks take care of the really vital data in most nations.
Shared lists slow down the flood of ip's per event but most of the work is done via trusted people in chat and getting a short list of interesting people to consider "travel" to a lovely safe fun holiday location ie the reality of extradition or rendition.
Domestic spying is now "Benign Information Gathering"
For whatever reason, the most negative people on /. always manage to get first posts. Some posters have already pointed out limitations but let's talk about the benefits of this.
If a bunch of hosts on my network start communicating in a way that they never have before, that me the sign that an infiltration has occurred. Inbound scanning looks for things trying to get through your firewall from the outside. But as we also point out on /. all the time, does almost nothing against social engineering attacks. It's not hard to get people to plug in USB thumb drives. (I've seen them given out by vendors at security conferences!) Nor can they resist attaching their personal devices to the corporate network.
Now you have a bunch of hosts making outbound connections to new places. Encrypted outbound traffic to IP addresses where FCrDNS fails. Worth investigating.
Of course this isn't a perfect solution, but it has at least some value. There is no claim that this is a replacement for other technologies only a supplement.
Yes, when you're a government or quasi-government organization and have taps on every internet backbone to collect every single bit of traffic, you can do this. The company compiling this list? Not so much.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
My ISP knows I pen-test for a living. I pay a premium for my bandwidth and they leave me alone as a result. I do get an email from them once in a while if I've been testing a new exploit, making sure I'm actually the one doing it and not an infected system. A portscan? I doubt theyd even blink. Connecting to a couple hundred or so IPs in the frame of an hour? No ISP would think twice about it, especially if you're connecting on 443. Normal browsing habits for most households.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
Oh, and for the record, one email I got from them was about my Time Capsule being used in a DDOS. Fucking Apple ships the things with SNMP on by default, with default communities and no security; and no version of AirPort utility that runs on an Intel CPU can change the setting. I had to boot up an old Windows machine to fix that.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
With the private sector been herded into helping or staff helping or collaborating or been called on to "collaborate and cooperate"...
Not much of the US private sector is really "free" anymore not to help make lists or have gov networks installed to track traffic in real time.
Cyber Information Sharing Act is just the start with its FOIA issues and domestic access beyond NSL (national security letter) or FISC (FISA Court) access.
Over time the entire US tech sectors private net logging is been guided into a legal, US court friendly government "collaborate and cooperate" database.
Every aspect of 'Foreign" logging is now going to be used "legally" domestically.
A collect it all list.
Domestic spying is now "Benign Information Gathering"
Meds. You're off yours. You act like this is all some conspiracy we don't all already know about. Anyone in tech already knows this stuff, it's not relavent to this discussion, though.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.