Malware and Botnet Operators Going ISP

Trailrunner7 writes to mention that malware and botnet operators appear to be escalating to the next level by setting up their own virtual data centers. This elevates the criminals to the ISP level, making it much harder to stop them. "The criminals will buy servers and place them in a large data center and then submit an application for a large block of IP space. In some cases, the applicants are asked for nothing more than a letter explaining why they need the IP space, security researchers say. No further investigation is done, and once the criminals have the IP space, they've taken a layer of potential problems out of the equation. 'It's gotten completely out of hand. The bad guys are going to some local registries in Europe and getting massive amounts of IP space and then they just go to a hosting provider and set up their own data centers,' said Alex Lanstein, senior security researcher at FireEye, an anti-malware and anti-botnet vendor. 'It takes one more level out of it: You own your own IP space and you're your own ISP at that point.'"

Comments

Filtering easier?

[Anonymous Coward]

If they own the IP block (or it's assigned exclusively to them) then wouldn't that make it a lot easier to block them? Why complain? Just find out their range and shitlist it.

Re:Filtering easier?

[JWSmythe]

    The article (and story here) are a bit deceiving.

    The LIR is usually the ISP. So, they're filling out the IP justification form to ask for a block of IP's, just like anyone with their own rack or cabinet would. Big deal. I once had over a dozen /24's, but it was for legitimate purposes, and I properly (and honestly) justified them.

    I watched spammers do that in the past. They'd get multiple T1's (at their location) or ethernet handoffs (in datacenters). They'd be able to do a spam run for about 3 days on a block of IP's. When they got the complaint, they'd simply switch to another line. Say they have 7 of these circuits. It would take 21 days before they rotated back around to the original provider. If one should (oh my gosh) cut them off for the illegal activity, they'd simply bring in new circuits under new names.

    By combining providers in a single rack, that saved them the money of needing more servers. They'd frequently have a few cabinets, in a few different datacenters. So, 4 racks, 7 circuits each, would give them 28 unique identities. At 3 days before the line is burnt, that would give 84 days before they'd rotate back around to the original line.

    They would let a line sit idle for 84 days. That would just be stupid. They'd run multiple campaigns at the same time, so they'd rotate through them. It was an art, playing providers and the spam traps. They'd send a nice apology to the provider when they got the notice to stop, saying some machine was compromised, and the complaints would stop after just a couple days, and no one would care.

    Of course, some legitimate traffic would be hosted on these lines also, just to make things look good. In a 40u rack, they may have 30u's populated with spam servers, and a couple u's with web servers and what looked like paying customers on them.

    It's just like a black market operation run by the mob. Sure, you can buy merchandise in the store front. You'd never see the mobsters counting out suitcases full of cash, or shelves full of stolen merchandise bound for other places. No one questions what you're doing, because your store front *looks* legitimate.

    All they're indicating is that the spammer crowd has realized that there is no money in spam any more, and they've migrated to malware.

    All in all, it's not hard to get a cabinet, nor a circuit or three, in a datacenter. You don't even need a legitimate company. You just need to *appear* that you have a legitimate company. $100 and a few minutes of your time will incorporate a company to use. Corporate address? A PO box somewhere. Company phone? A "magicjack" or throw away cell phone. The only things that would tie anything to anything would be who's signing the contracts, which can be anyone. For minimum wage, you can have an employee of your illicit corp sign off on papers as "CEO".

    At one job, I wasn't listed as an "officer" in the company, so I couldn't sign anything. I got annoyed with trying to deal with the provider, so the next time I called to do something, I was "Vice President of Information Technology", and suddenly I was allowed to make changes. It was with the CEO's blessing, so I wasn't doing anything wrong. It was just to get through the providers annoying "protective" measures. The CEO never even got a phone call asking if I was allowed to make the changes. He just saw it reflected on the next bill.

Re:Filtering easier?

[RobertM1968]

In addition to that, as many people seem to erroneously use the term, this makes them an OSP, and not an ISP.

That aside, virtually every ISP and OSP has an ISP they "report to" - thus this should in no way make shutting one of these company's/criminal's/site's internet access down any more difficult than in the past. Basically, unless you are a backbone owner, you're paying for a connection to the Internet via someone else and having lines installed by someone else.

In addition, I'd suspect it makes it easier to get them disconnected as they cannot claim (in the US) safe harbor if they are knowingly and/or through actions of their own; placing such botnets online on "their" network. The provisions of the law here are to protect those ISPs and OSPs who get snared in the actions of end-users (not their own malicious actions), only if and when they take appropriate actions to deal with it (those actions dependent on the infraction type... for instance, for copyright infringement, following the rules in the DMCA). In this case, they are causing two strikes to be against them from the get-go...

I'd surmise, that unless a botnet operator buys a big chunk of the Internet "backbone" that the Internet cannot survive without, that regardless of the number of IPs they own, following standard procedures against their ISP will result in the same ends as before. And I would further surmise that even if they did buy a big fat pipe, this would also make it easier to block them at peering points (which in some cases, if done drastically, would help convince their upstream provider to disconnect them even faster than the paperwork and complaints filed).

But that's just my guess... from I dunno... years in the business, including working for UUNet before they got entangled in the MCI-Worldcom debacle (you know, back in the day when besides running the 2nd largest (behind IBM) and then largest part of the backbone, they were actually the real provider for the majority of MSN's and AOL's networking and end user connections. So... as I said, it's just a guess... the Internet landscape has changed a lot from those days of antiquity... but I suspect my guess is pretty close to the true reality of the situation, thus meaning this article on threatpost is massively (and incorrectly) overstating the significance of this.

Then again, I haven't RTFA, so I am only going by a summary - even though my experience on /. has shown that's a bad idea... (but it is more fun having conversations about things that way). ;-)

Easier to block?

[phil reed]

Maybe I'm not being smart today, but doesn't that actually make it easier to block the bad guys, once their address space is identified?

Re:Easier to block?

[CannonballHead]

Out of curiosity... does that make that IP space sort of permanently black-listed? e.g., if the "bad guys" go out of business and "good guys" buy the IP space... how do the new owners clear the IP space of its bad name?

Seems like a shame to start throwing IP space away because there's no way to make it clean again.

Re:Easier to block?

[Demonantis]

In TFA it mentions that it starts to become spaghetti. As ISP get smart and start blocking that address block the criminal moves on to other things. The lease expires on the block and it is issued to a legit company and then problems happen because the blacklists are not updated by the ISPs. IPv4 also is a very limited size so you can't just rotate around the blocks you issue every 100 or so years (conservatively) and avoid this issue.

Re:Easier to block?

[Zerth]

That's why your lists should have a time component.

If you do something naughty, you're blacklisted for an amount of time, then greylisted for the next step up. If you do something naughty while greylisted, you get blacklisted for the remainder and greylisted the next step up again.

Mine goes 15 minutes/1 day/2 weeks/3 months/1 year. I've yet to blacklist anyone for a year.

Re:Easier to block?

[denis-The-menace]

Wouldn't they need to peer with someone?
If so, then that peer should become the new target for shutdown requests.

Am I right?

Re:Easier to block?

[mysidia]

There is a strong movement on the public internet registries such as ARIN, RIR, etc, supporting privacy of IP address allocation data. In the future, it is very likely that registry policy may shift in favor of these supporters of internet privacy.

The result will be you cannot do so much as a WHOIS lookup to find out who these spammers might be if the privacy advocates/spammer have their way, only with a court order...

Good luck getting that when the spammer lives in a different country, where spam isn't illegal.

No, because once every /24 in those f****ers block gets on enough blacklists, they get a few more hosts to justify a bigger block, fill out a form to RETURN the IP addresses they got. Their old IPs will be assigned to someone else, and after the exchange their old IPs for a fresh new block of IPs they have even more /24s than before, and none of them blocked.

Now only the new guy (that happens to be so unlucky as to get their old IPs) is blocked.

Of course the f'ers will pretend to be legitimate extremely well, and make it as hard as possible for people to see reason to ban their whole block.. (E.g. The "shell" ISP will create "fake" separation from spammers who "received space" from their block)

They may do all kinds of weird s**** to make it look like it's not just one spammer.

Alternatively, they just apply for more space, using more shell companies, lather, rinse, and repeat. Until IPv4 is exhausted, that is.

If they have no problem lying once... it's not the least bit difficult to create 30 more fake companies (or even, make them real companies -- if the spam effort is profitable enough).

This is all assuming they are getting the IPs from the RIRs in the first place, which I doubt is the most common.. that could be too easy to track, since these allocations generally get published very visibly.

LIR ips are just fine for them, and much easier to get.

Also, the RIRs are basically powerless to stop this. Contrary to the article, it's not necessarily about "LIRs being lax".

Once a block of IP addresses is assigned, it is not as if the LIR or RIR can revoke it and force its use to cease.

Revoking IP addresses doesn't magically make them unreachable on the internet -- once the spammer convinced their ISP to announce the address space, they don't need (any longer) to prove they got the IPs legitimately, until/unless they get more ISPs.

The article's terminology is wrong. An LIR is just another name for an ISP. Verizon is an LIR, Level3 is an LIR, Cogent is an LIR, AT&T, Sprint, etc, are all LIRs, any ISP that receives ISP allocations of addresses which are issued to them for the sole purpose of sub-delegating for use with their services, is called an LIR.

Maybe the article means the spammers are getting IP delegations from an ISP LIR, that would make sense. It is very easy to believe, they could do this en masse with very little effort, in fact.

If you buy internet services from an ISP like Verizon, and claim to have X hosts, they will have a very hard time rejecting a request from their customer for those IPs.

For a simple /24 or two, most won't ask for much documentation, as long as the price is right, it's not customer-friendly to try that.

The tough questions don't start getting asked, until a request for a larger number of IPs is made, which is sensible. Level of justification and documentation commensurate with the expected usage.

The LIR/ISP will SWIP the listing or list the claimed owner on their RWHOIS Servers, but it won't appear as public knowledge in the RSS feeds, that such and such /24 has been allocated.

ISP RWHOIS servers are commonly broken and poorly maintained -- the spammer's new subdelegation may not even become public knowledge.

Re:Easier to block?

[gknoy]

Do you have any helpful links to guides that would explain how to do that? I'm sure I am not the only network-care neophyte who would like to have a safe and spam-free system at home, so I'm sure it would get you modded informative.

Re:Easier to block?

[xous]

No, it doesn't.

We had a "customer" that had 15+ dedicated servers with us. This customer received tons of SPAM complaints. Each time they had a different excuse.

After I disabled the servers and refused to turn them back on without examining them. The "employee" said he wasn't supposed to give me the root passwords but after I said that they would stay down until I got them he reluctantly gave them to me. Upon cursory examination the systems seems clean as a whistle until I realized there were no services actually running. No mail, etc.

Where was the email coming from?

I then found that the customer had GRE tunnels configured. This allowed servers in other data-centers to generate and send the spam through our network without having anything of actual value hosted with us.

The "employee" that was our customer was so convincing that I could have believed that at least he thought his company was legitimate. He even tried to tell me that it was because they couldn't get IP addresses from their current provider they bought dedicated servers from us ($1500/mo) for IP space.

Obviously the customer was terminated as soon as I found the tunnels.

Re:Easier to block?

[mysidia]

No.. it's worse than that. IP addresses aren't bought or sold.

Once they are no longer using the IPs, once they cancel the connection, the IP delegation goes away.

If the IPs came from the ISP, that ISP has to re-use such IPs: they count against the ISP's ability to justify need for more IP addresses.

If the IPs came from a RIR, once the justification goes away, the IP addresses are supposed to be returned, or they get revoked when the recipient of the IPs stops paying their annual maintenance fees.

In any case, the IPs eventually go back to the free pool, and get allocated to someone else.

The registries aren't going to try and "clean" blacklists, neither will ISPs. The recipient of IPs inherits the problem, to deal with any connectivity issues caused by blacklisting.

For IPs received from an ISP though... you should be able to convince your ISP to get you new IPs and allow you to move, if you're willing to take the time and energy to renumber, and (for some ISPs), there may be fees involved in you making the change requests, for the time it takes the ISP to make changes.

In many ways, poorly-maintained blacklists are just as harmful to the internet and end-to-end universal connectivity, as the spammers and malware peddlers are.

Re:Easier to block?

[Hognoxious]

Then 100 years later, an IP that was spamming 100 years ago gets re-used... and can't connect to anyone......

No worries, everyone will be using IPv8 by then.

Re:Easier to block?

[mysidia]

Well, you probably broke quite a few laws by using coersion to gain access to a customer's servers. But I for one would overlook it, given the benefits to the world at large (still it could be risky).

Fortunately, given the use of GRE tunnels, the spammer probably broke more laws, and would probably be a bit hesitant to sue.

The scenario is atypical. From the sounds of it, most spammers are not buying the cabinet space from the same company that is providing the internet access.

Of course it's a breach of contract and likely a violation of SLA for a cabinet provider to power down anyone's equipment or start cutting wires, because they think they might be spamming.

The spammer might sue claiming loss of valuable data (due to an unclean shutdown of their server).

Industry standard terms are power can be disconnected at request of customer (for a fee of course), emergency, planned maintenance, and violation of wiring standards (e.g. many major colocation facilities will have many rules on how equipment can be plugged in). But I don't think there are many Enterprise rack residents that accept "We may disconnect you if we feel your servers are doing something suspicious"

Of course network connections are a bit different.

Well, if you buy TRANSPORT from point A to point B, such as a connection from your rack to an ISP, in a major datacenter, you can expect by contract the transport provider cannot examine any data crossing the wire. In fact, they cannot cut the cable, just because they suspect you might be sending spam over it.

Your OC-3 or Ethernet transport from "Point A" to "Point B" is not an internet service. It's extremely unlikely for an Enterprise to negotiate a contract that allows their transport provider to disconnect them.

Following industry standard terms, a transport provider cannot kill the link, even if you are spamming, in fact, even if an internet attack happens to be crossing the link, a transport provider has no right to kill your connection or detect the nature of the traffic that is being transported.

To do so would be breach of contract/SLA on their part, and subject them to unnecessary liabilities (they lose their common carrier status for links that they 'watch').

In most cases, the one and only party that can legally cut off such a professional spammer at the source is the upstream ISPs, transit providers, or peering exchange of the misbehaving party.

Naturally, this is assuming the ISP isn't the same company that provides the rack space. In other situations matters might be different.

And in a major datacenter, there might be a lot of different ISPs to choose from...

I guess, my point is just... the standard arrangements for such facilities can actually serve to protect spammers.

Just like they protect Enterprises (who wouldn't inhabit them otherwise -- if someone could just arbitrarily decide to power off their servers, because they didn't like a file on their website).

Re:Easier to block?

[nacturation]

Run spamd on OpenBSD or other OS that supports it. Works beautifully.

http://www.openbsd.org/cgi-bin/man.cgi?query=spamd&sektion=8
http://www.openbsd.org/cgi-bin/man.cgi?query=spamd-setup&sektion=8
http://www.openbsd.org/cgi-bin/man.cgi?query=spamd.conf&sektion=5
http://www.linux.com/archive/feature/61103

By default, email gets greylisted. In other words, the first two tries are rejected with a temporary failure message, the third try gets through. Real mail servers will retry, spammers often won't. Mail that gets through is whitelisted for that combination of sender, recipient, and IP for a month or so. You can also up-front blacklist IPs by whatever criteria you want -- published blacklists, country IP ranges, and so on. You can specify specific email addresses as spam traps, so you setup fromlamespammer@example.com on your mail server and put that as a hidden mailto link on your home page, and anyone who emails that obviously harvested it and their IP gets blacklisted.

Combine that with Bob Beck's greyscanner (google for it) which looks for individual IPs trying to send from multiple domains and blacklists them for a period of about a month. I've found it eliminates about 99% of all spam. You should still do things like proactively whitelist clients and mail servers which send from a pool of servers (otherwise it'll get delayed quite a bit). And the occasional spam that gets through should get its IP address blacklisted.

It has the additional benefit that if you run a busy mail server, running this in front significantly reduces the load on the mail server. So you end up with less spam, less wasted storage space, and a snappier mail server.

Re:Easier to block?

[xous]

Hi,

The SPAM was originating from our network which is an TOS violation which allows us to suspend services. I had already disabled the switch ports and the customer was trying to get it back online.

I had no obligation to waste my time trying looking into the problem to see how the spam was being sent. The customer could have easily went somewhere else instead of accepting the condition for turning the equipment back on.

I think what this "company" was doing had all their spam services in a data-center and only used their connection with them connecting to GRE tunnels.

Then they found smaller dedicated hosting companies that offered cheap servers ($100/mo) and tunneled all their traffic to their hosts at other networks.

It's not a bad tactic as it can sometimes take smaller companies a while to investigate complaints.

I thought...

[Darkness404]

I thought the entire reason why botnets were so hard to stop is because they could be on a huge range of IP addresses. With this isn't it trivial to see that Evilnet ISP is a botnet and has the IP addresses xxx.xxx.x.xxx- xxx.xxx.x.yyy and just block those? I mean, yeah, if they had enough bandwidth they could still flood you with requests that slow down the servers because they all need to be blocked, but shouldn't it make blocking them easier?

DNA samples/Chips in fingertips?

[e2d2]

No further investigation is done

And none should be. They're a potential customer buying IP addresses and hosting, not automatic weapons.

Pretty soon we're gonna be so "secure" it's gonna take an act of congress take a piss.

Re:DNA samples/Chips in fingertips?

[casings]

Mark Foley would probably like that idea.

Re:DNA samples/Chips in fingertips?

[Darkness404]

Sure, but the thing is IPv4 IP addresses are limited. Because of this, even if they started a botnet today and a year from now were gone, those range of IP addresses still might be blocked by various places.

I agree with your general feelings that you shouldn't need investigating to get a block of IP addresses, but it reduces a scares commodity and is in the best interests of those giving out blocks of IP addresses to check out the companies a bit more.

Hyperbole

[uassholes]

Having a block of IP addresses does not make one an ISP.

Re:Hyperbole

[Shakrai]

But they are providing internet service to the critically underserved market of phishers, extortionists and viagra salesman. I bet they even obey network neutrality and don't inject fake RST packets into your connections too. Clearly they qualify as an ISP ;)

Isn't this cool?

[DNS-and-BIND]

Remember back in the 90s when everyone was jizzing in their pants about Bruce Sterling and Neal Stephenson's writing, dreaming of actually implementing the ideas therein? Data havens, crypto-anarchism, impregnable anonymity, hackers making a decent living by a life of crime, and so forth?

Well, now the future is here. Kind of sucks, doesn't it? Careful what you wish for, you just might get it.

Re:Isn't this cool?

[JohnyDog]

Remember back in the 90s when everyone was jizzing in their pants about Bruce Sterling and Neal Stephenson's writing, dreaming of actually implementing the ideas therein? Data havens, crypto-anarchism, impregnable anonymity, hackers making a decent living by a life of crime, and so forth?
Well, now the future is here. Kind of sucks, doesn't it? Careful what you wish for, you just might get it.

In those cyberpunk visions the world, political and judicals systems are tightly controlled by corrupt mega-corporations and the net is anything but open. The very act of accessing the network or tampering with it may land you in prison, criticizing the rulers means you're dead and so on. Every piece of hardware is registered, so if you want to get any hacking done you have to turn in to black market (for stuff) and criminals (to get money for stuff), out of pure necessity. (it's the classical tale of occupied country's resistance movement working together with organized crime, right?)

Compare that to the reality we got: cheap ubiquitous internet, cheap ubiquitous hardware to access it, the net is *by default* free and open, and all attempts to any large-scale censoring has failed miserably. Anonymity is just one unsecured wifi hotspot away on every corner (so you don't need to pay a hacker to get you online), and any attempts at uncovering corruption and truth are met with public support. So the traditional heroes of cyberpunk stories can operate publicly or semi-publicly (think wikileaks), the worst that can happen to them is someone pulling the DMCA on the copied/leaked documents, which rarely results even in fines, much less prison time. The hackers are working on cool engineering projects instead of breaking into companies networks, and the criminals are, well, criminals - since they are no longer needed for the goals of the freedom fighters, all they do is disrupt the free information exchange (ddosing sites for greed, decreasing signal-to-noise ratio by spamming the hell out of everyone etc.), and so are frowned upon even by the neo-anarchists.

Is the address space for something else?

[damn_registrars]

Sure, we know a lot of the botnet activities that we care about - distributed spamming, distributed hacking, etc... But I suspect that isn't what they want the dedicated IP space for. People already pointed out that if the lion's share of your spam or hacking attempts came from a single IP block, it would be trivial to block it.

Hence I suspect the operators want the IP space for other uses. Consider your average spam - we'll say it asks you to buy viagra through joescheapdrugs.com. Now joescheapdrugs.com needs to be purchased, which requires a registrar. It also needs to be resolved via a DNS server somewhere (which isn't always done by the registrar or ISP). If joescheapdrugs.com were an average spamvertised site, it would likely be hosted in one continent, registered through a registrar in another, and resolved by a DNS in yet another.

The IP space would be useful because the DNS could be done in that range, and once the spammers establish an accredited registrar they could sell themselves domains from there too. We all know that .com, .org, .net domains not only are not restricted to sales to people/companies/organizations in the US, they aren't even restricted to being sold by companies in the US. So by owning IP space, they can actually keep more of their own money for their operations, thus increasing their profit margins. They can offer hosting, DNS, and registration services for anyone who wants to sell anything, and then sell them spamming services as well.

It becomes one-stop-shopping for vendors trying to make a fast buck (or those who don't know better).

Re:Old news

[Zocalo]

No it's not, several of the larger spam/malware gangs including the infamous Russian Business Network have been doing this for several years now. That's partly what prompted Spamhaus to create their solution to the problem: DROP. All it takes is a for the majority of the Tier 1 carriers to adopt the DROP list and it's pretty much game over for this this technique.

youtubers beware

[cl191]

"You own your own IP space and you're your own ISP at that point." I believe this sentence was designed to make youtube commenters' heads to explode......your you're you what?

Re:youtubers beware

[juliannoble]

Yo you, you're your youtube you, yet your youtube's yesterday's you.