Slashdot Mirror
Zero Errors? Spamhaus Flubs Causing Domain Deletions
Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.
Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.
On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)
So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.
A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.
Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.
Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:
DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.
Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.
Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.
Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.
So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.
Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."
Afilias does not have the intrinsic right to blackhole your DNS no matter what Spamhaus does. However, it is in your agreement when using an .info domain. An easy way out of this is to use a domain that is unaffiliated like .com/.net or out of the country like .me/.co/.it/.to
.info to death, run a persistent check on spamhaus and remove your domains from the list immediately instead of after Afilias finds out.
If you have the time, find better contacts at Afilias and get them to clarify their policy. If you have the money, call a lawyer. If you are really bored and love
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Spamhaus DBL is poorly run and full of spite listings and other garbage. Zero false positives? They mean zero legit entries. Spamhaus has become what it set out to oppose, and it's time they were exposed for what they are today. A disgrace to the anti-spam, anti-abuse community.
He claims that no illegal activity was taking place, but if he's running proxies that are open to the public via a mailing list, doesn't it seem entirely likely that a spammer may be making use of his mailing list to get more proxies that can be used for their operations? And, if so, isn't it entirely likely that that's exactly what got him blacklisted in the first place?
What evidence is there that his proxies weren't being used by others for illegal activities? Seems like he conveniently skirted that point in his entire write up.
Until the services their customers are trying to get around block his web page. Email works a bit better for this as it's not easily blocked (unless the people doing the blocking are going to block hotmail and gmail).
Your registrar sucks, its nothing to do with Spamhaus.
I want to delete my account but Slashdot doesn't allow it.
You should consider this a wake-up call. It's time to switch from mass-email to a web page with RSS.
If people really want your newsletter, they'll come to you.
...it's a proxy list.
how long do you think those sites would stay off chinas webfilters ?
a proxy list you can't get to is rather useless.
world was created 5 seconds before this post as it is.
It was a good idea in the beginning; Getting network and system administrators to share their stories of problems on the frontier. And for awhile, it was good. But as these services developed, they decided to start automating the process. And that's when the problems started. As an example, let's say all spammers use open relays. The logic here then is to test for open relays and block any that are found. Spam problem solved! Except it doesn't look at the reverse case: Namely, that not all open relays are used by spammers. In fact, it could be the case that the vast majority of open relays are perfectly harmless and have a legitimate reason for existing.
Now I'm not trying to discuss open relays from a technical standpoint, or the arguments for or against them -- what I'm trying to show is the logic problem in assuming that just because when 'A' is often found next to 'B', that means that 'B' is often found next to 'A'. That's the crux of the problem with the RBL and Spamhaus -- it's a logic fail of epic proportions.
Automation is attractive because it can catch things faster and with greater accuracy than humans can. But humans are better at making judgement calls, looking at the evidence, and problem resolution with other humans. Spamhaus and the RBL fail here because they implimented the automation and then because of their perceived success, they decided Automation Was God and made appealing the decision of its robot overlords increasingly difficult if not impossible. And that's when Spamhaus and the RBL became evil: The process stopped being overseen by humans, started to assume everyone was an evil spammer, and that the solution in every case was to follow the De Facto Anti-Spammer Laws as laid down by its robotic overlords. "Fix your open relay!" became the reply, instead of checking to see whether said open relay had actually sent any spam, or whether there was a good reason for its existance (again: No debates about open relays please! It's just the example!).
Of course, spammers got smarter and started coming up with more sophisticated methods of injecting their crap... which led to more complex robots, and as each new counter-measure was rolled out, the reply to hapless admins caught in the motorized wheels o spammy justice was "It's your problem, not ours!" My advice to system and network admins these days is to not use spamhaus or the RBL, or if you must, make sure your mailboxes and such are setup similar to how gmail and many exchange servers are: Have a separate spam folder, and give the user the option to whitelist anything your filters catch. Ultimately, you're providing a service to them... you have no duty or obligation to anyone else. Make sure they can use what you've given them.
#fuckbeta #iamslashdot #dicemustdie
Sooner or later people forget they signed up, stop giving a damn, or otherwise get tired of what you're sending.
If they can't figure out how to get out of it (because, really, who is going to respond to something they think is spam to make it stop), they'll flag you as spam.
Or, something automated comes along and decides that whatever you're sending is spam.
As long as it stops coming when people get tired of it ... they really don't give a crap about what happens to you.
Lost at C:>. Found at C.
If that's what happened, it sounds like a DOS attack waiting to happen.How long do you suppose it will be before someone sets up an operation to spam your competition's websites to get them plonked.
Don't talk to him like a noob, people. Bennett has been around a very, very long time. He has had a beef with DNS distributed blocklists for most of that time. Others publishing their opinions gets in his craw when it interferes with his operations. He comes in here periodically with his latest incident to rally the "freedom to do whatever I want" crowd into a frenzy. He also posts lots of other stuff worth reading. *grin*
If one considers the DBL a list of domains who have appeared in emails to spamtraps, then I would contend that it very possible that the "zero false positive" claim holds up because it very well might have happened. If it claims that all listed entities are domains owned by spam operators, then he might have an argument.
Haselton's fundamental gripe is that he should be free to communicate until a real person decides he shouldn't. The fact that automated systems now make the blocking decision, requiring human intervention to override them, is an inverted model compared to the "old internet." (The necessity came from the raw volume of spam) The death of the "old internet" began with Canter and Siegel. Some of our long-term, asylum residents just haven't accepted that fact.
That's great, but his list is a list of proxy servers. The purpose of those proxy servers is 'proxy avoidance'. My content filtering automatically filters pages in the category of 'proxy avoidance'.
Therefore, if someone wanted to use his proxy servers (which he's constantly adding new domains to to get around my attempts to keep my employees from avoiding my filters) he needs a way to get them those proxy servers and they need a way to find him. I'm not allowed to block email services, but I am allowed to block sites related to getting around my filters.
This is why email works better. They can sign up at home or on some page before I find and block it, confirm via email, then get updates even if I'm blocking the place where they signed up in the first place.
There is a problem with emails being blocked as well, but that is spam filtering not my active attempt to keep them from getting around my filters. Overall this is the fundamental problem with getting around content blocking/filtering. You have to be able to find the site that tells you how to get around the filtering before the people doing the filtering filter that site.
I do not believe it is possible to be contactable and run a blacklist. It would require an army of support people, and most of the blacklists just do not get the kind of income necessary to pay for that.
Blacklists are a pain to deal with in general. Some simply hold you for ransom. Yet it is also a pain to run a mailserver without blacklists, so... Spamhaus has fewer false positives than most, in my experience, but it is stupid of them to claim that any list has zero of them.
Finally! A year of moderation! Ready for 2019?
From NameSilo regarding an alleged malware domain:
From Spamhaus:
So in short - the registrar asked for evidence that the domain was violating their terms of service and spamhaus simply replies they are cybercriminals... trust us! After seeing other abuse reports from them, I can tell you that spamhaus has a very snub attitude and expects to be listened to. Once when Namesilo did not listen to them enough to their liking, they added namesilo.com to their RBL - they had me modify their MTA to route email around the block, but still - I think you can see the problem here - someone has to keep spamhaus in check.
I've yet to receive any piece of e-mail from a .info domain that wasn't spam. Simply matching on .info is the most reliable filter I've found for identifying e-mail from scumbags who deserve death.
Anyone else notice this?
Unfortunately nobody keeps Spamhaus in check, that's why they've become a degenerate network abuse source. Their DBL shows them for what they are now, something the rest of the Internet needs to shun.
Like the subject says, I have to challenge the claim that Spamhaus is wrong (full disclosure: I've been professionally involved in email and web security for more than a decade, but am not, and have never been, affiliated with Spamhaus. I do, however, hold them in high regard).
First of all, when I went to those domains, what was the first thing that caught my eye? "Get a green card" ads for usagc.org. I'm not specifically accusing usagc.org of spamming, but these sorts of businesses are most typically advertised by spam. I'm sure you've seen some.
Next, those sites are open proxies (by design). Anyone can create a URL like this: http://rootface.info/ojgnl.php?ZlQc9TMpAmsr3onaDWV0g=t1wn6QmM0TaAEo7rD%2F%2Bm%2Fy%2B365U2AwdnE4VH60DF8%2BU%3D (nothing dangerous, it goes to cnn.com, but of course, you shouldn't trust me) and send it out in spam advertizing whatever they want.
Finally, you do not appear to state anywhere in your article that Spamhaus said your proxy mailing list was the source of the spam complaints (although they would not tell you if it was), and I doubt that it was. The most likely scenario is that someone abused your proxies to send spam, and since running an open proxy (regardless of noble motive) makes you complicit in that abuse, Spamhaus listed those domains.
Whether the registry's actions were justified or correct is a separate consideration. Maybe they were, maybe they weren't, but you are claiming without evidence that Spamhaus made a mistake. I'm pretty confident they didn't, for the reasons outlined above.
GIven that his article was about him setting up 10 new proxies and emailing them out, it would seem that, at least for the domains relevant to this discussion, the OP was the owner/administrator and most definitely intended them to be used in that manner. Also, from context, it appears that he was running webproxies, not email proxies. They're generally used as anonymizers, or to circumvent geo-IP techniques, not to spam people.
Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face