Slashdot Mirror
Zero Errors? Spamhaus Flubs Causing Domain Deletions
Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.
Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.
On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)
So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.
A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.
Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.
Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:
DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.
Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.
Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.
Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.
So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.
Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."
Afilias does not have the intrinsic right to blackhole your DNS no matter what Spamhaus does. However, it is in your agreement when using an .info domain. An easy way out of this is to use a domain that is unaffiliated like .com/.net or out of the country like .me/.co/.it/.to
.info to death, run a persistent check on spamhaus and remove your domains from the list immediately instead of after Afilias finds out.
If you have the time, find better contacts at Afilias and get them to clarify their policy. If you have the money, call a lawyer. If you are really bored and love
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Spamhaus DBL is poorly run and full of spite listings and other garbage. Zero false positives? They mean zero legit entries. Spamhaus has become what it set out to oppose, and it's time they were exposed for what they are today. A disgrace to the anti-spam, anti-abuse community.
You should consider this a wake-up call. It's time to switch from mass-email to a web page with RSS.
If people really want your newsletter, they'll come to you.
an Afilias issue, not a Spamhaus issue.
Secondly, how sure are you somebody didn't forward your email to their own not-so-double-opt-in list which got reported as spam.
Your hair look like poop, Bob! - Wanker.
He claims that no illegal activity was taking place, but if he's running proxies that are open to the public via a mailing list, doesn't it seem entirely likely that a spammer may be making use of his mailing list to get more proxies that can be used for their operations? And, if so, isn't it entirely likely that that's exactly what got him blacklisted in the first place?
What evidence is there that his proxies weren't being used by others for illegal activities? Seems like he conveniently skirted that point in his entire write up.
Your registrar sucks, its nothing to do with Spamhaus.
I want to delete my account but Slashdot doesn't allow it.
Spamhaus always send an automatic notification to abuse@YourDomain.info, if they add you to the blacklist. I suspect you may not have configured an MX to receive mail on these domains. If you had, you would have received a notification.
Where administrators are gods of their personal fiefdomes and you have no say in anything unless you happen to own the wires or boxes yourself or are in the good graces of whoever does.
Suck it up.
It was a good idea in the beginning; Getting network and system administrators to share their stories of problems on the frontier. And for awhile, it was good. But as these services developed, they decided to start automating the process. And that's when the problems started. As an example, let's say all spammers use open relays. The logic here then is to test for open relays and block any that are found. Spam problem solved! Except it doesn't look at the reverse case: Namely, that not all open relays are used by spammers. In fact, it could be the case that the vast majority of open relays are perfectly harmless and have a legitimate reason for existing.
Now I'm not trying to discuss open relays from a technical standpoint, or the arguments for or against them -- what I'm trying to show is the logic problem in assuming that just because when 'A' is often found next to 'B', that means that 'B' is often found next to 'A'. That's the crux of the problem with the RBL and Spamhaus -- it's a logic fail of epic proportions.
Automation is attractive because it can catch things faster and with greater accuracy than humans can. But humans are better at making judgement calls, looking at the evidence, and problem resolution with other humans. Spamhaus and the RBL fail here because they implimented the automation and then because of their perceived success, they decided Automation Was God and made appealing the decision of its robot overlords increasingly difficult if not impossible. And that's when Spamhaus and the RBL became evil: The process stopped being overseen by humans, started to assume everyone was an evil spammer, and that the solution in every case was to follow the De Facto Anti-Spammer Laws as laid down by its robotic overlords. "Fix your open relay!" became the reply, instead of checking to see whether said open relay had actually sent any spam, or whether there was a good reason for its existance (again: No debates about open relays please! It's just the example!).
Of course, spammers got smarter and started coming up with more sophisticated methods of injecting their crap... which led to more complex robots, and as each new counter-measure was rolled out, the reply to hapless admins caught in the motorized wheels o spammy justice was "It's your problem, not ours!" My advice to system and network admins these days is to not use spamhaus or the RBL, or if you must, make sure your mailboxes and such are setup similar to how gmail and many exchange servers are: Have a separate spam folder, and give the user the option to whitelist anything your filters catch. Ultimately, you're providing a service to them... you have no duty or obligation to anyone else. Make sure they can use what you've given them.
#fuckbeta #iamslashdot #dicemustdie
Sooner or later people forget they signed up, stop giving a damn, or otherwise get tired of what you're sending.
If they can't figure out how to get out of it (because, really, who is going to respond to something they think is spam to make it stop), they'll flag you as spam.
Or, something automated comes along and decides that whatever you're sending is spam.
As long as it stops coming when people get tired of it ... they really don't give a crap about what happens to you.
Lost at C:>. Found at C.
In summary:
1. You run a mailing list
2 You *claim* that it's opt-in
3 Somehow Spamhaus gets your list in its honeypots
4. Spamhaus lists you
5. Afilias nukes you, all 10 of your domains.
6. You easily get your domains off Spamhaus by filling out a form
7. Somehow this is Spamhaus' fault and not Afilias for giving you the run-around
Spamhaus has servers that collect spam from the internet by just being on the internet. Spammers blindly send mail to addresses and the Spamhaus servers read the headers to see where they came from. Headers can be forged, but a good algorithm can do the same thing that a human does when reading a header - follow the chain of Received: until it hits the inevitably forged nonexistent or non-sequitur domain. The one before that gets listed at Spamhaus.
Spamhaus has no users on its honeypots that subscribe to lists. They are just "there" on the net silently collecting spam and they give no 5xx or 4xx errors (because, you know, why bother?). The only way for the honeypot to get messages from you is if your list actually contains the addresses of the honeypots.
Spamhaus has a good reputation. They are probably the most reliable blacklisting service out there and this maddens spammers to no end. There are others that shouldn't be used, but Spamhaus is used by nearly everyone who uses a blacklist because of its accuracy.
>If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list
It doesn't work that way. Clean up your list.
--
BMO
Don't talk to him like a noob, people. Bennett has been around a very, very long time. He has had a beef with DNS distributed blocklists for most of that time. Others publishing their opinions gets in his craw when it interferes with his operations. He comes in here periodically with his latest incident to rally the "freedom to do whatever I want" crowd into a frenzy. He also posts lots of other stuff worth reading. *grin*
If one considers the DBL a list of domains who have appeared in emails to spamtraps, then I would contend that it very possible that the "zero false positive" claim holds up because it very well might have happened. If it claims that all listed entities are domains owned by spam operators, then he might have an argument.
Haselton's fundamental gripe is that he should be free to communicate until a real person decides he shouldn't. The fact that automated systems now make the blocking decision, requiring human intervention to override them, is an inverted model compared to the "old internet." (The necessity came from the raw volume of spam) The death of the "old internet" began with Canter and Siegel. Some of our long-term, asylum residents just haven't accepted that fact.
I do not believe it is possible to be contactable and run a blacklist. It would require an army of support people, and most of the blacklists just do not get the kind of income necessary to pay for that.
Blacklists are a pain to deal with in general. Some simply hold you for ransom. Yet it is also a pain to run a mailserver without blacklists, so... Spamhaus has fewer false positives than most, in my experience, but it is stupid of them to claim that any list has zero of them.
Finally! A year of moderation! Ready for 2019?
From NameSilo regarding an alleged malware domain:
From Spamhaus:
So in short - the registrar asked for evidence that the domain was violating their terms of service and spamhaus simply replies they are cybercriminals... trust us! After seeing other abuse reports from them, I can tell you that spamhaus has a very snub attitude and expects to be listened to. Once when Namesilo did not listen to them enough to their liking, they added namesilo.com to their RBL - they had me modify their MTA to route email around the block, but still - I think you can see the problem here - someone has to keep spamhaus in check.
I've yet to receive any piece of e-mail from a .info domain that wasn't spam. Simply matching on .info is the most reliable filter I've found for identifying e-mail from scumbags who deserve death.
Anyone else notice this?
Unfortunately nobody keeps Spamhaus in check, that's why they've become a degenerate network abuse source. Their DBL shows them for what they are now, something the rest of the Internet needs to shun.
Like the subject says, I have to challenge the claim that Spamhaus is wrong (full disclosure: I've been professionally involved in email and web security for more than a decade, but am not, and have never been, affiliated with Spamhaus. I do, however, hold them in high regard).
First of all, when I went to those domains, what was the first thing that caught my eye? "Get a green card" ads for usagc.org. I'm not specifically accusing usagc.org of spamming, but these sorts of businesses are most typically advertised by spam. I'm sure you've seen some.
Next, those sites are open proxies (by design). Anyone can create a URL like this: http://rootface.info/ojgnl.php?ZlQc9TMpAmsr3onaDWV0g=t1wn6QmM0TaAEo7rD%2F%2Bm%2Fy%2B365U2AwdnE4VH60DF8%2BU%3D (nothing dangerous, it goes to cnn.com, but of course, you shouldn't trust me) and send it out in spam advertizing whatever they want.
Finally, you do not appear to state anywhere in your article that Spamhaus said your proxy mailing list was the source of the spam complaints (although they would not tell you if it was), and I doubt that it was. The most likely scenario is that someone abused your proxies to send spam, and since running an open proxy (regardless of noble motive) makes you complicit in that abuse, Spamhaus listed those domains.
Whether the registry's actions were justified or correct is a separate consideration. Maybe they were, maybe they weren't, but you are claiming without evidence that Spamhaus made a mistake. I'm pretty confident they didn't, for the reasons outlined above.
GIven that his article was about him setting up 10 new proxies and emailing them out, it would seem that, at least for the domains relevant to this discussion, the OP was the owner/administrator and most definitely intended them to be used in that manner. Also, from context, it appears that he was running webproxies, not email proxies. They're generally used as anonymizers, or to circumvent geo-IP techniques, not to spam people.
Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
>Why don't you just register a single .com domain and run your stuff from there
Because that would make it easy for China, Iran, and other regimes to block users from using his services. That's what he *does* -- he enables people stuck behind oppressive (often government-run) firewalls to get to blocked sites & surf without frustration and/or fear.