Slashdot Mirror
Maintaining a Publicly Available Blacklist - Mechanisms and Principles
badger.foo writes "When you publicly assert that somebody sent spam, you need to ensure that your data is accurate. Your process needs to be simple and verifiable, and to compensate for any errors, you want your process to be transparent to the public with clear points of contact and line of responsibility. Here are some pointers from the operator of the bsdly.net greytrap-based blacklist."
Use greylisting as a first defense - easily configurable in postfix, and it reduces the amount of spam dramatically. This relies on the behavior of the sender, rather than someone else's opinion of them.
I want to delete my account but Slashdot doesn't allow it.
And while we're at it, some hints on using a public blacklist with regards spam. The correct way is not to trust the blacklist 100%. Instead, you use it as one part of a comprehensive scheme (part of this complete breakfast). So, you may use a dictionary, and for every word in the dictionary you add 10 points (viagra, v1agra, v14gr4, etc.). You can use SPF and if it doesn't match, then that's worth 50 points, and if it's not there, maybe 20 points. And if the domain or IP address is on a blacklist, maybe 40 points. You assign the points as you like. Then, if you hit 100 points, you mark the email as "probably spam".
But you never reject or mark an email spam just because it's on some blacklist. That's just stupid. Now I'm off to RTFA.
----
OK if you have your own blacklist (perhaps a list of domains or IP addresses that have sent email to a catch-all, or that have fallen into a honeytrap), then you do what you want. But you probably should date entries and remove old ones (if they do not misbehave again), in case a legitimate user is now at that location.
HELP MY ACCOUNT HAS BEEN HACKED BY AN ILLIBERAL ART STUDENT SET TO DESTROY THE INTERWEBZ!
The nice thing about yelling at yourself is you can be pretty sure someone is listening, even if they don't agree with you.
. Your process needs to be simple and verifiable,
The process can't be simple because spammers are endlessly creative with how they try to get past the filters. And if it was verifiable, that would mean published -- and once published, becomes useless. Spammers can simply test their latest creation against your filter, and now you effectively have given them a way to bypass your entire process, making it worthless.
and to compensate for any errors, you want your process to be transparent to the public
The administrative process can be transparent, but the technical process, as outlined above, cannot.
with clear points of contact and line of responsibility.
The problem here is; how do you tell the liars from the rest? Responsibility is fine, clear points of contact are fine, but what's the criterion for delineating between 'spam' and 'marketing'? How about between 'spam' and 'opt-in' that the user no longer wants? How about between... you get the idea. There is some grey here, and odds are good you're going to find someone doing something with a legitimate and ethical reason, that by all appearances... isn't. And then you're going to make a decision based on those appearances (because what else can you go on?) and then you're going to burn a bridge down.
These problems can't be solved with a handwave and a post on an internet forum.
#fuckbeta #iamslashdot #dicemustdie
most of your spam problems will be solved by simply blocking all email from those countries except for your business partners
First off, because spam is so bad (80% of messages by some counts) just about ANYTHING that ANYONE does will reduce their spam (ignoring false positives).
Secondly, READ YOUR LOGS!
There are broad categories of how different groups use email (and their email infrastructure). So what works great for one group sucks for a different group.
So I recommend something like SpamAssassin where you can tweak the settings to what works for your specific circumstances (and the people/groups that you send/receive email with).
Greylisting is great, except when you try to greylist gmail servers. So know how the tools work and think about situations where they would fail and then adjust the knobs to deal with those potential failures.
And if you don't accept EVERY email sent to you (I don't) then make sure that you customize the rejection notice so that the SENDER can contact you if his server includes the rejection message (which most of them do). I include my phone number.
In my opinion, the more knobs that you can adjust the better it is.
... though they are not publicly-accessible; only accessible to our customers. Here's how they work:
Using our reputation-collection protocol, we receive a constant stream of events from our customers. An "event" is something like "IPv4 address x.y.z.w sent to a nonexistent recipient" or "IPv6 address abcd::1234 sent something that a human voted as spam"
Currently, we have a database of just under two billion events. Once an hour, we go through our database and categorize IP addresses as:
The whole system is 99.99% automated. The only manual intervention is when some requests delisting. If it seems that someone was the victim of a compromise and has now cleaned up his/her machine, we delist it for 45 days which is long enough for all events from that IP to expire. Then it goes back into consideration for automatic listing.
This system works really well. We have about 3.75 million IPv4 and 3300 IPv6 addresses on our lists; those are machines for which we have confidence that there's enough data to categorize them.
Another board I frequent, using the Drupal blogging software, is currently being overwhelmed with spam.
.
Our beloved webmaster is experimenting with Mollum spam retarding software
This software does have its faults, as it is hindering the posting of links by some of our most informative posters. A blogsite's "good folk" need to be whitelisted so they can post links unhindered. More often than not, the most informative content of a post is a link.
Anyone else having a blogsite overrun with crap might want to look into this. I do not think its the ultimate solution, but its a start.
"Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
Better solution: Stop trying to force email to be a reliable and concurrent source of information. It has never been reliable nor has it ever been concurrent protocol. Check the default settings for sending email - try every hour for up to 5 days before giving up. Wait one day before sending a trouble report.
That email now generally DOES deliver results in almost real time is no excuse to think it will ALWAYS deliver in real time. If your communication either critical and/or time sensitive, then email is the wrong tool to use.
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
0. Find a system that makes their blacklistings publicly available.
1. Send it SPAM.
3. See what gets through, send more of that from those IPs.
4. Tweak the stuff that didn't get through until it does.
2. V1AGR4 !!
5. Rotate IPs from your pool of thousands that aren't blacklisted.
6. Prophet.
7. GOTO 0.
Protip: Your public blacklist is part of the fucking problem, fool. Either use a whitelist if you can (+trust graphs), or if you can't then let those blacklisted contact you if they care.
Your post advocates a
(X) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
(X) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
(X) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
(X) Microsoft will not put up with it
( ) The police will not put up with it
(X) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
(X) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
(X) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
(X) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
(X) Susceptibility of protocols other than SMTP to attack
(X) Willingness of users to install OS patches received by email
(X) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
(X) Joe jobs and/or identity theft
(X) Technically illiterate politicians
(X) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
(X) Bandwidth costs that are unaffected by client filtering
(X) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
(X) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
Six years ago, I wrote milter-greylist. At that time I thought some kind of distributed spam traps would be useful. I wrote software for a P2P network of mail servers that exchange signed information on messages reaching spam traps. The thing turned to be useless: greylisting alone was enough. Today, greylisting with variable delays depending on sender reputation from various DNSRBL is still enough, even is the DNSRBL information is not very reliable: an error just means an extra delay in delivery.