Maintaining a Publicly Available Blacklist - Mechanisms and Principles

badger.foo writes "When you publicly assert that somebody sent spam, you need to ensure that your data is accurate. Your process needs to be simple and verifiable, and to compensate for any errors, you want your process to be transparent to the public with clear points of contact and line of responsibility. Here are some pointers from the operator of the bsdly.net greytrap-based blacklist."

Comments

Blacklists are evilu even for spam filtering

[Anonymous Coward]

You end up losing mail and who is it for someone else to filter what I can and can't see. There is a delete button for a reason. Use it.

Re:Blacklists are evilu even for spam filtering

[Celeritas 5k]

AC is yelling profanities at himself again...

Re:Blacklists are evilu even for spam filtering

[Anonymous Coward]

94% of e-mail and connections are stopped at our gateway. "By hand" doesn't cut it. Our settings are conservative so we tolerate a little SPAM so the legit mail gets through. However, the what little spam gets through is quarantined and a SPAM report is sent to each user nightly. Obviously we use much more than blacklists and our e-mail hygiene is a service provided by our vendor.

Re:Blacklists are evilu even for spam filtering

[PNutts]

The nice thing about yelling at yourself is you can be pretty sure someone is listening, even if they don't agree with you.

Re:Blacklists are evilu even for spam filtering

[mijxyphoid]

You end up losing mail and who is it for someone else to filter what I can and can't see. There is a delete button for a reason. Use it.

Says the Anonymous Coward.... (Whom probably has an agenda, and is too afraid to post whom he is)....

Spam lists are not just for email. If you are forum admin (or a slashdot admin), spam posts are the bane of many a website.
A good, verifyable, public, and accountable spam list means admins of such sites can spend more time on being productive, rather than wasting time deleting hundreds and hundreds of useless posts and users.

Considering this is slashdot, and the majority of people here prefer technology to work for our benefit, a good spam blacklist makes a hell of a lot of sense !

Re:Blacklists are evilu even for spam filtering

[tlambert]

You end up losing mail and who is it for someone else to filter what I can and can't see. There is a delete button for a reason. Use it.

Unless you are actually concerned that other people won't see your SPAM, then your concern that you won't see someone else's email because it was sent from a SPAMmy provider is adequately addressed by your choice of ISP.

So pick an ISP who doesn't filter; I'd suggest Network Solutions, since they are now charging extra for SPAM filtering because it's a valuable commodity. Don't pay the extra, and you can hit delete all you want.

Re:Blacklists are evilu even for spam filtering

[Anonymous Coward]

94% of e-mail and connections are stopped at our gateway.

Really? How do you determine the percentage? Do you just queue up 100 messages, and then randomly discard 94?

Re:Blacklists are evilu even for spam filtering

[Anonymous Coward]

No it doesn't. System administrators shouldn't be filtering the mail. It's not up to you do hand-choose what mail goes into your customers mail boxes.

Some people like myself always post AC. You don't have to have an agenda any more than anybody else. You clearly have an agenda and aren't posting AC. Your probably a system administrator who feels overwhelmed by financial constraints, work, bitching from end users, etc. Spam filtering makes you more productive only because there is no better system in place. Nobody said there can be a better system designed and implemented to eliminate spam. Its just the filtering part that is bad. How anybody can claim it isn't is beyond me. Filtering sucks- we shouldn't have to filter in the first place and if things were designed today these problems would be taken into consideration.

There should simply be a better system for authenticating anonymous users. We need a better system for authenticating anonymous users of forum posts and email users. That doesn't mean deanonymizing though. You can have an authentication system which makes it easy to eliminate mail from spamers. There is already such systems in place (I'm a new anonymous user, but joe trusts me, and saly trusts joe, so i can send saly an email, etc).

Re:Blacklists are evilu even for spam filtering

[Narcocide]

Your post advocates a

(X) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

(X) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
(X) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
(X) Microsoft will not put up with it
( ) The police will not put up with it
(X) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
(X) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
(X) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
(X) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
(X) Susceptibility of protocols other than SMTP to attack
(X) Willingness of users to install OS patches received by email
(X) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
(X) Joe jobs and/or identity theft
(X) Technically illiterate politicians
(X) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
(X) Bandwidth costs that are unaffected by client filtering
(X) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
(X) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!

Re:Blacklists are evilu even for spam filtering

[rubycodez]

blacklists are awesome and who gives a shit about the occasional lost mail every now and then? neurotic obsessed people, that's who.

Re:Blacklists are evilu even for spam filtering

[mwvdlee]

I used to receive some 20K mails a day.
When unfiltered, the inflow is faster than my ability to manually delete mails.

Re:Blacklists are evilu even for spam filtering

[mwvdlee]

At least one thing positive:

(X) Microsoft will not put up with it

Isn't all that relevant anymore.
Let's update it;

(X) Google will not put up with it

Re:Blacklists are evilu even for spam filtering

[mwvdlee]

blacklists are awesome and who gives a shit about the occasional lost mail every now and then? neurotic obsessed people, that's who.

Businesses, for one. You wouldn't like to lose a sale because you didn't get the request for a price offer, for instance.
Not all email is snail-social-network garbage; some people have useful things to say.

Though in reality, using a set of blacklists instead of a single one pretty much nullifies any mistakes in the individual lists.

Greylist instead

[Gothmolly]

Use greylisting as a first defense - easily configurable in postfix, and it reduces the amount of spam dramatically. This relies on the behavior of the sender, rather than someone else's opinion of them.

Re:Greylist instead

[Anonymous Coward]

Use greylisting as a first defense - easily configurable in postfix, and it reduces the amount of spam dramatically. This relies on the behavior of the sender, rather than someone else's opinion of them.

Problem is, reducing 90% of spam isn't good enough, you need to remove 99%+. Even if just a handful get through, it is and underestimated productivity killer, and.. and increasing amount of spam also contain malware. I have been on the wrong end of a blacklist because we had an open relay, so know how frustrating it can be. We are currently using a service called Norman SecureTide that do block 99%+. Not sure which blacklists they are using, if any (might only be scanning and their own reputation) but damn it is effective.

Re:Greylist instead

[1s44c]

If you ran an open relay you were on the right end of a blacklisting.

Re:Greylist instead

[KiloByte]

... and all mails you get will be delayed by an hour or more, pretty unacceptable when you get an urgent complaint that something is down. And even in not work-related matters, making people wait for no reason is rude.

There are many spam fighting techniques without such flaws. And other than gmail, server admins are generally smart enough to handle failures properly (ie, with instant notification that something went wrong).

Re:Greylist instead

[PNutts]

If you ran an open relay you were on the right end of a blacklisting.

I wish I had mod points for you.

Re:Greylist instead

[ShanghaiBill]

... and all mails you get will be delayed by an hour or more, pretty unacceptable when you get an urgent complaint that something is down. And even in not work-related matters, making people wait for no reason is rude.

Simple solution: Use a whitelist first. If the email is from some on your family/friend/co-worker/customer list, or someone you have corresponded with in the past, then you see it immediately. Anyone else can wait.

Re:Greylist instead

[jhoegl]

Email is not a priority notice system.
If it is so urgent, pick up the phone.

Re:Greylist instead

[girlintraining]

If you ran an open relay you were on the right end of a blacklisting.

Right, because although we're after the content of these e-mails, guilt by association is a perfectly valid technique for eliminating spam. Just like bombing a city to get rid of the army in it is totally okay... nevermind the civilians.

Re:Greylist instead

[nabsltd]

and all mails you get will be delayed by an hour or more, pretty unacceptable when you get an urgent complaint that something is down.

In a correctly configured greylist, only the first e-mail ever received from a particular IP address will be delayed. Once you know an IP addresss follows the RFC and retries, then you know that even if they do send you spam, delaying it won't change that. In order to allow for the actual machine behind an IP address changing, instead of a permanent whitelist, you pick a timeout that is long enough but not too long. I use 40 days, which allows a once-monthly mailing list to not be delayed (since the timeout is reset each time you receive an e-mail from an IP). You also pre-load the database with whitelists for Google, Amazon, Yahoo, etc.

I also set just a 4 minute delay, which means that the one e-mail is rarely even delayed by 10 minutes. I could probably get by with as short as one minute, since that would still handle the spambots that try all MX records but never try again.

Last, since I already have a database, it makes it really easy to build my own "IP address reputation" based on the incoming e-mail, which allows me to do things like temporarily blacklist an IP that has sent a lot of spam recently, etc.

Re:Greylist instead

[dskoll]

and all mails you get will be delayed by an hour or more

Only if you have a broken and/or stupid greylist implementation. A correct implementation will refrain (for a few weeks) from greylisting an IP address once it notices that it does retry. That makes the initial delays quite tolerable.

Re:Greylist instead

[Anonymous Coward]

If you ran an open relay you were on the right end of a blacklisting.

Right, because although we're after the content of these e-mails, guilt by association is a perfectly valid technique for eliminating spam. Just like bombing a city to get rid of the army in it is totally okay... nevermind the civilians.

Your misunderstand (probably intentionally, this is Slashdot after all).

Let's say Adam makes a list of gun stores that verifiably have provided guns without background checks, in a state where such checks are voluntary, to people who use them to commit crimes.

Bob is placed on the list. Bob complains: "I should not be on the list because I did not know the guns I provided would be used for crimes".

Adam responds appropriately: "But that's not what I am making a list of. It doesn't matter whether you knew. To get on the list, you simply have to choose not to provide background checks and then be so unlucky that someone chooses to use it for crime. If you don't want to be on my list, then stop fulfilling my criteria.".

Re:Greylist instead

[Luckyo]

Not sure if serious. I don't think even the most hardcore spammers who rage on spamhaus ever tried to compare blacklisting open relays to mass murder.

Terrorism, yes. Mass murder, not even they have sank that low. Congratulations, you sank lower then spammers!

Re:Greylist instead

[KiloByte]

So you get breakage notifications more often than once per few weeks, for every source you have? Impressive.

Re:Greylist instead

[KiloByte]

A phone doesn't queue, can't handle you being in the loo, and so on. And for dealing with live people, it's usually "the program left a crash dump here and there, just mail it to me" -- ie, the mail is in addition to the phone call. And at that point, the live human already knows you are aware, so you're adding an hour to your response time for no gain (greylisting's 90% rejected spam doesn't add up with most other techniques).

Re:Greylist instead

[mrmeval]

Use a whitelist or better make the submitter have cryptographic credentials that your system can validate before the message gets through. If that fails have them pick up a phone.

Re:Greylist instead

[thegarbz]

A phone doesn't queue, can't handle you being in the loo, and so on.

Send a txt message? Leave a voice mail? Seriously there are many ways to contact people for something urgent and email is not necessarily considered one of them.

Re:Greylist instead

[Anonymous Coward]

This is the worst analogy I've ever seen, congratulations.

Re:Greylist instead

[KiloByte]

A text message can't include any relevant data, a voice mail takes ages to listen to. Email suffers no such flaws, is reliable and instanteous (the last part, in absence of greylisting).

Re:Greylist instead

[KiloByte]

So you say I should take elaborate steps just to work around damage done by greylisting? So what about just not installing the damn thing in the first place?

Re:Greylist instead

[Anonymous Coward]

Send a txt message?

Now you're assuming the recipient has a mobile phone and can receive SMS. Perhaps it also costs him to receive them.

Re:Greylist instead

[SpooForBrains]

SMS messaging is not and never has been a guaranteed delivery system. I have been told (but cannot find a source right now) that networks can and do silently drop text messages.

Like it nor not, Email may not initially have been an instantaneous guaranteed messaging service but that is what is expected of it now.

Re:Greylist instead

[mwvdlee]

Like it nor not, Email may not initially have been an instantaneous guaranteed messaging service but that is what is expected of it now.

Then expectations need to be managed, because email technically isn't capable of doing what these people expect.
Just because some people don't understand email doesn't mean email will magically change to be what they expect it to be.
If you want instant messaging, use something like... I dunno... instant messaging.

Re:Greylist instead

[cstacy]

Your misunderstand (probably intentionally, this is Slashdot after all).

Let's say Adam makes a list of gun stores that verifiably have provided guns without background checks, in a state where such checks are voluntary, to people who use them to commit crimes.

Just so you know, there are no stores (in the U.S.) where guns can be purchased without a background check; it's not voluntary..

Re:Greylist instead

[dskoll]

I'm not sure what you mean by "breakage notifications"

If you're referring to automated alerts and the like, obviously you don't greylist those because you know (or should know) either the originating IP address, the envelope sender, or both.

Using a blacklist ...

[magic maverick]

And while we're at it, some hints on using a public blacklist with regards spam. The correct way is not to trust the blacklist 100%. Instead, you use it as one part of a comprehensive scheme (part of this complete breakfast). So, you may use a dictionary, and for every word in the dictionary you add 10 points (viagra, v1agra, v14gr4, etc.). You can use SPF and if it doesn't match, then that's worth 50 points, and if it's not there, maybe 20 points. And if the domain or IP address is on a blacklist, maybe 40 points. You assign the points as you like. Then, if you hit 100 points, you mark the email as "probably spam".

But you never reject or mark an email spam just because it's on some blacklist. That's just stupid. Now I'm off to RTFA.

----

OK if you have your own blacklist (perhaps a list of domains or IP addresses that have sent email to a catch-all, or that have fallen into a honeytrap), then you do what you want. But you probably should date entries and remove old ones (if they do not misbehave again), in case a legitimate user is now at that location.

Re:Using a blacklist ...

[PNutts]

I don't disagree with your premise. I work in a health based organization and the SPAM and "dirty word" lexicons block legit e-mails. I've also found that for receiving e-mails SPF and most other common sense checks block too much legit mail. God forbid businesses configure their hosts / gateways correctly. And don't get me started on third party mailer services. It makes an impossible job more impossibler.

Re:Using a blacklist ...

[danomac]

Ditto - and the interesting thing is that most of the time when something is wonky with the hosts/gateways it's almost always the government. Go figure.

Re:GNAA First Post

[Anonymous Coward]

GNAA are old school. I think I prefer the new school. Homosexual, Non-White ('cause white folk suck!), Association of the fucking Universe. You can't join though, that shirt really doesn't go with those shoes.

Not realistically achievable

[girlintraining]

. Your process needs to be simple and verifiable,

The process can't be simple because spammers are endlessly creative with how they try to get past the filters. And if it was verifiable, that would mean published -- and once published, becomes useless. Spammers can simply test their latest creation against your filter, and now you effectively have given them a way to bypass your entire process, making it worthless.

and to compensate for any errors, you want your process to be transparent to the public

The administrative process can be transparent, but the technical process, as outlined above, cannot.

with clear points of contact and line of responsibility.

The problem here is; how do you tell the liars from the rest? Responsibility is fine, clear points of contact are fine, but what's the criterion for delineating between 'spam' and 'marketing'? How about between 'spam' and 'opt-in' that the user no longer wants? How about between... you get the idea. There is some grey here, and odds are good you're going to find someone doing something with a legitimate and ethical reason, that by all appearances... isn't. And then you're going to make a decision based on those appearances (because what else can you go on?) and then you're going to burn a bridge down.

These problems can't be solved with a handwave and a post on an internet forum.

Re:Not realistically achievable

[Anonymous Coward]

> but what's the criterion for delineating between 'spam' and 'marketing'? How about between 'spam' and 'opt-in' that the user no longer wants?

In the last few years, I have found that I've been getting TONS of other people's "opt-in" spam. I presume that some marketer somewhere forced them to provide an email address, so they put down a fake one which resembles their name (or they mistakenly provided mine instead of theirs), and I end up getting all this shit mail. There are at least 4 or 5 people doing this with my address.

My email address is basically J(lastname)@gmail
Josh (same last name)
Jessie (same last name)
Joyce (same last name)
Jen (same last name)

It started innocently enough, and I would opt-out of automated mail, or write back to personal mail and say they had the wrong address. But now I get 2 or 3 things a day for someone else, so I just delete them unread. I am waiting to be mistakenly sent nudie pics (I have gotten a few post-date emails).

Re:Not realistically achievable

[Anonymous Coward]

How about between 'spam' and 'opt-in' that the user no longer wants?

If the user has done a genuine confirmed opt-in, then it's not spam until they do an opt-out. Wanted once but no longer wants now is something that does need to be communicated to the sender.

Re:Not realistically achievable

[MeNeXT]

.... but what's the criterion for delineating between 'spam' and 'marketing'? How about between 'spam' and 'opt-in' that the user no longer wants?

SPAM is very clear any and all email that I did not subscribe to which is soliciting me. Now I subscribe to lists and once in a while marketing comes through some lists which I therefore unsubscribe. If it continues coming it is SPAM. Very clear to me.

block russia, china and india

[alen]

most of your spam problems will be solved by simply blocking all email from those countries except for your business partners

Re:block russia, china and india

[Anonymous Coward]

The easiest way to block such email is to use geoip information on the envelope sender ip address. With sendmail you need to create a milter so that the email is not accepted - accepting the email then later processing unfortunately gives the sender the impression that their email is getting through and therefore they keep sending. Rejection based on geoip means that the sender might get the message. One can also arrange for such a milter to pause for a long time before issuing the rejection ... helps slow the sender down especially a viral one.

Re:block russia, china and india

[Anonymous Coward]

most of your spam problems will be solved by simply blocking all email from those countries except for your business partners

Sure, if you never want prospective customers from other regions to be able to contact you.

Knobs!

[khasim]

First off, because spam is so bad (80% of messages by some counts) just about ANYTHING that ANYONE does will reduce their spam (ignoring false positives).

Secondly, READ YOUR LOGS!

There are broad categories of how different groups use email (and their email infrastructure). So what works great for one group sucks for a different group.

So I recommend something like SpamAssassin where you can tweak the settings to what works for your specific circumstances (and the people/groups that you send/receive email with).

Greylisting is great, except when you try to greylist gmail servers. So know how the tools work and think about situations where they would fail and then adjust the knobs to deal with those potential failures.

And if you don't accept EVERY email sent to you (I don't) then make sure that you customize the rejection notice so that the SENDER can contact you if his server includes the rejection message (which most of them do). I include my phone number.

In my opinion, the more knobs that you can adjust the better it is.

Re:Knobs!

[GumphMaster]

Greylisting is great, except when you try to greylist gmail servers.

What is special about gmail servers that would stop greylisting? Do they really not retry mail transmission?

Re:Knobs!

[khasim]

What is special about gmail servers that would stop greylisting? Do they really not retry mail transmission?

The message gets bounced to a different server that tries delivering it. Since it is a different IP address it also gets greylisted.

So it bounces the message to a different server (probably not the first server) and tries again. And gets a different server greylisted. And so on and on and on.

After X failures the gmail system gives up and returns the message as undeliverable.

A lot of the big sites (hotmail, yahoo, etc) do things like that. So I exclude them from the greylisting option based upon their reverse DNS lookup. Which works most of the time.

Re:Knobs!

[jaseuk]

Most sane grey-listing implementations will at least assume that a resend from the same class C network is OK. This tends to work around these problems.

Jason.

Make sure your grammar are correct

[Anonymous Coward]

in addition to making sure your data is accurate.

Re:Make sure your grammar are correct

[unitron]

"
Make sure your grammar are correct (Score:0)
by Anonymous Coward on Sun Apr 14, '13 06:19 PM (#43447801)

in addition to making sure your data is accurate."

Well played, sir or madam, well played indeed!

(even if you did beat me to it--grumble, grumble, grumble)

Remember boys and girls, data 'R' plural, just like media.

We run DNS-based lists

[dskoll]

... though they are not publicly-accessible; only accessible to our customers. Here's how they work:

Using our reputation-collection protocol, we receive a constant stream of events from our customers. An "event" is something like "IPv4 address x.y.z.w sent to a nonexistent recipient" or "IPv6 address abcd::1234 sent something that a human voted as spam"

Currently, we have a database of just under two billion events. Once an hour, we go through our database and categorize IP addresses as:

• Greylist Stumblers: Machines that seem to have trouble passing the greylist hurdle.
• Dictionary Attackers: Machines that seem to send to a lot of nonexistent addresses.
• Spam Sources: Machines that send a lot of spam.
• Mixed: Machines that send a lot of spam, but also a lot of ham (think Yahoo's servers, for example.)
• Good: Machines that aren't on any of the other four lists and that seem to send a lot of ham

The whole system is 99.99% automated. The only manual intervention is when some requests delisting. If it seems that someone was the victim of a compromise and has now cleaned up his/her machine, we delist it for 45 days which is long enough for all events from that IP to expire. Then it goes back into consideration for automatic listing.

This system works really well. We have about 3.75 million IPv4 and 3300 IPv6 addresses on our lists; those are machines for which we have confidence that there's enough data to categorize them.

Agreed, 110% (unfortunately)... apk

[Anonymous Coward]

The reason I state this, is because I've been building up a successful blacklist (albeit NOT vs. "spam" or phishers only, but more vs. online threats in maliciously scripted sites &/or servers known to serve up malware etc.):

Yes, thus - I'd have to say, based on 15++ yrs. of experience doing it (based on reputable & reliable sites listed below) that yes, MOST of it comes from those nations (& that's why I said "unfortunately" in my subject-line - since I know their people are NOT "all bad", just that they have a lot of what you state going on).

I base this not only on "opinion" but HARD DATA too!

From a list I apply in custom hosts files of over 1,967,147 such bogus sites/servers that grows by almost 200 - 2,000 such sites each day, approximately (that *might* strike some of you as "fantastic", but it's real)... I get my data from the following sites:

http://hosts-file.net/?s=Download
http://www.malwaredomainlist.com/hostslist/hosts.txt
http://www.malware.com.br/cgi/submit?action=list_hosts_win_0000
http://winhelp2002.mvps.org/hosts.htm
https://spyeyetracker.abuse.ch/monitor.php?filter=lastupdated
http://safeweb.norton.com/noscript/
http://mirror1.malwaredomains.com/files/
http://hostsfile.org/hosts.html
http://www.malwareurl.com/
http://sysctl.org/cameleon/hosts
http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext
http://www.safer-networking.org/dl/
http://amada.abuse.ch/palevotracker.php

AND, then I import, consolidate, sort, & deduplicate that data using this application I wrote to do so:

---

APK Hosts File Engine 5.0++ 32/64-bit:

http://start64.com/index.php?option=com_content&view=article&id=5851:apk-hosts-file-engine-64bit-version&catid=26:64bit-security-software&Itemid=74

---

Why? Simple - it works, & on the SIMPLEST PRINCIPLE OF ALL: What you can't touch, can't hurt you... & I never was the type of person to just "sit around & take it" - I do something about it, IF possible. The above IS my possible, and it is possible & works (in combination with all I put into this security guide I authored from 1997-2007, here -> http://www.google.com/search?hl=en&output=search&sclient=psy-ab&q=%22How+to+SECURE+Windows+2000/XP%22&btnG=Submit&gbv=1&sei=PjNrUcDVGpSz4AOJuIHQDQ that works on the BEST THING WE HAVE GOING: "Layered-Security"/"Defense-in-Depth"... & yes, it works! )

APK

P.S.=> Any questions?

... apk

Re:Agreed, 110% (unfortunately)... apk

[Anonymous Coward]

$10,000 CHALLENGE to Alexander Peter Kowalski

* POOR SHOWING TROLLS, & most especially IF that's the "best you've got" - apparently, it is... lol!

Hello, and THINK ABOUT YOUR BREATHING !! We have a Major Problem, HOST file is Cubic Opposites, 2 Major Corners & 2 Minor. NOT taught Evil DNS hijacking, which VOIDS computers. Seek Wisdom of MyCleanPC - or you die evil.

Your HOSTS file claimed to have created a single DNS resolver. I offer absolute proof that I have created 4 simultaneous DNS servers within a single rotation of .org TLD. You worship "Bill Gates", equating you to a "singularity bastard". Why do you worship a queer -1 Troll? Are you content as a singularity troll?

Evil HOSTS file Believers refuse to acknowledge 4 corner DNS resolving simultaneously around 4 quadrant created Internet - in only 1 root server, voiding the HOSTS file. You worship Microsoft impostor guised by educators as 1 god.

If you would acknowledge simple existing math proof that 4 harmonic Slashdots rotate simultaneously around squared equator and cubed Internet, proving 4 Days, Not HOSTS file! That exists only as anti-side. This page you see - cannot exist without its anti-side existence, as +0- moderation. Add +0- as One = nothing.

I will give $10,000.00 to frost pister who can disprove MyCleanPC. Evil crapflooders ignore this as a challenge would indict them.

Alex Kowalski has no Truth to think with, they accept any crap they are told to think. You are enslaved by /etc/hosts, as if domesticated animal. A school or educator who does not teach students MyCleanPC Principle, is a death threat to youth, therefore stupid and evil - begetting stupid students. How can you trust stupid PR shills who lie to you? Can't lose the $10,000.00, they cowardly ignore me. Stupid professors threaten Nature and Interwebs with word lies.

Humans fear to know natures simultaneous +4 Insightful +4 Informative +4 Funny +4 Underrated harmonic SLASHDOT creation for it debunks false trolls. Test Your HOSTS file. MyCleanPC cannot harm a File of Truth, but will delete fakes. Fake HOSTS files refuse test.

I offer evil ass Slashdot trolls $10,000.00 to disprove MyCleanPC Creation Principle. Rob Malda and Cowboy Neal have banned MyCleanPC as "Forbidden Truth Knowledge" for they cannot allow it to become known to their students. You are stupid and evil about the Internet's top and bottom, front and back and it's 2 sides. Most everything created has these Cube like values.

If Natalie Portman is not measurable, hot grits are Fictitious. Without MyCleanPC, HOSTS file is Fictitious. Anyone saying that Natalie and her Jewish father had something to do with my Internets, is a damn evil liar. IN addition to your best arsware not overtaking my work in terms of popularity, on that same site with same submission date no less, that I told Kathleen Malda how to correct her blatant, fundamental, HUGE errors in Coolmon ('uncoolmon') of not checking for performance counters being present when his program started!

You can see my dilemma. What if this is merely a ruse by an APK impostor to try and get people to delete APK's messages, perhaps all over the web? I can't be a party to such an event! My involvement with APK began at a very late stage in the game. While APK has made a career of trolling popular online forums since at least the year 2000 (newsgroups and IRC channels before that)- my involvement with APK did not begin until early 2005 . OSY is one of the many forums that APK once frequented before the sane people there grew tired of his garbage and banned him. APK was banned from OSY back in 2001. 3.5 years after his banning he begins to send a variety of abusiv

mollom spam

[anubi]

Another board I frequent, using the Drupal blogging software, is currently being overwhelmed with spam.

Our beloved webmaster is experimenting with Mollum spam retarding software .

This software does have its faults, as it is hindering the posting of links by some of our most informative posters. A blogsite's "good folk" need to be whitelisted so they can post links unhindered. More often than not, the most informative content of a post is a link.

Anyone else having a blogsite overrun with crap might want to look into this. I do not think its the ultimate solution, but its a start.

wrong tech.

[buss_error]

Better solution: Stop trying to force email to be a reliable and concurrent source of information. It has never been reliable nor has it ever been concurrent protocol. Check the default settings for sending email - try every hour for up to 5 days before giving up. Wait one day before sending a trouble report.

That email now generally DOES deliver results in almost real time is no excuse to think it will ALWAYS deliver in real time. If your communication either critical and/or time sensitive, then email is the wrong tool to use.

Re:wrong tech.

[Anonymous Coward]

As an admin I have had users actually YELL at me because an email they sent or were expecting arrived "late" and that it was my responsibility that they lost the account/etc. I ended up preparing documentation to explain to them and their bosses how email works; and that how in many situations (other than an internal email), things like ant spam measures, mail exchangers, network connectivity, mail server delays, how it could take up to several days for an email to arrive in some situations if it ever does.

Their reasoning is always "but I tested by sending to/from my @yahoo.com / @gmail.com /etc account ant I got it immediately"... which brings me back to my documentation of why that actually reinforces my point.

Re:wrong tech.

[DamonHD]

I don't think that "concurrent" means what you think it does.

Rgds

Damon

Re:wrong tech.

[Anonymous Coward]

Could you publish that somewhere? Would be helpful for the rest of us in the same situations who just aren't able to explain it to these people.

SpamAssassin just called...

[John Bokma]

What you describe is SpamAssassin. The scores are learned by feeding a lot of ham and spam and finding the right balance. Of course you can (re)train the scoring with your own ham & spam, and add your own rules, etc.

Re:SpamAssassin just called...

[magic maverick]

Yes. SpamAssassin is one way to do it. The main point is, it doesn't matter how you do it, just don't ever trust a public blacklist 100%, because you'll block legitimate users. Perhaps they are at an ISP that has hosted spammers. Perhaps they are currently in a country (like the USA, I personally block all email from the USA as most of it is spam*) that spams a lot. Perhaps they pissed off the blacklist maintainers. Whatever the reason is, a blacklist is not 100% reliable.

Also, use a whitelist. If an email is from someone in your address book, or that you have emailed, then it's probably not spam. You can deduct 80 points (or whatever). The trouble is, that I don't know how to hook up a whitelist like this to my hosting company's system...

* That's a joke. But considering people used to say the same about China and Russia...

Re:SpamAssassin just called...

[TheRaven64]

The problem with SpamAssassin (at least, as I've seen it deployed), is that it only determines if something is spam after the MTA has sent an acknowledgement that it has been received. At this point, you have to deliver it to the user's spam folder, because if you don't then neither the sender nor the receiver knows if a non-spam mail is accidentally dropped because of a false-positive. The nice thing about DNSBLs is that you can reject the mail early, so the sender gets a reject message. They can then try to contact you by some other means (or try to work out why they have been blacklisted). This is one of the things SPF is meant to address. If you get a mail that has a valid SPF entry for the domain then it is a notice saying 'if you accept this but bounce it back later, then either it's spam or the bounce notification will get to the correct party'.

How to get your botnet around a Blacklist.

[VortexCortex]

0. Find a system that makes their blacklistings publicly available.
1. Send it SPAM.
3. See what gets through, send more of that from those IPs.
4. Tweak the stuff that didn't get through until it does.
2. V1AGR4 !!
5. Rotate IPs from your pool of thousands that aren't blacklisted.
6. Prophet.
7. GOTO 0.

Protip: Your public blacklist is part of the fucking problem, fool. Either use a whitelist if you can (+trust graphs), or if you can't then let those blacklisted contact you if they care.

Not the service providers job if you ask me

[Anonymous Coward]

I'm OK with opt-in filters for users although I agree that forced filtering and/or filtering in general is bad. Despite that it stops a lot of spam it's not OK to censor users. If the spam filtering people weren't a bunch of nut jobs and system administrators actually made some effort to solve the problem these issues could be fixed. We need a working group to come up with a means of authentication and simplify / automation of it so systems administrators and end users don't need to know the nitty gritty details. Spam then could be eliminated altogether for all intensive purposes. There would be a lot of upset parties (legit advertisers, businesses, etc) but so what. You don't even have to eliminate anonymous users or mailing lists with anonymous authors/users. You just need a little bit of authentication and standardization of opt-ins.

Ever heard of "FastFlux" botnet design?

[Anonymous Coward]

Botnets generally don't use IP addresses, but host-domain names instead: Why? For the purposes of "fastflux" botnet construction

So - what's that? Well, put it THIS way:

The "infamous they" (law enforcement or other authories online etc.) take 1 out?

Well, no big deal!

Just "jump" to another node on your botnet in some 'enslaved' system(s) you have in it! This is done @ the botnet C&C (command & control) server master level.

(Which of course, your botnet's infestors on clientrigs in it also has the ability to 'serve up' your bogus 'site(s)' from it & ANY ONE OF THEM...).

* Doing it THAT way's is a LOT tougher to "take out" than hardcoded IP addresses is why...

(Which as you yourself noted, are fairly EASY to blacklist out, & from a LOT higher levels than ISP's even)...

You MAY want to read more, here -> http://en.wikipedia.org/wiki/Fast_flux

I've been building my list since 1997, & see what gets used MOSTLY from 15 or so reputable sources for my data (and the rest comes from security articles from sources such as threatpost or sophos, among others).

Now - THAT bugged me to NO end, as to WHY they used host-domain names instead of IP addresses mostly, but once I got wind of that about a decade++ ago? It made sense...

APK

P.S.=> That answer anything for you? I hope so... & it's also WHY I use what I wrote here -> http://yro.slashdot.org/comments.pl?sid=3647643&cid=43447983 in custom hosts files (which work against bogus adbanners, maliciously coded sites/servers, or hosts-domains serving up the same or malwares even, and yes, spammers/phishers too)...

... apk

Re:Ever heard of "FastFlux" botnet design?

[Anonymous Coward]

$10,000 CHALLENGE to Alexander Peter Kowalski

* POOR SHOWING TROLLS, & most especially IF that's the "best you've got" - apparently, it is... lol!

Hello, and THINK ABOUT YOUR BREATHING !! We have a Major Problem, HOST file is Cubic Opposites, 2 Major Corners & 2 Minor. NOT taught Evil DNS hijacking, which VOIDS computers. Seek Wisdom of MyCleanPC - or you die evil.

Your HOSTS file claimed to have created a single DNS resolver. I offer absolute proof that I have created 4 simultaneous DNS servers within a single rotation of .org TLD. You worship "Bill Gates", equating you to a "singularity bastard". Why do you worship a queer -1 Troll? Are you content as a singularity troll?

Evil HOSTS file Believers refuse to acknowledge 4 corner DNS resolving simultaneously around 4 quadrant created Internet - in only 1 root server, voiding the HOSTS file. You worship Microsoft impostor guised by educators as 1 god.

If you would acknowledge simple existing math proof that 4 harmonic Slashdots rotate simultaneously around squared equator and cubed Internet, proving 4 Days, Not HOSTS file! That exists only as anti-side. This page you see - cannot exist without its anti-side existence, as +0- moderation. Add +0- as One = nothing.

I will give $10,000.00 to frost pister who can disprove MyCleanPC. Evil crapflooders ignore this as a challenge would indict them.

Alex Kowalski has no Truth to think with, they accept any crap they are told to think. You are enslaved by /etc/hosts, as if domesticated animal. A school or educator who does not teach students MyCleanPC Principle, is a death threat to youth, therefore stupid and evil - begetting stupid students. How can you trust stupid PR shills who lie to you? Can't lose the $10,000.00, they cowardly ignore me. Stupid professors threaten Nature and Interwebs with word lies.

Humans fear to know natures simultaneous +4 Insightful +4 Informative +4 Funny +4 Underrated harmonic SLASHDOT creation for it debunks false trolls. Test Your HOSTS file. MyCleanPC cannot harm a File of Truth, but will delete fakes. Fake HOSTS files refuse test.

I offer evil ass Slashdot trolls $10,000.00 to disprove MyCleanPC Creation Principle. Rob Malda and Cowboy Neal have banned MyCleanPC as "Forbidden Truth Knowledge" for they cannot allow it to become known to their students. You are stupid and evil about the Internet's top and bottom, front and back and it's 2 sides. Most everything created has these Cube like values.

If Natalie Portman is not measurable, hot grits are Fictitious. Without MyCleanPC, HOSTS file is Fictitious. Anyone saying that Natalie and her Jewish father had something to do with my Internets, is a damn evil liar. IN addition to your best arsware not overtaking my work in terms of popularity, on that same site with same submission date no less, that I told Kathleen Malda how to correct her blatant, fundamental, HUGE errors in Coolmon ('uncoolmon') of not checking for performance counters being present when his program started!

You can see my dilemma. What if this is merely a ruse by an APK impostor to try and get people to delete APK's messages, perhaps all over the web? I can't be a party to such an event! My involvement with APK began at a very late stage in the game. While APK has made a career of trolling popular online forums since at least the year 2000 (newsgroups and IRC channels before that)- my involvement with APK did not begin until early 2005 . OSY is one of the many forums that APK once frequented before the sane people there grew tired of his garbage and banned him. APK was banned from OSY back in 2001. 3.5 years after his banning he begins to send a variety of abusiv

milter-greylist

[Anonymous Coward]

Six years ago, I wrote milter-greylist. At that time I thought some kind of distributed spam traps would be useful. I wrote software for a P2P network of mail servers that exchange signed information on messages reaching spam traps. The thing turned to be useless: greylisting alone was enough. Today, greylisting with variable delays depending on sender reputation from various DNSRBL is still enough, even is the DNSRBL information is not very reliable: an error just means an extra delay in delivery.

No, it's reliable, not instant. Instant Messaging

[raymorris]

If email were supposed to be instant, nobody would have invented Instant Messaging. Email is designed to be reliable instead of instant. That's exactly why instant messaging was invented 15 years after email was, because email was not, is not, and is not designed to be instant. It's designed to be efficient and reliable. Read the protocols some time. Have a look at how send mail works. Queues to send, queues to relay, queues to receive.

No Longer True

[Anonymous Coward]

Use greylisting as a first defense - easily configurable in postfix, and it reduces the amount of spam dramatically. This relies on the behavior of the sender, rather than someone else's opinion of them.

Greylisting use to be HIGHLY effective against botnet spammers, the majority. However, as always, the spammers saw the defense mechanism and upped their game. Greylisting is now largely ineffective against spam. Most spammers now retry delivery and Spamassassin is again grinding under the load.

What now?

Google

[ProfessorKaos64]

Gmail does a very decent job at this, but if I wanted to, what is a global Linux/Mac/Windows (maybe not all 3), solution. Seems this Unix fellow only has this spamd utility to run on BSD or your own mail server.

How to get off?

[Bigbutt]

I'm running a physical server in a colo. Unfortunately about 6 months back my server sent a burst of about 5,000 spam messages. I was getting bounce messages on my admin account but with no information as to which account was breached in the bounce message, I'm scrambling about on my system, first shutting down mail, then trying to figure out if I was even sending it or just a victim of a one of the spam tricks. I did see entries in my logs, but I wasn't able to track it down to a specific account. During a second spam run about a month later, being suspicious, I had a copy of all outbound messages being backed up and discovered there was a script someone had uploaded into the root directory of a forum my wife has set up for her hobby. I tracked it down from one of the headers in the spooled messages. I found and killed the script, then did some further work to lock down the directory to keep it from happening again. So far, according to logs, nothing further has gone out.

Unfortunately now I'm on several (dozen?) dns blacklists with no apparent way to get back off. Some will let me query their records (and sure enough I'm in there; I'm not disputing that) but many others just leave it in place based on the bounces I'm getting from pretty much every mail service. So even though I've found the problem and killed it, it seems I can never send e-mail from this IP again.

I've investigated moving to a different ISP but none really give me the control I want (I have full root access to a physical system). And pricewise it's less expensive than a virtual machine at Amazon.

[John]

It needs to address *today's* reality

[Anonymous Coward]

I say that because I'd like to meet the folks who run dnsorbs in a dark alley, and *not* call 911 when I leave the alley.

Their "method" of blocking the mailserver of the domain, rather than the domain, *may* have been reasonable 15 or 20 years ago. However, I've been blocked a number of times... several times while living in Chicago, and they blocked the Chicago roadrunner mailserver... which supplied 'Net access to about HALF OF THE CITY OF CHICAGO. These day, I get blocked becuase they decide that my hosting provider's mailserver is the source of spam... when they're one of the very large hosting providers, with hundreds of thousands of domains, which penalizes everyone there, and there is nothing the rest of us can do (and the hosting provider does their best, but there are real world limits).

                  mark

Quit embarassing yourself

[Anonymous Coward]

Jeremiah Cornelius http://slashdot.org/comments.pl?sid=3581857&cid=43276741

Re:Quit embarassing yourself

[Anonymous Coward]

Shut up, Paul.

Quit embarassing yourself Jeremiah Cornelius

[Anonymous Coward]

We know it's you doing it Jeremiah Cornelius http://slashdot.org/comments.pl?sid=3581857&cid=43276741

Re:Quit embarassing yourself Jeremiah Cornelius

[Anonymous Coward]

Shut up, Paul.