FairUCE - the Smart Email Proxy

Jestrzcap writes "This just posted on Freshmeat: FairUCE (which stands for 'Fair use of Unsolicited Commercial Email') is an SMTP proxy, running between multiple instances of Postfix, that verifies email by attempting to verify the sender through lookups (a user customized challenge/response). It claims to be able to 'stop a vast majority of spam' without the need for content filters, and 'virtually eliminates spoofed addresses, phishing, and even many viruses with a few cached DNS look-ups and a couple of if/then statements'."

Comments

At last - a technological solution to spam!

[Anonymous Coward]

No way will the spammers ever find a way around this. It's solid!

Oh crap....

[Justice8096]

I've already had problems getting email from my government coworkers with spam validators like this. The military really doesn't like broadcasting who their email servers are... So they regularly get sent to Junk Mail.

forward and reverse

[gonaddespammed.com]

If MTA's on the Internet required the forward and reverse DNS lookups to match ~70% of spam (and viruses) would disappear. This requires ISP's to correcty configure their DNS, which unfortunately doesn't happen because people are lazy.

Re:forward and reverse

[Skapare]

The reverse DNS for email is NOT for determining a match between the sender email address domain, and the server itself. All that needs to match is the hostname of the mail server itself, thus identifying who administers it (not necessarily who gets to use it). If the ISP administers the mail server, then the hostname in the PTR record of the appropriate in-addr.arpa zone will be a unique name in an ISP domain. The forward lookup then prevents forged PTR records by making sure the domain owner acknowledges that name belongs to that IP address.

While most ISPs do have reverse DNS on their mail servers, when you focus on just the servers that spam houses run from, this changes over to most do not. But what would really happen if everyone blocked on lack of matching rDNS is that the spammers would adapt and use it. Then we'd know what domain they are using. But many of them are now registering bulk volumes of domain names (if you're making a million dollars a month abusing other people's networks, registering 100 randomly generated domains a month is just a tiny cost of business).

Will it be better than milter-sender?

[Matt Perry]

FairUCE looks interesting but I'd be curious if it'd do a better job than milter-sender. About a year ago, before I installed milter-sender, I was receiving about 200-300 spams per day. Since installing milter-sender in March 2004 and adding the spamhaus SBL-XBL checks to sendmail, I've received (checking spam mbox) 1568 spam messages.

Re:Italics!!

[Anonymous Coward]

No kidding, I hate people with slanted views.

Pyrrhic Victory?

[Jaysyn]

Doesn't this just create more traffic?

Jaysyn

Challenge Response Spam

[SnowZero]

One problem with challenge response is that Spammers not only send me spam, but send spam purportedly sent by me. I regularly get error messages about mail that could not be delivered. Now I'll get loads of challenge messages instead.

Of course if my MTA signed my messages with a random key, and the challenge message sent the key back, my MTA could filter out anything I didn't actually send. Unfortunately that requires coordination which the various email/spam task groups do not seem to be capable of.

Re:Challenge Response Spam

[fyngyrz]

One problem with challenge response is that Spammers not only send me spam, but send spam purportedly sent by me.

This is very common - and not just with a real users address. I have seen thousands of "bounce" messages come to the various domains I own as spammers use the domain prefixed by various random bogus names at whateverdomainitis.com.

Luckily (for us, anyway) we've now got the proper software written and configured to keep this crap from ever hitting a mailbox we own; however, a more serious problem here is the "do-gooder" problem.

It goes like this. Joe Spammer decides to use several_thousand_names@mydomainname.com as his assumed identity. A do-gooder site gets reports of that mydomainname.com is "sending" this spam to, oh, say a zillion people. They promptly "blacklist" my domain -- from whence, I hasten to point out, no spam has ever been, or will ever be, sent. However, my domain is a valid domain that I depend upon to make my living. Various ISP's, through a compounding of stupidity (but still with the intent to "do good"), promptly bounce our valid emails, because the do-gooders site says we are spammers.

The end result is that because some spammer out on the net has used our domain name, we, not the spammer, are penalized and in a real financial sense.

In the meantime, the spammer, who like any competent spammer watches the do-gooder's sites very carefully, notices that my domain is banned, and promptly switches to a new domain. Meanwhile, I can't send mail to my customers. Meanwhile, I get thousands of "bounce" messages that have to be handled by some layer of software or, Darwin forbid, by one of the legitimate users at my site. Random netizens out there have been temporarily "protected" from (typically) one spam email per email address they have, while our customers are cut off at the knees, as are we.

So what the do-gooder has accomplished is to cause the spammer to take another domain (probably from an automated list, no sweat off the spammer's brow whatsoever) and the do-gooder has hurt a legitimate net citizen who never spams.

Everybody's trying to do good here except the spammer. The do-gooder and the ISPs using the do-gooder list hurt our end users by blocking mail they should be getting; they hurt us by screwing up our commications channel to our customer base; but -- they don't hurt the spammer one flipping bit, and they do no permanent good for the average netizen who gets one of these spams. The spammer just restarts his list at the break point and begins with a new domain; the end user, after a short delay, gets a new spam with a new domain name, and the temporary respite for them is over -- and the net result of the do-gooder's blacklist is no good whatesoever has been done. Some users will get two spams if the spammer restarts the list back a little to make sure he doesn't miss anyone. Great, eh?

Obviously, do gooder blacklisting doesn't work, and cannot work. Mostly, it causes harm to legitimate parties.

IMHO, if Internet mail is going to be unregulated, then it needs to be just that -- unregulated. If spammers are going to be fined and/or jailed, then the govt(s) need/s to get the heck after it (and probably needs to close the international email borders to any non-co-operative country so that such a thing is possible.) The latter seems far too severe; the former is being degraded by do-gooders and the people they confuse into accepting their services in an area they should have no absolutely authority in to a degree that should be unacceptable to any thinking person.

The only good solution to spam I know of is to use whitelists and web submission entry gateways. If someone is on your whitelist, you get email from them. If someone is not on your whitelist, they get an auto-reply email telling them to mail you via a form on a website. The form, which has to be hand-filled out, mails you at a whitelisted address that is not publ

Re:Challenge Response Spam

[farnz]

I'd be interested to know which blacklists are by domain, not by sending IP address; I find that SpamAssassin's use of SPEWS and Spamhaus blacklists is enough to catch virtually all the spam I get, and both of those blacklists are done via sender IP, not by domain name.

So, I'd disagree with your conclusion that blacklisting doesn't work; if a spammer can use one of your IP addresses to spam, then you need to fix up your system to be more secure. A quick browse of mail logs will show any unexpected outgoing e-mail, and you can always feed your mailserver IP to spews.org and see if they list you (they're one of the most aggressive listing places).

If it's not coming from one of your IP addresses, then it doesn't affect mail sent from your domain, only from the spammer's IP addresses. Hence there is no fallout on you unless I use an aggressive list like SPEWS, and you are being blocked because your ISP hosts spammers himself.

Here we go again

[nsayer]

Your post advocates a

(X) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
(X) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
(X) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
(X) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
(X) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(X) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

Re:Here we go again

[physicsphairy]

Modularize this, extend its applicability, and we can replace 90% of slashdotters with a small shell script!

yet another waste of time

[mabu]

Have we not established a few basic tenets of the spamademic?

1. Spammers make money by using a disproportionate amount of bandwidth than what they pay for. Stopping spam from entering peoples' inboxes is less than half the problem. 70% or more of all SMTP traffic is UCE and everyone pays for that in higher costs and slower performance regardless of whether they have spam filters in place.

2. The majority of the anti-spam solutions (with the exception of RBLs) including the one related to this article, require extra time, bandwidth and resources on the part of innocent networks to deal with the spam problem. This is a step backwards.

If you want to stop spammers you have to stop them from stealing bandwidth. To date, the ONLY effective solution thus far has been relay blacklisting. This has several added benefits including: stopping propagating of worms/viruses, and forcing ISPs to police the illegal activities of their users and shut down nodes which are spamming through their network.

As an ISP, I have no interest in yet another costly anti-spam solution that I have to install that doesn't address the larger issue of the tons of bandwidth spammers waste on my network and every one in between. This system wastes even more resources by attempting to verify the source of every e-mail in an even more detailed manner than before, so the end result is: more computing resources needed, more bandwidth needed and slower mail service.

No thanks.

I'll patiently wait until the *inevitable* SMTP whitelist scheme that is the only true solution to stopping spam (unless the authorities decide to actually start prosecuting spammers for their crimes).

Yet another challenge/response system: *yawn*

[Antique Geekmeister]

"If we could just rewrite everybody mailer's with my new widget in illegible Perl or badly written C that breaks several RFC's I've never bothered to read, we will surely stop spam!" I've heard this sort of thing before, every few months for the past 10 years.

There have been dozens of these wildly espoused challenge/response systems over the years. They don't work because users hate them, because vital automated systems such as bill payment and delivery verifications can't get past them. Coupled with "sender pays" systems, they're almost always subverted within short periods and never can or will gain the acceptance of the user community enough to become effective.

Re:Yet another challenge/response system: *yawn*

[mjh]

There have been dozens of these wildly espoused challenge/response systems over the years. They don't work because users hate them, because vital automated systems such as bill payment and delivery verifications can't get past them.

I've been using Challenge/Response for nearly 3 years. And I disagree with your critiques. Let's take this point by point:

• Users hate them: There is a kernel of truth to this. Some users do hate them. Those users hate challenge/response so much that they instigate fights. They submit their IP addresses to RBLs for blacklisting. These are a very annoying, and vocal MINORITY. By far most users are agnostic. They deal with the challenge once and then they're done.
• automated systems can't get past them: Again, there's a kernel of truth here. If you have badly configured your C/R you're going to be in trouble. But a properly configured C/R has absolutely no problems.

  I use TMDA. I've got it configured so that any email I send to unknown addresses will be allowed to respond for 7 days. After that, they go into C/R. For my bill pay services, I give them a special address that allows them in forever, but that's tied to them so that I'll know if they ever hand it out to someone else.

• they're almost always subverted: Really? In the last month I've had over 4000 pieces of email delivered to me from unknown addresses. Only 10 of those have been confirmed. Of the ones that were confirmed 2 of them were spam. This was easily remidied by removing those 2 addresses from my whitelist and adding them to my blacklist.
• never will gain the acceptance of the user community enough to become effective: While C/R may never gain the acceptance of the user community, I don't think it's for the reasons that you cited. I think the reason is that it's too hard to set up correctly. But that being said, it doesn't need the acceptance of the user community to be effective. It works for me today whether or not you use it.

  Personally, I think it'd be better if the entire world started using C/R. It'd be better because then everyone would understand that sending email to an unknown party involves a formal introduction process. This would cut down on the number of people who get confused when they receive a challenge. But if this doesn't happen it's not that big a deal. The number of confused people is already small.

IMHO, what you don't know about C/R is quite large.

Re:Yet another challenge/response system: *yawn*

[DavidTC]

In other words, you sent out 3992 pieces of spam to forged or invalid addresses, pissing off 2 people who knew what was going on bad enough that they confirmed your C/R.

Re:Yet another challenge/response system: *yawn*

[DavidTC]

I don't know in what universe it's a useful point to mention that you're removing invalid email address before you send mail to them. That mail wouldn't go through anyway! It's the valid addresses that are a problem.

But, hey, you gave me the last number. So...5%. That's about 200 pieces of mail you sent. And you got 8 valid responses, and 2 invalid.

So you sent out, basically, 192 spam messages, barring the occasional legit C/R you sent out that was ignored. (Which is also a failure of the system, it's just a failure that isn't spamming.)

To get 8.

To get 8 fucking messages, you sent 192. For every legitmate message you receive, 24 other people had to look at a spam you sent them.

Well, you're the moral paradigm I've come to expect from C/R people.

Fucker.

Re:Yet another challenge/response system: *yawn*

[Malc]

If I buy airline tickets online and they don't tell me the source email address, how am I supposed to get the itineray (sp?), etc that get's sent out automatically. On a couple of occasions the domain has differed from that of the website I purchased from. On another occasion I sponsored a friend to walk 60km to raise money for charity - the PDF receipt I need for tax purpose was sent from a different domain... it goes on. In that latter case I would have had to whitelist the email address I provided. It's all extra work which is inconvenient to a technical user like me, and far beyond what I could expect my parents to use. I *hate* C/R systems - if somebody (even a friend) uses them I won't bother unlocking with a response, and I won't use email to contact them again. It's their loss, not mine.

Re:Naive at best

[Antique Geekmeister]

I'm sorry, you're wrong on a detail.

There is no reason to have port 25 open outbound on anything but the ISP's authorized SMTP servers. None whatsoever iin this day and age, except the convenience of people who like to run their own mail servers. Unfortunately, with the massive number of zombied and badly run home SMTP servers, most outbound SMTP from ISP users that does not go directly to their ISP's SMTP server for delivery as mail from that ISP is in fact spam or email worms.

So yes, it needs to be blocked outbound. You simply need to use SMTPAUTH on the road to get your email to your own ISP's SMTP server over port 587. Problem solved.