Spam Catchers Block Latest Crypto-Gram

An anonymous reader writes "Bruce Schneier sent out a note about SpamAssassin and possibly other spam filters blocking his excellent Crypto-Gram newsletter. Fortunately you can get it here (early no less!)." Schneier's email reads, in part "Tomorrow I will be sending out the February CRYPTO-GRAM, as I do on the 15th of every month. In the process of creating this month's Crypto-Gram, I discovered that SpamAssassin thinks that this issue is spam, probably because of certain links and descriptions of scams in the text. I have anecdotal evidence that other spam filters block Crypto-Gram as well. ... I'd apologize for the inconvenience, but I'm not sure what I could do to make it less so -- I don't intend to alter my content to accommodate spam filters."

Comments

um, i could be terribly wrong here

[Anonymous Coward]

but why not distro the newsletter encrypted? then the spam filters wouldnt have anything to trigger the filters, and id say the target audience have the knowledge to unencrypt it when it gets there..

Whitelist

[sean23007]

That's why most good spam blockers (especially OS X's Mail.app) use their filters but compare the senders to a whitelist so that your friends can send you whatever they want to. If you've been receiving CRYPTO-GRAM for a while, it should be on your whitelist, and the blocker should just let it by.

But you don't always want to get everything people send you (everybody has those people who send you things they think are funny but you just can't stand). So there should be levels of "friendship" in the whitelist, so that some senders can be considered dubious (their mail shouldn't be deleted like spam, but perhaps placed in a different "Uninteresting" folder).

Re:Whitelist

[whereiswaldo]

So there should be levels of "friendship" in the whitelist, so that some senders can be considered dubious (their mail shouldn't be deleted like spam, but perhaps placed in a different "Uninteresting" folder).

I, for one, would love to see a feature like this in a mail program!
Actually, I'd like to participate in the development of an existing open source email app if someone could recommend one. Java based would be nice.

This is a non-issue....

[MrByte420]

False-Positives should be a non-issue. Either you choose to run a spam filtering software and live with thoose limitations or don't run a spam filtering program and deal with the extra emails about enlarging various organs that you will receieve every day.
I do tech support for a webhosting company and people call us every day complaining about their spam but as soon as we offer blocking software based on lists, etc all we get is complaints that some more-valuable-than-gold email is going to get lost and ruin their entire business.

This is a simple choice and people have to learn they can't have their cake and eat it too.

Maximum size of spam

[waynemcdougall]

Spam tends to be short. The shorter the spam, the more messages they can put through. So spammers would be loathe to add 21 pages of text to their spam.

I have
Const maxspamsize = 42695
in my spam filter - I've only receive one piece of spam larger than than in the last 12 months (a giant promotion for a Korean trade show). It speeds up my spam filter processing and lets large newsletters (with false triggers like this) through without a problem.

Re:Maximum size of spam

[waynemcdougall]

My point remains valid. Because there is a direct cost to the spammer to adapt.

If they bulk up their spam that's going to slow them down, increase their costs (even if bandwidth costs aren't going to be passed back to them now, the more they use, the more visible they become). They become more visible.

Or they continue on their way. The reality is that they concentrate on the easy targets - you and I will never purchase their services so people taking this approach aren't really in their target audience anyway. I know this is (surprisingly) less true than one might think. Spammers do work to overcome basic obstacles, but that adds more costs and time - they don't work hard to avoid tar pits, because there are so few of them.

So I still see it as a win...large emails are very unlikely to be spam. If that changes, well so be it, but that will hurt the spammers. In the meantime I reap the benefit of fewer false positives and faster spam filtering.

Final comment - over the last six months I've seen spam get slightly larger (from about 32k peak size to about 45k peak size). But I haven't been analysing for any trends - just the outliers.

Re:The problem with blocking IP adresses is...

[Mr Bill]

So blocking untrusted servers doesn't make email unreliable? I find that very hard to believe. Considering that most of the time it is Net blocks that are blocked, not just individual IP addresses.

blocking IP addresses is also open to abuse... If I had a grudge against an ISP, I could fake some SPAM headers and send it to any of the IP blockers. Maybe send several copies from different accounts. Getting an IP listed is usually easier than getting it removed, so in the mean time many legitimate emails are being blocked...

I believe you have to attack the root of the problem, and that is stopping the SPAM at the origin. This is probably the more difficult approach, but it is the only one that will avoid dropping legitimate mail.

A possible solution to the spam problem...

[kcbrown]

Right now everyone is forced to accept email connections from anyone who sends email because it's not possible to tell ahead of time whether or not the connection is coming from someone who is reliable, right? And spammers take advantage of this by sending millions of messages from open relays. Blocking that is a virtual impossibility because which relays are open changes over time.

The first inclination one has would be to suggest that everyone close their open relays. But this depends on people doing the right thing all the time, and has proven ineffective.

Fortunately, there's another way.

Right now, everyone who receives mail has to listen to everyone who tries to connect. The problem is how do you separate the wheat from the chaff?

The solution is to take advantage of the information SMTP and TCP/IP give you when a connection is established. The fact that you're receiving a connection gives you the address of the sender. And during an SMTP transaction, one of the SMTP commands (the MAIL FROM command) gives you the domain of the email's sender, e.g. "MAIL FROM slashdot@sysexperts.com".

When you're sending email to someone else, you do so by looking up the MX records for their domain, which tells you which systems are responsible for receiving email for that domain. This gives us a possible answer to the spam problem.

Suppose instead of blindly accepting email from everyone, you were to take the domain given to you by the MAIL FROM command, look up the MXes for that domain, and reject the email connection if the IP address of the sender doesn't match one of the domain's MXes?

Now, suddenly, you would end up rejecting email sent from every unauthorized relay, because the owner of the domain can make any system that is allowed to send email on behalf of his domain into an MX (and, if he doesn't want that system to be used for delivering email, then he simply makes such systems the lowest priority MXes in the list and blocks outside port 25 connections to them ... something he's probably doing anyway).

Suddenly, the only systems that spammers can send email from are systems that they legitimately control and that are defined as MXes for a domain they control. Suddenly, spammers have to set up and maintain their own domains and their own boxes. The costs have just become a lot higher, which will get rid of most of the spammers.

And suddenly, blocking spam becomes orders of magnitude easier -- you only have to deal with spammers who have decided to pay the (now much higher) price for sending spam and who cannot use someone else's system to do their dirty work without permission.

Re:The problem with filters

[Lazy Jones]

This is exactly why content-based filters will never work: the professional spammer will take the time and run his e-mail through filters until he gets a good result (a negative answer). The non-spammer will not take the time to test his e-mail with all the spam-filters. Therefore, it is very likely that legitimate content will be filtered and professionally composed spam e-mails will not. So IMHO, Spam-Assassin and all the other content-based spam-filters are completely useless.

So let's send spam as Bruce Schneier

[marcink1234]

As a lot of people will probably whitelist cryptogram, if one wishes to spam technical people, he just needs to set From to Bruce.

Just shows that...

[forgoil]

This simply shows that newsletters and similar are not really sent by the right medium right now. EMail hasn't kept up with the times and as a result we see this endless amount of spam.

What is needed is a foolproof way of saying "I want this, please send it to me" and then being able to reject it safly without needing the other party to do it for you. For example:

I send a message to cryto-gram, including a key. This key can then be used to send it to me, and I accept it (key in combination with who send it and so on, I am sure someone with even more experience can figure out a fool proof way). Good stuff. But then I realise that I don't want this anymore, and I simply remove the acceptance of this key in my own software (and send a message that I don't want it anymore, no harm being nice to the nice), and it will be filtered away.

Or something along those lines, I can asure you that I haven't fixed up a foolproof and perfect system yet ;)

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:The problem with content filtering

[sysadmn]

Which is why most bayesian-type filters react to a subset of the keywords. By estimating based on the 10 or so best indicators the dilution is less effective...

False alarm?

[babbage]

I've just checked the headers for this month's Cryptogram, and the current version of SpamAssassin (2.44) did not flag it as spam. To wit (slightly reformatted because of Slashdot's "this Nerd site will not accept technical postings thankyouverymuch" comment filter):

X-Spam-Status: No, hits=2.0 required=5.0

tests=BALANCE_FOR_LONG_20K, BALANCE_FOR_LONG_40K, NORMAL_HTTP_TO_IP, OPT_IN, SPAM_PHRASE_01_02, SUBJECT_MONTH, SUBJECT_MONTH_2, US_DOLLARS_2, US_DOLLARS_4

version=2.44

X-Spam-Level: **

Note that SpamAssassin isn't on my whitelist or anything like that -- it just worked.

False alarm?

Re:SPEWS

[Skapare]

Or he might be

• A customer of UUNet which spews has listed because it disagrees with some of the content they host

UUNet has become one of the worst ISPs around due to their harboring of large numbers of spammers. And they do absolutely nothing to respond to complaints reported to them. They just let the spammers keep spamming.

NOBODY with a brain is using SPEWS anymore. Listing the largest commercial internet supplier in the US was simply idiotic. And it was done for completely illegitimate reasons.

There are completely legitimate reasons for blocking UUNet. It's the spam. You may be confusing SPEWS with some small-time renegade blocklist.

The whole blacklist concept boils down to vigilante tactics, use threats to keep people in line. The problem being that the people who run the lists tend to turn into self-important little tinpot dictators after a short time.

As soon as I see SPEWS operators "turn into self-important little tinpot dictators" I'll certainly stop using it. But I have not seen it happen. Feel free to point out any specifics if you are aware of them.

If anything, it is the very act of harboring spammers that is a vigilante tactic. Given that the costs of transmitting email are heavily slanted to the recipient end when spam is involved (because the spammers use special software to send email that scale up more effectively than ordinary MTA software), such a tactic could be in active use by some ISPs to drive up the costs for others (their competition).

Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech

Unture, with the exception of Limabaugh whose judgment in Nixon is opinionated nonsense the Federal courts have all rulled that the junk fax laws are constitutional.

Read my statement again, this time carefully. I said it is a violation of the principles. I did not say it is a violation of the Constitution and/or First Amendment itself (see the way the clause is written). Since the Constitution places restrictions on the government, it is the government that is the one that has to be sure not to restrict speech based on its content. You and I are free to do so within the context of our property rights and those of others. While it would be wrong for me to go delete your messages (that would be violating your property rights), you could certainly delete them yourself if you choose to. But I do fully believe in the principles the US Constitution was based on, and I practice my life that way. Thus, I do not use content based filtering. That's my choice.

Re:Seems like it worked fine....

[Eythian]

What makes it even funnier, the initial notice got flagged as spam by my spamblocker, but the actual crypto-gram didn't.