Slashdot Mirror


Spam Catchers Block Latest Crypto-Gram

An anonymous reader writes "Bruce Schneier sent out a note about SpamAssassin and possibly other spam filters blocking his excellent Crypto-Gram newsletter. Fortunately you can get it here (early no less!)." Schneier's email reads, in part "Tomorrow I will be sending out the February CRYPTO-GRAM, as I do on the 15th of every month. In the process of creating this month's Crypto-Gram, I discovered that SpamAssassin thinks that this issue is spam, probably because of certain links and descriptions of scams in the text. I have anecdotal evidence that other spam filters block Crypto-Gram as well. ... I'd apologize for the inconvenience, but I'm not sure what I could do to make it less so -- I don't intend to alter my content to accommodate spam filters."

238 comments

  1. Hopefully SpamAssassin didn't by Chris_Stankowitz · · Score: 5, Funny

    block that important e-mail I was waiting for on enlarging my....never mind, I have to check my e-mail now.

  2. Mozilla would learn from it by Anonymous Coward · · Score: 0

    And use the bayesian method to not block messages that fit this type as spam.

  3. um, i could be terribly wrong here by Anonymous Coward · · Score: 4, Interesting

    but why not distro the newsletter encrypted? then the spam filters wouldnt have anything to trigger the filters, and id say the target audience have the knowledge to unencrypt it when it gets there..

    1. Re:um, i could be terribly wrong here by Anonymous Coward · · Score: 0

      Yes, the word is "decrypt."

      knowledge to unencrypt... sigh...

    2. Re:um, i could be terribly wrong here by Feztaa · · Score: 3, Informative

      There would be a tremendously large problem with encrypting the message to all of it's recipients...

      See, when you PGP encrypt some text, it is only possible to encrypt it to one person (one public key). That's just how it works, it's inherent in the encryption methods used; however, PGP and GPG get around this by duplicating the entire message for each public key that it is encrypted to.

      My point is that if you had a mailing list with 1000 subscribers, and you wanted to encrypt it, you'd basically be increasing the size of the encrypted message 1000-fold, because you need 1000 copies of the message, each encrypted to a given recipient. Obviously, this isn't feasable...

      What they could do, though, is sign the messages. I know SpamAssassin, at least, reduces a message's spam score if there is a PGP signature attached to it.

      However, if you were just trying to obscure the contents of the mail from the spam filter but not the user, you could just gzip the message and make it an attachment. I don't know how well that would go over with the spam filter, but at least it wouldn't find your m/blow.*job/s in the message ;)

    3. Re:um, i could be terribly wrong here by TheRaven64 · · Score: 1

      then the spam filters wouldnt have anything to trigger the filters
      Really? A lot of HTML spam used to encrypt itself and then have a little piece of javascript that decrypted it embeded in the email. This was quite easy to block, since all a spam-blocker had to do was spot emails not containing any words of your favourite language. I'm not sure if SA does this, but I wouldn't be surprised if it blocks all encrypted messages.

      --
      I am TheRaven on Soylent News
    4. Re:um, i could be terribly wrong here by Ian+Jefferies · · Score: 1

      The message would be encrypted with Bruce Schneier's private key, and anyone with access to his public key would then be able to decrypt it and read it. There are three basic modes of operation here, relying on the fact that the private key is very difficult to obtain and the public key is well known:

      1) Identifying the sender of the message and controlling the recipient
      Both the public key of the recipient and the private key of the sender are used to encrypt the message. This message can only be decrypted by someone possessing both the recipients private key and the public key of the sender. This limits availability of the content of the message to the recipient, and at the same time ensures that the recipient can confirm who sent the message.

      2) Control recipient of message only
      Encrypt the message with the public key of the recipient. Only the recipient has the private key that can access the message, but there is no means of identifying who sent the message in the first place (this is what you suggested).

      3) Identify the sender of the message
      Encrypt the message with the private key of the sender. Anyone with access to the public key can obtain the message, and confirm who sent it. The sender doesn't know who reads the message, and doesn't care, it just establishes the identity of author of the message (this is how the message should be encrypted for the mailing list).

      Hopefully I didn't miss anything important, I'm not a crypto expert.

      Ian.

      --
      A physicist is an atom's way of thinking about atoms
    5. Re:um, i could be terribly wrong here by SN74S181 · · Score: 1

      All valid suggestions, and I am sure there are a half dozen Spammers working on implementing it now.

      How many spammers are now going to try mimicing Schneier's newsletter? Seems like a cool idea for me if you're a spammer and want to confuse the filters. Imitate an uber-kewl hacker's newsletter, something leetos will DEMAND the spam filters let through. Can the spam imitating the comp.risk newsletter be far off?

    6. Re:um, i could be terribly wrong here by mpe · · Score: 1

      See, when you PGP encrypt some text, it is only possible to encrypt it to one person (one public key). That's just how it works, it's inherent in the encryption methods used; however, PGP and GPG get around this by duplicating the entire message for each public key that it is encrypted to.
      My point is that if you had a mailing list with 1000 subscribers, and you wanted to encrypt it, you'd basically be increasing the size of the encrypted message 1000-fold, because you need 1000 copies of the message, each encrypted to a given recipient. Obviously, this isn't feasable...


      Actually it's trivial. You encrypt with a private key then anyone who has the public key can decrypt it.
      This is how PGP/GPG signing of a message works. You have a checksum encrypted with a private key, when you receive the message the software attempts to decrypt the checksum then compare it with what it has calculated the checksum to be. If the decryption fails the message isn't from the claimed source, if the checksum fails it has been altered.

    7. Re:um, i could be terribly wrong here by justins · · Score: 1

      Actually, that would not work to circumvent at least one of the popular spam filters, Cloudmark SpamNet.

      SpamNet users elect to "block" (or unblock) certain messages, a checksum of the message is sent to their central server, and when a certain message has been blocked a few times the software itself will automatically move those messages to a "spam" folder on end users' machines. Users could just as easily block encrypted mails.

      I've had a few legitimate mailing lists blocked by dumb SpamNet users in the past. The Motley Fool list comes to mind. Presumably people let the time run out on the free service but are too lazy to unsubscribe from the list, and so just tried to use the "block" function.

      --
      Now before I get modded down, I be to remind whoever might read this that what I am saying is FACT. - bogaboga
    8. Re:um, i could be terribly wrong here by Phil+Gregory · · Score: 2, Informative
      See, when you PGP encrypt some text, it is only possible to encrypt it to one person (one public key). That's just how it works, it's inherent in the encryption methods used; however, PGP and GPG get around this by duplicating the entire message for each public key that it is encrypted to.

      Incorrect. When PGP or GnuPG encrypts a message with a public key, they really just encrypt the message with a symmetric cypher and a sufficiently long, random key. Then they encrypt the key with the public key. (The reason for this is that public key cryptography is much, much slower than symmetric key stuff.) So for sending to multiple recipients, all that needs to be added is some additional header data for each recipient.

      -rw-r--r-- 1 phil phil 212358 2003-02-16 13:01 original
      -rw-r--r-- 1 phil phil 90343 2003-02-16 13:02 one-recipient.gpg
      -rw-r--r-- 1 phil phil 90893 2003-02-16 13:04 three-recipients.gpg

      A better solution would still be to encrypt the message with a particular public key for which the private key was widely available. Encrypting the message with Bruce Schneier's private key makes sense cryptographically, but I don't believe PGP and GnuPG support that sort of behavior.


      --Phil (Far too much of a crypto geek)
      --
      355/113 -- Not the famous irrational number PI, but an incredible simulation!
    9. Re:um, i could be terribly wrong here by rthille · · Score: 1

      If I remember correctly, PGP encrypts the entire message with 'normal' (not public-key) encryption, then encrypts the key with public-key encryption. So, you could send the message to 1000 people, and not have the message included 1000 times. You would however have a 1000 copies of the key (1024 bits?), all encrypted with a different private key.

      --
      Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
    10. Re:um, i could be terribly wrong here by greenrd · · Score: 1
      A better solution would still be to encrypt the message with a particular public key for which the private key was widely available. Encrypting the message with Bruce Schneier's private key makes sense cryptographically

      So, you're saying Bruce Schneier's private key is widely available? ;-)

  4. Seems like it worked fine.... by telstar · · Score: 4, Funny

    So he sends out the Crypto-Gram newsletter, then he sends out a note about the Crypto-Gram newsletter. 2 emails to cover what should've been sent as 1. Seems like the spam filter is doing just fine ...

    1. Re:Seems like it worked fine.... by Eythian · · Score: 2, Interesting

      What makes it even funnier, the initial notice got flagged as spam by my spamblocker, but the actual crypto-gram didn't.

  5. Re:Finally by GMC-jimmy · · Score: 1, Offtopic
    --
    __________________________________
    Free your mind - Flush your toilet
  6. White List by SealBeater · · Score: 4, Insightful

    That's easy to fix, add the crytogram address to a whitelist. Every spam
    filtering software I've ever run, including spamassasin (which I like a great
    deal) has a whitelist option. If you're running some kind of filtering
    software, it behooves you to keep an eye on what it's blocking, hence, I am
    sure that people are aware of it and have adjusted their software accordingly.

    SealBeater

    --
    -- Its survival of the fittest...and we got the fucking guns!!!
    1. Re:White List by tbmaddux · · Score: 1
      If you're running some kind of filtering software, it behooves you to keep an eye on what it's blocking...
      I agree; it is disturbing to get hit by a false positive, never know about it, and then later find out that you missed an email from someone that you would rather have read. So you'd like to review what gets rejected. Unfortunately, at some point that puts you right back to reading spam again!
      --
      Can't you see that everyone is buying station wagons?
    2. Re:White List by Chelloveck · · Score: 1
      I agree; it is disturbing to get hit by a false positive, never know about it, and then later find out that you missed an email from someone that you would rather have read. So you'd like to review what gets rejected. Unfortunately, at some point that puts you right back to reading spam again!

      That's why I like SpamAssassin so much. It flags spam by altering the subject line (to prepend "***SPAM***") and adding a couple header lines to let you know why it thinks it's spam.

      Then I have Pine set up to automatically mark all spam as "deleted". This doesn't remove it from my INBOX immediately, so I have a chance to skim the spam for anything useful. But, without any further action from me, the spam all disappears when I exit Pine (or manually expunge the INBOX). If there is a false positive I can catch it and undelete it. (And add the sender to my whitelist!)

      Most of the newsletter-like lists that I'm on get flagged as spam before I whitelist them. My mom actually sent me a message the other day that got marked as spam (one of those "internet postcard" things), so she's in the whitelist now.

      For me, this technique balances my desire to avoid manually evaluating and deleting each piece of spam with my desire to avoid missing anything hit by a false positive. And I applaud Bruce for not compromising his content to satisfy the filters!

      --
      Chelloveck
      I give up on debugging. From now on, SIGSEGV is a feature.
  7. Whitelist by sean23007 · · Score: 5, Interesting

    That's why most good spam blockers (especially OS X's Mail.app) use their filters but compare the senders to a whitelist so that your friends can send you whatever they want to. If you've been receiving CRYPTO-GRAM for a while, it should be on your whitelist, and the blocker should just let it by.

    But you don't always want to get everything people send you (everybody has those people who send you things they think are funny but you just can't stand). So there should be levels of "friendship" in the whitelist, so that some senders can be considered dubious (their mail shouldn't be deleted like spam, but perhaps placed in a different "Uninteresting" folder).

    --

    Lack of eloquence does not denote lack of intelligence, though they often coincide.
    1. Re:Whitelist by Plix · · Score: 1

      A mailing list usually has a dedicated address which can be whitelisted without having to worry about such extraneous nonsense sneaking through.

    2. Re:Whitelist by whereiswaldo · · Score: 2, Interesting

      So there should be levels of "friendship" in the whitelist, so that some senders can be considered dubious (their mail shouldn't be deleted like spam, but perhaps placed in a different "Uninteresting" folder).

      I, for one, would love to see a feature like this in a mail program!
      Actually, I'd like to participate in the development of an existing open source email app if someone could recommend one. Java based would be nice.

    3. Re:Whitelist by SimplyCosmic · · Score: 2, Informative


      Well, in terms of Spamassassin, you could create rules which subtracts a particular number of points from the spam score of any particular message, rather than letting it through automatically, which gives it a better chance to go through if it's a pretty un-spam-like content.

    4. Re:Whitelist by Anonymous Coward · · Score: 0

      > Java based would be nice.

      Let me get this straight: you want to write a program that's got to process between 10-100+ mails a minute (which is the incoming mail stream at the mail server at our department, which handles mail for about 40 people) ... in Java? In dogslow Java? You want to do classification, perhaps cross referencing with sources on the net, ... in Java. In memory-pig Java?

      Why?

      Crazy, crazy I say.

    5. Re:Whitelist by whereiswaldo · · Score: 1

      Actually, I was thinking more of a mail _client_, not a server, but I'll bite anyway. :)

      Java could easily handle 100+ emails a minute on a decent PC. Java is not dogslow. If you think it is, you haven't tried it in a looong time. The JIT compiler makes it run extremely fast, actually. What's slow is the startup time which is unimportant unless you're writing Java CGI's which nobody would do anyway for a high load app.
      Classification - that's just filters. Cross-referencing sources on the 'net - Java's networking support is great.
      Now memory - you got me there. Java does use a lot of memory compared to a native app, but these days with RAM prices so low I'm not too concerned about memory.

    6. Re:Whitelist by zcat_NZ · · Score: 1

      Well, in terms of Spamassassin, you could create rules which subtracts a particular number of points from the spam score of any particular message, rather than letting it through automatically, which gives it a better chance to go through if it's a pretty un-spam-like content.

      You mean like this?

      USER_IN_WHITELIST (-100.0 points) From: address is in the user's white-list

      --
      455fe10422ca29c4933f95052b792ab2
    7. Re:Whitelist by vondo · · Score: 1
      So there should be levels of "friendship" in the whitelist, so that some senders can be considered dubious (their mail shouldn't be deleted like spam, but perhaps placed in a different "Uninteresting" folder).

      I, for one, would love to see a feature like this in a mail program! Actually, I'd like to participate in the development of an existing open source email app if someone could recommend one. Java based would be nice.

      SpamAssassin already does something like this. First, it comes with a set of whitelisted addresses, like ebay.com, etc. It's quite possible that Cryptogram will be included in that list next time around.

      But better than that, SA has something called autowhitelisting that keeps track of the average spam score for people who sucessfully get mail delivered to you (through the filter). This means that a good friend, who's mails are normally fine, can send you a spam-ish mail that gets through, or if your friends are borderline, like above, they may only get a few points for being a friend and a spam-ish mail will trigger the filter.

  8. Habeas SWE? by balamw · · Score: 1

    You could always sign up for Habeas SWE and put their little haiku "warrant" in your headers. This will stop most spam tools from filtering you out, unless of course you violate the terms and send UCE including the warrant.

    Balam
  9. SpamAssassinAssassin by Anonymous Coward · · Score: 5, Funny

    SpamAssassinAssassin could look at the folder where you put your filtered mail and learn what to pull back out, and flush the rest to /dev/null.

    I'm sure Paul Graham will be glad to write it in lisp.

    Or, of course, we could just do what the obvious solution is: get in a P.O. Box, send out spam for herbal viagra and penis enlargement, and when you get the checks in the mail HUNT THE CUSTOMERS DOWN AND KILL THEM.

    It's simple, really.

  10. Filters are so 2002 by Anonymous Coward · · Score: 0

    Get with the times! Regular filters with whitelisting are old news. New filters are smarter than that.

  11. This is a non-issue.... by MrByte420 · · Score: 4, Interesting

    False-Positives should be a non-issue. Either you choose to run a spam filtering software and live with thoose limitations or don't run a spam filtering program and deal with the extra emails about enlarging various organs that you will receieve every day.
    I do tech support for a webhosting company and people call us every day complaining about their spam but as soon as we offer blocking software based on lists, etc all we get is complaints that some more-valuable-than-gold email is going to get lost and ruin their entire business.

    This is a simple choice and people have to learn they can't have their cake and eat it too.

    --
    If religous zealots don't believe in Evolution, then why are they so worried about bird flu?
    1. Re:This is a non-issue.... by Anonymous Coward · · Score: 0

      This is a simple choice and people have to learn they can't have their cake and eat it too.

      So what to hell do you do with your cake? Is it the same thing you do with your apple pie?

    2. Re:This is a non-issue.... by Elwood+P+Dowd · · Score: 4, Insightful

      Thank you. Also, if all the bayesian filtering advocates are right, then the users should be able to mark the Cryptogram as non-spam, and the filter should adapt. More to your point, though, is that lack of spam-filtering software can cause false-positives in your own personal, analog, spam filtering algorithm. Many of my users have deleted important, non-spam, automated emails manually because they thought it was spam. Sometimes, the machine might have less false positives than they would.

      Huh. It occurs to me that it seems like some spam filters might pass a turing test if the only output is their spam judgment. Wow. The future is now, dude.

      --

      There are no trails. There are no trees out here.
    3. Re:This is a non-issue.... by 1u3hr · · Score: 3, Insightful
      Either you choose to run a spam filtering software and live with thoose limitations or don't ...

      Except if it's done upstream from you, perhaps even without your knowledge (eg a few months ago it was found that Mac.com was aggressively filtering, with a lot of false positives).

    4. Re:This is a non-issue.... by NilObject · · Score: 1

      Which my be a good thing. I Used to recieve about 20 spams a day before iTools became .Mac, and now I hardly recieve anything at all! Errr, I mean, hardly any spam.

    5. Re:This is a non-issue.... by Anonymous Coward · · Score: 0

      False positives are never a "good thing".

  12. The problem with filters by markfletcher · · Score: 5, Insightful
    This illustrates one of the big problems with filters. They will never be perfect, spammers are always adjusting to them (even the Bayesian ones), and the way many are implemented, they make email unreliable (by deleting suspected spam messages and not bouncing them). Blocking untrusted servers by IP address avoids these issues.

    obPlug: This is why I created Trustic.

    1. Re:The problem with filters by Lazy+Jones · · Score: 2, Interesting

      This is exactly why content-based filters will never work: the professional spammer will take the time and run his e-mail through filters until he gets a good result (a negative answer). The non-spammer will not take the time to test his e-mail with all the spam-filters. Therefore, it is very likely that legitimate content will be filtered and professionally composed spam e-mails will not. So IMHO, Spam-Assassin and all the other content-based spam-filters are completely useless.

      --
      "I love my job, but I hate talking to people like you" (Freddie Mercury)
    2. Re:The problem with filters by carsten · · Score: 2, Informative

      Well guess I only get spam from non-professional spammers then. I run spamassassin on my server and almost never get any spam into my Inbox. I get maybe 5-10 spams a day and they all get tagged by spamassassin and procmail filtered into a folder where I check them for false positives before deleting. The only false positives I get is a news letter from the airline KLM, for which I am too lazy to set up a procmail filter since I never read it anyway.

      I have filters for all my mailing lists and so forth in my .procmailrc and then the spamassassin filter at the end. Works like a charm for me.

  13. The problem with content filtering by Leeji · · Score: 4, Insightful

    This is exactly the problem with most content filtering approaches.

    It is very hard to discern the difference between talk about sex, spam, viruses, etc and talk from sex, spam, viruses, etc. Newsletter authors go as far as writing "v*rus" and "sl*mmer" so that pitiful content filtering blocks don't trash them.

    It gets even worse for email lists that use inline text ads. The ads alone would constitute spam, but they're nestled within several paragraphs of high-quality discussion.

    The problem is that content filtering approaches usually only analyze the "spamminess" of a piece. They usually don't analyze the "goodness" of a piece. So if I put "hot teens go crazy for debt-free viagra while earning $$$ from home" in the middle of some fine Shakespeare, that will get flagged as spam.

    The new "bayesian" approaches are finally dealing with this problem -- something can look an awful lot like spam, but it will be saved if it looks even more like legitimate email.

    In this case, spam doesn't generally run for 21 pages with words like "cryptography," and "full disclosure."

    --
    It all goes downhill from first post ...
    1. Re:The problem with content filtering by Forgotten · · Score: 2, Funny
      The problem is that content filtering approaches usually only analyze the "spamminess" of a piece. They usually don't analyze the "goodness" of a piece. So if I put "hot teens go crazy for debt-free viagra while earning $$$ from home" in the middle of some fine Shakespeare, that will get flagged as spam.

      Nor would you be wrong to insert that, since that's roughly the Cliff's Notes reduction of several Shakespeare plays.

    2. Re:The problem with content filtering by Tricot · · Score: 5, Funny

      ...if I put "hot teens go crazy for debt-free viagra while earning $$$ from home" in the middle of some fine Shakespeare, that will get flagged as spam.

      eMerchant of Venice. Act I Scene IV, right?

    3. Re:The problem with content filtering by Forgotten · · Score: 1
      In this case, spam doesn't generally run for 21 pages with words like "cryptography," and "full disclosure."

      Well, that I'm not so sure of. That pretty much does describe some of the spam I've received. I'm not sure there's any machine-detectable characteristic (or set of them) you can say wouldn't be in spam - that's why current approaches lean so heavily toward positive indicators. I'm not saying your idea doesn't have merit - I think it does - but it's tougher than it sounds to tell an MTA or client how to implement it. If we could predict everything worthwhile that would ever appear in mail, there'd be no point having mail. ;)

    4. Re:The problem with content filtering by 1u3hr · · Score: 3, Insightful
      In this case, spam doesn't generally run for 21 pages with words like "cryptography," and "full disclosure."

      The problem with that is that if you score mail by the percentage of spam, rather than the absolute amount, the obvious response by spammers is to ADD 21 pages cribbed from a crypto newsletter to the end of their penis-enlarging spam. Maybe even fake the headers to make it look like it came from a respected source.

    5. Re:The problem with content filtering by Lt.Hawkins · · Score: 1

      but that would be worth reading!!!

      --
      -- My Sig is a P228.
    6. Re:The problem with content filtering by anubi · · Score: 1
      Oh yes... filters.

      As far as I go, nobody who sends me a personal message ever uses HTML.

      So I just code all incoming files with embedded HTML as spam.

      If anybody wants to contact me.. just do NOT embed HTML in it.

      --
      "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]

    7. Re:The problem with content filtering by Leeji · · Score: 1

      You're right, it's not easy. SpamAssassin does indeed have measurements that suggest a mail is not spam (the full list of tests are here, the negative ones,) but there very few.

      I worked on spam filtering for a project in University, based on the computational linguistics field of genre classification. I first ran tons of spam and good email through a decision tree maker (using many statistics: word length, verb tense, number of possessive pronouns, and more) Once I let the _computer_ decide which statistics were important, I could run any mail through this decision tree and find out two things: the probability that it was spam, but also the probability that it was good email. This proved very effective, as does the simpler keyword-based prediction of bayesian filtering.

      Determining statistics for good email is harder than determining statistics for spam, but it is possible.

      --
      It all goes downhill from first post ...
    8. Re:The problem with content filtering by Anonymous Coward · · Score: 0

      > It is very hard to discern the difference between talk about sex, spam,
      > viruses, etc and talk from sex, spam, viruses, etc.

      well the last time I got an email from sex..

    9. Re:The problem with content filtering by donheff · · Score: 1

      The real thing wouldn't get through.

      Here is Thisbe talking to "WALL" in Midsummer Night's Dream:
      "My cherry lips have often kiss'd thy stones,
      Thy stones with lime and hair knit up in thee."

    10. Re:The problem with content filtering by Zeinfeld · · Score: 1
      As far as I go, nobody who sends me a personal message ever uses HTML.
      So I just code all incoming files with embedded HTML as spam.

      And how exactly would you know? After all if I send you HTML mail you will never see it.

      About 50% of my non-spam personal email is HTML. Of course the statistics might be off as I did help write HTML.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
    11. Re:The problem with content filtering by wheany · · Score: 1

      The new "bayesian" approaches are finally dealing with this problem -- something can look an awful lot like spam, but it will be saved if it looks even more like legitimate email.

      In this case, spam doesn't generally run for 21 pages with words like "cryptography," and "full disclosure."


      Well, as a matter of fact, my bayesian filter marked the message spam, when I test-sent the html-file as an attachment to myself.

    12. Re:The problem with content filtering by NineNine · · Score: 2, Insightful

      Spammers won't do this. Why? The number of people using something like Spamassasin are so small, it's not worth their time. Besides, those customers aren't going to buy, anyway.

    13. Re:The problem with content filtering by sysadmn · · Score: 2, Interesting

      Which is why most bayesian-type filters react to a subset of the keywords. By estimating based on the 10 or so best indicators the dilution is less effective...

      --
      Envy my 5 digit Slashdot User ID!
    14. Re:The problem with content filtering by Leeji · · Score: 1

      when I test-sent the html-file as an attachment to myself

      That should explain it, although I could be wrong. If your filter doesn't look inside of attachments (which I think is the norm,) it could very well look like spam. If it does look inside of attachments, I think it's time to review the mail you trained your "spam" on. Many others, even those using SpamAssassin with its default threshold, have mentioned that the CryptoGram gets through.

      --
      It all goes downhill from first post ...
    15. Re:The problem with content filtering by wheany · · Score: 1

      Nope, it looked through the attachment. It was attached as quoted-printable text/html, and the discriminating words were from the attachment. The only other word in the body of the mail was "testi."

      The three spammiest words were "million" "trust" and "reports."

    16. Re:The problem with content filtering by rookkey · · Score: 2, Informative
      The number of people using something like SpamAssassin are so small, it's not worth their time.


      Not for long. Filtering software such as SpamAssassin is now being used at the server level to recognize junk email for thousands of clients.

      For example, the University of Colorado at Boulder now uses SpamAssassin to scan all incoming student email. This means SpamAssassin handles the spam filtering needs of a student population of 30,000. There is no doubt that as the spam problem increases, filtering solutions will begin to appear at the ISP level.

    17. Re:The problem with content filtering by anubi · · Score: 1
      Zeinfeld, I owe you an explanation.

      I certainly do not mean to push against HTML. It is by far the most easy-to-use form of linking and page layout I could ever imagine. I think you guys did a great job on it. The main thing I get miffed over is how all these guys come in after the fact and make their own bastardized versions that require all sorts of specialized plug-ins to make their content viewable. But there are no telling if these "plug-ins" are honest, or if they are trojans.

      The thing I love so much about HTML is that its quite easy to open it up in any ascii text editor and verify what it was supposed to do. No under-the-covers sneakystuff. Although you guys obviously went to great lengths to cover all the bases, there are those who want to do the exact same thing, but different. Like, why would I have to have yet another "plug-in" to listen to an MP3 if I already have a trusted MP3 decoder on my end? Thats not an HTML issue.. thats an issue of somebody that wants to coin yet another file format and use the flexibility of HTML to implement it.

      As for my personal stuff, all of it is plaintext ascii quickies. No formatting. No pictures. The sender did not want to spend a lot of time with the tags. He just wants to tell me a quick note or show me a page link. Netscape itself is intelligent to recognize a link and display it as such. Typically the email I get takes maybe a minute or so to type, and contains maybe 20 words.

      I code quite happily here on Slashdot, and much prefer the elegant HTML coding scheme over anything else, and quite happily invest the extra time to insert the tags because I know many of the posts I generate may be at least viewed by thousands of people using dozens of different browsers. The idea I can use different fonts, italicize, hotlink to other sites, whatever, is quite powerful as far as my trying to present a readable complex document that will display on whatever pulls it up. But it also means I am willing to take the time on my end to save many people time on their end by neatly formatting the document and making it easy to read. And by the time I post here, the document is much more complex than something like

      "Jan's brother is coming along with us to Hemet. See you at 7:30. Diane. "

      By no means would I want to blemish HTML. Its just that people who spam me often use HTML as they have a complex document, often with hundreds of kilobytes of linked images. The people who coin these spams often are sitting right on top of a server with high speed links to their editing terminal, and are quite unaware of the several minutes their message is going to take to transfer if they were on a dialup. Even big companies can be completely unaware of how much time their advertising automails can take on a dialup. I even have my own ISP mailing me from an automailer and using the power of HTML to cause lots of image downloading into my machine. Sure it looks pretty, but did I really want to spend two or three minutes downloading that image of the "pleased person's face" expressing exhuberance over yet more stuff thats going to take yet more time to display? But then, I remember, this is a big company I am dealing with. Trying to change them is like trying to change a bank by pushing on it. They are ad-men. They must be paid by the byte. They have no concept of the value of time. They just find someone who is impressed by pretty pictures, and they get funded to push those pictures onto everyone else via spam. HTML just gives them the tools to do it with.

      Also, many of them want confirmation that their email spray was productive, as just opening the email at the client end invokes retrieval, hence display of my IP in their server logs, of images linked to in their HTML spam. This confirms to the spammer that he hit someone who opened up and read his spam. It would not take much on his end, especially if he crafted his spams individually, to find out which email addresses spammed results in a hit on his server. This concentrated set of email addys could be valuable to him to sell to other spammers, each doing the same. I am just trying to nip it in the bud by refusing to look at stuff that the mere fact of opening it confirms to the sender that it was read.

      HTML is a very powerful tool, but in the wrong hands it can be a nuisance. Just as any other good tool.

      --
      "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]

    18. Re:The problem with content filtering by duncf · · Score: 1

      I suppose you haven't been following the SpamAssassin mailing lists then. :-) There are numerous cases of spammers adapting to specifically evade SpamAssassin.

    19. Re:The problem with content filtering by EvilStein · · Score: 1

      And your source is....?

      There are probably a lot more places using SA than you think.. :-)

  14. So evolve the filters? by ghutchis · · Score: 1

    No, there's no reason to change the newsletter. (And certainly CRYPTO-GRAM isn't the only e-mail newsletter that's run into problems with spam filtering.)

    IMHO, spam filters are always going to be an evolving act--more obviously so with pseudo- and full Bayesian methods.

    So let's treat this as part of the corpus of "non-spam" or "ham" or whatever you'd like to call it and code accordingly. If this brings more focus to improving filters and addressing non-technical methods (i.e. legal action, petitions, legislative advocacy), great.

    But do we need another discussion of spam filtering?

    -Geoff

    (In my case, I simply filter off known good messages first before sending it to the Junk Mail filtering.)

  15. Am I the only one by ehintz · · Score: 0, Troll

    Who thinks this is an utter waste of a FPP? I mean, why the hell would anybody even submit this to /.? And if they did, why the hell would it be posted fer crissake?

    Wow. Some spam filters may have a false positive. How groundbreaking. News for nerds. Stuff that matters.

    --
    ehintz
    1. Re:Am I the only one by Anonymous Coward · · Score: 0

      You think this is bad? Try reading the Ask Slashdot articles. At least half of them could be answered in seconds with Google, but instead they go to the bother of writing up an article and convincing an editor to put it up. The other half is generally people asking how to do their job, with the occasional rare interesting question.

    2. Re:Am I the only one by Anonymous Coward · · Score: 0

      mod parent up, i agree.

  16. SPEWS by some1somewhere · · Score: 3, Insightful

    At least he is only on Spamassassin which tends to be run on the client-side, so statistically less people would not see the newsletter. If he were on the SPEWS's blocklist, he'd never get out!

    http://www.antispews.org/ the SPEWS fansite (not!)

    Personally I see less problem with client-side blocking, as there is less chance that any 2 people would use exactly the same combination of blocklisting/priorities/etc. Plus, programs like Spamassassin use quite a lot of processing power, so large mail servers (eg. for an ISP) would need significant additional resources to handle this. Thus it is best to move such individualized and resource-intensive applications to the client-side anyway.

    YMMV.

    --
    **FREE** Track and view your phone's via CellID and/or WIFI and/or GPS :- http://tinyurl.com/la6fhd
    1. Re:SPEWS by harlows_monkeys · · Score: 1
      If he were on the SPEWS's blocklist, he'd never get out!

      Unless he sends actual spam, or is on an ISP that supports spamming, he wouldn't end up on the SPEWS list in the first place.

    2. Re:SPEWS by letxa2000 · · Score: 1
      Thus it is best to move such individualized and resource-intensive applications to the client-side anyway.

      The problem is that for that to happen the client has to have already downloaded the spam. Granted with faster connections that doesn't always take TOO long, but the idea is to protect the client from spam--if they've already downloaded it then you're just not showing everything that was downloaded.

      Additionally, having that on the client side means all the clients have to have the filtering software. All the clients need to update the filters or approaches as necessary. This is a perfect application for a server-side system.

      Baynesian can and should be done on the server. A big help will be when commonly used email programs have a feature to automatically submit missed spam to their server so the server can refine their Baynesian statistics--but the solution is on the server, not on the client.

      IMHO.

    3. Re:SPEWS by some1somewhere · · Score: 1

      But what about the person that actually enquired about the penix enlargement?

      --
      **FREE** Track and view your phone's via CellID and/or WIFI and/or GPS :- http://tinyurl.com/la6fhd
    4. Re:SPEWS by Anonymous Coward · · Score: 0

      If he were on the SPEWS's blocklist, he'd never get out!

      Generally it helps if you actually bother to check how something works first rather than rely on a website run by spammers.

      From the SPEWS FAQ:

      Q42: My IP address/range is being listed by SPEWS but I'm not a spammer and I just signed up for this/these address(s). What can I do to be removed from the list?

      A42: SPEWS is just an automated system, if spam or spam involvement (hosting spammers, selling spamware) from your IP address/range ceases, it will drop out of the list in time. Normally the listing involves spam related problems with your host and the first step you need to take is to complain to them about the listing, in almost all cases, they are the only people who can get an address/range out of the SPEWS list. If there is a spam related problem with your host, their IP address/range will not be removed until it is resolved. If your host or network is certain a listing mistake has been made, ask them to read this FAQ then post a message in a public forum mentioned above with the SPEWS record number (eg. S123) and/or the IP address/range information in it. Placing the text "SPEWS:" in the subject can help a SPEWS editor or developer see the message and they may double check the listing - note that, although others may, no SPEWS editor or developer will ever reply to the posting. Will this get your IP address/range removed from a SPEWS listing? Again, not if there are currently spam related problems with your host. Be aware that posting ones email address to any publicly viewable forum or website makes it instantly available to spammers. If you're concerned about getting spammed, change or "mung" the email address you use to post with.

    5. Re:SPEWS by c0ntempt · · Score: 1

      sounds like someone got listed in spews... ;P

    6. Re:SPEWS by Skapare · · Score: 3, Insightful
      If he were on the SPEWS's blocklist, he'd never get out!

      And this is why the SPEWS blocklist is so effective and so good. If he were on it, then that would mean that he and/or his network fell into one of the following categories:

      • Is a spammer
      • Is an ISP harboring a spammer (or an upstream ISP thereof)
      • Is a customer of an ISP harboring a spammer

      Because spam causes abuse to email servers, even when the mail is refused either for reasons of an IP based blocklist, or for content filtering ... abuse in the form of higher costs for the server operators and recipients ... the proper goal is to get the spammer not just blocked from being able to get mail into your mailbox, but fully disconnected from the internet to prevent these kinds of costly abuses in the future. And since only the ISP hosting them can actually disconnect them, it will be the job of that ISP to do so. Most ISPs will when they realize the situation. A few ISPs refuse to, and that's when it comes time to put pressure on the ISP by expanding the blocking of the ISP's network, forcing them to consider that their legitimate customers will be leaving if they do not disconnect the spammer. SPEWS gradually expands listings so that the point where the ISP finally understands this can be reached with the minimum of so called collateral damage (which is not really, because these are customers who are paying money to an ISP which harbors spammers, so they share in the guilt).

      Bruce Schneier's mail server happens to not be listed by SPEWS. So it can be said that he is not a spammer, is not running an ISP that harbors spammers, and is not using an ISP that harbors spammers. That is a good thing and shows that SPEWS not only works, but works better than content based filtering.

      Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech (although the actual amendment only applies to restrictions imposed by the government and does not apply to private businesses in most cases, if not all). Infringement of free speech happens when the decision is based on what the content is. When restrictions are not affected by the content, then such restrictions are considered fair since any content can be passed when the behaviour that evoked the restrictions is not done. And the whole spam issue is about behaviour, not content. The bad behaviour is the act of inappropriately choosing multiple recipients for sending the message ... e.g. unsolicited bulk email (UBE).

      Of course on your own mail server you have a right to use whatever methods you deem appropriate based on how you want to balance your costs, the quality of your service to your customers, and how much cost you want to pass on to your customers. Obviously you have to be in contractual agreement (possibly implied) with your customers about what methods are chosen. If you only offer one kind of service and your customer does not want that kind, by being properly aware of what you do offer, they can go elsewhere. Or you can offer a diversity of services the customer can choose from (e.g. a customer control panel to control the methods of spam filtering for their email accounts). So the choice of what method to use to block spam is strictly a relationship between a provider and its own customer.

      In the case of a network owned by a business only to serve that business function, then it's simply the commercial version of "my server, my rules".

      --
      now we need to go OSS in diesel cars
    7. Re:SPEWS by _xeno_ · · Score: 0, Troll
      "Yes, your honor, the bomb did kill 158 innocent civilians. But it also killed the two terrorists!"

      Come on - is it really your fault if you accidently find yourself "a customer of an ISP harboring a spammer?" Do you deserve to be punished too? Do you really think that blocking the entire netblock of people who may be using the service because they have no other choice is really a good method to stop spammers?

      I doubt many people blocked due to a single spammer are going to think "oh, well, I may not be able to send e-mail to my most important client - but at least while I'm losing thousands of dollars, I know I'm helping to fight spam!" Most, I'd bet, would just call up the offending receiver and complain that they're getting bounce messages when they try and send e-mail and that the receivers should fix their mail servers as soon as possible.

      So, I guess if costing a few hundred people a hundred bucks to move to another service is "helping to reduce spam," it's a cost that they should be glad to pay...

      If you want to use SPEWS for your own personal webserver, then go ahead. If you expect anyone doing any buisness with e-mail to use it regardless of the risks of blocking important e-mail, then you're out of your mind. If you think that blocking entire netblocks is going to encourage companies to use SPEWS, then you're insane. If you think harming many to bring justice to a few in the group is morally just, then I must question your morals.

      (I suppose a better analogy would be "Yeah, the gas may have put hundreds of innocent civilians into the hospital for a month, but it also put the three-man scam out of operation!" in that people that are blocked by SPEWS can become unblocked, and hence are only "wounded." It's still harming many to eliminate a few.)

      --
      You are in a maze of twisty little relative jumps, all alike.
    8. Re:SPEWS by GammaTau · · Score: 2, Informative

      http://www.antispews.org/ the SPEWS fansite (not!)

      Heh, this antispews.org money-making scam is a rather funny one. Strangely enough the Hostway Corporation started hosting the site three days after t3marketing lost their lawsuit against Joe McNicol. The Hostway Corporation is behind the t3marketing and many other "direct marketing" buggers. So it's no wonder that they are listed in SPEWS and using every possible way - sue spamfighters, spread FUD, etc. - to help them to continue poison our mailboxes.

      That being said, I'm not sure if the SPEWS way of doing things is such a good idea but the antispews.org site is still run by spammers and should be treated as such.

    9. Re:SPEWS by Anonymous Coward · · Score: 0

      If you expect anyone doing any buisness with e-mail to use it regardless of the risks of blocking important e-mail, then you're out of your mind.

      My workplace does business with e-mail, we've been using SPEWS for over a year, it's blocking about 90% of the spam, with 0 false positives so far. And you're analogy is bad: those 158 "innocent" civilians are supporting the criminals.

    10. Re:SPEWS by Skapare · · Score: 1
      "Yes, your honor, the bomb did kill 158 innocent civilians. But it also killed the two terrorists!"

      We're not talking about 158 innocent civilians. We're talking about customers of an ISP. The ISP harbors spammers, and the customers are being pressed to get the ISP to stop that bad practice. At any point in time the ISP can do the right thing and disconnect the spammers. This isn't like a war where the leaders refuse to consider diplomacy and negotiations. The bad guys can stop being bad guys at any time and this will resolve the situation. And the customers are not being blocked anywhere they go; they can deliver mail via a 2nd ISP, or they can pull up stakes and move.

      Your analogy is flawed because blocking email via a particular ISP does not prevent the customers from being able to use another ISP to send mail, but killing people prevents them from doing anything forever. These things simply are not equivalent.

      Come on - is it really your fault if you accidently find yourself "a customer of an ISP harboring a spammer?" Do you deserve to be punished too? Do you really think that blocking the entire netblock of people who may be using the service because they have no other choice is really a good method to stop spammers?

      No, it is not your fault to find yourself a customer of an ISP harboring a spammer, if that was not going on when you started with the ISP. But, the customer can choose to move (at least their mail sending operation) to another ISP, and bill (or sue) the guilty ISP for the costs of doing this.

      Very few cases exist where the customers of an ISP have no choice. Even in areas where that might exist in terms of connective access, there is also the option to acquire the services of a remote server (thousands of ISPs available all over the world, most of which are not listed in SPEWS), tunnel securely to it, and send mail out from that server.

      I doubt many people blocked due to a single spammer are going to think "oh, well, I may not be able to send e-mail to my most important client - but at least while I'm losing thousands of dollars, I know I'm helping to fight spam!" Most, I'd bet, would just call up the offending receiver and complain that they're getting bounce messages when they try and send e-mail and that the receivers should fix their mail servers as soon as possible.

      The customers that are not "inconvenienced" by this won't be complaining to their ISP about the spam problem. Then there will be no pressure on the ISP to disconnect the spammers. Then the spammers will continue to abuse millions of email servers all over the world even when those servers are rejecting messages because the spammer is listed in SPEWS or some other DNS blacklist, or bouncing them due to content filter rejections.

      There is no way to put pressure on an ISP that they can understand other than through their customer base. Remember, this involves ISPs that are not voluntarily disconnecting spammers, probably because of their greed for the pink money the spammers are paying them.

      Do you have a better idea that will place this pressure on the ISP? I've asked this question of many people, and have gotten no direct answers, only whining about punishing innocent customers.

      So, I guess if costing a few hundred people a hundred bucks to move to another service is "helping to reduce spam," it's a cost that they should be glad to pay...

      I won't say they will be happy about it at all. I expect them to be quite pissed off. But that anger should be directed to billing or suing their ISP for these costs. Since the ISP could have corrected the problem by disconnecting the spammers, they cannot hide behind "... due to conditions beyond our control".

      And keep in mind that the spammers involved are spewing out millions, sometimes even billions, of copies of junk, costing the end recipient mail server operators money in terms of server resources abused and wasted (e.g. running a process to carry out the SMTP protocol, check the rejection database, analyze the message content, or whatever it has to do). And this cost is several times more than the sum of what the spammers make, the ISP makes, and the customers lose to move. Do you think the recipient mail server operators are going to say "it's a cost I'm glad to pay to ensure no email is ever lost"?

      If you want to use SPEWS for your own personal webserver, then go ahead. If you expect anyone doing any buisness with e-mail to use it regardless of the risks of blocking important e-mail, then you're out of your mind. If you think that blocking entire netblocks is going to encourage companies to use SPEWS, then you're insane. If you think harming many to bring justice to a few in the group is morally just, then I must question your morals.

      Based on the numbers of complaints about "SPEWS is blocking my email" it's rather obvious that quite a lot of networks are using it. Of course not everyone will. And it is also possible to configure certain addresses to not make use of SPEWS, such as the sales department or the abuse department. Still, not everyone understands the costs against the mail servers, so they don't know about a need to get spammers disconnected, and thus they may not be motivated to use SPEWS.

      Again, you have the opportunity to stop being negative (being negative is saying what not to do while not offering an alternative that achieves the same goals) and offer a better idea which will get spammers disconnected.

      (I suppose a better analogy would be "Yeah, the gas may have put hundreds of innocent civilians into the hospital for a month, but it also put the three-man scam out of operation!" in that people that are blocked by SPEWS can become unblocked, and hence are only "wounded." It's still harming many to eliminate a few.)

      Were these civilians supporting the scam operation? Were they used as human shields? I don't even see an analogy here, so I suspect you still have no clue about the scale of the spam problem.

      --
      now we need to go OSS in diesel cars
    11. Re:SPEWS by Anonymous Coward · · Score: 0

      "And you're analogy is bad: those 158 "innocent" civilians are supporting the criminals."

      Just like the Iraqi people are supporting Sadam Hussein. That's why the sanctions against them are just. If they don't like it, they can just rise up and overthrow him. If they don't do what we want then we have every right to punish them.

    12. Re:SPEWS by nehril · · Score: 1
      And this is why the SPEWS blocklist is so effective and so good. If he were on it, then that would mean that he and/or his network fell into one of the following categories:

      Is a spammer

      Is an ISP harboring a spammer (or an upstream ISP thereof)

      Is a customer of an ISP harboring a spammer


      uh, this is exactly why things like blacklists *are* broken. There are plenty of spammers not on any blacklist, so don't think of (!blacklisted) as equal to (whitelisted). Also, (blacklisted) != (spammer) as well, since alot of these list ops don't care about false positives or collateral damage.

      Secondly, consider your "is a customer of an isp harboring a spammer" rule. The point of antispam efforts is not to block out all spam. (redirect all email to /dev/null would accomplish THAT goal). The point is to allow genuine communication. That means a perfect antispam would allow 100% of "useful" communication (whatever you define "useful" to be) and deny 100% of everything else. Blocking "customers of ISPs" goes directly against that: purposefully denying non-spam traffic is a broken concept. Blacklisters tend to justify such behavior as "zero tolerance," and "putting pressure on ISPs," but I think attacking innocent bystanders is extremely offensive, ineffective and just plain wrong.

      So what if your favorite blacklist decides to stuff the entire 64.*.*.* IP address range? you will cut a lot of spam but suffer enormous collateral damage. Find a spammer, block the spammer. but don't bomb his whole neighborhood "to prove a point."

    13. Re:SPEWS by Zeinfeld · · Score: 2, Insightful
      And this is why the SPEWS blocklist is so effective and so good. If he were on it, then that would mean that he and/or his network fell into one of the following categories:

      Or he might be

      A customer of UUNet which spews has listed because it disagrees with some of the content they host

      NOBODY with a brain is using SPEWS anymore. Listing the largest commercial internet supplier in the US was simply idiotic. And it was done for completely illegitimate reasons.

      The whole blacklist concept boils down to vigilante tactics, use threats to keep people in line. The problem being that the people who run the lists tend to turn into self-important little tinpot dictators after a short time.

      Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech

      Unture, with the exception of Limabaugh whose judgment in Nixon is opinionated nonsense the Federal courts have all rulled that the junk fax laws are constitutional.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
    14. Re:SPEWS by Zeinfeld · · Score: 1
      Come on - is it really your fault if you accidently find yourself "a customer of an ISP harboring a spammer?"

      Oh SPEWS and its ilk have gone further than that. Several of the Blacklists have blacklisted whole countries. In particular China and Korea.

      I doubt many people blocked due to a single spammer are going to think "oh, well, I may not be able to send e-mail to my most important client - but at least while I'm losing thousands of dollars, I know I'm helping to fight spam!" Most, I'd bet, would just call up the offending receiver and complain that they're getting bounce messages when they try and send e-mail and that the receivers should fix their mail servers as soon as possible.

      This happened to us as we are customers of UUNET which SPEWS listed because they don't like some of the content they host. Switching ISPs was never considered, we simply used the backup feed to send out an email to all the mailing lists we host stating that we had been blacklisted, have no intention of changing feeds and people who wanted to participate in those working groups could fix their mail servers pronto.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
    15. Re:SPEWS by Heinrich · · Score: 1
      If he were on the SPEWS's blocklist, he'd never get out!

      And this is why the SPEWS blocklist is so effective and so good.

      The problem with SPEWS is the refusal to consider appeals. Out of their FAQ:

      Q16: I'm not a spammer or spam operation... heck I hate spam, but my email is getting bounced by someone using SPEWS, or I can't access a website due to SPEWS based blocking.

      A16: You maybe part of the rare "inadvertent blocking" that can occur when a spam friendly provider is listed in spews. Your best option is to try and educate your provider or switch to one who is not listed in SPEWS as spam friendly. SPEWS aims to avoid listing any non-spammer or non-spam support areas if possible - we just want to stop spam.

      [...]

      Q41: How does one contact SPEWS?

      A41: One does not. SPEWS does not receive email - it's just an automated system and website, general blocklist related issues can be discussed in the public forums mentioned above.

      Every blocklist has sooner or later false positives. When there is no way to handle complaints then this list is more harmful than good.

    16. Re:SPEWS by david+duncan+scott · · Score: 1
      Thus it is best to move such individualized and resource-intensive applications to the client-side anyway.

      Clearly you're not on dial-up. By the time I've downloaded 100 spams, half the annoyance is over.
      --

      This next song is very sad. Please clap along. -- Robin Zander

    17. Re:SPEWS by SN74S181 · · Score: 1

      'Unless you are a real criminal and belong in prison, you will not find yourself arrested.'

      'Unless you are a henious criminal and deserve to die, you will not find yourself on death row.'

    18. Re:SPEWS by Skapare · · Score: 1
      uh, this is exactly why things like blacklists *are* broken. There are plenty of spammers not on any blacklist, so don't think of (!blacklisted) as equal to (whitelisted).

      No anti-spam method is perfect. It is unlikely any will ever be. Don't expect some clever new spam to be blocked until the blockers get more clever. It's a game of one-upsmanship.

      Also, (blacklisted) != (spammer) as well, since alot of these list ops don't care about false positives or collateral damage.

      Go back and read my original post. These are not false positives or collateral damage. They are intended. When the customers of an ISP are listed and blocked in order to pressure the ISP to stop its support of the abuses by spammers, that is not an error, not a mistake, not a false positive, or collateral damage. It is in fact intended and for the described purpose.

      In war, we speak of collateral damage as the UNintended targets of things such as bombs. Blocking customers of ISPs is not that. It is more like trade sanctions. The trade sanctions don't work in Iraq because most of the people cannot switch to living in a different country. But most customers of an ISP that doesn't get the clue, can switch to another ISP. And it has accomplished the intent in many cases.

      Secondly, consider your "is a customer of an isp harboring a spammer" rule. The point of antispam efforts is not to block out all spam. (redirect all email to /dev/null would accomplish THAT goal). The point is to allow genuine communication. That means a perfect antispam would allow 100% of "useful" communication (whatever you define "useful" to be) and deny 100% of everything else. Blocking "customers of ISPs" goes directly against that: purposefully denying non-spam traffic is a broken concept. Blacklisters tend to justify such behavior as "zero tolerance," and "putting pressure on ISPs," but I think attacking innocent bystanders is extremely offensive, ineffective and just plain wrong.

      You're missing a goal. The other goal is to keep the communications cost effective. Consider that an onslaught of spam, even though it is not going to be delivered for whatever reason, can overload a mail server, possibly even crashing it, and deny other communications. Spam attacks can deny the timeliness of communications. There won't be any form of 100% perfect useful communications until every spammer is gone. That goal cannot ever be realized given human nature, but we can get very close by making sure that ISPs deal with the issues of spam that they should be doing. Once they are doing that, then we'll at least have 99.9999% usefulness.

      We obviously disagree. In my opinion, what you call attack is nothing more than a boycott. And remember that it is the recipient mail system operator making that decision to use SPEWS or some other blocklist. If they believed as you do, they would not use it (I presume you do not).

      So what if your favorite blacklist decides to stuff the entire 64.*.*.* IP address range? you will cut a lot of spam but suffer enormous collateral damage. Find a spammer, block the spammer. but don't bomb his whole neighborhood "to prove a point."

      First of all, picking that specific address range is stupid. SPEWS will not expand a listing to an unrelated ISP. The 64.0.0.0/8 block is broken up into many allocations by ARIN. But maybe you can use the 12.0.0.0/8 network instead, since it is allocated entirely to one ISP.

      It's not bombing a neighborhood. If enough people were to use SPEWS, then the ISP would eventually realize that they will lose more money by legitimate businesses leaving than they get from spammers. Bombing is lasting damage that has to be rebuilt over. Blocking an ISP is fixed by a very simple action of disconnecting the offending spammers.

      I'm sure you would be quite pissed off if your mail bounced because your ISP was listed in SPEWS. But consider that the operator of the mail server used by the party you tried to send the mail to is equally pissed off at your ISP for letting one of its customers continue to attack his server. Actually, it is more likely he will be even more pissed off, because the costs in terms of resources consumed and wasted at the recipient server exceeds the money the ISP makes from the spammers, the money the spammers make for themselves, and the cost to the customers to switch ISP, combined.

      --
      now we need to go OSS in diesel cars
    19. Re:SPEWS by Skapare · · Score: 2, Interesting
      Or he might be
      • A customer of UUNet which spews has listed because it disagrees with some of the content they host

      UUNet has become one of the worst ISPs around due to their harboring of large numbers of spammers. And they do absolutely nothing to respond to complaints reported to them. They just let the spammers keep spamming.

      NOBODY with a brain is using SPEWS anymore. Listing the largest commercial internet supplier in the US was simply idiotic. And it was done for completely illegitimate reasons.

      There are completely legitimate reasons for blocking UUNet. It's the spam. You may be confusing SPEWS with some small-time renegade blocklist.

      The whole blacklist concept boils down to vigilante tactics, use threats to keep people in line. The problem being that the people who run the lists tend to turn into self-important little tinpot dictators after a short time.

      As soon as I see SPEWS operators "turn into self-important little tinpot dictators" I'll certainly stop using it. But I have not seen it happen. Feel free to point out any specifics if you are aware of them.

      If anything, it is the very act of harboring spammers that is a vigilante tactic. Given that the costs of transmitting email are heavily slanted to the recipient end when spam is involved (because the spammers use special software to send email that scale up more effectively than ordinary MTA software), such a tactic could be in active use by some ISPs to drive up the costs for others (their competition).

      Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech
      Unture, with the exception of Limabaugh whose judgment in Nixon is opinionated nonsense the Federal courts have all rulled that the junk fax laws are constitutional.

      Read my statement again, this time carefully. I said it is a violation of the principles. I did not say it is a violation of the Constitution and/or First Amendment itself (see the way the clause is written). Since the Constitution places restrictions on the government, it is the government that is the one that has to be sure not to restrict speech based on its content. You and I are free to do so within the context of our property rights and those of others. While it would be wrong for me to go delete your messages (that would be violating your property rights), you could certainly delete them yourself if you choose to. But I do fully believe in the principles the US Constitution was based on, and I practice my life that way. Thus, I do not use content based filtering. That's my choice.

      --
      now we need to go OSS in diesel cars
    20. Re:SPEWS by Zeinfeld · · Score: 1
      There are completely legitimate reasons for blocking UUNet. It's the spam. You may be confusing SPEWS with some small-time renegade blocklist.

      Since the SPEWS maintainers refuse to answer any correspondence whatsoever there is no way you can possibly know what criteria they are applying. They state that their criteria for listing UUNET is content hosted by a UUNET customer. Of course SPEWS could be lying.

      The realtime blacklists are simply not transparent.

      Comparing UUNET to vigilantes is simple sophistry.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
    21. Re:SPEWS by Skapare · · Score: 1

      UUNet does harbor spammers. As long as UUNet believes you'll stay despite spammers, they will continue to harbor spammers and the spammers will continue to abuse other people's mail servers.

      The blacklists that list all of a whole country or a whole ISP are not the same thing as SPEWS. SPEWS doesn't do that kind of thing. You must be referring to http://www.blackholes.us/.

      --
      now we need to go OSS in diesel cars
    22. Re:SPEWS by Skapare · · Score: 1

      As long as spammers remain, listings remain. When spammers are gone, eventually this is detected and the listings fade out. But there is a vanue for reporting errors. It is the news.admin.net-abuse.email newsgroup. And it works. I've seen it work. Yes, there have been a few errors, and they get reported and corrected.

      --
      now we need to go OSS in diesel cars
    23. Re:SPEWS by Skapare · · Score: 1

      SPEWS will list the ISPs that provide services for spammers other than the IP routing service. For example an ISP who provides only the web site hosting for a spammer who is spamming on some other network can be listed. Another ISP that provides only the DNS service to a spammer can be listed.

      --
      now we need to go OSS in diesel cars
    24. Re:SPEWS by _xeno_ · · Score: 1
      How about this - you are saying that it is justifiable for terrorists to kill civilians because the country they happen to be in supports something they disagree with. Spam may be a large problem (although it's easy to block for me, since most of it is in Chinese for some reason), but that does not justify harming people who just happen to have the misfortune of being near a spammer.

      I'm just going to make this point: blocking non-spammers will only hurt SPEWS in the long-run, as its current effectiveness is based on the majority of people using it. As people find that they either have to turn to some other source than SPEWS or accept that occasionally people they must communicate with cannot send them email without a whitelist, they will not think "yay, we're helping to eliminate spammers!" and instead think "Goddamned broken SPEWS thing accidently blocking valid people - remove it!"

      Eventually, SPEWS will have caused enough problems that no one (except a small core of vigilantes on their own servers) will use it. And then it will have no power over the vast majority of spammers. SPEWS would be more effective if it only blocked spammers and many people used it - it could help make spam an ineffective method of contacting people. Force the cost of spamming up, not the cost of happening to use an ISP that hosts a spammer. All you're going to do is create things like this, and force people against you. SPEWS does not help the cause by being a vigilante and trying to force people who have no buisness with the spammer to take action against their ISP.

      Besides - which is easier: getting the ISP to drop the spammers acount, or telling people who want to communicate with you to stop using SPEWS? The path of least resistance is likely to be followed... which may include "ok, we'll just ignore the bozos using SPEWS."

      --
      You are in a maze of twisty little relative jumps, all alike.
    25. Re:SPEWS by Skapare · · Score: 2
      How about this - you are saying that it is justifiable for terrorists to kill civilians because the country they happen to be in supports something they disagree with. Spam may be a large problem (although it's easy to block for me, since most of it is in Chinese for some reason), but that does not justify harming people who just happen to have the misfortune of being near a spammer.

      How about not. That's not at all what I said. There is no equivalency between killing and being listed in SPEWS and being blocked by network operators who choose to use SPEWS.

      Are you having to make up things you think I said so you have a better chance to argue against it?

      I'm just going to make this point: blocking non-spammers will only hurt SPEWS in the long-run, as its current effectiveness is based on the majority of people using it. As people find that they either have to turn to some other source than SPEWS or accept that occasionally people they must communicate with cannot send them email without a whitelist, they will not think "yay, we're helping to eliminate spammers!" and instead think "Goddamned broken SPEWS thing accidently blocking valid people - remove it!"

      Just imagine if the majority of people were to realize what the problem with spam really is, and join in the movement of SPEWS. The end result would be, of course, some period of time where a few people cannot communicate. But it would also force the minority who are indirectly supporting spam (by directly supporting the ISP that harbors spammers) to find a better ISP. Then the bad ISPs will go out of business or if they get a clue quick enough, change their behaviour, kick out the spammers, and be a good ISP everyone can use.

      Eventually, SPEWS will have caused enough problems that no one (except a small core of vigilantes on their own servers) will use it. And then it will have no power over the vast majority of spammers. SPEWS would be more effective if it only blocked spammers and many people used it - it could help make spam an ineffective method of contacting people. Force the cost of spamming up, not the cost of happening to use an ISP that hosts a spammer. All you're going to do is create things like this, and force people against you. SPEWS does not help the cause by being a vigilante and trying to force people who have no buisness with the spammer to take action against their ISP.

      SPEWS would be totally INEFFECTIVE if it just blocked the spammers. The goal would not be met. Spammers would continue to abuse mail servers unabated. ISPs would continue to provide services to them.

      Spammers survive on less than %0.01 response rate. When those who won't respond anyway are blocking the spammers, they will still get most of their response from a few idiots who are influenced by them. They will still make enough money to continue spamming. They will still keep spamming their full lists and abusing everyone else's mail servers. They will still be increasing the costs of running mail servers. They will still be denying email recipients the full facility of their mail server resources.

      For all I know, you may be the person who set up that site.

      And those people do have a business with spammer, in an indirect way. They are in effect saying to their ISP, "we don't care if you keep the spammers, we'll keep sending you money for services and you can stay in business". If an ISP is only getting 1% of its revenue from spammers, then it will only take the fear of losing more than 1% of legitimate customers for the ISP to do the right thing. And those few that never will (I know of a couple which are 100% listed in SPEWS), need to go out of business.

      Besides - which is easier: getting the ISP to drop the spammers acount, or telling people who want to communicate with you to stop using SPEWS? The path of least resistance is likely to be followed... which may include "ok, we'll just ignore the bozos using SPEWS."

      People are certainly free to NOT use SPEWS. One result of that decision is a lot more spam. If you genuinely think just blocking spammers alone is going to work, then start your own blocklist service which does just that. Then see what it's like when you find that you're playing whack-a-mole with spammers changing addresses within the same ISP all the time. See what it's like tracking a moving target. See what it's like when some spammers sue you for claiming they are spammers when all they were doing is giving everyone a chance to opt out of their mailings. If your idea is the right one, then why is no one doing it?

      --
      now we need to go OSS in diesel cars
    26. Re:SPEWS by Doomdark · · Score: 1
      Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech (although the actual ... ).
      Infringement of free speech happens when the decision is based on what the content is.

      Actually, you are missing the actual active ingredient of infringement. That would be that censoring is done by a third party. Whether that is the case depends on who is using the content based filtering.

      Perhaps this is just nitpicking, but if end-user is the one who decided to use such a filter, no infringement can occur. Although people are free to speak their mind, no one is forced to listen to them.

      What is not nitpicking, however, is pointing out that in this respect both content and sender-based filtering done by a third party WOULD indeed constitute an infringement (if done by gov't etc. etc.). There is no difference whatsoever.

      --
      I like paying taxes. With them I buy civilization -- Oliver Wendell Holmes
    27. Re:SPEWS by Skapare · · Score: 1

      Of course there is no legal infringement if the party doing the censoring is the recipient. Just as you say, no one ie required to listen (read, etc). Although I am not required to, I personally choose to not use content based filtering because of both that principle the First Amendment is based on, and the fact that I really don't care what the content is.

      And yes, if the government were to get involved in this process, it could end up (depending on what they do) being an infringement. That's why I won't depend on them doing it. It will be very hard for the government to make decisions about what is spam. The one safe-harbor decision they could make, that spam is defined as unsolicited bulk email, goes against the wishes of one of the patrons of Congress: the DMA. I'm not counting on the government ever doing anything right in this regard, and I fear them doing anything as it probably will make things worse.

      --
      now we need to go OSS in diesel cars
    28. Re:SPEWS by haraldm · · Score: 1
      Content based filtering also is a direct violation of the principles of the US First Amendment right to free speech

      Well - sort of. The 1st Amendment may give you the right to speak but it does not make me liable to listen. I still have the choice to listen or not. I can filter locally whatever I want, based on whatever criteria.

      Sender and ISP based filtering is more of a 1st Amendment violation because it affects innocents who happen to be the same ISPs' customers. Like filtering *@hotmail.com or something.

      Besides, about 80% of internet users are not US citizens, and thus not 1st Amendment "users". Besides (2), mail headers are easily forged, and there are so many open relays around that sender/ISP based filtering may be very unreliable. This is where RBLs come in, and then you are very close to Spamassassin again.

      --
      open (SIG, "</dev/zero"); $sig = <SIG>; close SIG;
    29. Re:SPEWS by haraldm · · Score: 1
      No, it is not your fault to find yourself a customer of an ISP harboring a spammer, if that was not going on when you started with the ISP. But, the customer can choose to move (at least their mail sending operation) to another ISP, and bill (or sue) the guilty ISP for the costs of doing this.

      Nonsense. 99% of all internet customers are far to clueless to understand what happens. Some of their e-mails don't reach the recipient. They will never understand why.

      Apart from that, your philosophy forces people who just want internet access for homebanking or e-mail to fight a war which isn't theirs, and which they don't understand. While this may be an option for the current US government it definitely isn't an option for long-term internet users who are used to tolerance and freedom.

      --
      open (SIG, "</dev/zero"); $sig = <SIG>; close SIG;
    30. Re:SPEWS by Anonymous Coward · · Score: 0

      Yeah right.

      That's where the tinpot dictator comment comes from (the way people conduct themselves on that particular Usenet group).

      Anyways, going on n.a.n-a.e is not very fun, even if you're not the ISP responsible but just a client of UUNet (or whatever most hated target of the week is). You'll get flamed and told you might get off the list, or that you should get another provider.

      SPEWS is an example of blacklists at their worst. A couple of lists that I can recommend are Spamcop (very automated, and listings actually do age out, unlike what is claimed on spews.org) and Spamhaus, where you have an organization you can actually correspond with (though I believe a few people who are involved with Spews are also involved with Spamhaus).

    31. Re:SPEWS by Doomdark · · Score: 1
      Ok, perhaps I misunderstood what you were saying... did you not claim that sender-based blocking is ok (or less evil) than content based one, in regards to free speech issues? I don't think that really is true; I don't think it really matters what the blocking (aka censoring) is based on, but more on who is doing blocking.

      One could also argue that discriminating based on origin of communication is "more evil" than one based on content... After all, there are restrictions on free speech, but all are indeed based on content (the tired old "don't yell 'fire' at a crowded theatre" etc), and none restrict based on who is doing talking (except for individual court orders... which are interesting exceptions). I don't think there's huge difference one way or the other, but YMMV.

      Of course this has nothing do with effectiveness of various blocking methods, which was your main point. :-)

      --
      I like paying taxes. With them I buy civilization -- Oliver Wendell Holmes
    32. Re:SPEWS by Skapare · · Score: 1

      If I restrict communication based on who sends it by having judged them on the basis of the content of what they have sent or what I feel they might send, then I would agree that is a form of content based censorship (although practiced by the recipient, so not technically subject to the free speech clause of the US Constitution First Amendment). And, yes, I have done this before in a few cases.

      However, if I make that restriction on the basis of how they have selected who to send a communication to, then I cannot necessarily agree. Of course this is a difficult judgement here. What if all that someone is doing is sending me a private reply to a posting I have made? Of course, that reply having been on topic would not cause me to list them, even if I disagree with what they say (that happens a lot). However, if they scrape my address from Slashdot and send me a private message on how I might enlarge my penis, then (except for a few articles on Slashdot :-) that would be off topic. Now here is where the semantics play an important part. I might block them, not for what they say specifically, but instead for having chosen the wrong audience for what they have to say. This is unsolicited.

      Now, the next issue is whether it is bulk or not. What if they chose to do so just to one recipient and no others? That wouldn't be bulk. And because it is not bulk, it would not have nearly as much of a scaling problem as bulk email does. So there is less motivation to block them because it's not something that has the potential to be a problem. Of course if they were sending it to me once each day, I'd get annoyed, call it bulk for sure, and block them (and report them to their ISP if they don't stop after getting a couple bounces). If all they send is one, I really don't know if it is bulk. However, it usually is bulk in these cases (still keep in mind the judgement is on how they have chosen the recipients and not specifically the content).

      Published lists like SPEWS have very good tools to measure both the unsolicited and bulk aspect of a given sender. They have some number of "spam trap" addresses (maybe hundreds, maybe thousands; I don't know). These addresses are placed where spammers often scrape their lists, such as incidental locations in web sites and public forum postings. I put a non-existant address on one of my Slashdot postings a long time ago and it got spammed. So spammers even scrape their addresses from here (probably the trolls, don't you think). With so many of these spam trap addresses, SPEWS and other lists can measure whether a given sender is indeed meeting the unsolicited bulk email metric. Then list them.

      Listing a spammer isn't the big controversy for SPEWS in most cases. The controversy of SPEWS is the practice of later on adding to the listing gradually expanding address ranges of the ISP. That does affect customers who aren't spammers. When seen as a content based issue, that doesn't seem right. But there is a behaviour aspect involved here. Would you sign up for service with an ISP that is a known abuser of the internet? Would you sign up for service with an ISP that knowingly has one or more customers that abuse the internet? I wouldn't, and certain members of the anti-spam crowd don't think others, should, either. But consider why an ISP allows a spamming customer to stay on ... money. As long as they believe that they will make more money by having spammers than by not having spammers, and as long as their investors (vulture capitalists) are overriding any moral sense the ISP operators might have about it, they won't disconnect that spammer. Since even with the spammer blocked, the cost of the abuse is real for the intended recipient's mail server, this becomes a take-down issue. But with an ISP reacting only based on the money involved, then by that ISP's decision, money becomes the necessary means to get them to take the spammers down. One potential tactic would be to tell all the customers of that ISP that they are harboring a spammer. But that would itself be spam, and so it is ruled out. Another is to advertise on various media. But not only is this expensive, it also depends on the good will of those customers, who themselves might be influenced by vulture capitalists, or see a specific cost in breaking a contract (there being no degraded service, breaking contracts is much harder to do).

      That leaves the (controversial) tactic of blocking the ISP's customer addresses. That gives the customers real incentive to do something about it (cost vs. cost), the ability to address the concerns of the vulture capitalists (make more money), and even the opportunity to break contracts (the ISP's service is now degraded, and since it is not beyond their control, their "beyond our control" contract clauses will not apply). If the customers raise the issue with the ISP, demand the spammers be kicked out, and make it clear the ISP will lose the revenue from the legitimate customers if not, then all it takes is enough customers and even a greedy ISP will comply. The risk is if the ISP already gets more than 50% from spammers, in which case the situation probably cannot be resolved and the ISP will eventually be 100% blocked as a few are now.

      SPEWS is measuring behaviour based on factors such as sending unsolicited bulk email (and not even commercial ... politicians and non-profit organizations have been listed as well), and for supporting in some way those who do that. This is why I feel using SPEWS to block email on my servers, and recommending it to my clients (some do, some don't), does not violate the principles of free speech (which the US Constitution First Amendment is based on).

      --
      now we need to go OSS in diesel cars
    33. Re:SPEWS by Skapare · · Score: 1

      There isn't much we can do about clueless customers, besides educate them. But even clueless customers might discover that mail gets through more with one ISP than with another. They might, for example, ask a friend who uses a different ISP to try sending for them. When it works, they ask what ISP they are using. Then the clueless customer might switch to the other ISP. That will have the desired effect even if that customer still remains clueless.

      How is that any different than an ISP running a poorly configured and/or programmed mail server that occaisionally loses some mail? At least with blocking, there is going to be something come back with a reason (even if it is hard to reach because of header clutter).

      The spam war really is theirs, unless they have no regard for how much they spend for internet access. Spammers pounding away on other mail servers causes the costs to go up. Small ISPs have to buy a new mail server sooner than they otherwise should. Large ISPs have to have perhaps twice as many mail servers as they otherwise should. Postmasters and administrators have to do more work to manage more servers and the increased jam ups that happen with spam. And in many cases that have even reached the news media, mail servers at even large ISPs have been so swamped with spam they crash, denying total mail services for all the customers.

      The government is one of the entities I would recommend to not use SPEWS for blocking, at least on public facing email addresses.

      --
      now we need to go OSS in diesel cars
    34. Re:SPEWS by Skapare · · Score: 1

      I guess we'll have to agree to disagree on this. I have constructed my argument why it is the other way around elsewhere in this thread. It would be just the same argument again here. I think you don't understand why the 1st Amendment exists. Of course if the government restricted the right to speak without considering what the content is, the argument may be raised it is such a violation. But the government can also defend that it is not in certain cases, such as prohibiting the posting of signs in the middle of a highway. The foundation of the 1st Amendment comes from the idea that governments of the past have suppressed speech on the basis of what is said, and this they wanted to make sure would not happen in the United States. That is the principle I refer to.

      --
      now we need to go OSS in diesel cars
    35. Re:SPEWS by Skapare · · Score: 1

      This morning I found in my logs a large number of spam attempts that hit my mail servers overnight. They all came from host database.datacommarketing.com[65.242.117.12] which is within a huge UUNET address block. Many of these attempts were made to a variety of standard role accounts in several domains I own or run. The rest were made to what look like message-ids (they look like email addresses and typically fool address scrapers) in some of my domains. I did not get any content whatsoever, so I cannot say what the content is. But based on the selection of addresses, there is no doubt whatsoever that this was spam using the unsolicited bulk email definition (UBE).

      None were delivered because SPEWS already has them listed, and I do use SPEWS to block them.

      I then scanned the SPEWS record for all addresses with that huge UUNET address block. There is a very large number of current ongoing spamming activity from many addresses in that block. If UUNET is going to allow spammers to impose a costly burden on other mail server operators, then it is only fair that many of these mail server operators get together to push that cost burden back on the ISP. That is what SPEWS is all about.

      --
      now we need to go OSS in diesel cars
  17. In principle, yes, in practice, no. by zabieru · · Score: 2, Insightful

    Sure. Assuming Schneier has the public keys of all his subscribers, AND the processing power to encrypt everything in a reasonable span of time. That second is a big if, considering the number of subscribers. It would be possible to use a symmetric algorithm and include the key in the message, but while most readers would have the knowledge to decrypt it, they would likely not have the software to do so easily, and so it would be much more convenient for them to just get the announcement and go check the website, as opposed to spending half and hour trying to find and configure software.

    1. Re:In principle, yes, in practice, no. by atam · · Score: 1

      Is it possible to encrypt the Crypto-Gram article with Schneier's private key, then every one receiving it just use Schneier's public key to decrypt it?

    2. Re:In principle, yes, in practice, no. by RedWizzard · · Score: 1

      Schneier could encrypt it with his own private key, which would allow anyone with access to his public key to decrypt it. This would also prove that the email is from him, provided you could trust the public key as being his.

    3. Re:In principle, yes, in practice, no. by Rabidbunnylover · · Score: 1

      I think you're missing the point. It's not necessary to actually securely encrypt the letter. It just needs to be modified enough so that it doesn't set off spam alerts. Hell, "encrypt" it using ROT-13.

    4. Re:In principle, yes, in practice, no. by zabieru · · Score: 1

      Sure, which is great for *nixers. I'll bet the majority of Schneier's subscribers are on Windows machines and will need to go find/write something to ROT13 it. Or do it by hand. Both of which are more difficult than just finding it on the Web.

    5. Re:In principle, yes, in practice, no. by zabieru · · Score: 1

      Cryptographically, yes. I'll even assume for purposes of discussion that this can be done securely within RSA, or else by using DSA. I'm not sure of either of those, but we'll go with it. I do know that it's not been implemented in PGP/GPG. Which makes the whole thing a moot point, since that's what most folks use, and so if they can't it becomes more convenient to get it off the web.

    6. Re:In principle, yes, in practice, no. by Mr.+X · · Score: 2, Informative

      This is one of the key features of PGP/GPG.. It's called signing a message, and there is an option to encode the entire message and not just its hash.

    7. Re:In principle, yes, in practice, no. by MortimerK · · Score: 1
      I'll even assume for purposes of discussion that this can be done securely within RSA, or else by using DSA.

      Just FYI -- it's quite possible with RSA but not with DSA. DSA can be used for digital signatures only, not for general encryption.

    8. Re:In principle, yes, in practice, no. by kasperd · · Score: 1

      Is it possible to encrypt the Crypto-Gram article with Schneier's private key

      Then it would not be an encryption but a signature. A poorly implemented signature that is. I'd rather use a more decent signature implementation like hashing the message and attaching a signature of the hash. I think signatures is one of the tools we can use against spam. In particular spam with forged sender address.

      --

      Do you care about the security of your wireless mouse?
    9. Re:In principle, yes, in practice, no. by Anonymous Coward · · Score: 0

      > I'll bet the majority of Schneier's subscribers
      > are on Windows machines

      ROTFL!

      Which planet are you from, sounds... interesting.

    10. Re:In principle, yes, in practice, no. by BlueUnderwear · · Score: 3, Insightful
      Then it would not be an encryption but a signature.

      You are right that it would not be encryption in the sense that it doesn't protect privacy of the message (indeed, in order to read the message, you only need Bruce's public key, which is indeed, uhmm, public...).

      However, it would still fulfull the goal of evading spamassassin, because, as far as I know, spam assassin is not yet smart enough to figure out that the message has been "encrypted" with Bruce's private key, and to fetch the public key from the Bruce's webserver to decrypt it.

      But then again, rot13 would probably be enough to evade spamassassin too... as long as you don't mispell inventive as ivntenive that is...

      --
      Say no to software patents.
    11. Re:In principle, yes, in practice, no. by X00M · · Score: 1

      This discussion shows another need for Encryption needs to become mainstream and i'd say less geeky
      Anyone have any good ideas on how to do this?

      Xoom

    12. Re:In principle, yes, in practice, no. by Anonymous Coward · · Score: 0

      Just FYI -- it's quite possible with RSA but not with DSA. DSA can be used for digital signatures only, not for general encryption.


      Yes, but there's something called El-Gamal encryption that's as good as DSA (in fact it's DSA running "backwards" so it encrypts instead of signing). GPG does El-Gamal.
    13. Re:In principle, yes, in practice, no. by zabieru · · Score: 1

      I stand corrected. I'd always assumed that option encrypted and signed, so never bothered, since that's basically redundant.

    14. Re:In principle, yes, in practice, no. by zabieru · · Score: 1

      Sure, I know that seems laughable to an open-source zealot. However, remember that most slashdot readers are on Windows machines. Also remember that a lot of people would prefer to use *nix, but for whatever reason it's not practical (they're at work, they need to do stuff for work and so need Windows apps, they like to play a lot of games that haven't been ported...)

    15. Re:In principle, yes, in practice, no. by Anonymous Coward · · Score: 0

      as long as you don't mispell inventive as ivntenive that is
      That's funnier than it looks. The last three letters ROT13ed are Latin for man, as in the root of virility.

    16. Re:In principle, yes, in practice, no. by Anonymous Coward · · Score: 0
      Yeah, "viagra man", yeah!

      Nice boat, btw!

  18. er... by inode_buddha · · Score: 0

    Thanks, but I already knew that (the hard way). Must be something to do with enlarged things....

    --
    C|N>K
  19. False Positivies by Anonymous Coward · · Score: 0

    Spamassassin also marks the slashdot ( text only ) and freshmeat ( html ) newsletters as SPAM.

  20. Spam Assassin does not block spam by Anonymous Coward · · Score: 1, Insightful

    Spam Assassin does not block spam. It just marks it as spam so you can do your own sorting/filtering with your email client. Anyone doing this should periodically review their "spam bucket" where they route such spam-marked articles.

  21. Whitelist, header matchups and viruses by jasonrocks · · Score: 1

    Unfortunately, I have executed a virus and now get quite a few emails trying to get me to run the program again because it uses search techniques to tell who my friends are and sends me a message from one of them with a wrong IP address. Fortunately, this virus wasn't written by a spamme (to my knowledge).


    Unfortunately, informing individuals that their system has been compromised can be a very time consuming process. Does anyone have any suggestions.
    --

    void
    1. Re:Whitelist, header matchups and viruses by kasperd · · Score: 2, Insightful

      Unfortunately, I have executed a virus

      We often see viruses and spam being send with spoofed sender address, and some spammers are clever enough to even use sender addresses from the same domain, which would be more likely to be on the whitelist. It would be nice to combine the whitelist with signature checking, if you know the senders public key, you simply filter anything unsigned.

      --

      Do you care about the security of your wireless mouse?
  22. darwinist approach by eimi · · Score: 1

    if the spam blockers will come as standard on ISP-level, how much time spam will need to adapt? I fear time, when my email box will be flooded with hundreds letters, each mimicking [slashdot] stories :) according to saint thomas, in this universe every essentia needs its ente. even spam.

  23. economics by dubiousmike · · Score: 1

    I don't intend to alter my content to accommodate spam filters.

    Some of us aren't so lucky. The rest of us actually need eyeballs on our newsletters and try to test our content through filters before sending it out. I am consistantly amazed at the little things that flag my newsletter as spam.

  24. Can someone run it through SpamAssassin? by Leeji · · Score: 2, Informative

    When you run SpamAssassin in test mode, it tells you what rules got hit. You can also look at the headers in "Spam-Tagged" email to see what rules got hit. I looked for "Spam Testing" pages on the 'net, but had no luck.

    Could someone run the Crypto newsletter through SA to find out what cased its evaluation?

    As an aside, Counterpane could have done this to find out what the problem was, too. Not that they should have to, but they could have.

    --
    It all goes downhill from first post ...
    1. Re:Can someone run it through SpamAssassin? by gleam · · Score: 1

      i ran the text of the newsletter through spamassassin, but not the actual email newsletter itself (i'm not subscribed).

      as a result, it'll look different than someone subscribed to the list, since spamassassin does rely a bit on the headers, not just the text:

      Oh, and as a side note, when i tried to paste this, unedited, slashdot spat the following at me:

      Lameness filter encountered. Post aborted!
      Reason: Please use fewer 'junk' characters.

      the irony is thick. as is, of course, the irony that i could completely bypass it by formatting my message as "code". Now, back to the show:

      X-Spam-Status: Yes, hits=5.1 required=5.0
      tests=BALANCE_FOR_LONG_20K,BALANCE_FOR_LONG_40K,DA TE_MISSING,
      FROM_MISSING,MISSING_HEADERS,NORMAL_HTTP_TO_IP,OPT _IN,
      SPAM_PHRASE_01_02,SUBJ_MISSING,SUPERLONG_LINE,US_D OLLARS_2,
      US_DOLLARS_4
      version=2.44
      X-Spam-Flag: YES
      X-Spam-Level: *****
      X-Spam-Checker-Version: SpamAssassin 2.44 (1.115.2.24-2003-01-30-exp)

      SPAM: This mail is probably spam. The original message has been altered
      SPAM: so you can recognise or block similar unwanted mail in future.
      SPAM: See http://spamassassin.org/tag/ for more details.
      SPAM:
      SPAM: Content analysis details: (5.10 hits, 5 required)
      SPAM: FROM_MISSING (-0.0 points) Missing From: header
      SPAM: DATE_MISSING (0.8 points) Missing Date: header
      SPAM: SUBJ_MISSING (0.3 points) Subject: is empty or missing
      SPAM: OPT_IN (1.5 points) BODY: Talks about opting in
      SPAM: US_DOLLARS_4 (0.4 points) BODY: Nigerian scam key phrase ($NNN.N m/USDNNN.N m/US$NN.N m)
      SPAM: US_DOLLARS_2 (0.1 points) BODY: Nigerian scam key phrase ($NNN.N m/USDNNN.N m/US$NN.N m)
      SPAM: BALANCE_FOR_LONG_20K (-0.7 points) BODY: Message text is over 20K in size
      SPAM: BALANCE_FOR_LONG_40K (-0.1 points) BODY: Message text is over 40K in size
      SPAM: SPAM_PHRASE_01_02 (0.5 points) BODY: Spam phrases score is 01 to 02 (low)
      SPAM: [score: 1]
      SPAM: SUPERLONG_LINE (0.0 points) BODY: Contains a line >=199 characters long
      SPAM: NORMAL_HTTP_TO_IP (1.3 points) URI: Uses a dotted-decimal IP address in URL
      SPAM: MISSING_HEADERS (1.0 points) Missing To: header

      hope this helps,
      gleam

      --
      this .sig is not a .sig.
    2. Re:Can someone run it through SpamAssassin? by MavEtJu · · Score: 2, Informative


      SPAM: FROM_MISSING (-0.0 points) Missing From: header
      SPAM: DATE_MISSING (0.8 points) Missing Date: header
      SPAM: SUBJ_MISSING (0.3 points) Subject: is empty or missing
      SPAM: MISSING_HEADERS (1.0 points) Missing To: header


      See this posting for one with the headers, which shows that SpamAssassin doesn't tag it as spam anyway.

      --
      bash$ :(){ :|:&};:
    3. Re:Can someone run it through SpamAssassin? by gleam · · Score: 1

      yeah, which is pretty much what I noticed too.. of course, spamassassin has a configurable threshold, and configurable weights, so maybe one particular configuration of spamassassin will mark it as junk..

      but it looked like it would have made it to my inbox with no problems.

      -gleam

      --
      this .sig is not a .sig.
    4. Re:Can someone run it through SpamAssassin? by Zeinfeld · · Score: 1
      SPAM: OPT_IN (1.5 points) BODY: Talks about opting in

      Go read the DNSSEC mailing list, there has been a considerable amount of discussion about OPTIN.

      Or read any of the privacy mailing lists where the term opt in is used in the exact same context

      The big problem with developing a SPAM solution is that nobody wants to hear any solutions, start describing something and they will interrupt your first sentence to tell you their idea. Then when you explain that the idea is not new and has severe drawbacks they assert that it works for them so it should be good enough for anyone.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
  25. Maximum size of spam by waynemcdougall · · Score: 2, Interesting
    Spam tends to be short. The shorter the spam, the more messages they can put through. So spammers would be loathe to add 21 pages of text to their spam.

    I have
    Const maxspamsize = 42695
    in my spam filter - I've only receive one piece of spam larger than than in the last 12 months (a giant promotion for a Korean trade show). It speeds up my spam filter processing and lets large newsletters (with false triggers like this) through without a problem.

    --
    Recycle PCs and build a wireless community network www.hillsborough.org.nz
    1. Re:Maximum size of spam by 1u3hr · · Score: 1
      It speeds up my spam filter processing and lets large newsletters (with false triggers like this) through without a problem.

      Yes, it works now. But if your criterion were widely used, spammers would just bulk up their spam. (Now they often add some gibberish to the end, or even within HTML tags, so that it's not normally visible, and breaks up trigger words.)

    2. Re:Maximum size of spam by waynemcdougall · · Score: 3, Interesting

      My point remains valid. Because there is a direct cost to the spammer to adapt.

      If they bulk up their spam that's going to slow them down, increase their costs (even if bandwidth costs aren't going to be passed back to them now, the more they use, the more visible they become). They become more visible.

      Or they continue on their way. The reality is that they concentrate on the easy targets - you and I will never purchase their services so people taking this approach aren't really in their target audience anyway. I know this is (surprisingly) less true than one might think. Spammers do work to overcome basic obstacles, but that adds more costs and time - they don't work hard to avoid tar pits, because there are so few of them.

      So I still see it as a win...large emails are very unlikely to be spam. If that changes, well so be it, but that will hurt the spammers. In the meantime I reap the benefit of fewer false positives and faster spam filtering.

      Final comment - over the last six months I've seen spam get slightly larger (from about 32k peak size to about 45k peak size). But I haven't been analysing for any trends - just the outliers.

      --
      Recycle PCs and build a wireless community network www.hillsborough.org.nz
  26. In other news.. by Anonymous Coward · · Score: 0

    President Bush was walking today and bent over and tied his shoe in a double knot... who cares

  27. Why newsletters at all? by PenguinOpus · · Score: 1, Offtopic

    I realize its tradition and there's a certain extra bit of personal ownership involved in having a copy of the contents arrive entirely in my mailbox, but... Why do newsletters waste the bandwidth of sending out copies of the content to everyone on the list? A small email with a single link to the contents of the new newsletter would work just as well and only those people that read it would spend the bandwidth. In addition, the author could then scrutinize the logs and see what links and what sections generated interest and perhaps better serve the (sometimes not fully understood) audience.

    1. Re:Why newsletters at all? by Anonymous Coward · · Score: 0

      His newsletter reads like a Slashdot digest.

  28. yea by Anonymous Coward · · Score: 0

    nigger

  29. mod this up by Anonymous Coward · · Score: 0

    its funny laugh

  30. Same thing happened to IACR newsletter in 1991 by SiliconEntity · · Score: 1

    The Risks digest reported in 1991 that the email newsletter from the International Association for Cryptographic Research was being blocked by spam filters. One of the IACR board members was a crypto expert with the unfortunate name of Don Beaver. And there were some references to "hardcore bits" and LaTex. It was all too much for the filters.

    1. Re:Same thing happened to IACR newsletter in 1991 by Zeinfeld · · Score: 1
      The Risks digest reported in 1991 that the email newsletter from the International Association for Cryptographic Research was being blocked by spam filters.

      I somehow doubt it, Canter and Segal didn't send out their mass mailling till 1993.

      One of the IACR board members was a crypto expert with the unfortunate name of Don Beaver. And there were some references to "hardcore bits" and LaTex. It was all too much for the filters.

      I think you are describing a censorship filter rather than a spam filter there.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
  31. (Bayesian Stat Filters)++ by McG33k · · Score: 1

    Well, it gets through our spam filter just fine. It got a weight of 33% spam which is way below our threshold. I guess spam assassin and other such thingy-whuzzits need to relax a little. Nobody has sent this cryptogram off to razor, have they? That would cause a headache!

    --g33k

    1. Re:(Bayesian Stat Filters)++ by Treylis · · Score: 1

      In fact, somebody did send it off to Razor. That's a large part of the problem.

  32. Did anyone see Crypto Snake Oil? by jasonrocks · · Score: 1

    At the bottom of his article is mentioned a blurb about a company that manufactures software that uses 1 million bit encryption. Yup, it's snake oil, but the problem is that they're making money,
    They build an alternate reality where every cryptographic algorithm has been broken, and the only thing left is their own system. "The weakening of public crypto systems commenced in 1997. First it was the 40-bit key, a few months later the 48-bit key, followed by the 56-bit key, and later the 512 bit has been broken..." What are they talking about? Would you trust a cryptographer who didn't know the difference between symmetric and public-key cryptography? "Our technology... is the only unbreakable encryption commercially available." The company's founder quoted in a news article: "All other encryption methods have been compromised in the last five to six years." Maybe in their alternate reality, but not in the one we live in.
    They've got pseudo-scientific gobbledygook galore, including paragraphs like this: "Stated simply, the content of the message is not sent with the encrypted data. Rather, the encrypted data consists of pointers to locations within a virtual matrix, a large (infinitely large in concept), continuously changing array of values." I just love stuff like this. It almost just barely makes sense. It's as if someone took a cryptography book, had it machine-translated from language to language to language, and then tried to write similar-sounding text. Some of the words and phrases are scientific, but the paragraph makes no sense.
    THE WORST PART IS THE FOLLOWING:
    According to a press release on their Web site, the U.S. Department of Labor recently gave them $4M. Various smaller companies are supposedly using this stuff. SC Magazine gave them a five-star rating, for goodness' sake! I am amazed at the sheer stubbornness that can be exhibited by a company that simply refuses to accept reality.

    --

    void
  33. Re:The problem with blocking IP adresses is... by Mr+Bill · · Score: 2, Interesting

    So blocking untrusted servers doesn't make email unreliable? I find that very hard to believe. Considering that most of the time it is Net blocks that are blocked, not just individual IP addresses.

    blocking IP addresses is also open to abuse... If I had a grudge against an ISP, I could fake some SPAM headers and send it to any of the IP blockers. Maybe send several copies from different accounts. Getting an IP listed is usually easier than getting it removed, so in the mean time many legitimate emails are being blocked...

    I believe you have to attack the root of the problem, and that is stopping the SPAM at the origin. This is probably the more difficult approach, but it is the only one that will avoid dropping legitimate mail.

  34. spamassassin's "known mailing list" rules by perlchild · · Score: 1

    With the readership of the crypto-gram, why isn't it just in the "known mailing list" list?
    I'm sure it would save a lot of trouble to everyone.

  35. Unless they use netscape. by Anonymous Coward · · Score: 0

    Of course, if they use Netscape, ROT13 is a right-click away.

  36. Awesome sig OFFTOPIC, I know by sawilson · · Score: 1

    Do you remember where that quote is from?
    It reminds me of another good one:

    Arrogance is compensation for a lack of intelligence.

    that I think was the same person, but I can't seem
    to find either of those quotes.

  37. spam filters are the wrong solution by Anonymous Coward · · Score: 0

    Spam filters are the wrong way to go, because of the false positive problem. A better way is to charge the sender a penny for every email they send.

  38. A possible solution to the spam problem... by kcbrown · · Score: 4, Interesting
    Right now everyone is forced to accept email connections from anyone who sends email because it's not possible to tell ahead of time whether or not the connection is coming from someone who is reliable, right? And spammers take advantage of this by sending millions of messages from open relays. Blocking that is a virtual impossibility because which relays are open changes over time.

    The first inclination one has would be to suggest that everyone close their open relays. But this depends on people doing the right thing all the time, and has proven ineffective.

    Fortunately, there's another way.

    Right now, everyone who receives mail has to listen to everyone who tries to connect. The problem is how do you separate the wheat from the chaff?

    The solution is to take advantage of the information SMTP and TCP/IP give you when a connection is established. The fact that you're receiving a connection gives you the address of the sender. And during an SMTP transaction, one of the SMTP commands (the MAIL FROM command) gives you the domain of the email's sender, e.g. "MAIL FROM slashdot@sysexperts.com".

    When you're sending email to someone else, you do so by looking up the MX records for their domain, which tells you which systems are responsible for receiving email for that domain. This gives us a possible answer to the spam problem.

    Suppose instead of blindly accepting email from everyone, you were to take the domain given to you by the MAIL FROM command, look up the MXes for that domain, and reject the email connection if the IP address of the sender doesn't match one of the domain's MXes?

    Now, suddenly, you would end up rejecting email sent from every unauthorized relay, because the owner of the domain can make any system that is allowed to send email on behalf of his domain into an MX (and, if he doesn't want that system to be used for delivering email, then he simply makes such systems the lowest priority MXes in the list and blocks outside port 25 connections to them ... something he's probably doing anyway).

    Suddenly, the only systems that spammers can send email from are systems that they legitimately control and that are defined as MXes for a domain they control. Suddenly, spammers have to set up and maintain their own domains and their own boxes. The costs have just become a lot higher, which will get rid of most of the spammers.

    And suddenly, blocking spam becomes orders of magnitude easier -- you only have to deal with spammers who have decided to pay the (now much higher) price for sending spam and who cannot use someone else's system to do their dirty work without permission.

    --
    Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
    1. Re:A possible solution to the spam problem... by kcbrown · · Score: 1
      This will work for most senders, but it is entirely possible that the addresses sending mail are different to those which can receive mail.

      I addressed this problem in the original proposal: set up the systems that are sending mail as MXes anyway (at low priority, so all connections will try to go to the valid MXes first), and drop inbound port 25 connections to your systems that are email senders. You're dropping those connections anyway if you've got any brains at all (since your explicit intention is to not receive inbound email to those boxes from the outside world), so it's not like you have to make any drastic changes to the way you do things.

      --
      Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
    2. Re:A possible solution to the spam problem... by Pharmboy · · Score: 1

      This will work for most senders, but it is entirely possible that the addresses sending mail are different to those which can receive mail.

      Correct. We are one of those. We have our own mail server (rented rack in Dallas) but use Speakeasy SMTP servers (Our SDSL provider located in NY) for outgoing opt-in newsletters. We don't HAVE to do it this way, but for technical reasons, its faster and easier for us. And it doesn't count toward our bandwidth on the rack.

      Another server we have, on another SDSL line was blacklisted because it was SDSL, effectively shuting down all mail services. The server has been up on that IP for years, and averaged 10 to 50 emails per day, so we surely were not spammers. Our SMTP server are not open relays. It was blacklisted purely because it was on SDSL ip ranges.

      --
      Tequila: It's not just for breakfast anymore!
    3. Re:A possible solution to the spam problem... by Malc · · Score: 1

      Your approach is too simplistic. I've been using my yahoo.com email address for years, but never once have I relayed through a Yahoo SMTP server. In fact, a previous ISP of mine wouldn't even allow that. I do not want to be forced to use an email address with ISPs domain in it either.

      I have a static IP at home hosting a personal domain and mail server. The hostname for my static IP doesn't match that of my domain, although it is an MX. Thus the email addresses from my domain don't match the domain of the IP address (is that relevant??). More troubling for me though is that I've configured my MTA (Exim) to do something along the lines of what you said. I've seen in my logs messages being rejected. For example, an old friend tried to contact me through FriendsReunited.co.uk - my mail server rejected the message because it couldn't verify the email address of the sender, which had a different domain.

    4. Re:A possible solution to the spam problem... by 0x0d0a · · Score: 1

      Yes, this approach is well known about, and used by some people. It's also a pain in the ass for everyone else.

    5. Re:A possible solution to the spam problem... by kcbrown · · Score: 1
      Your approach is too simplistic. I've been using my yahoo.com email address for years, but never once have I relayed through a Yahoo SMTP server. In fact, a previous ISP of mine wouldn't even allow that. I do not want to be forced to use an email address with ISPs domain in it either.

      Then use your own domain! An ISP that won't let you send outbound email from your system is one that should be dumped, IMO. But let's assume that they won't, and they force you to use their system to relay email.

      The email address that appears in the "From:" line is under your control. The email address that appears in the SMTP "MAIL FROM" command is under the ISP's control. They can easily set it up so that whenever you send email through their SMTP relay, the address that appears in the "MAIL FROM" command is that of your email account with the ISP (and can be gotten a couple of ways, including the authentication method you're using to the IP address you're using). They can also set it up so that what appears in the "MAIL FROM" command is a constant (e.g., "MAIL FROM mta@yourisp.net"). But no matter how you look at it, the problem you're talking about won't be an issue.

      I have a static IP at home hosting a personal domain and mail server. The hostname for my static IP doesn't match that of my domain, although it is an MX. Thus the email addresses from my domain don't match the domain of the IP address (is that relevant??).

      If you mean that the reverse lookup of your static IP address won't resolve to the domain you're using then no, that won't matter at all. That's something I tried to be very careful to avoid, because it's rare that someone has control over the result of a reverse lookup of his IP address.

      All that matters is that when the receiver looks up the MX list for the domain mentioned in the "SMTP FROM" command, one of the MX IP addresses matches the IP address of the machine you're connecting to them with. Since you have control over your domain (and a static address...lucky bastard :-) you can add your IP address as an MX for your domain. If you already have other systems designated as MXes for it you'll probably want to list your static IP as a low-priority MX so that mail always gets delivered to your real MXes first. Then you just have to block incoming SMTP connections to your static IP address, unless you want it to actually act as an MX for your domain.

      More troubling for me though is that I've configured my MTA (Exim) to do something along the lines of what you said. I've seen in my logs messages being rejected. For example, an old friend tried to contact me through FriendsReunited.co.uk - my mail server rejected the message because it couldn't verify the email address of the sender, which had a different domain.

      That's not quite the same thing, because you got that information by trying to reverse the IP address of the sender, right? That's a method which is a lot more error prone because people don't have control over their reverse zones (thanks to how DNS is set up). But they do have control over their forward zones, which is why the scheme I'm promoting uses it.

      The method I'm suggesting is a convention. Like any convention, it's not going to work if people don't adopt it. But aside from spam filters, there is no method that doesn't require the adoption of a convention, so the method I'm suggesting isn't any worse than any of the others in that regard.

      --
      Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
    6. Re:A possible solution to the spam problem... by eclectechie · · Score: 1

      Suppose instead of blindly accepting email from everyone, you were to take the domain given to you by the MAIL FROM command, look up the MXes for that domain, and reject the email connection if the IP address of the sender doesn't match one of the domain's MXes?

      I am doing something like this now, with a patched version of Postfix.

      But I am not doing it globally; I have a list of commonly-forged domains (hotmail, yahoo, earthlink, et al), and if the MAIL FROM is in the list, the sender's domain must match. It certainly stops a lot of mail; only my osirusoft test catches more. And, no false positives (yet).

      --
      "The empty vessel makes the greatest sound." -- William Shakespeare; Henry V, 4. 4
    7. Re:A possible solution to the spam problem... by cyways · · Score: 1
      I use a similar strategy to filter out spam, though I don't require all sending servers to match the SMTP MAIL FROM: header. Instead I apply these rules to a limited number of domains which are commonly forged by spammers like aol.com, msn.com, excite.com, yahoo.com, etc. In cases where the MAIL FROM claims the message is sent from joe323529@msn.com, but it doesn't come from an MX host in the msn.com domain, I reject it. This avoids the problem described by some other posters who forge their From headers for a good reason.

      By the way, I use the old, but reliable, secure smtpd daemon from Obtuse. Among other security features, it has an excellent rule-based language to filter messages during the SMTP exchange.

  39. Crazy talk ... that's user error by blinq · · Score: 2, Informative
    SpamAssassin works the way you tell it to work. If you feed it all your mail and don't bother to pre-filter or whitelist known good mail, it's your fault if SA flags things such as newsletters as SPAM.

    I use procmail with SpamAssassin in this manner:

    • add procmail filters to put messages from family members and close friends into my INBOX
    • add procmail filters to sort out messages from mail lists and newsletters
    • adjust individual scores for SpamAssassin rules if necessary (usually I adjust them so a matched rule's score is higher than the default score)
    • whitelist addresses from family members and close friends in SA's user preferences (a redundant mesaure just for the heck of it)
    • let any mail that isn't sorted by my procmail filters be checked by SpamAssin
      • messages flagged as spam by SA are put aside into a spam folder
      • messages not flagged as spam by SA make it to my INBOX

    It only takes a little bit of thought and minimal configuration to keep your mail from incorrectly being flagged as SPAM. For me, using this method has led to zero (0) false positives on messages from known sources, for two years. Every once in a while a SPAM message sneaks into my INBOX (a couple a year), but then I submit it to a SPAM database used in SA's checks (like Razor), or adjust any particularly annoying rules' scores, and it doesn't make a repeat appearance for me.

    If your find that any particular newsletter is being treated as SPAM by your mail filters, there's probably a very simple way for you to make sure it isn't filtered out. Use the tools you have wisely, and you won't be disappointed.

    --
    ~Chris
  40. I subscribe to crytpogram by Brawyin_Neytal · · Score: 1

    I also run the spam filters for my employer. I use spamassassin and mailsweeper with a ton of custom rules. It is my responsibility to be the human interface that examines the caught spam spool every day and delete or forward the emails. If people subscribe to a mailing list that has advertising link or other spam triggers I let tell them they need to mail the list admin and tell them to stop. Otherwise it sucks to be them. I have no tolerance anymore. Do not let your mail have the characteristics of spam if you want it to reach people. wtf.

  41. WOAH WOAH WOAH Slow down now... by The+Notorious+ASP · · Score: 1

    ROT-13? Not so fast!
    here
    Maybe YOU want to risk the jail time, but me, I'll pass!

  42. SpamNet by Anonymous Coward · · Score: 0

    I'm using SpamNet and while I never subscribed to the letter, it's interesting to note that *if* SpamNet users are not receiving it, it is only because a significant number of people who are receiving it don't want it.

  43. Bayesian filtering by Anonymous Coward · · Score: 0

    I highly recommend you read up on this. Even if you don't go for the gory statistical details, read Paul Graham's overview:

    http://www.paulgraham.com/better.html

    It works quite well, even when spammers try to evade it using techniques like you mentioned. For example, a message with this:

    Highten S/e/x/u/a/l Satisfation, 1 0 0 % Safe ... was easily caught and filtered, even though every keyword is mispelled or mangled, and even though the body of the message was seemingly spam-innocent.

  44. oops one more thing by Anonymous Coward · · Score: 0

    Oh I should also mention I'm using this filtering in Mozilla 1.3 beta. They are implementing the algorithm described in that link.

  45. So let's send spam as Bruce Schneier by marcink1234 · · Score: 3, Interesting

    As a lot of people will probably whitelist cryptogram, if one wishes to spam technical people, he just needs to set From to Bruce.

    1. Re:So let's send spam as Bruce Schneier by Anonymous Coward · · Score: 0

      So Bruce just needs to sign the message, and then you can filter messages which are From: Schneier but not signed, not on the 15th of the month, etc.

  46. Funny until it's something trully important... by Prof.Phreak · · Score: 2, Funny

    An employer of mine sent out a very important e-mail with "IMPORTANT - MUST READ" in the title, and guess how many people got it? All thanks to wonderful e-mail filters...

    --

    "If anything can go wrong, it will." - Murphy

    1. Re:Funny until it's something trully important... by spells · · Score: 1

      Was this an internal email? Who filters local email?

      Well, come to think of it, I would like to, but it just wouldn't look good :)

  47. Free Speech by Skapare · · Score: 0, Offtopic

    Free speech isn't about simply being able to speak something. It's about being able to speak about any topic you choose to. If you want to speak about voting out the incumbent president, or recommending penis enlargers, that should be your right. Infringements on free speech are those that take into account what the speech subject is, to decide whether to suppress it or not. This is the kind of infringement that content filtering does. Perhaps the content filtering is simplistic and looks for "penis" in the message. Or perhaps it is very sophisticated and approaches a conceptual understanding of the message. But regardless of how good it is, by being based on the content, this is infringing against free speech.

    Of course for your own mail server, whatever you choose to use is up to you. The US First Amendment only applies to restrictions imposed by the government. But I happen to choose to not restrict based on content; I choose to restrict based on the behaviour of the sender who is sending unsolicited bulk email (UBE) regardless of the content.

    --
    now we need to go OSS in diesel cars
    1. Re:Free Speech by Anonymous Coward · · Score: 1, Insightful

      free speech doesn't mean you have to listen...

    2. Re:Free Speech by Zeinfeld · · Score: 1
      If you want to speak about voting out the incumbent president,

      How can we vote him out? Thy wouldn't let the votes be counted when it came to voting a president in.

      The US First Amendment only applies to restrictions imposed by the government.

      And in any case will be suspended in the next few weeks, after all they have already suspended due process and habeas corpus so it is only a matter of time before the first ammendment goes as well.

      One of the problems with the death penalty is that its popularity with the voters means that opponents are considered unelectable. So you end up with politicians like Clinton who know that it does not work but will pay lip service to it to get elected, or you get psychopaths like Bush.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
  48. A Simple question... by Pathwalker · · Score: 3, Insightful

    Am I the only one that has all of the mailing lists I subscribe to bypass SpamAssassin?

    For each mailing list I subscribe to, I use a special address suffix just for that list, that bypasses all of my spam checks (including SpamAssassin ), and just goes right into the mailbox that I use for that mailing list.

    No problems with false positives, and it saves me the overhead or running SpamAssassin on every incoming message from a busy list.

    it just seems like common sense, no one should have a problem with SpamAssassin misclassifying incoming newsletters if they just think about how they organize their email.

    1. Re:A Simple question... by babbage · · Score: 1
      Am I the only one that has all of the mailing lists I subscribe to bypass SpamAssassin?

      I was going to do this, except that at least one of the lists I'm on -- as it happens, the one I'm most interested in reading -- itself tends to get spam every now and then. As long as lists have open membership rules -- and in general I think that's the right policy -- then spammers will be able to sign up to deliver unwanted messages. Hence, SpamAssassin has to scan that mail as well.

      As it happens, the only list that I exclude from SA is a Craigslist classified ads list, since a lot of the legit mail from it is actually people making TRULY GREAT OFFERS with INCREDIBLE DEALS and WHOLE LINES OF SHOUTING about things that I SIMPLY CANNOT MISS! As annoyingly over the tp some of these postings are, as a whole it is actually mail that I want to receive, so I don't want SA to flag it as spam. And because Craigslist's software sets the From header to that of the person sending out the posting, as opposed to nobody@craigslist.org like they used to, it was easier to just exclude CL mail from SA filtering than trying to come up with a whitelist rule based on CL's mail servers.

      C'est la guerre :-)

  49. initial analysis for Bruce by Daniel+Quinlan · · Score: 5, Informative
    I'm one of the SpamAssassin (SA) developers and I asked Bruce to send me a copy of the newsletter after hearing about his note of warning a few days ago.

    Aside from the spot-on comments that people have made regarding adding a whitelist entry Crypto-Gram (an obvious candidate for whitelisting if there ever was one, given that it frequently discusses spam, scams, and probably even includes text straight out of some spams), here is my initial analysis and response to him.

    Oh, first one other comment: SpamAssassin does not block content. SpamAssassin only flags probable spam. What the site or user does with that flag is their own business. Some mail administrators misuse SpamAssassin to block email, but we do not recommend blocking email. Really.

    ------

    [...] One false positive (or a related set of false positives) is not really a statistically useful sample size. To get to a high rate of filtering, most filters do have some false positives. You can get fewer false positives with customization of one form or another (personalized Bayes training, whitelists, rules, automatic learning algorithms). Our goal (everyone's goal, I think) is to get the best ratio of false positives to false negatives. It's a difficult balance sometimes and some legitimate content has a harder time.

    On to the data:

    I checked your newsletter with two versions of SpamAssassin: the current stable version (2.44) and the very-soon-to-be-released development version (2.50).

    A score of 5.0 is the default threshold to be flagged as spam.

    In SA 2.44, your mail receives a score of 3.20 (2.40 as I received it, but I believe the score would be about 3.20 for most people). That's on the high side, but has bit to go before being flagged as spam. The score is the same with network tests (DNS blacklist tests and Razor).

    In SA 2.50, your message would probably receive a score of 1.90 without network tests and 1.00 with network tests. Note that the test scores may change a bit before the final release of 2.50, but those are better scores, more what we like to see for non-spam content. They would be even lower when using Bayes (part of SA 2.50). Those lower scores are not unexpected because... well, 2.50 is better. :-)

    Based on these results, it's not clear to me why yesterday's newsletter was flagged as spam. Some possibilities:

    • your newsletter is routed through blacklisted hosts for some people
    • some people are using a old or misconfigured versions of SpamAssassin (extra rules, additional blacklists, many possibilities here)
    • the newsletter as received by some subscribers is substantially different than what you sent me
    • something else?

    Can you give me more information about the false positive that you experienced or was reported to you?

    Thanks.

    Dan

    ------

    If I find out more of interest before the thread is closed to comments, I'll try to post a follow-up to my post.

    1. Re:initial analysis for Bruce by libertynews · · Score: 1

      I'm running v2.44 and it passed the CryptoGram newsletter just fine. I'd bet that the report came from someone who has tweaked their SpamAssassin settings to be non-default (as mine are).

      Not a problem.

      bcl

      --
      Remember Lexington Green!
    2. Re:initial analysis for Bruce by linux11 · · Score: 1
      Oh, first one other comment: SpamAssassin does not block content. SpamAssassin only flags probable spam. What the site or user does with that flag is their own business. Some mail administrators misuse SpamAssassin to block email, but we do not recommend blocking email. Really.


      You may want to have a talk with the maintainers of amavisd-new about the default behavior of their program when using SA. Other contrib applications such as spamgate.py also promote the concept of a quarantie/spamtrap address which email gets redirected too. As these contributions continue to grow it will be harder for end users to figure out what is supposed to be the vanella behavior of SpamAssassin and what is behavior that is just common to contrib works for using SA.



      Does SpamAssassin have any "use guidelines" for developers that want to integrate SA into their code?

    3. Re:initial analysis for Bruce by linux11 · · Score: 1

      As an additional side note about SA not blocking content, the feedback I got regarding the default for mime_defang (a configuration option still not listed online) was that SA "might as well have deleted the HTML email" since it made it just as unreadable from their prospective.

    4. Re:initial analysis for Bruce by Daniel+Quinlan · · Score: 1
      Just a short follow-up.

      It looks like the problem are the distributed checksum tests, specifically Razor2 and DCC.

      I realize DCC is not a spam test, but a "bulkiness" test, but we use it as one of our rules and let our GA (genetic algorithm) figure out how worthwhile it is. Unfortunately, in this case, it helps drive Crypto-Gram into the probable spam region. Crypto-Gram is a bit different than other newsletters in that it contains a lot of clippings from spams, scams, security-related excerpts like JavaScript, etc. that trigger some SA rules.

      As far as Razor2 goes, it seems like their trust metric needs some work. Crypto-gram isn't showing up in the Razor2 database now, but it only has to be there when you receive the mail. (Brings to mind delayed re-checking, but that's an idea for later implementation.)

      If you are a Crypto-Gram subscriber, my advice is to either whitelist it or write a rule that matches the newsletter and assign the rule a negative score. Also upgrade to SA 2.50 when it is released.

      Dan

  50. Just shows that... by forgoil · · Score: 2, Interesting

    This simply shows that newsletters and similar are not really sent by the right medium right now. EMail hasn't kept up with the times and as a result we see this endless amount of spam.

    What is needed is a foolproof way of saying "I want this, please send it to me" and then being able to reject it safly without needing the other party to do it for you. For example:

    I send a message to cryto-gram, including a key. This key can then be used to send it to me, and I accept it (key in combination with who send it and so on, I am sure someone with even more experience can figure out a fool proof way). Good stuff. But then I realise that I don't want this anymore, and I simply remove the acceptance of this key in my own software (and send a message that I don't want it anymore, no harm being nice to the nice), and it will be filtered away.

    Or something along those lines, I can asure you that I haven't fixed up a foolproof and perfect system yet ;)

    1. Re:Just shows that... by Heinrich · · Score: 1
      What is needed is a foolproof way of saying "I want this, please send it to me" and then being able to reject it safly without needing the other party to do it for you.

      This can be done today if you are able to create email addresses on the fly. This is supported by several mailing systems (Qmail or Postfix, for example):

      1. Create a new email address. Include a secret key in it if you want to be sure that is cannot be easily guessed. In case of Qmail:

        maildirmake ~/mailbox/cryptogram
        echo './mailbox/cryptogram/' >~/.qmail-cryptogram1234567'

      2. Subscribe that address and handle that address separately, i.e. avoid any spam filtering for it. In case of Qmail:

        echo | QMAILSUSER=cryptogram1234567 qmail-inject \
        crypto-gram-subscribe@chaparraltree.com

      3. If you want to unsubscribe and you encounter difficulties to do this directly, just let all messages bounce to the specific email address you have created. In case of Qmail:

        echo '|bouncesaying "Good bye!"' \
        >~/.qmail-cryptogram1234567

    2. Re:Just shows that... by Ivan+Raikov · · Score: 0, Troll

      What is needed is a foolproof way of saying "I want this, please send it to me" and then being able to reject it safly without needing the other party to do it for you. For example:

      It's called Usenet. You post something to a newsgroup, and anybody who is interested may read it without their mailbox getting stuffed with crap. Electronic mailing lists are for people who are too dumb to figure out how to configure a newsgroup reader.

    3. Re:Just shows that... by Silas · · Score: 1
      What is needed is a foolproof way of saying "I want this, please send it to me" and then being able to reject it safly without needing the other party to do it for you. For example: I send a message to cryto-gram, including a key. This key can then be used to send it to me, and I accept it...But then I realise that I don't want this anymore, and I simply remove the acceptance of this key in my own software...and it will be filtered away.

      This can be done via TMDA, a whitelist-centric anti-spam package. You can create sender-based addresses so that the originating org has a direct pipe to you inbox, but that anyone else trying to use the address will have to confirm their identity first. You can remove that direct pipe at your leisure.

  51. Comment removed by account_deleted · · Score: 2, Interesting

    Comment removed based on user account deletion

  52. This employer by Rhinobird · · Score: 1

    This employer, he didn't happen to have a funky, some might say pointy, hair cut, did he?

    --
    If Mr. Edison had thought smarter he wouldn't sweat as much. --Nikola Tesla
  53. Ancient Procmail Secret... by WWWWolf · · Score: 2, Informative

    ..."Ancient Gurus srb and guenther say, 'Sort your mailing lists to the folders before you filter your spam.'"

    Crypto-Gram isn't the only mailing list that gets hit by misunderstandings - all automatic mail handling is always confused about automailers and mailing lists. And even due to usability factors, it makes sense to sort mailing lists to folders anyway, and use a client that supports multiple specific folders.

    1. Re:Ancient Procmail Secret... by chriskenrick · · Score: 1

      ..."Ancient Gurus srb and guenther say, 'Sort your mailing lists to the folders before you filter your spam.'"

      This works well, except that sometimes the mailing lists can be spammed too (eg lists which don't require subscribtion to post).

  54. yes, annoying by fattybob · · Score: 1

    Yes, i was using this shared network spam killer, which was great apart from the fact that it shared headers with other no nothings who subsrcibe to newsletters, then log them as spam. It also had problems whenever it updated itself - very annoying, anyway, it got itself removed from my system - also not an easy task, and I have the very neat and fun "Popfile" spam killer / email sorter - just what i always wanted!

  55. Only need procmail by conufsed · · Score: 1

    I've found my own answer, spammers rarely put my e-mail in the To:/CC: fields, so I have set procmail to deliver all my mail to the 'unknown' folder by default and have a rule which puts e-mail sent to all my known addresses put in my inbox folder, and mailing lists go to their own folders. I also allow mail delivered to anything @mydomain to deliver to my mailbox via procmail, and use a different address for each site/vendor/whatever and therefore I can block bad e-mail addresses forever

  56. please no by upper · · Score: 5, Informative
    A "solution" like that would trash my outbound mail. I forge my From: addresses routinely.

    My primary mailbox is with a small, local ISP. I can't buy broadband from them, so I get my connectivity via cablemodem. I do have a mailbox in the cablemodem company domain -- that's the one I give out when I expect abuse. (I do it this way because I expect to be dealing with that ISP long after the cable vendor has either ceased to exist or has treated me badly enough that I left.)

    So I want my outbound mail to appear to have come from the ISP. Setting Reply-To is usually adequate, but not always -- when a human is looking for the address, they could easily grab the wrong one. And it creates potential confusion I don't want to create. So I set my from address to name@isp.com.

    I can't relay through the ISP's relays, because I'm outside of their IP range. (If they did some form of authenticated SMTP, such as SMTP-after-POP, they could let me.) And the cable vendor's mail relays won't send mail out with some other domain name on it. So I send everything out directly, no relays.

    If you look at many headers, I suspect you'll find that I'm not the only one forging my From: address for legit reasons. The presence of the X-Authentication-Warning header some MTAs add correlates fairly weakly with spam. (Some details of it -- e.g. no valid reverse DNS for the sending machine's IP -- could be useful indicators.)

    1. Re:please no by kcbrown · · Score: 1
      A "solution" like that would trash my outbound mail. I forge my From: addresses routinely.

      What appears in the From: line and what appears in the SMTP "MAIL FROM" command are often two entirely different things. The former is set by you, the latter is usually set by your MTA. Generally, if you invoked your MTA then it'll use your username and whatever domain you've assigned to your box in the "MAIL FROM" command, if I'm not mistaken.

      In any case, you should have enough control over your own box to control what appears in the "MAIL FROM" command, and that's the point: spammers do their deed by using systems that they do not control.

      If you acquire your own domain and make your IP address an MX for your domain, then you'll be able to get past the scheme I described without any problems -- no matter what you set your "From:" line to. You can do this even if your IP address is dynamic; there are a number of dynamic DNS services out there that will do this for you, for your own domain.

      I can't relay through the ISP's relays, because I'm outside of their IP range. (If they did some form of authenticated SMTP, such as SMTP-after-POP, they could let me.) And the cable vendor's mail relays won't send mail out with some other domain name on it. So I send everything out directly, no relays.

      With the scheme I described, you won't need to relay through anybody. In fact, you won't be able to relay through anybody, except those systems that you control, or those systems belonging to your ISP that will generate an SMTP "MAIL FROM" command based on the "From:" line you use in your email -- and that means that you'll be forced to use their domain in your "From:" line if you want to use their relays. That's a benefit, not a drawback: it forces everyone to be honest, which is very much what the spammers do not want.

      --
      Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
    2. Re:please no by grolim13 · · Score: 1
      What appears in the From: line and what appears in the SMTP "MAIL FROM" command are often two entirely different things. The former is set by you, the latter is usually set by your MTA.

      No. The vast majority of the time, the MUA gives the MTA a MAIL FROM address which is identical to the From: line in the mail.

    3. Re:please no by kcbrown · · Score: 1
      No. The vast majority of the time, the MUA gives the MTA a MAIL FROM address which is identical to the From: line in the mail.

      But if you have control over the box the MTA is running on then you supposedly have control over what the MTA will do in that situation. I suppose that's not necessarily true, since you may be running a proprietary MTA. In that event I'm not sure what to tell you, except perhaps to always use a From: line that refers to a domain that is under your control.

      The biggest problem with the convention I described is that right now, people don't use systems that are declared MXes to send outbound email. It's something that would have to change, but to deal with the spam problem effectively something will have to change anyway.

      --
      Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
    4. Re:please no by nautical9 · · Score: 1
      Absolutely. You're not the only one "forging" the From: address at all, since it's the default thing to do in a lot of Mail programs.

      Example: Mozilla Mail - I have three different IMAP servers that I talk to (Work, Personal, and my ISP's). When I reply to a mail from any of those, Mozilla correctly changes my "From:" field to the appropriate setting so the emails will come back to whichever account I was sending from.

      However, with Mozilla, you can only specify ONE outgoing SMTP server, which in my case is my local machine. This would mean almost all of outgoing emails would get rejected, as the reverse lookup of my Mail-From: header wouldn't match my local IP.

      Plus, it's not uncommon for large ISPs or organizations to have a different set (IP block) of outgoing SMTP vs. incoming SMTP, to distribute load.

  57. GPG sign it. by David+McBride · · Score: 1

    ... SpamAssassin actually _lowers_ the spam rating of a particular mail if it's got a GPG signature on it.

    1. Re:GPG sign it. by hughk · · Score: 0

      It isn't the GPG signature, it is the random string. Some spammers try to defeat signature tracking systems by adding a random sequence to the text.

      --
      See my journal, I write things there
    2. Re:GPG sign it. by Anonymous Coward · · Score: 0

      No. It's the GPG signature. For future reference, if you don't know what you're talking about you'll look less a fool if you keep your mouth shut.

    3. Re:GPG sign it. by hughk · · Score: 1

      False info. Spamassasin doesn't mark the GPG doen just the random string.

      --
      See my journal, I write things there
  58. White list by Fuzzums · · Score: 1

    make a list of validated e-mail addresses and move them to the inbox before you run the spam-filter.

    --
    Privacy is terrorism.
  59. DNSbl operations by Anonymous Coward · · Score: 1, Informative
    Considering that most of the time it is Net blocks that are blocked, not just individual IP addresses.(sic)

    But most of the time does not really matter, what matters is the DNSbls upon which your handling is based. After a brief foray into listing /24s, SpamCop has returned to its original practice of listing only the offending IP addresses.

    If I had a grudge against an ISP, I could fake some SPAM headers and send it to any of the IP blockers.

    And you could get your right to submit spam revoked when the ISP complained.

  60. What about ROT13? by Anonymous Coward · · Score: 0

    I doubt any content filter will ROT13 any message just to see what is in it. A line not ROT13 will be enought to the ones that don't get it when first looking at it.

    --
    I have no .sig

  61. Spamfilters seem pointless.. by Wonda · · Score: 1

    ..to me, what's the use? to avoid missing the emails that were false positives, you have to go through the rejected mails, so you see the spam anyway, may as well have it in my inbox, saves me switching folders. or am i missing something here?

    1. Re:Spamfilters seem pointless.. by NM156 · · Score: 1
      ..to me, what's the use? to avoid missing the emails that were false positives, you have to go through the rejected mails, so you see the spam anyway, may as well have it in my inbox, saves me switching folders. or am i missing something here?

      Here's what you're missing... SpamAssassin tagged emails go into my /usr/local/spam directory, instead of my inbox. Consequently, my new mail notification doesn't even go off, and I'm not bothered by all that crap that is being sent to me. Once a week, I go to that directory, and run the following:

      grep From: * | grep -v points

      If I don't recognize any of the From: addresses, I simply nuke everything, and the problem is gone. This is a little different from going through the rejected emails and I still get the benefits of spam filtration. In case I do find the rare false-positive, I add that address to my whitelist, and the problem never happens again. I've been using SpamAssassin for about year and half now, and using this technique, I'm now up to over 99% accuracy.

  62. Agreed (needed even more at university) by smcv · · Score: 1

    I have a domain at a dedicated hosting company (i.e. not an ISP), which I want to use as my primary address. At the moment the host has a mail relay (using POP-before-SMTP to prevent abuse), so while at home, I ignore my ISP's mail services completely and use my web host's POP3 and relaying; when my domain was with a different host a while ago, I couldn't even do that, since they provided POP but not SMTP, so I had to use a "forged" header as you describe.

    At university this is even more necessary, since my university blocks port 25 at most of their routers - the only exception is that anyone in the university can connect to a "server" (actually a load-balancing cluster) which acts as a central relay. This means it's impossible to send mail unless it's either tunnelled in some way (not an option for me, my web host charges extra for ssh), or through this relay server. The relay accepts mail with any faked From address, on the basis that some people (including some departments) need this functionality, and if someone spams through it, they have it logged and know who to blame.

    (Before you ask whether my uni gives me an e-mail address: yes it does, but I do game modifications, and I don't want to use my uni address for that. Also, my domain is more permanent than an address that disappears when I graduate)

  63. Irony by Anonymous Coward · · Score: 1, Funny

    I don't intend to alter my content to accommodate spam filters

    But spammers will.

  64. Re:Whitelist You want POPFile by Anonymous Coward · · Score: 0

    I really liked spamassasin but after using POPFile, I don't think I will ever go back. http://popfile.sourceforge.net/

    BTW the "magnets" feature seems just what you want. Give it a look, the saftware is free as in beer and speech and runs on perl (for *NIX like systems/servers) or you could use the binary distro for Win32 if that is more to your liking. And finally if you love it you can "feed" the author. http://sourceforge.net/forum/forum.php?forum_id=21 3876

    pingmeep

  65. Blocked this email....not gonna alter it? by Lioner · · Score: 1

    Who cares....if you want to avoid looking like spam do it. If you don't I won't see it! Big frecking deal....

  66. Re:Awesome sig OFFTOPIC, I know by sean23007 · · Score: 1

    Actually I invented my sig 3 years ago in my freshman English class. I was trying to comprehend the idiocy of my classmates, and the phrase just wrote itself down on paper.

    I do not know who invented the quote you mentioned (but I'm pretty sure it wasn't the same person... ie "not me" ;) ), but I would certainly subscribe to it.

    And by the way: if anyone can find a classical source that actually invented my sig, I would be interested in knowing it. As far as I know, I invented it, but if someone else came up with it first, I plagiarised it completely unknowingly. If I didn't really come up with it, I would like to give credit where credit is due.

    --

    Lack of eloquence does not denote lack of intelligence, though they often coincide.
  67. you just blocked MSN email by ChrisCampbell47 · · Score: 1
    As far as I go, nobody who sends me a personal message ever uses HTML.

    So I just code all incoming files with embedded HTML as spam.

    It was reported just a few days ago that MSN emails now have ONLY an HTML mime part -- no more plaintext. Just like spam. Sure, it's fun to say things like "well, I don't, and wouldn't, have any friends on MSN", but that's just being juvenile, isn't it?

    1. Re:you just blocked MSN email by anubi · · Score: 1
      Well, you are right there.. I do not have any friends on MSN yet. I guess when I do, we will have to work something out.

      I just may be unreachable from Microsoft. It would not be the first time as I am rapidly losing compatiblity with Microsoft products as they make their changes. I can no longer see a lot of stuff that was produced in a Microsoft system. If its that important, I'll read it at the College- they have a Microsoft system - and all the support people to keep it going.

      I just have to accept that Microsoft is a big boy's game, and I just don't have the ante to play.

      --
      "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]

    2. Re:you just blocked MSN email by jovlinger · · Score: 1

      I read my email using a text mode browser. No matter how childish it may be, I can't read html-only emails unless I sit down and manually read through the source. Effectively, MSN has decided to shut me out of the loop.

      Whenver someone sends me html-only email, I inform them that I can't read their email. If they refuse to send text, what can I do? They're the ones sending non-standard email formats.

  68. My Results by tweakt · · Score: 1
    (Note: AWL == AutoWhiteList)

    Headers added by spamassassin:

    X-Spam-Status: No, hits=-2.4 required=5.0
    tests=AWL,BALANCE_FOR_LONG_20K,BALANCE_FOR_LONG_40 K,
    MIME_LONG_LINE_QP,NORMAL_HTTP_TO_IP,OPT_IN,
    &nbsp ; SPAM_PHRASE_01_02,SUBJECT_MONTH,SUBJECT_MONTH_2,
    US_DOLLARS_2,US_DOLLARS_4
    version=2.40-cvs

    It still sounds like I should upgrade to 2.5 when it comes out, sounds like some very nice features. Keep up the good work.

  69. False alarm? by babbage · · Score: 2, Interesting
    I've just checked the headers for this month's Cryptogram, and the current version of SpamAssassin (2.44) did not flag it as spam. To wit (slightly reformatted because of Slashdot's "this Nerd site will not accept technical postings thankyouverymuch" comment filter):
    X-Spam-Status: No, hits=2.0 required=5.0
    tests=BALANCE_FOR_LONG_20K, BALANCE_FOR_LONG_40K, NORMAL_HTTP_TO_IP, OPT_IN, SPAM_PHRASE_01_02, SUBJECT_MONTH, SUBJECT_MONTH_2, US_DOLLARS_2, US_DOLLARS_4
    version=2.44

    X-Spam-Level: **

    Note that SpamAssassin isn't on my whitelist or anything like that -- it just worked.

    False alarm?

  70. Bad news, it's in Razor by imroy · · Score: 3, Informative

    I just got the email today and it failed. I'm running 2.44 from Debian and haven't yet looked at tweaking any of the rules.

    Here's the verbose banner that SA put on my copy:

    SPAM: Content analysis details: (5.90 hits, 5 required)
    SPAM: SUBJECT_MONTH_2 (-0.5 points) Subject contains a month name - probable newsletter (2)
    SPAM: SUBJECT_MONTH (-0.5 points) Subject contains a month name - probable newsletter
    SPAM: OPT_IN (1.5 points) BODY: Talks about opting in
    SPAM: US_DOLLARS_4 (0.4 points) BODY: Nigerian scam key phrase ($NNN.N m/USDNNN.N m/US$NN.N
    m)
    SPAM: US_DOLLARS_2 (0.1 points) BODY: Nigerian scam key phrase ($NNN.N m/USDNNN.N m/US$NN.N
    m)
    SPAM: BALANCE_FOR_LONG_20K (-0.7 points) BODY: Message text is over 20K in size
    SPAM: BALANCE_FOR_LONG_40K (-0.1 points) BODY: Message text is over 40K in size
    SPAM: SPAM_PHRASE_01_02 (0.5 points) BODY: Spam phrases score is 01 to 02 (low)
    SPAM: [score: 1]
    SPAM: NORMAL_HTTP_TO_IP (1.3 points) URI: Uses a dotted-decimal IP address in URL
    SPAM: RAZOR2_CHECK (3.9 points) Listed in Razor2, see http://razor.sf.net/

    It looks like some dumbass has entered it into Razor. Unfortunately, some people (and yes I did this originally) had their procmail setup to enter an email into razor if it is deemed "spam" by SA or something else. Those 3.9 points are what puts it over the threshold.

  71. you are wrong by Anonymous Coward · · Score: 0

    that's called signing, not encrypting, you dolt.

  72. is he a registered... something something by AssFace · · Score: 1

    I know that if I get e-mail from Half.com or Ebay.com or a few others, there is a thing that shows up in the scores that says something about it being a registered sender and therefore gets through.

    also, if you are subscribed to a mailing list and don't have it on your whitelist... well, then dhuuuuuuur

    --

    There are some odd things afoot now, in the Villa Straylight.
  73. www.habeas.com by Xandir · · Score: 1

    Just start using the headers from habeas.com, and you should not have these problems. I know spam assasin has a rule for this, as should the rest of the spam apps.

  74. A filter that learns is better by bigberk · · Score: 1

    It seems to me that this is why a self adapting filter, such as spamprobe (free *nix filter that uses Bayesian analysis) is superior in the long run. It builds a database and associates words and multi word phrases with spam or non-spam. At the start, its guesses are rather crude but I found that after 'training' the filter with about 50 emails, the accuracy is incredible.

  75. The future of e-mail by ziegast · · Score: 2, Insightful
    The message below will get around just about every spam filter...


    From: schneier@counterpane.com (Bruce Schneir)
    To: reader@slashdot.org (Nutcase)
    Subject: Monthly Cryptogram newsletter

    The February 2003 newsletter is out!

    http://www.counterpane.com/crypto-gram-0302.html


    It has some other advantages too:
    1. Instead of blasting out 20K messages to all of the recipients at once, he blasts out a bunch of 1K messages, cutting down on his 95th percentile bandwidth. People will come back to read the articles, and when they do, web caching servers/software between users and his server will cache anything static. Eg: 5000 AOL users will get the article from the AOL caches instead of his site, but a bug in the HTML will get a 1x1 gif from his site directly.
    2. Everyone sees exactly the same newsletter as Bruce intended to publish it (he probably doesn't make exceptions of Opera 7 ;^) instead of worrying about hoiw to accommodate HTML into everyone's broken mail reader.
    3. It keeps from filling up countless mailboxes for something we'd probably go to his website for anyway.
    4. If he has advertisers that want to post on his website, they get more eyeballs, and it's less annoying than being sent an ad as part of your mailbox. Conversely, like Slashdot, subscribers can pay Bruce not to put ads into the newsletter by giving him the annual subscription fee.
    5. Bruce can tell exactly how many people read his article (web logs).


    I learned this from the electronic greeting industry. Similar to Usenet 2 and Internet Mail 2000, messages semaphores will become the future of e-mail. People will create web content as easy as they create e-mail messages now and semaphore the recipients (using IM or email) to look at their content. Recipients who are interested will click on the URL in the semaphore. Recipients who want mail from Bruce, will open it. Bruce might even (G)PG(P)-sign the announcement notice so that spammers can't pretend to be him.

    Then again, why should Bruce have to mail anyone at all? If his newsletter is so good, his readers will bookmark his page and read it every now and then, just like I do with DaemonNews or ArsTechnica.

    The Internet is evolving, and Bruce is whining along the way. Mass-mailed newsletters are going the way of the dino-WAIS-server (just like FTP ;^).

    -ez
  76. Sure, chain me to a platform by Anonymous Coward · · Score: 0
    but why not distro the newsletter encrypted?

    That is a great idea until you realize that maybe I am reading my e-mail on my cell phone, or PDA, or a blackberry, or my friend's machine using hotmail or anything else other than a PC running Windows or Unix where I can install software at a whim.

    For most /. people, they sit shackled to their PC all day and cannot imagine life any other way.

    The rest of us spend very little time at a desktop.
  77. SPEWS concept -- obnoxious by Anonymous Coward · · Score: 0
    And this is why the SPEWS blocklist is so effective and so good. If he were on it, then that would mean that he and/or his network fell into one of the following categories:
    * Is a spammer
    * Is an ISP harboring a spammer (or an upstream ISP thereof)
    * Is a customer of an ISP harboring a spammer.

    Could you imagine this justification in any other context?
    * Your car won't start because you bought gas from Amoco, who also sold gas to a traveling salesman
    * You cell phone won't work because Verizon also sold a cell phone to a telemarketer
    * The Postal Service refuses to deliver your mail because another postal customer was doing direct mail
    * Your power goes out because the power company had a customer who sent out spam

    This justification for shutting down mail servers is one of the most perverted things I have heard in a long time.
  78. open source java mail client by dryeo · · Score: 1

    I, for one, would love to see a feature like this in a mail program! Actually, I'd like to participate in the development of an existing open source email app if someone could recommend one. Java based would be nice

    Checkout the polarbar mailer (www.polarbar.org). Opensource, written in Java.
    Dave

    --
    https://en.wikipedia.org/wiki/Inverted_totalitarianism
    1. Re:open source java mail client by whereiswaldo · · Score: 1

      Checkout the polarbar mailer (www.polarbar.org). Opensource, written in Java.

      Sweet! Just what I wanted. Thank you very much.

  79. Isn't that the entire problem? by Kjella · · Score: 1

    I forge my From: addresses routinely. (...) If they did some form of authenticated SMTP, such as SMTP-after- POP, they could let me.

    In other words, you're sending a mail that appears to come from a ISP address, without ever being in contact with that ISP. Assuming there are rouge boxes out there, how could you possibly fix it without breaking your setup? I think the solution is fair - if you want to send with a @domain address, you must authenticate with the @domain servers somehow. If your ISP doesn't offer it now, I'm pretty sure they would if there was real demand for this feature. And if your tell your ISP that this will help reduce spam, they'll like it. They're not more fond of spam than you are (except those ISP profiting more from having spammers than getting spam, but they're few)

    Kjella

    --
    Live today, because you never know what tomorrow brings
  80. Personally... by Anonymous Coward · · Score: 0

    personally, spam filtering at work is a royal pain in my butt. Corporate HQ is in charge of the spam filtering (they own the external mail gateways), so we have no control over the lists..

    So, when we are trying to quote a customer for a "9-inch mirror" (we make optics), it gets filtered...

    and, for some reason, "Take your next left" in an email to a customer giving them directions to our facility gets it blocked as spam...

    I go through one or two a week, being tech-support, trying to figure out what is blocking the user's emails. Its a royal pain.

  81. Sure, Bruce has nothing better to do... by Xtifr · · Score: 1

    I send a message to cryto-gram, including a key. This key can then be used to send it to me

    You want the newsletter, then it's up to you to make sure you can receive it. I think there's pretty near zero chance that Bruce is going to waste his time jumping through hoops for your benefit and your benefit alone. Well, ok, I won't speak for Bruce, but speaking as a Debian developer, if you send me a question or request for help, and my response bounces because I'm not on your whitelist, I'm simply going to delete your question/request, and will probably add your name to my killfile, just so I don't have to deal with that crap in the future.

    1. Re:Sure, Bruce has nothing better to do... by forgoil · · Score: 1

      Newsletter is one thing (and I certainly am not interested in any solution which uses email as its carrier) and I certainly meant for the system to be easy to use and with minimum (read: no) need for maintenance. Stop being so awfully negative and try to see possibilities instead of just how to either change what you have or simply come up with 20 reasons why someone elses idea suck.

      This was not a suggestion geared towards personal communication, nor would I be so dumb as to set up a whitelist and then not add the person I was emailing (including a little tail saying that you could only reply from the address I sent you, otherwise you have to notify me) so I couldn't get a reply. Don't assume everyone is an idiot, makes you look like one yourself.

  82. Re:Bad news, it's in Razor - And DCC too by mkettler · · Score: 1

    The SpamAssassin Developers have already opened a bug to discuss this issue, but the "heavy scoring" contributors to it being spam-tagged would appear to be Razor and DCC.

    Of course, it's always struck me odd that anyone would use DCC on any system they didn't want FP's on. DCC is a "bulk" email tracker, not a "spam" email tracker (ie: any mass-mailing should be in it, solicited or not).

    At any rate, if you want to monitor the SpamAssassin bug regarding the Crypto-gram newsletter, you can read it here:

    http://www.hughes-family.org/bugzilla/show_bug.cgi ?id=1490

    --
    -Matt