Slashdot Mirror
Aggressive Email Filtering Blocks Political Debate
Stephen writes "Many of us have spam blockers operating on our mail. But according to this BBC article, when British members of parliament starting having their emails filtered last month, it stopped them talking about genuine political business such as the Sexual Offences Bill, and prevented them receiving some constituents' emails." This problem has bit me on the bum a few times too. About 1 message in every 250 spam is a false hit. Course thats about once a day :(
These types of incidents may be good in the long run - if it makes law makers "wake up" to the problem of spam.
...We can only hope... Perhaps we could even start bombarding law makers with spam ourselves? - that would raise their awareness!
I can just imagine the outrage if this happened to the bush administation.
'what do you mean no one got my emails?'
'It seems your.. uh... last name is causing some issues with spam filters sir'
'That's it.. lets bomb the spammers'
I am the lord of the pun. Dance Knave!
...to eliminate all the dupe stories!
I think polititians shouldn't have any filters on their e-mail.
After about 2 weeks of what the average person goes through, we'd see stronger anti-spam legislation/penalties.
Is it possible to make your SMTP server look for a certain subject line, and if it doesn't see it, bounce the email with a message that basically says:
Please resend this message with yyf6d55s in the subject line.
Change this magic key each week/month, and only the first email from each person that they send each month will need to be resent.
Get your own free personal location tracker
that the best anti spam method is to block certain IPs. No filtering based on content.
Sometimes filtering CC entries works pretty good.
it stopped them talking about genuine political business
thats because they no longer knew how to enlarge their penises and missed being notified that some russian woman wanted them so badly that it hurt.
that would certainly stop our gov't, at least..
xao
xao
http://TheHillforum.hopto.org
Do any spam filters work (as in NOT throwing out legit mails) other than ourselves?
the problem is that just by knowing there could be a false positive, you have to examine all your filtered spam, which makes the spam filter useless in the first place.
I can easily see why this may be happening. The types of filters that use keywords can easily fall into this.
I understand that keywords and phrases such as
'free money' 'zero percent financing' 'win
million dollars' 'sex xxxxx pictures!' and so
on can trigger many filters.
I would like to think that the better designed
filters would use a combination of key words as
well as suspicious domain names and/or IP
address blocks to do filtering.
The spam filter that is used on my email account does not filter out, but it does add the word
'SPAM?' into the subject line of the email message. I can then see right away if it is
really spam or is something mistaken by the filter for spam. The message is not blocked, though.
Mark
Cleara
Rome taught me patience and assiduous application to detail. Virtues which temper the boldness of great, general views.
how to balance open access to constituents without being overwhelmed.
Perhaps Parliament could consider some of the steps that the American Congress has taken. The American Congress has a de facto filter built in to prevent Joe Random crazy from flooding their representatives with spurious requests. Most Congressional requests, letters, phone calls, faxes and emails are tossed out unless they come from certain designated people known as lobbyists. These lobbyists have worked hard to cultivate contacts in the Congress, and can get better results from one office visit than 1,000 letters from voters. In a way, they're professional access voters.
So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.
A similar problem happens with free Webmail or adversiting-supported e-mail accounts. The small advertisements attached to the bottom (I call them "spamlets") will sometimes trigger mail filters.
Watch out for this if you're sending a message from e.g. Yahoo! to Hotmail, who both attach spamlets and both filter incomming mail. They also will not send rejection notices to the sender, so you may never know if you message got through.
Perhaps they should start using pgp encrypted/signed stuff and filter out all non-encrypted/signed emails?
*shrug* That's what I do.. I hate getting email from somebody I don't know...
Just when you make it idiotproof, some idiot builds a better idiot.
You're confusing the right to free speech with the privilege of being heard.
Like woodworking? Build your own picture frames.
It isn't so insecure if you use PGP
HallmarkOrnaments.Com
Because most of the people they should be talking to , i.e. constituents, also use this insecure system. In reality, most of the politicians I know use email aren't discussing state nuclear secrets or troop movements. If certain politicians are likely to leak sensitive information this way I would be far more concerned about what gets picked up from the far more insecure system - VoA (Voice over Air)
It's obvious that they're NOT using encryption and that is point of my email.
All the best,
--Achmed
Swaribabu Consulting Inc. -- We code so you don't have to
Specifically for the parliment, I dont see why they dont just whitelist all other parliment members.
People who think they know everything really piss off those of us that actually do.
...but not perfect.
Ok, here's the situation as I see it.
We have a problem: Spam
We need a solution.
So far filtering has been working good and is slowly getting better, but there's always gonna be the chance for false positives.
so how do we stop this?
I have no clue.
We should probably start cracking down on open relays, even use governmental pressure if needed (on spammers in our countries and on the governments of other countries). They serve no real purpose other than facilitaing spam.
What else can we do? Go after spammers legally. We need to make them pay. I bet if 1000 people sued ralsky for $500 a piece he'd start to take notice, but he still wouldn't learn. Some states, like washington, are doing that, and it seems to be working, or at least getting the spam recievers a little extra cash. If I lived there, I know I'd try it at least once. Hell, I might even pay for my braodband connection with the money I got from spammers:)
I've heard people recommend opt-out lists like they use for telemarketers- that's not gonna work because spammers are much more slimy- they'll use the opt-out list as a verified list.
We're not left with many choices, besides educating people to simply delete spam and DON'T buy from it. make it cost spammers money. if they sell even one thing, they they're winning.
I took a slightly fun approach. I'm building a list of 'legit' companies that sell your email address to spammers. What I did was bought a domain, and whenever I signed up for something, I used the companies name@ the domain, and had it all forward to one account. so when I get spam to musiccity@mydomain.com, I know that musiccity sold my email address (which they did).
Does anyone else have any Ideas how to stop spam? if so, save the redundant mods and reply.
Looking for Book Reviews? Check out Literary Escapism.
Without spam, how else would I be able to sit home every day and make $1,000 a week watching TV while playing with my 12 inch penis?
from Scunthorpe.
We offer SpamAssassin at the college where I work. I always tell new users that any spam blocking system, no matter how good, will eventually block something that was legitimate. That's why I don't write procmail recipies that redirect mail flagged as spam to /dev/null. You gotta put it in a seperate folder and you are asking to get burned if you don't skim the subjects and senders every couple days. Also, they should be whitelisting messages from addresses in their domain.
I don't see how this is news. It's just an example of bad system administration.
I'm sure the filters caused many problems with the "Hot, horny housewife" bill and the new "Extra six inches" tax debate.
Unfortunately, I am not Wil Wheaton
By definition filters are hit-and-miss and non-deterministic. I get almost exclusively SPAM with spoofed return addresses. How about this solution:
1. Sending mail server generates a tx content key based on the contents of an e-mail being sent.
2. Sending mail server uses the tx content key with a private key to create a confirmation key.
3. Sending mail server sends the e-mail, along with the confirmation key to the receiving server.
4. Receiving mail server generates a rx content key from the e-mail contents.
5. Receiving mail server sends the rx content key and the confirmation key back to the sending mail server.
6. Sending mail server uses its private key plus the rx content key to re-generate the confirmation key.
7. Sending mail server compares the confirmation keys.
8. If the keys match, the receiving mail server allows the mail to enter the recipient's mailbox.
9. If the keys don't match, the mail is bounced.
The keys are in place to keep the SPAMmer from tagging along on a valid return address with mail that address didn't send. This technique also keeps the second transaction to a minimum exchange of keys. The keys add traffic, but the eliminated SPAM traffic more than makes up for the penalty. As more and more mail servers are updated with this feature, spoofing is all but eliminated. The remaining "spoofable" domains can be explicitly severed from the net or blocked.
Xesdeeni
create a text file [&] zip it
Unless the recipient is expecting this they should just delete the message. I routinely delete any email that has zipped attachments unless I have previously agreed with the sender to send it that way. (That's assuming the recipients mailserver doesn't routinely strip zip files off as an enterprise virus protection measure in the first place.)
But one way your suggestion could be modified that will work for anyone whose email can view HTML is to print your message to graphic file, convert it to a GIF and embed it into a simple a webpage.
The reader will open the file and see what looks like a text message, but it actually will be the GIF image of your message.
Most filters don't block HTML and GIF files.
Work for Change & GET PAID!
i got in a fight with an ex-girlfriend and we ceased speaking for awhile
;-(
i became further incensed because she never contacted me after the fight
we didn't talk for 2 months
finally, i contacted her and said "why didn't you get back to me??!!"
she said, "you didn't get my email?"
i looked, and there it was, 2 months back, in my spam folder (yes, i keep all of my spam, the folder is gigantic)
although you could make a joke about emails from girlfriends being called spam, in this particular case, considering the chance at reconciliation that was lost and the feelings involved, it was definitely not funny at all
so i can say, with certainty, that my personal life has been greatly and adversely affected by spam.
you can hate spam for all sorts of reasons, but for me, it's personal.
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
If this were to hapen in the US, they would say "spam isn't really an issue these days" and no spam laws...
Boy would that be bad. Slow progress, is better than "no problem at all".
Lets push for a "no spam filter for Congress until Congress passes a no-spam law"
Then again, wouldn't be needed if enforced.
According to the article the system was implemented without prior warning. What they should do is educated the users on how to implement spam filtering on their machines and not stop messages from going through at all.
In my e-mail client spam is marked in a different color, and by now the success rate seems pretty good, but I still don't trust it enough to auto-delete them. Spam sucks, but false positives not getting through might be worse than boobie mail getting blocked. In this case members of a governing body are affected. They should be working on legislation against spam, instead of having their hands held by the IT department.
Hank! White!
I just recently started to use popfile, but so far it looks good!
I know it's a pain but sometimes you have to bite the bullet and get another email address. You would soon gain the time spent migrating back through having less time spent sifting through SPAM.
Anyone getting the idea that I'm sick of seeing the complaints too?
--------
interested in wallpapers, coverings, interior decorating in Australia?
Perhaps it's coincidental, but the Yahoo spam filter consistently marks conservative political email I receive as spam (GOA, Texas GOP, Federalist, stuff like that).
Filters suck.. always have always will..
Instead of actually dealing with the REAL problem they decided.. lets filter it..
Kinda like a college kid that can't do breast cancer research at the skool library because they've got a net-nanny enabled that says the word breast is bad.. same thing goes with grandma at hte library looking up a recipe for fried chicken breasts..
They should go after the spammers and not filter stuff.. isolating yourself from the problem isn't the same as actually fixing the problem.. hopefully they'll wake up and notice this..
probably not though..
...I thought the Right Honourable member for Scunthorpe was just being rude.
Seriously though, I used:
FaxYourMP.com
when I wanted to complain about the entitlement/ID cards scheme. I got a reply from my MP (a copy of a letter sent to our Incompetent Home Secretary), on House of Commons headed notepaper in the post 3 days later. For once I feel slightly included in the political process...
# init 5
Connection closed.
Oh...
There really needs to be a better way of blocking email besides by subject line or sender, but that's really the only way unless something is done on the server side. But the filters could be much better than what is available today. Here's what I propose. The blocker would have a list of valid senders. When you get mail, the blocker would check the list. if it's there, fine the mail passes, otherwise an email is sent back to the sender with a verification code. The sender would then have to reply back with the code and then the original message would go through. That would eliminate the ficticious email addresses that many bots use. And if it was a valid email the bot would have to be much more intuitive. Of course you could always add your own senders and preapprove certain email addresses that are created by good bots (an amazon.com tracking number email)
bit me on the bum
Taco, ol' Sod, I see you're hard at work addressing those complaints from our brothers overseas about the persistent American slant of SlashDot.
Good On Ya, Mate!
That said, and out of fear of being mod'd OT, let me add that I have had success training Evolution's filter system to recognize spam not based on the subject but on the domain name. Without ever bothering with public blacklists, I've just patiently built out my own Enemies List over the years. The "keywords," if you will, in so many of the spammers' domains are remarkably similar -- "email" "deals" "free" etc. Combine that with whitelisting based upon my address list, and I think I've had maybe 2 false plucks for as long as I can remember (receiving on the order of 150 spams daily)
Many constituents who have had perfectly reasonable emails blocked may not pursue the issue further.
No shit.
Does anyone here who blocks spam actually indicate to the spammer that they've been blocked? That would be counterintuitive to say the least. Seems to me the constituents have no way of knowing if their message was denied as a false positive.
At least they're being honest.
When the FTC decided to try filtering
So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.
Works well for who? I don't see how it helps the average joe citizen who wants to get his point across unless he donates money somewhere. Corporations have tons of cash to throw at it. So if Jimmy Lobbyist has more access than Joe Sixpack, thats a problem. repetition and filtering be damned. It is the duty of a representative democracy to represent those they are representative of, and if they aren't willing to take into account every email and letter and fax and phone call they get in their decisions, then it's a stone's throw away from not having elections at all, especially when you consider that when voting the only two candidates who generally have a chance is a lesser of two evils situation.
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
Hey, that's an interesting idea. wouldn't a filter be perfect for recognizing a duplicate story? First, run a program that builds a 'word map' for a particular article (using keywords, for instance), then compare that map to an article you wish to submit.
The problem is when there are multiple "unique" stories about the RIAA or Microsoft. Even if the word maps are different the story is the same...
no, really. If people want to send you e-mail, make them send it pgp. If a message is sent non-pgp, have them re-send it pgp.
That, or have your mailserver put e-mail from unrecognized e-mail addresses into a waiting pool and have it bounce an e-mail back to the sending address as confirmation that there is a live human being at the other end of the address. If you're expecting e-mails from addresses with machines on the other end, look in the spam cesspool for them or add the originating e-mail address to your mailserver's "ok" list.
...to eliminate all the dupe stories
-
ping -f 255.255.255.255 # if only
Comment removed based on user account deletion
Would you block my email? I run exim and use it instead of my ISP's SMTP because the SMTP server is slow and unreliable. Exim gets the job done but my return to address does not match my sending address. I'd love to run my own mail server, but the cable company blocks inbound mail requests. Would your filter label my mail as spam?
DMCA, Hollings, Palladium. What might have sounded like paranoia is now common sense.
I've been enjoying spamassassin now and then I thought about ways around it, just as a matter of curiosity.
the image approach that you mention there is the first thing that I thought of, and it made me want to work on a way around it.
I know enough about image recognition that I could write a script to use that image as input and scan it and treat it as text - then run it through spamassassin.
it could be an add on to spamassassin...
but at this point, not enough spam is coming in that format to me (meaning "none") for me to worry too much about it.
There are some odd things afoot now, in the Villa Straylight.
....to eliminate all the dupe stories
Cover your eyes and click this link!
Anti-SPAM SPAM
I wonder how effective it would be to implement spam filters that rank messages instead of filtering them?
Generally, I'd be satisfied with having the most important emails simply appear before the spam, rather than have the filters delete it entirely, risking some false positives.
Usually, mail is sorted in order of arrival (at least in my box). Perhaps it's getting to the point where the content is more important than the timing?
bayesian filtering works. i've gone 3 months (after about a month of training) with 0 false positives at about 120 spams per day. i'm down to about 1 false negative per month as well. training is conveniently done by adding keybindings to mutt. when i get a false negative in my inbox, i hit a key and bogofilter learns that it's spam. another key corrects false positives. training is fun, too. i couldn't be happier with this setup.
I know what you're getting at with that sentiment, but I have to disagree. I would rather have a few false positives, than to be frequently interrupted with false negatives. The difference, upon arrival "real" e-mail causes biff to get excited and sends a text message to my phone, all of which lead me to take a break from whatever I am doing to check my e-mail.
A spam, or a false positive, goes into my spam folder. A few times a day, or whenever, I check my spam folder, sending the true spam off to spamcop, and refiling the false positives. False positives are rare, and in my case they have almost always been non-critical things, such as shipping confirmations, and notices my frequent flier statement is ready.
So utterly useless, no. You are falling into the common trap of thinking that because something isn't perfect, it is useless. Certainly my method can be a bit annoying, but so far is better than the alternatives, which are either letting myself be interrupted by every spam (12 so far today, and its not even 11am), or not noticing important e-mails until long after they arrive.
Why not have the anti-spam filter reply to the message sender with a message explainig that his message has been filtered and requesting that he resends the message to a specific, one time only, e-mail address? Spammers would get all their e-mail back and someone trying to contact you would have a way of sending you an e-mail even if he gets filtered.
An example:
- jonh_doe@example.com sends an e-mail to joane.doe@example.com
- For some reason Joane's e-mail reader rejects the e-mail as spam and sends a reply to John telling him to resend the message to joanne.doe-617131243@example.com
- John does has told and the message his delivered
- The generated e-mail address is deleted
This solution combines the best of spam filters and systems requiring the user to prove he is human (like Spam Arrest). Could this work?
I attended the conference on spam at MIT. The conference would have been more accurately labelled a 'solving spam with the hammer we know about' conference since no other solutions were accepted - although several people besides myself submitted authentication based papers.
The big problem with the Bayes approach is false positives. Lots of great statistics were quoted but the claims were simply not credible. I don't believe that Spam is such a simple problem that the performance of naive Bayesian techniques is several orders of magnitude better on that problem than any other.
So really the trick is to swing the problem arround. START from the problem of making sure that anyone with a legitimate reason to contact me can do so without interference from statistical filtering techniques. The proper place to apply those is on the mail I cannot authenticate in that way.
I dislike the bounce-back loop as a filter for personal correspondence. I think it is great for the purpose of a lightweight authentication mechanism for mailing list subscriptions. I get very irritated when people use it to filter email, particularly since all my email is signed. People should not substitute their ad hoc authentication mechanisms without first supporting deployed standards.
The other problem with call back loops is that if they are used widely they will become a bigger problem than the spam, this is why I have been urging Microsoft et. al. NOT to support them. The trick that the spammers have developed to get round the callback loop is to steal addresses off mailing list archives and send forged messages to the other members of the list. So work out the effect that deployment of the naive bounceback hack would have.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Hey, this article proves they are ultimately detrimental to society and politics, and therefore should be prosecuted.
SatireWire headline from last year: "House sends spam bill to Senate; Senate spam filter deletes it".
When all you have is a hammer, everything looks like a skull.
a ./ reader with a girlfriend! /end joke/ :-)
Here's a thought for the future. What if the number of spammers (thorough email, push advertising, etc) trying to spam you gets so high that the only way to wade through it is to use filtering. Combine this with "intelligent agents" (sarcasm) sifting the data that's comming to your door, and you might never see a single deep discussion or controverial thought again.
Will spammers, not the government, cause us to censor our own experience of the interent (and all other news sources)?
The British political process is supposed to be open to public scrutiny!
Oh? Many people set their filters to tag'n'bag (or simply dump) any non plain-ascii email. I treat any email with HTML, base64, or an attachment of any kind as probably spam and potentially dangerous, and inspect it before reading it.
One line blog. I hear that they're called Twitters now.
haha, so the politicians relied on their email to remind them of the "genuine political business" they should be discussing?
so without their email, they were all sitting around saying to each other "we need to realign our political strategy to centralize what is decentralized into a whole new paradigm"
One of the many things I hate. thingsihate.org
And both of them with the great honour of actually being _listened_ to.
Real Daleks don't climb stairs - they level the building.
You matter how good your filter is, it seemes as if it always makes some mistakes.
It might be a better idea to rate spam by what the odds are that it is spam. Then you rate all you email but how sure you are that it is spam. Start reading your mail till you get to the spam, after that you can be pretty sure everything that comes after it is also spam. You get to define the final boundry, the ratings just help you find it.
Spam Seive, and other filters come close but still still try eliminate any grey area so you will always have false positives, and spam that gets through.
Silence Bossy Meat Creatures!
So, *that* is why Pete Townshend never heard back from his MP!
I hate to do this because it's only partially complete. But I have a concept worked out on how to handle spam that works extremely well and removes the chance of false positives, especially from Real People.
It's not a money-making scheme, but it is prior-art <grin>.
The idea is a hybridization of SpamAssassin and tmda (tagged message delivery agent) wherein you accept all email into your inbox and the spam goes into a spam mailbox. Nothing New...
The cool part comes in when you start automating the spam_mail similar, at least conceptually, to what I have on my website. Shameless plug here
The idea is that you send out an email confirmation, similar to tmda, for only that email which is considered spam (by SpamAssassin). This means that most of your regular communications would go unhindered. But it would also make casual contact via email the easy and simple function that it is supposed to be.
These notions of having an email list of only your known contacts is a pain in the arse and most times met with extreme hostility. This is especially true if you are attempting to contact someone privately from an email list, or from a solitication from their website.
I have to warn you that if you use the code as described on my website you will probably break your server in the first day. I've rewritten it to scale much better (1,000 spams every 10 minutes). But I haven't had the chance to post the new code. But conceptually it rocks!
I've processed something like 20,000 emails without taking a single false positive, unless the original sender vegged... but then he didn't really want to talk to me anyways now did he?
The point is, it places the responsibility of delivering spammy mail to the sender. I do not have to receive it. However it allows the non-spammer to go about the internet unhindered.
at this point, not enough spam is coming in that format to me (meaning "none") for me to worry too much about it.
I'm surprised by you not getting SPAM in that format since I get it all the time, of course there is a good chance that the spammer is including enough text in the message that the ones you are getting are being caught by the filter.
FYI, trying to find an example in my Yahoo based account I use for spam-at-risk correspondence, I see that they have now added an option to not display embedded images in email by default - if you select that option on your email preferences page. After changing to that option, I find that most of the spam in my inbox all looks the same with a big empty space where an image would normally be embeded in an HTML table, followed by some fine print that says "if the image doesn't display on your screen CLICK HERE" and then by "If you would like to unsubscribe from [insert name of spamming service du jour here] then please send an email to this address (or paint a big fat target on your chest and try storming Saddam's Palace - which is the most certain way to avoid future spam).
Work for Change & GET PAID!
TMDA
Regular mail is anything SpamAssassin rates less than 5, I get a few false negatives- no real biggy.
Spam Low is anything SpamAssassin rates less than 8.
Spam High is anything SpamAssassin rates 8 or more.
My ratio of Spam High to Spam Low is 30:1. I easily scan my Spam Low in seconds. I glance at my Spam High only for entertainment before trashing it.
The false positives that end up in Spam Low are usally mailing lists that I have not white listed. When I spot one, I adjust my white list.
I am eagerly awaiting SpamAssassin 2.5 which has Bayes filtering to eliminate the very few false positives I get. As I understand it, this filtering in combination with SpamAssassin means I need to provide no feedback to the filter. Yet, spammers will have a different Bayes filter, therefore they will be unable to adapt their spam to go through my filter.
Net result: SpamAssassin Rocks!
If you get a lot of e-mail from the same (good or bad) sources filtering works well. But it seems if your account gets a lot of e-mail from a lot of previously unknown senders each of those people has a different writing style and uses different words (except AOL users haha), and filters (especially bayseian) have a harder time distinguishing good from bad.
Politicians or anybody receiving "unsolicited" but legit e-mail from the public should definitely question whether filtering is a good idea, assuming they want to read every valid e-mail.
Then they can write some bots that generate random inane chatter in their posts.
Oh....
....never mind.
Anyway, if you filter spam by hand you get false positives as well. Spent any time clicking delete checkboxes in yahoo mail. How difficult is it to click one extra title as mistake? Once your filter has a better rate than you, you are better off letting it do the job even if it deletes some of your normal e-mail.
As various forms of "content filtering" become widespread, expect more of this in the future.
If your children ever found out how lame you are, they'd murder you in your sleep
Alright. render your 5k text message into a 980KB jpeg file and send it away.
No, the problem isn't voting. The problem is believing voting is the only say you have in government. Anyone can and should fax/write their representatives about issues they care about. If they think a lot of people would agree with them, they should put the word out for those people to do the same*. The Congressional Vacuum works both ways - if the only people talking to them are lobbyist, those are the only 'facts' they can base decisions on.
Of course, the level of success this has will very from representative to representative, and how close the issue is to what they ran on and their parties are taking stands on. For example, it would take a lot more voter pressure to get a candidate to flip on taxation, which they generally are well defined towards, as opposed to the DMCRA vs. the DMCA, of which they've only heard about from Big Businesses.
* Cheap example plug
R: That voice. Where have I heard that voice before? B: In about 365 other episodes. But I don't know who it is either.
Don't every give out your primary email, unless trusted friend. For all others make a catch all account on your domain, and give the email organization name_SPAM@domain.com Then you can track where the spam is coming from and who they sold it to etc...
Hardly, had you read my post rather than being a rather obvious shill for the product you mention you would have seen that I don't think that bombarding everyone with callback messages is an acceptable solution. There are better forms of authentication available than email callback loops that do not require end user intervention.
If everyone used this method the spam senders would simply extend a technique they are already using, taking email archives from the internet and forging messages purporting to come from one of the people on the list to the other list members. The person whose address was stolen would then get a massive attack of loopback messages.
If you want to authenticate senders the solution is S/MIME, an IETF specification that is designed to do email authentication without user intervention.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
If only I could filter out all the anti-Microsoft FUD this place spews forth
Manipulate the moderator system! Mod someone as "overrated" today.
You shouldn't be rejecting spam! You should redirect it to some folder/mail store and get your end-users to check it every once in a while. Simple.
And it's obvious that they should use PGP to encrypt emails from huffy constituents or emails that contain draft language of a law that will be public anyway?
Imagine if that email complaining about a 4% increase on beer tax fall into the wrong hands...
I am willing to be that the SAS doesn't send emails to MPs (or anyone for that matter) outlining their next secret operation.
"When it rains, it pours." --Morton's Salt
Only 6,000,000? Surely your mail server can handle more than that?
Looks like you've got some spare bandwith there buddy. Can I possibly interest you in a new diploma, penis enlargement, nude pictures of various Hollywood babes, a new credit card, a low, low, rate mortgage and a my exclusive guide in how to make a million on the stock market?
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
Yes, the guy really needs a better spam filter.
giel.y contains 2 shift/reduce conflicts
I deeply sympathize with ISPs who are battling spam, and with the blocking lists advocates, but collateral damage from blocking can get personal.
Lately I can't send email to my wife!
About 5 days ago, her ISP blocked all email from bellsouth.net, which is my ISP. Of course, they didn't tell her that, or me. Just out of the blue anyone who sent messages to any account at her ISP, via bellsouth.net, was told "Access Denied."
A lot of the reports of collateral damage in the anti-spam war focus on legitimate mailing lists, newsletters and the like. But it is also individuals who are finding their established means of communication cut off, and very abruptly so. Imagine a journalist who wants to interview my wife (she is the author of Network Security for Dummies). He sends her a message via an account at bellsouth.net, which is a large ISP (and not what the average person thinks of as "flaky"). The response "Access Denied" suggests the account is closed or unpaid for. Connection and opprotunity are lost.
Do I think we need to abandon blocklists? No, but we need whitelists as well, that allow mail through from verified and approved sources that are held to a high standard and thus given privileges (instantly revocable if need be for violations). This is technically possible. And it can be achieved. Think about the blocklist infrastructure--who would have predicted how quickly it could arise and refine itself to current levels. What we need now are more pieces of the puzzle to be put in place.
Stephen
A few weeks ago the Scunthorpe town council decided to implement a nasty words filter on all email received, just to reduce the volume of abusive email they were receiving.
The email filter worked out very well indeed - well, too well. Absolutely no mail was delivered. It took a while for them to realise that their own town name contained one particular rude word, and considering that their town name was part of their email address, all email had to have a certain word in it.
Irene KHAAAAAAN!
Your message was here?
Perhaps, with a flood of spam,
I deleted it.
I started using a filter called Spammunition a while ago. It's a free Bayesian filter for MS Outlook. (Not my ideal mail client but it's what we use at work). It's great. No false positives, and catches all my spam.
I believe that this practice is not that good --- especially when the mail was NOT sent by a real human, but ie. mailing list subscription robot, some account creation verification mechanism sending you password by mail etc. Also you may easily annoy those who write you a mail by bothering them with this automation, it can be easily considered as impolite. And the real fun starts when two people with this filtering try to get in touch; you protect yourself from mails not generated by a real human, but at the same time you produce such mails yourself.
I think the middle path is the best one: automatically add the spam negatives to the approved sender db and send the verifications only to senders of mails identified as spams. You will get no false positives, in 99% of cases you will be able to even subscribe to a mailing list easily, and you won't bother the other people that much (and if you will specifically mention in the verifications that they are sent because the mail looked as spam, I think they will be much less angry at you).
It's not the fall that kills you. It's the sudden stop at the end. -Douglas Adams
I use a similar this method as a filter, and could care less wether or not an email is signed or not. If you are not in my contact list or if I have not flagged you to send me mail, it will require you to include a daily random password on the subject line. I have had lots of people use this over the last couple of years. I have only recieved positive feedback about it, and just as importantly I have recieved virtually no spam for a very long time. The only disadvantages I found were:
A) Mailer daemons use nonstandard replies, and do not include message identifiers. My autoreply is frequently sent to bad addresses and triggers mail daemon responses. This means that the only way to determine if mail I send was undeliverable is to comb through a long list of messy mailer daemon replies. This is a small price to pay for the advantage of recieving zero spam, and not having to worry about false positives. Especially considering that it isn't often that I have to even check to see if someone recieved my mail because most people send me replies when I mail them.
B) It was vulnerable to being used to mailbomb. I fixed this by adding a flood protection to limit the number of autoreplies it can send to one address each day.
Regarding the mailing list archives.. I have been a member of mailing lists for years and never recieved spam masquerading as one. That is not to say that it can't or won't happen though. That seems like it would be a problem with mailing lists disclosing the member addresses though doesn't it?
I don't know how Apple does it, but the Mail client that came with OS X has some amazing anti-spam widget running. With essentially no training it misidentifies maybe 1 spam in a hundred, or two hundred (I get about 50 spam a day). I keep looking for good email in the spamcan and almost never see anything there except announcements from companies I've done business with in the past (pseudospam). I haven't had to deal with any really nasty spam in my inbox for months, no joke.
Does anyone know how Apple has pulled this one off?
=^..^= all your rodent are belong to us
I can't believe that they get so little spam. Only about 5% of my e-mail is legit. Therefore, I need a very good spam filter system and uses a combination of procmail pattern matching, scoring and Bayesian statistics.
and one that is in use today, is to not accept any public email at all.
The US Senate and House of Representatives have their member's websites with a contact page utilizing a web form to submit letters. Since this email address is hidden by the web server, the only spam that could possibly get to senators is someone specifically writing a program to submit information for that specific web form.
Since no spammer would need to spam senators (unless someone tries to mail bomb them, but that is an other issue all together), nobody would spam them.
This also solves the problem with the post office mail and anthrax problems that happened just after 9/11. The quickest way to contact your senator is by fax, but even this web form is higher priority than snail mail.
Two infinite things: your stupidity and mine. But I'm not sure about the latter. If my sig offends you, I'm sorry.
If governments find spam unacceptable, and resort to spam filtering, and then find that unacceptable because of false positives, the next recourse is spam legislation. Therefore, false positives are good.
... "Give me a woman who loves beer and I will conquer the w
lol
after awhile, the trolls get kind of funny, as they are so predictable.
troll: say nice things, because if you say negative things, do not be surprised if someone disagrees with you, especially the original poster on an emotionally sensitive subject.
more empathy, less negativity.
you will find your web posting experience to be more enjoyable.
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
I believe that this practice is not that good --- especially when the mail was NOT sent by a real human, but ie. mailing list subscription robot, some account creation verification mechanism sending you password by mail etc. Also you may easily annoy those who write you a mail by bothering them with this automation, it can be easily considered as impolite. And the real fun starts when two people with this filtering try to get in touch; you protect yourself from mails not generated by a real human, but at the same time you produce such mails yourself.
TMDA has thought about all of these issues, and has solutions to many of them. That's as good a reason as any for using TMDA as the confirmation manager rather than reinventing the wheel.
It is important to distinguish between rejecting a message (in which case the sender gets a "550 spam" indication) and discarding a message (in which case neither the sender nor the receipient is notified). Only the SMTP server can reject a message, it is too late by the time it has gotten to the message user agent (client).
If the anti-spam software rejects a message it is usually trivial for the sender to modify the message or find another delivery method and little is lost. If a message is discarded, the damage might be much greater.
Bayesian spam filters usually run on the client, and have to discard messages but there is no particular reason they couldn't run on the server.
The client can't reasonably return a "DSN" via email since the envelope from (even if known to the client) is probably a forgery, so responding would just be creating more spam. The SMTP server
can reject the message before it is accepted with an error code, it doesn't have to send an email with the error message.
From the article:
We receive over half a million incoming e-mails a month - so far the filter has blocked about 900 a week, which is about 1 in 180, much less than 1%
If only 1 in 180 messages are classed as spam, why are they using the filter in the first place? If the average amount of spam received across the board is less than 1%, then those MPs who complained of being inundated with spam must be few in number.
Why should the whole system suffer because of those MPs? They should implement their own filters if they have a problem.
The helpdesk has only received a handful of unblocking requests.
Not surprising. How are people supposed to know they're missing out on important e-mail messages if they never receive them because of the filter?
Incidentally, my ISP uses a spam filter which is completely transparent to the user. Any messages that get filtered, legitimate or otherwise, I never even know about. Most users don't even know the filter is in place. I'll be leaving them when my contract is up, being sure to first check up on the practices of any new ISP I choose.
Read, could care less how much aggravation you cause others.
If the message is signed then you know that it came from that individual. You should not require further authentication, you might however decide to use authorization.
I have had lots of people use this over the last couple of years. I have only recieved positive feedback about it,
Duuhhh, hardly suprising since you filter out any negative responses.
The point is that this type of scheme is only going to be socially acceptable if few people use them. As soon as lots of people do so they will rapidly become an annoyance. In effect you are reducing your spam by .
It looks like you are also one of the major assholes who sends a callback message every time they get a message from someone you consider insufficiently important. Most people would think that responding to one callback should be enough to be whitelisted by default.
Regarding the mailing list archives.. I have been a member of mailing lists for years and never recieved spam masquerading as one. That is not to say that it can't or won't happen though.
It probably just means that you have been lucky. This behavior has been happening for quite some time. Of course, since you claim to have an email hack to stop spam the problem would be hitting everyone else.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
If the message is signed then you know that it came from that individual. You should not require further authentication, you might however decide to use authorization.
So by that logic, if a spammer sends me mail that is signed, I am obliged to accept it. I don't think so.
Duuhhh, hardly suprising since you filter out any negative responses.
You obviously don't understand what I was explaining. My autoreply contains a password that you include in the mail. This only requires user intervention which prevents the vast majority of spam. If you wanted to send me a complaint about it, you simply include that in the tagline. duhhhh.
It makes you look like an asshole to make assumptions about people. Despite your rude presentation, you did make a point. That is that this is only acceptable because it is not in use on a large scale. For that I don't have an answer. You sound like you do though.
Comment removed based on user account deletion
SMTP is junk, no question about it.. there is almsot no way to effectively eliminate 100% of all spam without impeding others efforts to contact you, so i suggest that a new protocol be drafted up.. a common suggestion, all it would require is TRUE authentication of where the mail came from. Of course SMTP wouldnt go away, the 2 networks could operate simultaneously... i have a feeling SMTP would die out rather quickly though and become something nobody ever uses
You almost had it, but not quite.
Here in the US, they toss out all letters, faxes, emails and ignore phone calls. Congress has a very simple way to get their requests from lobbyists. Each "representative" hosts a "fund raiser" where the lobbyists have to pay thousands of dollars to get in. Each lobbyist gets a turn sitting next to the "representative" so he or she may discuss political issues "critical to the general public." No need to mess around with sorting through countless junk! ;-)
Hello, and thank you for your generous offer!
I am a very interested in participating in this venture. I have recently made a lot of money from a very good harvest of dental floss, and have been looking for a way to invest my money.
Please hurry to contact me with more information.
Irene KHAAAAAAN!
So by that logic, if a spammer sends me mail that is signed, I am obliged to accept it. I don't think so.
Seems you don't bother to read what other people write either.
Your callback loop is performing authentication. What you are talking about is authorization.
If spamers start signing their messages an appropriate response would be to have a whitelist of people who you have authorized to send you email. If you have a strong authentication mechanism you can include or exclude people on the basis of their domains, for example allow mail from anyone in mit.edu, exclude anything from goatse.cx etc.
It makes you look like an asshole to make assumptions about people. Despite your rude presentation
I happen to consider your callback hack rude so now you know.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Dick van Dyke -- Who could be more harmless, or more likely to be screened?
"All that is required for evil to triumph is for good men to do nothing." - Edmund Burke
...is that the MPs aren't filtering their e-mail, it's under the centralised control of Parliament's IT Services Dept.
Consequently, MPs are not receiving mail about e.g. the Sexual Offences Bill silently. They can't periodically check their "junk mail" folder for false positives, they have to know (via out-of-band communication) that they've had a false positive block and then go cap-in-hand to the IT Dept to ask for the mail to be released. Anything that gets caught that they don't know about, well, they won't know about.
This is why all spam filtering should be within the control of the user.
I apologize for people who might actually want to talk to me about mortgages, casinos, or sluts. But I figure it's okay because I'm not interested in those things.
"Democracy" means "government by the people". When the people's voice is simply thrown out because its volume is too much, that is ignoring the will of the people. When focusing instead on those special interests that have worked hard to earn a special status of favor with politicians, that is a form of aristocracy. (Okay, so it isn't necessarily government by the "rich" in terms of money, but those lobbyists are certainly rich in politician-attention-grabbing resources that the average American is forced to live without.)
I would recommend that the UK continue on its current course. A better plan might to be to hire several politically qualified persons to act as email secretaries -- save the controversial, coherent, and funny letters and essentially keep the officials' inboxes down to manageable levels. (I'd imagine that something like this is already in place anyway.)
When other filters fail, set up your own rules to overrule them.
Like I said, I have never recieved any complaints, and if a spammer forges your address and a couple of messages get bounced to you before my flood protection kicks in then I apologize in advance, but the likelyhood in that happening is slim, but I am not the one pretending to be you, and I'm not sending a nasty note or reporting you to your isp or anything. I am simply saying that if it is truly you who attempted to contact me, then please confirm this by performing this operation.
No, because it prioritises your mail. Mail from certain adresses is always real, mail from certain adresses is always spam. So, I don't check mail which looks like spam as often as I check mail which looks like spam. When I reply to someone who was treated as spammed, I tell them why the reply was delayed which should help the sensible ones avoid other people's fiulters. (don't mail HTML is a good rule for instance).
Automated trusted corespondents ensures that a real human, accountable for his or her actions, sent the email.
This would just make me give up emailing the person. Not useful if I was trying to buy something from them.
_O_
.|< The named which can be named is not the true named
After years of scratching my head over that Monty Python sketch with the bloody vikings, the joke about spam makes much more sense to me.
Eternity: will that be smoking, or non-smoking? I Corinthians 6:9-10
Examination of RFC821 and RFC2505 shows that any 55x error is (paraphrase) a permanent negative result code indicating that the receiving system cannot accept the message.
The third digit allows for fine gradation of replies, and "552" is a common return code from a spam filter.
This is a design decision -- in many cases, you don't want your anti-spam filter to return a result that could be interpreted to mean:
I do not deploy Linux. Ever.
How it works:
... and then there were none
I currently get one or two spams every day, so I don't bother with filtering. But if I someday deside to do it, it's very easy: I can just set up a really agressive filter and not worry about losing any important mail.
Here's why: An average spam filter won't catch any finnish (or any other non-english language) messages, and there's no such thing as finnish spam, so basically I'm all set. Well, maybe whitelisting @myworkplace.com would be a good idea.. Now, what I'd be really interested in would be a filter to weed out all messages which are boring, irrelevant to me, I've seen already or just plain stupid. I'd pay for that, actually.
Many legitimate people unwittingly send html emails. You may not want to receive emails from those kinds of people, but I don't know what the big deal is. Personally, I have KDE mail, I set it not to render html, and whenever I do receive some spam, I filter it by the email which sent it to me. This way, I don't get false positives, I do receive some *new* spam once every month, but I certainly don't receive enough to lose any sleep over it.
I'm not sure what you mean by that. You do realize that most spammers forge the From address (and frequently use someone else's actual email address)?
It really depends on volume. If you have the bad luck to get added to a number of "millions" CDs, or if you use it on a web page or Usenet posts, you'll get a LOT of spam. (I use a sacificial hotmail mailbox for that, and forward the occasional non-spam to my real mailbox.)
One line blog. I hear that they're called Twitters now.
I do realize this and I don't have a problem with blocking a strangers' email address, even if it's a forgery. Right now, I have 30+ specific filters set up, so I know my email is being passed around on the internet, but I guess I really don't have that much spam.