Slashdot Mirror
Ask Slashdot: How Effective Is Your ISP's Spam Filter?
An anonymous reader writes with the question in the title: does your ISP do a decent job culling spam? The reason I'm asking is that my ISP is Verizon and the Verizon spam filter is next to useless. It only blocks 15% of spam while also blocking 5% of legitimate emails. I've tried calling Verizon support a couple of times and the experience is about as pleasant and productive as banging my head on a wall. At this point I think my best move is to change ISP, but before I go around changing my email address at probably dozens of web sites I'd like to be sure that a new ISP would actually be better.
Uhmmm, why are you using your ISP's email in the first place? It's far better to use a third party email provider, so that you can switch ISPs at will without having to change your email address.
...si hoc legere nimium eruditionis habes...
>> my ISP is Verizon and the Verizon spam filter
Not too many people 'round here are dumb enough to use their ISP as their email provider. Fix that problem first. (Closes ticket.)
Don't tie your email account to your ISP. Decide how you are going to get your email independently, then your ISP is just the pipe.
Two benefits:
- You can change your pipe without causing problems- your email address doesn't change
- You have a lot more options for email providers than most people have options for ISPs.
I didn't even know my ISP had a spam filter. I have spam filters on my accounts and the junk folder fills up constantly. In fact, my ISP is one of the worst spam offenders, sending me constant offers for great deals if I just sign up for their cable TV and other "special deals".
You are welcome on my lawn.
Even if they had the world's most perfect spam filter, there's some very strong reasons to not use the same provider for both internet access and email.
Don't fool yourself on this one.
You can set up a filter that removes (what you consider to be) an acceptable TP:FP ratio, but it won't be effective for long. The Spammers are constantly adjusting their tactics to get around filters. Eventually the noise will take over and you will either lose an unacceptable amount of non-spam email or you will receive an unacceptable amount of spam email.
You cannot win with filters, period.
The truth of the matter - that a lot of people seem to either not be aware of or not be concerned with - is that spam is an economic problem. Spammers don't send out spam to piss you off, they send it out to make money. No amount of filtering or criminal prosecution will change that; in fact it generally just increases the total volume of spam that traverses the internet continuously. We all pay for this spam to be transmitted, stored, processed, downloaded, etc, even if we never buy any spamvertised product. We pay for it in that it increases the consumption of internet bandwidth, it increases the consumption of storage at ISPs, and has other downstream impacts as well
If you want to make a difference on spam, you need to go after the only thing spammers care about - money. The most effective tactics ever used against spam have been the ones that prevented spammers from getting paid, nothing else - not even the sum total of all the filters ever installed worldwide - has had an impact even remotely near it.
Stop thinking about filters an start thinking about solutions.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I use both Gmail and Google Apps for my own domain email and their spam filtering is very good.
They would much rather you join the 21st century and use any email service except theirs. Take the hint, choose Microsoft or Google and move on...
With the increasing growth of outsource email, it's getting really hard for spam detectors to distinguish between real spam, and email sent on behalf of one company by some outsourced mail/customer contact management company.
Here's the technology my ISP uses: http://www.escom.com/ (Disclosure: The developer is a friend-of-a-friend of long standing.)
Email spam control is a full time job. If you host your own domain and mailserver you can have complete control over your filter rules, but it's a good bit of maintenance. You can host your domain at a 3rd party mail provider and take advantage of their rules. Google purchased postini years ago and incorporated their email filtering into their products, one of the reasons gmail has rather good spam filtering. I've never had good luck with an ISP's mail filtering, except for an old local shop.
When I operated an ISP we had an absurdly effective mail filtering system. 2000 users in the domain, on average 1,000,000 spam messages blocked per day (the owner had sold the customer mail list several times (somuchate)). This required LOTS of work and honestly wasn't worth the effort.
I've had a GMail acct since 2004, haven't had an issue since. Left that ISP job, been through 2 other ISPs since then. Haven't had to change anything about external accounts, still have mail archival going back over a decade now and very few spam messages get through, very few legit emails blocked.
I've also got very little spam on my Gmail address as well..
- This sig deliberately left blank. Nothing to see, move along.
Boy, the number of comments has really fallen on all stories since Dice's last "upgrade."
You can set up a filter that removes (what you consider to be) an acceptable TP:FP ratio, but it won't be effective for long. The Spammers are constantly adjusting their tactics to get around filters. Eventually the noise will take over and you will either lose an unacceptable amount of non-spam email or you will receive an unacceptable amount of spam email.
Disagree. I have used gmail for quite a while and I very rarely see spam outside of the spam folder. This has been the case for many years now. I honestly cannot remember the last time I had a false positive (non-spam sent to the spam folder) and false negatives (spam that gets to my inbox) are fairly rare - less than 10 a month usually. It's good enough I don't even bother to check my spam folder anymore. When one does slip through I just flag it and the problem goes away. Spam effectively almost doesn't exist for me. While I do agree that no filter is perfect it isn't that hard to have one that is highly effective. With enough people flagging spam filters can be very useful in automating spam removal. It doesn't entirely solve the problem but it has made it manageable.
You cannot win with filters, period.
I have no illusions that I am going to eliminate spam entirely. The ISPs are the only ones really in a position to do something about the problem. So far nobody has come up with a credible and effective solution and I don't see that changing anytime soon.
My spam filters are very effective, but they're not on my ISP's servers. My email comes in through my own custom domain name sitting on an 'Nix Apache CPanel shared web host that I rent space on. I get to setup. This is very effective. Then my MacOSX Mail App does the next level of filtering. I have each level set for whitelists, blacklists, keywords so that there is very little in the way of false positives and only about 0.1% to 0.001% of spam is getting through (I have stats). There are surges where it rises to the 0.1% level when the spammers try something new but then the system adapts, recognizes them and zaps them.
Verizon is probably not a good choice for a spam filter as they do not have a lot of incentive to care.
Do not use ISP mail. Use a better free one. Or pay a little and get something with a highly configurable filter that works. I use fastmail.fm and they work beautifully.
Comcasts Spam filter is crap, but who in the world uses their ISP email? Gmail and call it done so you can change ISP without disruption to email.
Do not look at laser with remaining good eye.
I use gmail, and it frequently sends my forum registrations and password reminders to the spam folder, and delivers actual spam to me, then offers to send it an unsubscribe notice to let it know that my address is valid.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Like everybody else is saying -- Why are you using your ISP's email? They should only be your pipe. I personally stopped using any ISP's email in the 90's... It was after the first switch over that I figured this problem out.
Originally I ran my own domain and spam filtering. I was on the first batch of first spam from those lawyers. Fuckers. Anyway...
Have since migrated domain email to Google apps -- not free anymore for you unfortunately, but on a user @gmail.com basis is still very free.
For speed Google wins -- never even came close to matching their speed for users with gigs and gigs of email they refuse to delete. Not that I'm one to talk.
Their spam filtering beats anything I've seen. I always had too many false positives on my setup; Google has really had one problem in the last decade with that -- false positives from the COPIERS (they have their own accounts and in the domain mailing to same domain users). Annoyingly I had to add a filter to each user to fix that problem.
Otherwise their spam filters are dead nuts on for me. One, maybe two spam messages will hit my Inbox in any given year. My account will @ the .com variant of the domain will get 2-5,000 spams a DAY...
Use Google.
Why tie your email to your ISP? I ignore my ISP's email service except for email from them about my account. If I were you I'd set up an email account with another service and use that as my primary email. That way when I change ISPs (and I will, whether because I moved or because I got fed up with crappy Internet service) I don't have to worry about changing my email address everywhere. In fact it might not hurt to have accounts at more than one email service, so you have an established backup in case it's needed.
I honestly cannot remember the last time I had a false positive (non-spam sent to the spam folder) and false negatives (spam that gets to my inbox) are fairly rare - less than 10 a month usually. It's good enough I don't even bother to check my spam folder anymore.
If you don't check your spam folder than you cannot make a statement of your FP rate, until someone tells you that they sent you an email and you realize you never saw it - but you may be too late to do anything about it by then as the spam folder retention is not particularly long on gmail.
The ISPs are the only ones really in a position to do something about the problem
No. By the time spam makes it to your ISP you've already paid for it. At that point you have to pay your ISP to accept it, analyze it, tag it, and store it. Sure, they do it all at such high volumes that the cost is pretty minimal relative to your monthly bill but the cost is not zero either. As the volume of spam goes up - and filters only cause it to go up - the cost of dealing with it goes up as well. You have to pay for your ISP to keep more storage around just to hold spam so people can check their spam folders. You have to pay for your ISP to upgrade CPUs and RAM to process spam more quickly.
But if you like paying for spam, your ISP is happy to charge you for it.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
IIRC, mine was Time Warner, and it was just as bad! I don't think gmail existed yet. That's probably when I switched to yahoo, and it was better than TW.
Slow down, cowboy! It has been 4 hours since you last posted. You must wait another few hours.
I use both ISP e-mail and Gmail over IMAP with a proper e-mail client The ISP Spam filter works extremely well, always has, which is one reason I still use it. For years when someone has mentioned spam, I often said: "what is this spam thing you speak of?" For those who decry using ISP e-mail as a trap to keep you subscribed. Well perhaps, but since that ISP has had the fastest speeds and better service than any other local provide, I'm not worried that much. They did drop USENET though.
Gmail's filter, while effective at stopping spam, has a much higher false positive rate. I think it filters BEFORE my own filters kick in, when I'd rather it filtered AFTER.
and so dose
microsoft and yahoo and "fill in the blank"
they ALL!!!!!!! do for selling advertising
BUT it takes the nsa or gchq or any other Org to REALLY read the mail
"I don't pitch OpenSUSE Linux to my friends, i let Microsoft do it for me
I only route their daily digest to a folder because maybe every YEAR OR TWO I check for a false positive that may or may not be there.
What gets through is always a question. If you've purchased from a company and might do so again, are their weekly, biweekly or daily ads spam? I have an extensive local filter so most ads go to an "ads" folder for more frequent removal and pure spam gets added infrequently as needed.
If you don't check your spam folder than you cannot make a statement of your FP rate, until someone tells you that they sent you an email and you realize you never saw it - but you may be too late to do anything about it by then as the spam folder retention is not particularly long on gmail.
I would hear about a missed message if there were one and every once in a while I check just to be sure but my statement stands. It's good enough that I really don't feel the need to bother checking.
No. By the time spam makes it to your ISP you've already paid for it.
Wrong ISP. It needs to be handled by the outgoing ISP, not the incoming one. Yes it will cost money but less than later in the delivery process. You aren't going to eliminate spam but you can minimize the economic impact of it.
In any case the notion that you are somehow going to get greedy a-holes to stop sending spam is delusional. If there is money to be made (and there always is) then the problem will persist. It is quite correct that it is an economic problem but there is no viable economic solution. All we have a imperfect technical solutions. If you have an actual solution to this problem then by all means share with the class but there is a reason why there is a form letter for people who think they have the solution for spam. This is nothing new.
Just use Gmail. The SPAM filter there is quite good. Yes, on occasion something gets through when they haven't learned it yet. Yes, on rare occasions something gets put into Spam that shouldn't (so just check your spam folder every week or so). But overall I can't complain
Verizon FIOS (aka Verizon broad band) has what they call "SPAM Filters" but they are pretty much worthless. It used to be pretty good, but they have apparently not managed the filters for a few years and now they are really not effective at all. It is so bad that I'm almost wondering if they actually HAVE a filter anymore, or is it just a check box on the web screen they put there to keep people from complaining....
I've been seriously considering buying a domain and setting up my own server just so I can get some reasonable filters back, but who's got time for that? What I do now is just use GMail for everything and I just use a throw away account for most things with specific filters to forward only what I want to see and keep my *real* e-mail address for only the most important things and close friends.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
I honestly can't remember when I last used my ISP's email address, but I think it was in the 90's.
Why are you using it? There's not a single good reason I can think of.
That's still much better than my ISP.
Signed,
a Canadian.
Get free satoshi (Bitcoin) and Dogecoins
ISP email is typically anemic in both storage and security. Why in hell would you ever use it?
Run your own MTA, problem solved.
Yeah, you'll never get ANY mail if you're not on the white lists. Problem solved!
I strongly discourage any friends or family that I notice are using local ISP email accounts from doing it. I wouldn't trust it to keep the good email. LOL, for a while, our local cable co was spam binning their OWN newsletter. I do a lot of work with an opt-in email newsletter, so I've monitored a test email box with the cable company for years - it's bad, bad, bad. Coincidentally, they've recently changed something, and now a lot of spam emails don't even hit the junk folder, they simply vanish.
Ditto all the stuff about not using your ISPs email server.
Personally, I'd experienced a declining state of affairs in email hosting at a price that I thought was reasonable. I eventually got to the point where I hacked together my own (receiving only) email server in Python. (Also using pieces from Django to connect to an SQL backend.) My outgoing email is sent via a cheap hosting package I have that also doubles as a backup if I have a major incoming server problem... I can just point my MX records back to them.
https://github.com/marvinglenn/asnn-mda (Open Source licensed, free as in both speech and beer)
I'm getting a few false positives in blocking, but that's more due to incompetent configurations by legitimate companies. For example, US Cellulars SPF records don't clear their own sending servers. A few other business, my bank in particular, use a registrar that I find to be such a spam source that I reject email merely by being registered there.
This software is not for anyone that doesn't have at least a modicum of hacker spirit, but I've gotten it to the point where it's been extremely effective for me, hopefully not too hard to set up for others, and gives me the cathartic release of dropping F-bombs on spammers during the SMTP transaction.
The whores get mad when the sluts give it away for free.
what about the free email address your college gave you when you graduated? why aren't you using it?
I see a lot of comments here telling the OP to settup his own domain, email service and spam filter. That's a lot of work and cost. Since the OP is using ISP mail, he probably isn't wanting to go the full monty route being proposed by most respondents. Yahoo mail works pretty good. At least as far as spam filtering is concerned. I get a couple a week maybe, if that. Very few false positives as well.
I've been running my own MTA for 15 years. Not on any whitelists. No problems.
I don't know what platform you use but if it's Windows just use the mail client of your choice and something like ESet Smart Security which includes a really good spam filter; I have any number of customers using it and they are all very satisfied.
As far as customer service goes, I've deal with Comcast and Verizon many many times and it's always the same. The person you're talking to seems to have no idea how to address your problem; they put you on hold multiple times while they apparently run around looking for someone who has a clue and in the end you may be transferred multiple times and after spending what seems like days on the phone you either get no help or you get disconnected. Sometimes you could swear these people are paid to be as unhelpful as possible rather than to actually help the company's customers with their problems.
There's a really good reason Comcast was voted "Most hated company" and I'm sure that Verizon and Bank of America were runners up.
Spend a couple bucks grab a VM buy a domain name for 10 bucks. Use google or MS if ya realy need to.
No sir I dont like it.
My company does emails quite often and the emails affect the bottom line of the companies they are going to. Most of the companies that they are going to are small and old, and a large portion use their isp's email. Versison has decided to mark our emails as spam. They claim they aren't etc etc.
Isn't interfering with someones mail a federal offence?
I've got an account at a small local ISP and I know that they use Cisco's IronPort spam filtering and it works great. Additionally, I pull the e-mails down and it's even got my subscriptions (newegg, local brew store, etc) categorized with some mail headers and I can procmail them into folders. I've also got Gmail (it's now $5/mo for your own domain) which works well and I have another e-mail address that I host and use spamassassin. I get a few dozen each week on the one that I host because I'm too lazy to set it up right.
they send it all, spam included, i use seamonkey's junk filter which is decent enough
Politics is Treachery, Religion is Brainwashing
your ISP filtering your mail? I don't, and prefer to filter my own, for completely obvious reasons.
Create a GMail account and POP from your ISP's crappy system.
GMail's SPAM filter is not perfect, but it's very easy to live with.
ISP email is pretty crappy.
"I believe in Karma. That means I can do bad things to people all day long and I assume they deserve it." : Dogbert
That's great unless, for anyone of a number of reasons, you don't want to be thatgeek@college.edu for the rest of your life.
What free e-mail address? My university canned all my accounts several years after I finally got around to graduating. That's a lot of overhead for them to have to deal with I wouldn't expect that to live forever.
ISPs have no incentive to offer spam filtering, and indeed very little incentive to even give out email addresses. Email is a huge PITA and there are plenty of free providers to choose from : Gmail, Hotmail, Yahoo and probably dozens of lesser-known ones.
Also, ISPs have a pretty small profit margin on consumer service, so any costs they can cut, they will cut. So don't bother with an ISP email account.
I have a work email, a gmail account and a couple of ISP-based accounts that I've had for years. The gmail account blocks about 85% of spam, but blocks a lot of non-spam. The big problem is that the gmail account draws a tremendous amount of spam, so the volume of garbage to look through searching for the legitimate emails that were blocked is huge. Because of this, I have given up using gmail for anything that matters. Additionally, I consistently get gmail intended for at least four people who share my uncommon last name, two of whom share my first initial and two of whom don't. Some of this is quite personal and some is financial. The bottom line is that I don't trust gmail at all anymore. All I use it for now is getting grocery store coupons My personal opinion is that webmail sucks. I know other use it happily, but I'm not one of them. Right now I'mm happy with IMAP, and I will probably switch to my own hosted solution eventually.
Some mornings it's hardly worth chewing through the restraints to get out of bed.
How would you expect them to filter spam without reading the messages?
The right to protest the State is more sacred than the State.
Our Computer Science alumni get to keep their email account on the departmental server. (At least until we replace that server.) Students elsewhere on campus? Not so much.
As a matter of fact, for the last 5 years or so, non-CS students aren't provided with any university associated email address -- they have to provide their own. To be fair, this was largely because most of them already had an address of their own and weren't responding to emails sent to the campus address.
Ignorance killed the cat. Curiosity was framed.
I too have had several pieces of E-Mail that came in to my mailbox, but when I attempt to send the exact same E-Mail Verizon declares it spam. I have called and talked to Verizon on this issue several times with no effect. The help people say to send the E-Mail to; spamdetector.update@verizon.net A lot of good that does, as in NOTHING!!! I had to use another E-Mail provider to get around the issues.
What whitelist?
The only problem with self-hosting a SMTP-server is you often get blocked because your server's IP-address is being blacklisted because it's a residential customer's IP address. I have solved this configuring my mailserver to use my ISP's SMTP-server as a smarthost. Works fine.
Another stupid slasdot piece of drivel.. Go ask your 10 year old how to get rid of spam. or load up your own linux server, become your own isp and can your own spam.... FFS is this the most interesting thing we can post on here?
I do not because it seems like bragging when I do use it. I still use it for contact with old friends and professors. I also still subscribe to some of their mailing lists. I have to wonder if the primary purpose of the mailing lists is to get me to donate money for my alumni but I donate anyways even though they certainly have lots of money. Anyhow, it seems like I am bragging if I use my .edu address. I was fortunate enough to have graduated from a prestigious school and posting such seems to detract from discourse as I (I have used it in the past and learned my lesson) am asked about the school, how I got in, what I majored in, what my degrees are in, etc... So I have learned not to post it as it, as I have mentioned, is not productive.
"So long and thanks for all the fish."
Dreamhost
Spambayes
I see a number of highly-rated comments recommending using Google for mail rather than the ISP's mail service.
This surprises me, given the privacy implications. I can reasonably assume my ISP won't read my mail other than for spam filtering. Google, on the other hand, will use your mail as input for their advertising machine.
How Effective Is Your ISP's Spam Filter?
So effective I didn't see this
Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.
As best I can tell, many colleges and universities do indeed offer a free e-mail address to alumni. Most of the ones I'm aware of cost the institution nearly nothing and don't have much overhead. I'll note that very few of these actually host your email - they do indeed can your student account within a fairly short time of graduation. Instead, they'll provide you with a permanent address, usually of the form user@alumnus.college.edu, that is really just an alias/relay that forwards to your choice of email account. Of course, you have to figure out how to set your return address correctly, but this can be useful because if and when you change your "real" email provider for any reason you just need to log in to the alumni server, update your preferred email address, and don't need to ask all of your contacts to update your information themselves.
dmarc tells you about false positive, not false negative. if you send mail from a dmarc server, it will be more likely to reach its target.