How Pervasive is ISP Outbound Email Filtering?

Erris writes "A member of the Baton Rouge LUG noticed that Cox checks the text of outgoing email and rejects mail containing key phrases. I was aware of forced inbox filtering that has caused problems and been abused by other ISPs in China and in the US. I've also read about forced use of ISP SMTP and outbound throttling, but did not know they outbound filtered as well. How prevalent and justified is this practice? Wouldn't it be better to cut off people with infected computers than to censor the internet?"

Comments

Profit comes first

[techno-vampire]

"Wouldn't it be better to cut off people with infected computers than to censor the internet?"

If they did that, it would lower their income and cut into their profits. Filtering outbound email costs less, at least in the short run and that's all the typical MBA is interested in. Their idea is to move to a new company before the long-term damage they've caused becomes evident. (I'm not just wanking, here; I asked an MBA about it once and that's what he told me.)

Re:Profit comes first

[SeaFox]

If they did that, it would lower their income and cut into their profits.

That's assuming they actually close the customer's account or credit for the time out. Some ISPs do not, since the issue is generally a virus or other malware on the customer's PC (in other words, not the ISP's fault).

But you response overall is still correct. If they keep mucking around with the email, they still save money because eventually the customer gets sick of it and gets a Yahoo account instead. Now Comcast is still getting the same $40/month, but without having to provide mail services.

Looking further...

[Spazmania]

Digging further into the Cox situation, the Cox subscriber said:

I tried to send an email. The email only contained text. The text Cox
objected to was "http://my_homebox_IP_number/"

I haven't checked the Cox TOS lately, but don't they prohibit running a home web server like all the other residential internet providers? Hasn't this been the case since for essentially the same length of time that the Internet has been a commercial venture?

Amen

[davidwr]

ISPs should ask you what services you really need when you sign up for a new account:
"I'm a normal user, let me have what normal users get"
"I'm a power user, please turn on ___, ____, and ___"
"I'm a power user and I really really really know what I'm asking for, please turn on everything."

Then let them change it at any time, either permanently or, if they only need it for awhile, for an hour, a day, or a week.

Once you do that you can hold customers responsible for things like letting bots run loose spamming the planet over an available outgoing port 25.

Where, exactly, is the story?

[pongo000]

It's not clear to me that Cox actually scanned the message body in its determination that the e-mail in question was spam. There could have been any number of indicators that caused Cox to reject the outbound message.

I also note that Cox's TOS specifically prohibits the hosting of servers:

Servers. You may not operate, or allow others to operate, servers of any type or any other device, equipment, and/or software providing server-like functionality in connection with the Service, unless expressly authorized by Cox.

A more accurate title for this story would be: "User in violation of Cox TOS upset over Cox efforts to enforce TOS."

My advice to said user? Buck up and get business-level service, or find yourself a real hosting service for your mail server.

Holy WTF?!?

[Just Some Guy]

I can understand and am sympathetic to ISPs who force outbound traffic to go through their servers. I'm not saying I agree with it, but I really do get what they're trying to accomplish. I also understand ISPs having spam filters on their outbounds, and think that's actually a pretty good idea. If you really need to send a virus so someone, then you should be technically competent to encrypt it or otherwise shield it from a scanner.

But never in a million years can I even remotely condone actually scanning the text of emails and rejecting ones an ISP doesn't like. That's just Evil.

Re:Holy WTF?!?

[bughunter]

But never in a million years can I even remotely condone actually scanning the text of emails and rejecting ones an ISP doesn't like.

I recommend that the phrase "Cox Blocked" be granted status as the official 'net jargon for any message blocked by an outgoing content filter.

Re:Not Comcast

[Bender Unit 22]

I'd say that every ISP should do that, that is, if you could get it unblocked if you requested it or via some online account management.
99% of all people wouldn't need it anyway(except the bots on their machines) and the ones who do, would know how to open it. Of course it is a not the ideal way to solve the problem, but it's all we got for now.

Re:Phrases

[grcumb]

Since when is giving a person advice on species "spam-like"? Maybe it's the latin I don't know.

Cialis vincit disfunctio penilis!

Kudos to Cox Communications

[merc]

I would like to first state that I am a Cox cable internet subscriber in the Phoenix area. I also happen to wear the abuse desk hat for Arizona's oldest ISPs.

I can say without question that the amount of spam we get from cox is almost NIL. I constantly see spam coming out of Comscat's network, also Verizon and from time to time Time Warner but RARELY Cox. In fact I can't remember the last spam I received that originated from their network.

I don't mind that my egress SMTP port is blocked forcing me to use a MSA (mine is configured to use SMTP AUTH with TLS, which works nicely). The fact is that Cox has their act together in my opinion. The fact that they are a white hat in the abuse category makes me want to continue doing business with them. I don't think what you're seeing here is intentional censorship. It would actually be irresponsible for Cox not to filter outbound mail traffic, since they are bound to have customers that run malware infected / zombied host computers.

Anyway, I say "good job Cox" :)

P.S. I work for an ISP that is NOT Cox--which one might think after reading my glowing statements (in fact we compete against Cox)

Re:Kudos to Cox Communications

[rmerry72]

It would actually be irresponsible for Cox not to filter outbound mail traffic, since they are bound to have customers that run malware infected / zombied host computers.

You know, I'm getting sick of the prevailing attitude that ISPs and other "institutions" should limit legitimate activities with a technology and filter everybody's behaviour because some customers are bad apples (either intentially or through ignorance).

Don't penalise me and limit my activities - limit those who are adversly behaving. ie, block those that do have malware infected machines not mine! I do the right thing and protect my systems. Why should I should I be penalised by the highest common ignorance factor?

You are promoting this attitude by saying "We will do business with them because they bottled up their customers nicely and it saves us work" instead of "They have lower quality customers and have to bottle them. Not going to touch that crowd".

What am I saying? We live in a moddle-coddled world where nobody takes responsibility for they're own actions but instead focuses on fretting and controlling everybody else's actions. Arse above tit. With liberty comes responsibility.

Re:Kudos to Cox Communications

[rmerry72]

So it would be better for Cox to allow any old botnetted-computer to spew spam?

No. Kill the connection of those computers. Don't block and filter my computer because Joe Idiot has malware. Cut him off and make it his responsiblity to clean his property. If I had a spiking phone that was causing disruption to the telephone network they'd disconnect my phone not start filtering your phone conversations. If my car was a defect I wouldn't be allowed to drive.

If your mail situation is that important, buy a business-class account.

Come on, are you telling me sending an email is an add on to the basic funtionality of the internet, and optional extra? "Oh, you want "clean" water? Well I suggest you upgrade to our business service. Our residential water pipes only deliver untreated effluent."

Re:Not Comcast

[DCTooTall]

that's actually been a pretty common net-wide standard for awhile to block port 25. Logic being that many old spam virus's used to set up an smtp server on the infected machine and start spamming directly from the infected computer bypassing the isp's SMTP server. By blocking port 25 on the outskirts of the ISP network and forcing customer to use their SMTP it allowed better access controls to prevent spam. and more importantly, kept entire ranges of Dynamic IP's from getting blacklisted due to spam.


In the past few years with the increase in teleworking, remote access of email, and personal domain names, as well as the evolution of the spam-virus, that ISP's have moved to allow access to port 25 outside their network, instead doing IP access controls on their outgoing SMTP server, and using SMTP Auth to allow people to connect from outside their network.

ISP != Evil (not necessarily, anyway)

[cbone00]

I own part of a small ISP and CLEC in the South.
We do not use spy on our customers phone calls or throttle their P2P traffic. We are not considering monitoring their Internet traffic for copyrighted (or any other) data.
Maybe some of the big boys are out there using these draconian tactics, but your average, everyday, garden variety, small ISP is just trying to make a living providing a quality alternative to the behemoths out there.
Please don't lump us in with those guys.

All that said... We *do* filter inbound email traffic for viruses and SPAM. We do block inbound port 25 to our dynamic IPs.
We view these actions as our duty to our customers and to the rest of the Internet to do our small part to help at least slow down the rampant propagation of SPAM on the Internet.
We currently block about 95% of the email that hits our domains - and that number is slowly climbing. Do we occasionally throw out the baby with the bath water? Probably so, but it is rare. I can't even remember the last complaint we have gotten about this, so this tells me that our filters are highly effective.
As for blocking port 25, we do this to guard our address space against our own customers being irresponsible with their PC's and not keeping virus software up to date. Getting our address space blacklisted would effect ALL of our customers.

It is not about getting rich. Hardly so. Email is the probably the biggest drain on resources that any ISP faces. If we didn't take these steps, we probably would not be in business.

Everyone wishes we had the less evil Internet of yesteryear back, but it isn't going to happen. The Internet is a cesspool. We have to defend ourselves in the best way we know how.

ILOVEYOU

[AEton]

About ten years ago, it became impossible for me to send e-mails to my girlfriend with the subject line "ILOVEYOU."

The error message from Comcast -- something about rejection -- was pretty classic.

Re:Not Comcast

[squallbsr]

I also have Comcast, I was able to send email over SMTP (port 25) any time I pleased. That was until my brother decided to bring over his virus ridden, spam spewing, zombified windows machine over and hook it up to my network (while he was house sitting). They promptly blocked port 25, I got home and couldn't send email.

I had to call their very rude Security Something Department, they said my options were
1. 'Use a different port because other ports can be secured while port 25 cannot be secured.'
2. Use the Comcast alternate port SMTP-AUTH Server (of which I don't know my login/password for)

I told them I wanted option 3:
3. Re-open port 25.

They decided to tell me that they could as a ONE TIME courtesy re-open the port, but 'it will probably be blocked again because the problem that caused it to be blocked probably wasn't fixed' (even after I told them that I had found the problem and fixed it, in addition to monitored all transmissions over port 25 for an hour)... So I fixed my OpenBSD firewall pf rules to only allow 'trusted' computers to only be able to contact MY email server, and access the whole internet unfettered, the 'guest' machines have access to web and a handful of other ports (none of which is 25)...

Moral of the story: Stop using windows... /flamebait

Yep

[Sycraft-fu]

Cox does have business level cable and I've been quite happy with it. Used to use Speakeasy DSL but got spooked when Best Buy purchased them and switched to Cox. Thus far (little over a year) it has been great. I run 3 servers which do a moderate amount of traffic (maybe 50-100GB up a month) and have heard not a peep out of them. No ports are blocked that I can see, the servers run HTTP, HTTPS, SSH, IMAPS and SMTP between the group of them and it all works fine. They even have an SLA such that in extended downtimes you get monetary credit.

The difference, of course, is that I pay a good bit more. I'm not sure what a consumer level cable connection costs for 10mb/1mb but my understanding is it is somewhere in the range of $50/month. I pay more like $150/month for the business grade with 8 static IPs (the IPs do add a good portion of that).

However I'm ok with that. My usage is much in excess of what you'd get from a normal consumer, I'm ok with the fact that I have to pay for that. It's still not a bad price all things considered.

If you want the cheap consumer connections, then you need to deal with the consumer restrictions which usually include "no servers". It isn't as though they are being assholes and saying "No you can't ever do this," they are just saying "If you want to do this, you need a more pricey service."

Re:Not Comcast

[SCHecklerX]

You may have at one point been flagged as being 'infected with a virus'. This is when my comcrap connections always got nuked (I host a mailing list). But instead of filtering just outbound, they would kill everything.

I got tired of fighting with them (and after the headaches they caused with my overpriced business class connection when they took over for the ISP they bought out I was not going to pay for that service again), and discovered DynDNS's mailhop outbound and mailhop relay services. Problem solved. You can have stuff forwarded in on a nonstandard port and sent out that way too.

http://www.dyndns.com/services/mailhop/outbound.html
http://www.dyndns.com/services/mailhop/relay.html

Re:Not Comcast

[Anonymous Coward]

25 blocked ubiquitously here too. Instead of using cox's smtp service, I use the SMTP relay service at http://www.smtpport.com/ to tunnel regular smtp to my own company server through a nonstandard port. A decent workaround for when you don't have shell access or secure smtp. So far cox hasn't filtered or blocked it.

Re:Not Comcast

[cheater512]

Blocking every port under 1024 and having a touch tone phone interface to unblock them would be ideal.
That way there is no way for a bot to automate it (ok maybe if they still have a analog modem but unlikely) and its pretty easy to unblock yourself while keeping the ISP's workload low.

That would cut out a lot of the net's problems overnight and make it extremely difficult to bypass.

You forgot about the US government

[soren100]

However, filtering also raises the "you are now liable for what they say to an extent" issue that the whole Safe Harbor thing was suppose to fix for ISPs and could definately cost a huge pile more than just cutting access and losing customers. People have raised that idea as wel about AT&Ts plan to filter their network for copyrighted material.

The answer I have to that is "9/11 Changed Everything".

Seriously -- when the US government asked the telcos to commit surveillance crimes against the US citizens, only Qwest refused. Usually, breaking the law is a bad thing, but the US government was offering lots of money to the telcos, and presumably the promise not to prosecute. So the only company that got in trouble was the one following the law. And somehow the Qwest CEO that refused the deal ended up in jail. Meanwhile Dick Cheney is desperately trying to get immunity for the cooperating telcos for their crimes. See how that works?

So on the surface of things scanning and filtering our email might seem to be a bad busines move. But if the same US Government that got illegal telephone surveillance of US Citizens is also going for illegal surveillance of our emails, email filtering starts to make much more business sense.

It used to be that the idea of the US government secretly finding out what was in your emails was in the tin-foil hat realm. But the illegal surveillance of telephone calls would have been as well, along with secretly torturing people in secret overseas prisons. As well as "constitution-free" zones such as Gitmo that are paid for by US taxpayer dollars.

So if you have a government that scans your telephone calls, email, and web-surfing habits, you get very close to a goal of "total information awareness", which was one of the government's programs that was renamed and shuffled around after the public got very upset.

Re:Not Comcast

[Matt Perry]

Moral of the story: Stop using windows...

I'd say the moral is don't let people to connect devices to your network without your approval and possible oversight. It's not Windows' fault that your brother connected his infected machine to your network.

Re:Not Comcast

[Matt Perry]

It is, however, Windows' fault that for a long time in the late '90s and early '00s Windows was a festering pit of security holes that practically begged spammers and other maltards to abuse it.

That may be true, but we aren't talking about the distant past. Windows may still have security issues but that doesn't mean that a person can make it reasonably secure: keeping up to date with patches, using anti-virus, avoiding insecure software such as Internet Explorer, etc. Plenty of people use Windows without it getting infected. And my point still stands. The fact that he allowed his brother to connect an infected machine to his network isn't the fault of the OS.

Re:Not Comcast

[konohitowa]

A friend of mine uses Comcast in the Indianapolis area. I talked to him on the phone and he was surprised that I hadn't received an email from. We went through several tests and concluded that Comcast was indeed scanning his outbound email and filtering items that hit some type of keyword filter. He was able to send the email only when he slightly altered the subject text. The annoying part of it was that it was a "silent" filter - he got no indication that the email had been rejected. It just went straight off to /dev/null (so to speak).

Please that's not 1/2 as bad

[failedlogic]

My IS fil ers my o -bound pac ets to many we ites. Ju t make it har er to re d wh t I wri e. I'm a re ly a go d spell er trust me.

Re:Phrases

[ClickOnThis]

Anybody got any ideas of what phrases are being poof'd by cox? It may be that they are looking for repeated phrases in several successive e-mails as a sign that someone is sending spam. I say this because of a personal experience with Cox in May of last year, when I was e-mailing resumes as part of a job search. Then one day, Cox started rejecting my outgoing e-mails that contained my resume and cover letter. I contacted customer service and got this incredibly unhelpful canned response:

Dear XXXXX,

Thank you for your e-mail. I understand you are experiencing
difficulties sending e-mails stating messages are being rejected by the
server. I am really sorry for this inconvenience.

Our messaging team is adding functionality to the email platform that
will have the ability to detect spam emails and notify the you that you
are attempting to send spam, and that it will not be sent. Therefore,
when a your email has been identified as a spam, you will see an error
message. Please visit the link below for more information:

http://coxagainstspam.cox.net/

I hope you have found the information above useful. If the difficulty
persists or if there are any further inquiries you would like to
address, do not hesitate to contact our dedicated department for further
assistance.

Have you tried our customer support site? Visit

http://support.cox.com/

to find answers to many of your Cox High Speed Internet questions FAST,
including "click to fix" automated solutions and LIVE online chat
support 24/7!

Thank you for choosing Cox Communications as your friend in the digital
age. In other words, Cox said "Yep, your outgoing e-mails were flagged as spam and not sent, and we don't care. Have a nice day." Sheesh.

I was able to get around the problem by sending my resume as an attached RTF instead of DOC (both created with OpenOffice.) I'm guessing this change was enough to convince their filter that the messages with RTFs attached were not the same as the previous ones with DOC files.

Eventually the problem went away, and happily I did find a job. Still, I was pretty dismayed at how dismissive and unhelpful their "dedicated department" was.