How Pervasive is ISP Outbound Email Filtering?
Erris writes "A member of the Baton Rouge LUG noticed that Cox checks the text of outgoing email and rejects mail containing key phrases. I was aware of forced inbox filtering that has caused problems and been abused by other ISPs in China and in the US. I've also read about forced use of ISP SMTP and outbound throttling, but did not know they outbound filtered as well. How prevalent and justified is this practice? Wouldn't it be better to cut off people with infected computers than to censor the internet?"
If they did that, it would lower their income and cut into their profits. Filtering outbound email costs less, at least in the short run and that's all the typical MBA is interested in. Their idea is to move to a new company before the long-term damage they've caused becomes evident. (I'm not just wanking, here; I asked an MBA about it once and that's what he told me.)
Good, inexpensive web hosting
I use Comcast, and my outbound tcp/25 is blocked entirely. I can _only_ go to their SMTP relay.
25% Funny, 25% Insightful, 25% Informative, 25% Troll
If an ISP doesn't filter their outgoing email to make sure that it's own users aren't spamming, they WILL get blocked. I'm on a super-secret anti-spam mailing list which I can't tell you about, and everybody there cheerfully admits to blocking their own users' outgoing spam. It's necessary.
Don't piss off The Angry Economist
Digging further into the Cox situation, the Cox subscriber said:
I tried to send an email. The email only contained text. The text Cox
objected to was "http://my_homebox_IP_number/"
I haven't checked the Cox TOS lately, but don't they prohibit running a home web server like all the other residential internet providers? Hasn't this been the case since for essentially the same length of time that the Internet has been a commercial venture?
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
They could do inline virus filtering easier, cheaper, and still not be intrusive. IMHO they are being rude when they could be helpful.
Having to work for a living is the root of all evil.
No, but it is worrying. For example, I often have to resort to emailing people using PDF's which contain the bulk of my message because their stupid ISP marks me as spam. I think it is because a lot of my emails involve giving people advice on plant species names--which always makes me go "wtf" when my email bounces because it is "spam-like". Since when is giving a person advice on species "spam-like"? Maybe it's the latin I don't know. I don't use my ISP for outgoing email (I run my own email servers) but for those people who do... their emails better not be innocent because they'd probably be filtered as spam. Much better to write a long message about penis enlargement than something serious--it's more likely to pass through the filters.
I will no longer be able to point to my home server on these lists because Cox
rejects such messages as spam. The message given when I try is:
Sending failed:
Could not write file The message content was not accepted.
The server responded: "ID_INTENTIONALLY_REMOVED This message was
undeliverable. This message has been found to be a potential spam message,
and has therefore been blocked. Please visit http://coxagainstspam.cox.net/
for more information.".
Disk full.
The message will stay in the 'outbox' folder until you either fix the problem
(e.g. a broken address) or remove the message from the 'outbox' folder.
The following transport protocol was used:
smtp.east.cox.net
. . .
I could care less that their disk is stuffed and suspect it is misdirection.
This censorship is only a minor inconvenience but the message it sends is
ugly. It says, in so many words, that the internet is for your consumption
not participation. Changing messages to point to my physics page gets around
the immediate problem, but most people do not have such a thing nor should
they be forced to host things on someone else's computers. I'm paying for my
bandwith, why can't I use it for what I want? Finally, subscribers now know
that every word of every message sent is filtered. Will they filter my IM
conversations next?
You can't talk about Wikipedia's flaws on Wikipedia
hmm thats strange, i'm using comcast in the atlanta area and can easily do smtp to other hosts on the internet.
ISPs should ask you what services you really need when you sign up for a new account:
"I'm a normal user, let me have what normal users get"
"I'm a power user, please turn on ___, ____, and ___"
"I'm a power user and I really really really know what I'm asking for, please turn on everything."
Then let them change it at any time, either permanently or, if they only need it for awhile, for an hour, a day, or a week.
Once you do that you can hold customers responsible for things like letting bots run loose spamming the planet over an available outgoing port 25.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I also note that Cox's TOS specifically prohibits the hosting of servers:
A more accurate title for this story would be: "User in violation of Cox TOS upset over Cox efforts to enforce TOS."
My advice to said user? Buck up and get business-level service, or find yourself a real hosting service for your mail server.
I can understand and am sympathetic to ISPs who force outbound traffic to go through their servers. I'm not saying I agree with it, but I really do get what they're trying to accomplish. I also understand ISPs having spam filters on their outbounds, and think that's actually a pretty good idea. If you really need to send a virus so someone, then you should be technically competent to encrypt it or otherwise shield it from a scanner.
But never in a million years can I even remotely condone actually scanning the text of emails and rejecting ones an ISP doesn't like. That's just Evil.
Dewey, what part of this looks like authorities should be involved?
I'd say that every ISP should do that, that is, if you could get it unblocked if you requested it or via some online account management.
99% of all people wouldn't need it anyway(except the bots on their machines) and the ones who do, would know how to open it. Of course it is a not the ideal way to solve the problem, but it's all we got for now.
Cialis vincit disfunctio penilis!
Crumb's Corollary: Never bring a knife to a bun fight.
Some antivirus packages also block some outbound email as well. At a previous company I worked for, we had to send out numerous survey invitations. Norton would quietly queue and scan all the outbound data (going to port 25)-- which worked in many cases. Except that it was slow. And there was now way of knowing how much data (if any) was still queued. And if the computer was restarted before Norton finished processing the queue, the data was silently lost (even though a "Accepted for delivery" message was returned to the sending program).
These limitations wouldn't be hit by your normal 1-or-2 emails at a time users. But for the rare legitimate high volume senders, like us, it was a problem. IT wouldn't let us turn off Norton alltogether (and rightly so, as we'd seen virii on our network in the past), and there was no way to selectively disable that "feature". Eventually we forced to make our outbound mail server listen on a different port, so that Norton wouldn't scan/lose the data.
At least with COX you get a notification saying that the message couldn't be sent, with Norton, the messages might just quietly disappear.
Online Starcraft RPG? At
Dietary fiber is like asynchronous IO-- Non-blocking!
I would like to first state that I am a Cox cable internet subscriber in the Phoenix area. I also happen to wear the abuse desk hat for Arizona's oldest ISPs.
:)
I can say without question that the amount of spam we get from cox is almost NIL. I constantly see spam coming out of Comscat's network, also Verizon and from time to time Time Warner but RARELY Cox. In fact I can't remember the last spam I received that originated from their network.
I don't mind that my egress SMTP port is blocked forcing me to use a MSA (mine is configured to use SMTP AUTH with TLS, which works nicely). The fact is that Cox has their act together in my opinion. The fact that they are a white hat in the abuse category makes me want to continue doing business with them. I don't think what you're seeing here is intentional censorship. It would actually be irresponsible for Cox not to filter outbound mail traffic, since they are bound to have customers that run malware infected / zombied host computers.
Anyway, I say "good job Cox"
P.S. I work for an ISP that is NOT Cox--which one might think after reading my glowing statements (in fact we compete against Cox)
It's true no man is an island, but if you take a bunch of dead guys and tie 'em together, they make a good raft.
that's actually been a pretty common net-wide standard for awhile to block port 25. Logic being that many old spam virus's used to set up an smtp server on the infected machine and start spamming directly from the infected computer bypassing the isp's SMTP server. By blocking port 25 on the outskirts of the ISP network and forcing customer to use their SMTP it allowed better access controls to prevent spam. and more importantly, kept entire ranges of Dynamic IP's from getting blacklisted due to spam.
In the past few years with the increase in teleworking, remote access of email, and personal domain names, as well as the evolution of the spam-virus, that ISP's have moved to allow access to port 25 outside their network, instead doing IP access controls on their outgoing SMTP server, and using SMTP Auth to allow people to connect from outside their network.
I own part of a small ISP and CLEC in the South.
We do not use spy on our customers phone calls or throttle their P2P traffic. We are not considering monitoring their Internet traffic for copyrighted (or any other) data.
Maybe some of the big boys are out there using these draconian tactics, but your average, everyday, garden variety, small ISP is just trying to make a living providing a quality alternative to the behemoths out there.
Please don't lump us in with those guys.
All that said... We *do* filter inbound email traffic for viruses and SPAM. We do block inbound port 25 to our dynamic IPs.
We view these actions as our duty to our customers and to the rest of the Internet to do our small part to help at least slow down the rampant propagation of SPAM on the Internet.
We currently block about 95% of the email that hits our domains - and that number is slowly climbing. Do we occasionally throw out the baby with the bath water? Probably so, but it is rare. I can't even remember the last complaint we have gotten about this, so this tells me that our filters are highly effective.
As for blocking port 25, we do this to guard our address space against our own customers being irresponsible with their PC's and not keeping virus software up to date. Getting our address space blacklisted would effect ALL of our customers.
It is not about getting rich. Hardly so. Email is the probably the biggest drain on resources that any ISP faces. If we didn't take these steps, we probably would not be in business.
Everyone wishes we had the less evil Internet of yesteryear back, but it isn't going to happen. The Internet is a cesspool. We have to defend ourselves in the best way we know how.
About ten years ago, it became impossible for me to send e-mails to my girlfriend with the subject line "ILOVEYOU."
The error message from Comcast -- something about rejection -- was pretty classic.
We recently had heard in the office over one of the Yellow Machine that's made by Anthology Solutions.
I also have Comcast, I was able to send email over SMTP (port 25) any time I pleased. That was until my brother decided to bring over his virus ridden, spam spewing, zombified windows machine over and hook it up to my network (while he was house sitting). They promptly blocked port 25, I got home and couldn't send email.
/flamebait
I had to call their very rude Security Something Department, they said my options were
1. 'Use a different port because other ports can be secured while port 25 cannot be secured.'
2. Use the Comcast alternate port SMTP-AUTH Server (of which I don't know my login/password for)
I told them I wanted option 3:
3. Re-open port 25.
They decided to tell me that they could as a ONE TIME courtesy re-open the port, but 'it will probably be blocked again because the problem that caused it to be blocked probably wasn't fixed' (even after I told them that I had found the problem and fixed it, in addition to monitored all transmissions over port 25 for an hour)... So I fixed my OpenBSD firewall pf rules to only allow 'trusted' computers to only be able to contact MY email server, and access the whole internet unfettered, the 'guest' machines have access to web and a handful of other ports (none of which is 25)...
Moral of the story: Stop using windows...
Sleep: A completely inadequate substitution for Caffeine.
In the Boston area, comcast fuckers are blocking port 25. So, even though people have legitimate uses for the internet connection they pay for, these companies are taking it on themselves to block standard connection protocols.
/. crowd already knows.
First its port 25, because of spam. Then it will be P2P because of copyright. Then it will be ssh because of terrorism. Then it will be, inspired from the new york story yesterday, filtering web content to prevent false alarms.
Fuckers. Bury your head america.
When people talk about fascist Germany, they focus on the extermination of jews and the holocaust, and while those were horrific acts, they are not what the Nazi party was about. They were the result of the acts of fanatical and arguably insane men who had gained power in the Nazi party, not the Nazi party, per se'
The Nazi party was about power and the exercise of it. It was about bringing pressure on the citizens from all aspects of society to conform to it. It used social structures and industries and laws to bring people under control. It is EXACTLY what is happening in america today. Its all the little things slowly picking away at the big things, until the big things crumble. Freedom of speech? Nope, now we have "free speech zones," where no one will hear you. I could go on, but the
Just like the Reichstag fire in 1933, the world trade center in 2001 gave the neocons the ability to enact limits on freedom. After that, industries which were once regulated in order to protect the citizens are now deregulated and destroying citizens who do not conform, RIAA, MPIAA, walmart, etc.
ISP censorship is just one more piece of it. The internet is becoming the primary conduit of communication and fascist america must have its citizens controlled, just lake Nazi Germany needed its citizens controlled.
All this isn't a conspiracy theory either. No conspiracy theory need exist. Our government (of the people, by the people, bla bla) is supposed to protect us. If it stops protecting us from big companies, those companies will naturally do the work for their own gain.
Now everyone in the USA is afraid. Some of terrorists, some of losing heath care, some of losing their job, their house, what ever. Fear, as the nazi's will tell you is a powerful tool to harness.
Welcome to neocon amaerica where companies sue their customers because they can. Companies dictate what you can do with your property, because they can, and if you do anything about it or protest, you can lose your job which means your house and health care.
Sorry for the rant, but I can't be the only one who sees this whole thing in this way
I tried clicking on your link but I think my ISP is somehow blocking it.
Cox does have business level cable and I've been quite happy with it. Used to use Speakeasy DSL but got spooked when Best Buy purchased them and switched to Cox. Thus far (little over a year) it has been great. I run 3 servers which do a moderate amount of traffic (maybe 50-100GB up a month) and have heard not a peep out of them. No ports are blocked that I can see, the servers run HTTP, HTTPS, SSH, IMAPS and SMTP between the group of them and it all works fine. They even have an SLA such that in extended downtimes you get monetary credit.
The difference, of course, is that I pay a good bit more. I'm not sure what a consumer level cable connection costs for 10mb/1mb but my understanding is it is somewhere in the range of $50/month. I pay more like $150/month for the business grade with 8 static IPs (the IPs do add a good portion of that).
However I'm ok with that. My usage is much in excess of what you'd get from a normal consumer, I'm ok with the fact that I have to pay for that. It's still not a bad price all things considered.
If you want the cheap consumer connections, then you need to deal with the consumer restrictions which usually include "no servers". It isn't as though they are being assholes and saying "No you can't ever do this," they are just saying "If you want to do this, you need a more pricey service."
disfunctionem erectilem, I believe, is the correct ending to your spam.
You may have at one point been flagged as being 'infected with a virus'. This is when my comcrap connections always got nuked (I host a mailing list). But instead of filtering just outbound, they would kill everything.
I got tired of fighting with them (and after the headaches they caused with my overpriced business class connection when they took over for the ISP they bought out I was not going to pay for that service again), and discovered DynDNS's mailhop outbound and mailhop relay services. Problem solved. You can have stuff forwarded in on a nonstandard port and sent out that way too.
http://www.dyndns.com/services/mailhop/outbound.html
http://www.dyndns.com/services/mailhop/relay.html
Or server-like functionality?
So, what exactly, defines a server? When you think about it, there's just traffic between two points. From a semantic perspective, posting to /. could be seen as "serving" text to a remote computer...
But, I think this kind of highlights the apparent Cox conceptual model of the internet:
The optimist in me hopes I'm wrong on some of the above points, but the pessimist knows to suspect the worst.
The society for a thought-free internet welcomes you.
25 blocked ubiquitously here too. Instead of using cox's smtp service, I use the SMTP relay service at http://www.smtpport.com/ to tunnel regular smtp to my own company server through a nonstandard port. A decent workaround for when you don't have shell access or secure smtp. So far cox hasn't filtered or blocked it.
Blocking every port under 1024 and having a touch tone phone interface to unblock them would be ideal.
That way there is no way for a bot to automate it (ok maybe if they still have a analog modem but unlikely) and its pretty easy to unblock yourself while keeping the ISP's workload low.
That would cut out a lot of the net's problems overnight and make it extremely difficult to bypass.
The problem with an ISP using SMTP-auth for connections outside their network is that SMTP-auth is only as secure as the least secure password used in your customer base. Given that people are generally lazy and prioritize convenience over security, that means odds are that any decent sized ISP *will* have at least one (and probably very many more) weak passwords, and *that* means that the ISP's mail server *will* be an open relay as soon as the spammers figure it out.
This isn't just theory -- at an ISP I used to work at, we saw this happen. We started getting UCE complaints from other ISP's, but couldn't figure out how spammers were relaying through our server. We traced it down to one customer's e-mail account, but couldn't figure out how hosts from outside our netblock were relaying through our server. Finally, one of the admins noticed that SMTP-auth was turned on (it wasn't supposed to be). I've lost all faith in SMTP-auth on an ISP server since.
MCSE? No, sir...I don't do Windows. Yes, I am an idealist. What's your point?
According to the NANOG list (North American Network operators Group), Comcast has been discarding emails that include a link created using EasyURL, one of many services designed to provide shortened URLs for email links. This could be an anti-spam policy, as URL forwarding through these services is sometimes used by phishing scams to obscure the link's true destination.
The answer I have to that is "9/11 Changed Everything".
Seriously -- when the US government asked the telcos to commit surveillance crimes against the US citizens, only Qwest refused. Usually, breaking the law is a bad thing, but the US government was offering lots of money to the telcos, and presumably the promise not to prosecute. So the only company that got in trouble was the one following the law. And somehow the Qwest CEO that refused the deal ended up in jail. Meanwhile Dick Cheney is desperately trying to get immunity for the cooperating telcos for their crimes. See how that works?
So on the surface of things scanning and filtering our email might seem to be a bad busines move. But if the same US Government that got illegal telephone surveillance of US Citizens is also going for illegal surveillance of our emails, email filtering starts to make much more business sense.
It used to be that the idea of the US government secretly finding out what was in your emails was in the tin-foil hat realm. But the illegal surveillance of telephone calls would have been as well, along with secretly torturing people in secret overseas prisons. As well as "constitution-free" zones such as Gitmo that are paid for by US taxpayer dollars.
So if you have a government that scans your telephone calls, email, and web-surfing habits, you get very close to a goal of "total information awareness", which was one of the government's programs that was renamed and shuffled around after the public got very upset.
Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
I use an alternate-port SMTP service: my mail doesn't go through my ISP's server. That was after my outgoing mail got blocked and their customer service (I use the term loosely) people couldn't tell me why. I was just told that the problem should "correct itself" in a week or so. Well, it eventually did but by then I'd taken steps to never be in that position again. Now I just poll their mailbox for the occasional notification but I haven't sent a message through my ISP's SMTP server in years.
The higher the technology, the sharper that two-edged sword.
In this day and age, with most busy mailservers fending off about 60% of their load as Mass Spam storms, it is almost negligent to allow all of your customers unlimited access to smtp to any destination. Yes, there will always be outcry about 'censorship' and 'big brother'. It's a shame it's not the same crowd that shouts about the torrent of Spam and viruses that comes from high bandwidth, unaware mom & dad users (and us techies too - I can't remember the last Open Relay I saw configured by a mom & dad!) incidentally, scanning for and removing http://ip.ip.ip.ip/ links from Email is a pretty good way of detecting and blocking the outbound phishing attempts that each year result in millions of dollars being drained from the bank accounts of the uninitiated. Censorship is designed to prevent a particular content, subject or message from being propgated. I'm pretty sure you can re-write an Email in such a way as it does not get blocked. I'm pretty sure that if you want to run an SMTP server, you can get permission. if however, you happen to be a virus, you're hopefully s**t out of luck.
Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
It's not you being a grown-up, it's your idiot neighbors who click everything under the sun without regard to security. I think the solution is to block by default, and have a mechanism to open it up, as other posters have stated.
That's not incompetence, that's by design. The RFC for 587 submission states that it requires the use of SMTP-AUTH, rendering it useless for most forms of spam-spewing malware; an incompetent ISP will filter it, not open it.
A friend of mine uses Comcast in the Indianapolis area. I talked to him on the phone and he was surprised that I hadn't received an email from. We went through several tests and concluded that Comcast was indeed scanning his outbound email and filtering items that hit some type of keyword filter. He was able to send the email only when he slightly altered the subject text. The annoying part of it was that it was a "silent" filter - he got no indication that the email had been rejected. It just went straight off to /dev/null (so to speak).
But by the time you detect the spew, how many sites have already blacklisted your server?
MCSE? No, sir...I don't do Windows. Yes, I am an idealist. What's your point?
There was a recent article that showed that the performance of anti-virus s/w has got worse over the past year or two. People who think that Windows can be secured are in denial! The basic problem is that it is difficult to run as a limited user. Quickbooks requires administrator rights, I recently came across video capture and editing s/w that requires admin rights (despite Studio running on the same machine perfectly well for limited users). I am sure there are other programs. Yes, I know about "run as", but my claim is that it is difficult.
The real "Libtards" are the Libertarians!
My IS fil ers my o -bound pac ets to many we ites. Ju t make it har er to re d wh t I wri e. I'm a re ly a go d spell er trust me.
Wow. That reads like a curse from Harry Potter fan fiction.
Oooh, yeah let's regulate it. What would be the mechanism to open it up?
We had that a while back. It was called ARPANet. Progress is a circle and we improve by going backwards.
How about this: If you are an idiot who clicks on everything GET OFF THE DAMN TRAIN! A leave it for us grown ups.
There is a mechanism already - its called money. Pay more, get more. Nothing to do with security or idiot neighbours, purely about making more profit. Like everything these days.
We do not inherit the Earth from our parents. We borrow it from our children.
Thank you for your e-mail. I understand you are experiencing
difficulties sending e-mails stating messages are being rejected by the
server. I am really sorry for this inconvenience.
Our messaging team is adding functionality to the email platform that
will have the ability to detect spam emails and notify the you that you
are attempting to send spam, and that it will not be sent. Therefore,
when a your email has been identified as a spam, you will see an error
message. Please visit the link below for more information:
http://coxagainstspam.cox.net/
I hope you have found the information above useful. If the difficulty
persists or if there are any further inquiries you would like to
address, do not hesitate to contact our dedicated department for further
assistance.
Have you tried our customer support site? Visit
http://support.cox.com/
to find answers to many of your Cox High Speed Internet questions FAST,
including "click to fix" automated solutions and LIVE online chat
support 24/7!
Thank you for choosing Cox Communications as your friend in the digital
age. In other words, Cox said "Yep, your outgoing e-mails were flagged as spam and not sent, and we don't care. Have a nice day." Sheesh.
I was able to get around the problem by sending my resume as an attached RTF instead of DOC (both created with OpenOffice.) I'm guessing this change was enough to convince their filter that the messages with RTFs attached were not the same as the previous ones with DOC files.
Eventually the problem went away, and happily I did find a job. Still, I was pretty dismayed at how dismissive and unhelpful their "dedicated department" was.
If it weren't for deadlines, nothing would be late.
5000? In Canada, ISPs won't let more than 400 out *per day* through their mail gateways even on a commercial line. You have to set up your own mail sending system. Standard practice is to force all mail through their gateway. Checking message content (no I haven't read TFA) seems reasonable. You want privacy, that's your business, but average use is going to get checked out all the time. Nobody talks on 25 in Canada unless you pay commercial rates.
I don't know how Comcast detects it, but if they see spam or receive a spam report involving your modem they block port 25. No warning, apparently Comcast will refuse to lift the block. It has happened to a few of my clients and friends.
All 19 hijackers were known terrorists 09-10-2001. Lack of FBI intelligence does not justify warrantless wiretaps..