Hotmail & Yahoo Mail Using Secret Domain Blacklist

Frequent contributor Bennett Haselton writes: "Hotmail and Yahoo Mail are apparently sharing a secret blacklist of domain names such that any mention of these domains will cause a message to be bounced back to the sender as spam. I found out about this because — surprise! — some of my new proxy site domains ended up on the blacklist. Hotmail and Yahoo are stonewalling, but here's what I've dug up so far — and why you should care." Read on for much more on how Bennett figured out what's going on, and why it's a hard problem to solve.

On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.

After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.

At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:

----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable

After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)

But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)

I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.

I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).

So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.

Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.

I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.

So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.

But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.

In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.

This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)

Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)

On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.

I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.

Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.

Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.

Comments

conspiracy ! deliver my spam

[Anonymous Coward]

If it wasn't for Viagra ads generation US $50 a day, you could pretty much have unfetter delivery.
Do the proxies on your list relay smtp?

Re:conspiracy ! deliver my spam

[Anonymous Coward]

If it wasn't for real Viagra costing $25 a pill, there wouldn't be as hot a market for the spam.

Re:conspiracy ! deliver my spam

[multicoregeneral]

Dude, you're spelling it wrong. That's going to be flagged for sure.

Re:conspiracy ! deliver my spam

[Anonymous Coward]

I found out about this when I emailed my friend to motherless.com at his yahoo account. yahoo never delivered it because motherless.com was in the body. fuck yahoo.

Summary

[sorensenbill]

Is there a summary of the summary available?

Re:Summary

[TheMMaster]

According to TFA his list is opt-in only, so unless he's lying about that he doesn't appear to be a spammer.

I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.

I can assure you I have never sent a single spam email in my life.

This is the whole point of TFA though, there's no incentive for companies running mail services to ensure that legitimate mail gets delivered. It's simply cheaper to not bother with false positives at all because the cost of non-delivery is placed squarely on the shoulders of the sender.
This is why Spamhaus could easily force me to switch ISPs, it doesn't cost them anything to put my IP range on a shitlist, but it cost me money and effort to migrate my service.

Re:Summary

[TubeSteak]

Is there a summary of the summary available?

We call them "titles"
Here's one example: Hotmail & Yahoo Mail Using Secret Domain Blacklist

Re:Summary

[rudy_wayne]

so unless he's lying about that >

Well, there you go.

Re:Summary

[preaction]

No, but I can summarize the summary of the summary: People are a problem.

Re:Summary

[AlphaWolf_HK]

That's "hear hear", as in "hear him, hear him!" (which is where that phrase is rooted.)

Re: Summary

[Urza9814]

As a long-time subscriber to his list (at least 6 years), no, he's absolutely not. He provides a fantastic service and does a damn good job of ensuring only those who want the messages are receiving them. And I get less than one message per month from that list. If he's a spammer, so is literally every single person or organization that has ever sent me an email.

Re:Summary

[Anonymous Coward]

"Opt-in" covers multiple sins. Sometimes, it's a genuine "I want you to send me this stuff". Sometimes, it's "enter your email address to have a chance to win a magic unicorn. By the way, we'll also send you some emails", and sometimes it's an "opt-in" with the box checked by default, the way that everyone tries to get you to install browser toolbars or demo antivirus products.

Then you get people, who generally get sent marketing junk from any number of companies that they may or may not remember having dealt with. Typically, people don't go to any effort to try to unsubscribe from these lists, but just hit th e"report as spam" button because they don't want to read it.

Re:Summary

[TheRealMindChild]

Spamhaus != 0 false positives. This guy sends the same email out to tens of thousands of people who tend to use Yahoo or Hotmail. They both block the messages as spam.

Just FYI, I seen this guy bitching about it MONTHS ago. Apparently he still hasn't made a lot of headway. However, if you operate like a spammer (sending the same email to multitudes of folks, while relaying information about open proxy servers as information), then you will be treated like a spammer

Re:Summary

[afidel]

Why? By definition he is NOT a spammer since his messages are neither unsolicited nor commercial. It should be fairly easy for the responsible parties to verify he following best practices and whitelist him but apparently that's too much work for the postmasters at the big 3 webmail providers. Basically the postmasters at yahoo, gmail, and hotmail aren't doing their jobs. I know if our email admin was so bad at rectifying false positives he wouldn't be here for long but because of the scale of these organizations that pressure isn't happening.

Re:Summary

[girlintraining]

I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.

I've had problems like that with them as well. The thing is, Google et al. do provide very good spam filters. Out of the thousand or so spam messages that hit my mailbox every month, only about 5 make it through. A 99.95% success rate is nothing to sneeze at, so credit where credit is due. But the problem here is still architectural -- very few people respond to spam so the odds are very high that responses are to legitimate e-mail. Higher, I would think, than the 99.95% rate above. Multiple responses to the same address should override any spam-rating system they have automatically, and if not, there should at least be a 'white list' option for users to bypass the filter in the event of a failure such as this.

Neither option exists, and there is no remediation pathway available. The author (correctly) concludes this is deliberate and not merely a process oversight. Such is the nature of operations where the profit margins are so tiny that any support would obliterate it. Google only provides gmail so it can mine keywords and phrases from your e-mails to build a marketing profile and then target advertisements at you. Despite the very low rate of success here, it still beats the cost of the hardware maintenance and bandwidth when aggregated over a few hundred million regular users. But the only support incentive here is customer retention, and the support provided is very minimal and highly automated (as the author has discovered). This guy isn't a google customer -- he's trying to contact google customers, which places him in the "liability" column, not the "asset" column. Unless this guy can show that hundreds of thousands of Google customers are impacted and the impact is severe enough for them to switch, or consider switching, to another provider, there is no incentive for Google to even read his complaint, no matter how justified or rational, or easy to fix.

That's the free market problem he's run into: He thinks he's a customer, but he isn't. He's a service. And one that costs google more to support than any potential revenue that may be generated. The business decision here is clear, if not very friendly.

Re:Summary

[fifedrum]

yeah: guy discovers cloudmark domain blacklist is used by two cloudmark customers. At least, that's my opinion. this information isn't new, this list has been around for years, and you don't get on it easily. It takes multiple reports from multiple accounts before they add you.

Re:Summary

[jdavidb]

I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.

"Wasn't well liked" == "complaints had been received that they allowed their customers to send spam."

I agree with spamhaus. This puts pressure on ISPs to police their customers, or else their decent customers will leave. And everyone can choose whether they want to use providers that allow all contact through, or providers that filter out contact from ISPs that don't police their customers.

there's no incentive for companies running mail services to ensure that legitimate mail gets delivered

Well, there's some incentive in that if their customers truly want the mail and aren't receiving it, they'll have to pick a different provider. I purchased a product once to be emailed to me and had to acquire an alternative email address because the seller wouldn't do business with gmail, yahoo, or hotmail addresses. I didn't waste time arguing with him; I just got an email account that would get his mail through.

it cost me money and effort to migrate my service.

That's the price of offering a service. If enough people want it, they will more than make up for the cost of you going with an ISP they consider reputable. If not, the world has no obligation to keep your costs low enough to keep you in business. A much cheaper thing to do would've been to quit offering your service.

Re:Summary

[TheRealMindChild]

As a consumer of email, I would rather the 1% find a better way to communicate rather than stupify the email system even more to accommodate them

Re:Summary

[SomePgmr]

I'm not on his list so I don't know how it actually operates, but there's a lot of variation in these things. I've opted in to lists that don't provide a real opt-out, or in other ways don't comply with the old can-spam guidelines. I usually see this from foreign companies, which will require me to log in to an account on their website to get off the mailing list. Those get the spam button, every time, no questions asked.

Spamhaus does BL domains for no apparent reason, though. We're talking about properly configured mail servers, no open relays, no backscatter, appropriate DNS, with opt-in recipients only and working, simple unsub options right in every email. It is not a perfect service. My experience correlates with his on this though, in that you can have yourself removed pretty easily the first time.

This all ignores a pretty simple issue, though. it's easy to be too optimistic about the reliability of email delivery. It has never been great, there are no simple (and free) solutions, and things like this are going to happen. It's not because the providers are evil or conspiring to keep you from getting your job done, but because they're trying to make an implicit trust system usable over the internet.

And unfortunately, smtp implementations are not going to change in a way that fix all the present shortcomings.

Re:Summary

[Anonymous Coward]

... very few people respond to spam so the odds are very high that responses are to legitimate e-mail. Higher, I would think, than the 99.95% rate above. Multiple responses to the same address should override any spam-rating system they have automatically

"Reply to this email with "unsubscribe" to unsubscribe from this list." There, I probably just broke your idea.

Re:Summary

[afidel]

Why? Listservs are older than SMTP and have always been one of the use cases for electronic communications. Plus it's not like those providers are blocking all listservs, just those that don't pay their friends stupid high monthly fees for the privileged of emailing their users.

Re:Summary

[genner]

Is there a summary of the summary available?

It's a pain in the butt to get yourself removed from yahoo's email blacklist.
The end.

Re:Summary

[Anonymous Coward]

Well, if you read the first little bit of the summary, he uses email verification. So they have to respond to an email to get added to the list. So it's opt-in in the way most people mean it.

Re:Summary

[Onymous Coward]

Is the Cloudmark list of MTAs or URIs?

Maybe Cloudmark provides both kinds of lists now.

Re: Summary

[Anonymous Coward]

Most likely his domains are blocked because (gasp!) most of his subscribers are spammers exploiting his free URL wrapping service. If 99.999% of the time an emailed URL that uses his proxy lands on a spam landing page, then the filters are going to do the right thing and block the whole domain.

Re:Summary

[MrNaz]

1. Email blacklists are a terible idea, and I really sympathise with this guy's plight. I've been at the nasty side of a Spamhaus issue with my own mail server and I can tell you, those guys are nothing but a bunch of digital thugs who have managed to get themselves a nice big stick that they use to hit people randomly with. My server, being private, had just about every conceivable spam prevention mechanism turned on. SSL only connections, authorised SMTP-submission sending only, properly set up SPF records, PTR records correctly registered against the IP to allow reverse lookup. It got registered with Spamhaus and it took me a LONG time to get them to play ball. I'm still listed with a few older BL's but oh well.

2. If someone in a country wishes to circumvent government censors, why on Earth would they use a proxy? Why would they not just use Tor, which can't be blocked or filtered in that manner? If the government is doing deep packet inspection and will infer illegality from mere encrypted traffic, surely transferring illegal content in the clear is worse? Furthermore, setting up Tor is not materially more difficult than setting up a proxy. Not trolling, genuinely interested to know why one would choose the proxy path over Tor.

Re:Summary

[TheRealMindChild]

Because, AS A CONSUMER OF EMAIL, I don't care about your conspiracy issues. Communication backward and forward offers alternatives (like you said). Forcing people in the world to deal with even more spam so those technologically inept can get their message to you is not a cost that I, AS A CONSUMER OF EMAIL, and the rest of us care about.

As a matter of fact, I would quite welcome the same with paper mail. The only ones that would bitch at such a proposal would be those that want their stuff to get to you, even if you don't want it.

Re:Summary

[mystikkman]

According to TFA his list is opt-in only, so unless he's lying about that he doesn't appear to be a spammer.

Except that if even a few people viewing the email click on the "THIS IS SPAM" button in their email client/website, you're going on their shitlist regardless of whether people opted in or not. People tend to easily forget what they signed up for, and in some cases even if they remember, hitting the spam button is way easier than figuring out how to unsubscribe, even if the email has a link to do that.

Re:Summary

[afidel]

Let's use your physical mail analogy, under your idea charitable organizations would not be allowed to mail people who have signed up as supporters unless they went through a commercial mass mailing company paying a huge fee per piece mailed. While that's kind of the status quo for poorly run charities with a high overhead cost none of the charities I choose to support are so stupid, why you would want to reduce the amount of money reaching deserving causes and feed the commercial mass mailers I have no clue.

Re:Summary

[Anonymous Coward]

So your solution to the school bully demanding you give him your lunch money everyday is to just hand over the money? And then just consider that the cost of going to school? Maybe the easier solution in your eyes would just be to quit going to school all together.

Re:Summary

[fafaforza]

Spamhaus does have some false positives, but dealing with the amount of data they get, it's inevitable. Every time I've dealt with them, they were fairly responsive. Small blocks are removed automatically. Larger ones (like a /19 for example) take an email or two of back and forth.

As far as incentive, if companies want to retain email users, then of course they have incentive to minimize false positives. At the end of the day, it's a balance. It isn't always easy to tell spam from non spam if you aren't looking at your own, personal mailbox that you're familiar with.

Re:Summary

[TheRealMindChild]

Because my concern for the charities that can't go door to door, cold call, etc (just as effective as junk mail), is Nil. Just the same as most everyone on the receiving end of this garbage. Cost/benefit

Re:Summary

[Onymous Coward]

5 fails out of 1000 is 99.50% success.

My personal setup gets 99.78% success (9 spam delivered v. 4139 spam attempts in November), so we should set the bar higher for large corporations. I bet their rate is well above your miscalculation.

Not to dismiss the rest of your post. I thought it was insightful.

Re:Summary

[lipanitech]

Summary is I have yahoo mail its bad lets hope this helps.

Re: Summary

[Anonymous Coward]

Yeah, but speaking as an e-mail administrator, pretty much every single thing he's doing is what spammers do and the fact that his is opt-in isn't important to the filters. Heck, I can't even get to the Circumventor site because it's blocked by my OpenDNS settings. So if OP isn't a spammer then he's in the middle of a crowd that is. If enough recipients click "This is not SPAM" the filters will adjust.

Re:Summary

[Anonymous Coward]

There is incentive for these companies to provide a higher quality experience. Ultimately, these three email providers are the only ones that are relevant. They serve their users, the senders, and most importantly, advertisers. Their goal is to collect a bigger share of users, which ultimately results in higher advertising revenue. The problem is that if all three are using the same blacklist provided by the same group (whether internal or external), then there is no free market. It makes it difficult to impossible to improve their service without improving the service of their competitors, which results in a net loss since they spent the investment while their competitor received the same gains.

Re:Summary

[Stan92057]

"The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list."
He said NEW members that might only be 2-3 people hes a spammer giving vague details crying he cant spam us. These ISPs and free email providers are not stupid and have been fighting spammers for years now. Fact if your running a legit list then it will follow the Can spam laws meaning no fake headers a legit unsubscribe address in every email. Since hes not liget he cant get his spam delivered well boo hoo for him

Re:Summary

[Stan92057]

9 spam emails delivered VS 4139 spam attempts is NOT a 99.78 success rate. Its 99.78 % failure rate. or dam close.

Re:Summary

[Onymous Coward]

I'm not sure I understand you. Are you saying a denied spam delivery attempt is a failure?

Oh, from the spammer's perspective.

I didn't think I needed to be explicit. I do not want to receive spam. My filters are designed to deny spam. 4130/4139 spam attempts blocked is largely a success for me.

Re:Summary

[justin12345]

It sounds to me like he might just be getting blocked due to not having SPF records set up properly for all his domains that he's mailing on behalf of. Many mail servers block email from domains that don't bother with SPF.

Re:Summary

[Anonymous Coward]

Here's one: Bennett Haselton is spam, and has finally been flagged as such.

Re:Summary

[clintp]

Let's use your physical mail analogy, under your idea charitable organizations would not be allowed to mail people who have signed up as supporters unless they went through a commercial mass mailing company paying a huge fee per piece mailed. While that's kind of the status quo for poorly run charities with a high overhead cost none of the charities I choose to support are so stupid, why you would want to reduce the amount of money reaching deserving causes and feed the commercial mass mailers I have no clue.

Once the charities reach the size that the volume of mail they send raises the hackles of the post office, then they've already become part of the "conspiracy". The Iron Law is already in effect, regardless of their donation/overhead ratio. They just need to own up to it and formally join the cabal.

To the original article: a mailing list of 400,000 addresses isn't a community, it's a nation bigger than Iceland or Belize.

Re:Summary

[BitZtream]

Sure:

He runs a collection of open proxies and emails a list of them to people to use.

Yahoo and Hotmail are aware of this and blacklist his domains because he is effectively running spam sources.

He's too stupid to understand why they are doing exactly what they should be.

That should sum it up nicely.

Re:Summary

[BitZtream]

This has nothing to do with HIS list being SPAM.

The problem is that he runs OPEN PROXIES which then are naturally then used by spammers to spam people.

If you intentionally run an open proxy and you're surprised that you're blacklisted, you're pretty much the definition of stupid.

Re: Summary

[BitZtream]

Its not about who he mails the list to, its about who uses the services provided by the list.

The list provides a nice collection of proxies to use to send spam.

He could post the list on a website and never email it to anyone and it would STILL be blacklisted since spammers would still find and use his proxies.

Re:Summary

[Anonymous Coward]

Please stop trolling. Take a moment to read the OP and afidel's posts. This is *not* about cold calling at all. This is about getting requested content to those who have asked for it.

Re: Summary

[Urza9814]

They're not regular proxy servers; just an apache server hosting PHP proxy and some other CGI web proxy scripts. I don't believe there's any way to use them to proxy spam.

Re:Summary

[squiggleslash]

Oh bollocks.

Spammers have no problems whatsoever with this spamless utopia you espouse where legitimate emailers can't send email because they're running their own mail server. My mailbox is full of this crap all the time, and I've met people who work for companies that send spam and do everything they can to stretch the rules as far as possible, resulting in their largely unsolicited "Wait, I don't remember signing up for this" crap getting through.

You are the problem. You are the problem because you accept any idiotic solution to spam control no matter who it inconviences, and no matter how ineffective it actually is. Objectively, nothing this article is about concerns any legitimate means of blocking spam. Yet you're in favor of it, because that's the justification.

What you espouse, your support and your willingness to give full throated apologia for this crap, is undermining the email system. You reduce its effectiveness as more and more legitimate applications become impossible, while spammers continue to find ways around it.

Go away.

Re:Summary

[fafaforza]

That would be a fairly poor decision to make since not very many people bother with SPF. If anything, DKIM would be a better choice, especially for Yahoo.

Re:Summary

[Anonymous Coward]

here, here!

There, there.

Re: Summary

[ranulf]

This can't click "this is not spam" if they never get the message in the first place.

Re:Summary

[thedarknite]

My domain (a .edu.au) started being blocked by the hotmail servers yesterday with BAY0-MC2-F2.Bay0.hotmail.com #550 OU-002 (BAY0-MC2-F2) Unfortunately, messages from 203.xxx.xxx.xxx weren't sent. Please contact your Internet service provider since part of their network is on our block list and it was due to not having an SPF record.
After having one created mail goes through, but they seem to automatically be sent to the Junk folder with This message looks suspicious to our SmartScreen filters even though I don't see how. Maybe it's because I haven't had my domain registered with their third party "Safe Senders" list.

Re:Summary

[tricorn]

He's just commenting on the math, 5 failures (delivering spam) out of 1000 is 99.50% success (of stopping spam), not 99.95% as was stated in the post he was replying to.

Re:Summary

[Obfuscant]

According to TFA his list is opt-in only, so unless he's lying about that he doesn't appear to be a spammer.

But then he mentions the main reason he cannot use Constant Contact is because he sends different email to subsets of his full list, not that Constant Contact is a spammer almost beyond compare and won't remove someone from the lists they spew to even when both the recipient AND the sender tell them to.

I'm on two Constant Contact operated lists and there is absolutely nothing I can do to get off, including getting the companies that put me on to remove my address. At this point, I simply filter all Constant Contact email into the bit bucket.

Re:Summary

[Omnifarious]

Not trolling, genuinely interested to know why one would choose the proxy path over Tor.

Tor is frequently very slow. Totally worth it if you want industrial strength anonymity and use it correctly. But if you just care about your own government censors it's overkill.

Also, you can get very strange an unpredictable results from geographic targetting of Internet services. Oftentimes things will ignore any information you give them about what language you want to see the site in and decide that you should be seeing it in German because the IP you came from was in Germany. But then the next page load will be shown in Russian because the next connection came from a Russian IP. Which is very odd because all the session information is the same. But it still happens.

Re:Summary

[Entropius]

He's not sending mail from the open proxies. He's sending mail telling people where the open proxies ARE.

Re: Summary

[complete loony]

If the email providers a adding "golflanding.com" to their black list, stop sending the entire domain name on one piece. Construct a simple Turing test that the end user can solve to construct the name. If the text of the Turing test starts getting blocked as spam, you may be able to modify the test. Reducing the risk of the name of the proxy becoming well known or getting blocked automatically.

Perhaps he needs to build a new communication channel with his subscribers as his current approach has some obvious downsides. When a user first connects to a proxy give them a couple of proxy names randomly selected from your new list. Perhaps some client software that can silently refresh its list in the background, provided that one link still works.

Re:Summary

[Anonymous Coward]

How the Great Firewall of China is Blocking Tor: http://www.cs.kau.se/philwint/static/gfc/

Re:Summary

[jdavidb]

So your solution to the school bully demanding you give him your lunch money everyday is to just hand over the money?

That analogy doesn't fit at all.

Re:Summary

[girlintraining]

And he'd be right. Look, I'm normally a night owl but this week they got me in training and I have to wake up at 4 in the fucking morning... so if I'm off by a decimal point here or there, it's not surprising. I'm not going to care too much about 0.45% when we're talking about something as petty as spam. :)

Re:Summary

[Anonymous Coward]

He runs a collection of open web proxies.
You hate open email proxies because they are used by spammers.
You're too stupid to understand the difference, so you hate the guy for something he isn't doing.
That should sum it up nicely. Dumbass.

Re:Summary

[TheRealMindChild]

So. Let the shit fly because it is already flying. Typical

Re:Summary

[Anonymous Coward]

Because almost no one outside of this forum has heard of it.

Re: Summary

[Anonymous Coward]

Spammer landing urls like canadianpharmacy.com get blocked by filters.
Solution: wrap the url in a proxy redirector and stuff the new url into an email.
Over time filters start thinking that the proxy domain is a spammy domain. The filters are blocking these urls because real users are receiving wrapped URLs in their inboxes and junking the message because it really is spam.

Re:Summary

[Anonymous Coward]

2. If someone in a country wishes to circumvent government censors, why on Earth would they use a proxy? Why would they not just use Tor , which can't be blocked or filtered in that manner?

Do you actually believe that? Seriously? Tell me you don't really believe it.

Re:Summary

[trancemission]

Nailed it.

I also recall this story - simple point to take from this:

The reason the emails and hence his servers are being 'blacklisted' is due to the *content* of the message.

I am sure there are *many* companies out there that send emails out on this scale (and much more/often). I am to lazy to cite but think about it....

Please bring back the wild west of the internet, not much is free in this world now and I don't just mean in the monetary sense.

Why is this person being censored?

Political
Ineffective algorithms
Incompetence
+ Add your own

*Checks tin foil hat*

Re:Summary

[MurukeshM]

And I report your mail as spam or junk or (if the option is given) phishing. Screw you over, since my response is likely to get your emails to others classified as spam or - worse - phishing. If it takes me more than two clicks to unsubscribe, you're marked spam. Most people won't respond, some will be vindictive like me and very few will be willing to send an email.

Re:Summary

[TheMMaster]

My main issue is that there is no recourse. I did everything right, I had a /19 with all the correct WHOIS information, my server had nothing illegal on it or things that spamhaus doesn't like. (those two things aren't necessarily the same, that's the other thing that really annoys me)

And to pressure my ISP they decided to make it impossible for ME to use the server I paid for. I understand what they are trying to do, but the way they are doing it leaves a lot to be desired. If they actually cared about the damage they did they would have unblocked my /19 for at least a reasonable period of time for me to migrate. It's not like I got a warning or anything, I only found out I was shitlisted after people started to complain.

After that it took me several days to get everything moved over, DNS changed etc, and again: no recourse, no way of temporarily getting my service restored. The only thing I got was a warm fuck you from Spamhaus.

Re:Summary

[TheMMaster]

that was a /29 not a /19 whoops

Re:Summary

[fifedrum]

both, they've provided both for as long as I can remember, at least three or four years. MTAs are covered under the RBL, the URIs under the antispam "cloudmark authority" engine. One blocked at the connect phase, one at the end of the data phase.

Re:Summary

[Anonymous Coward]

I'm on two Constant Contact operated lists and there is absolutely nothing I can do to get off, including getting the companies that put me on to remove my address.

What part of the phrase, "constant contact," was unclear? They're your stalker now.

Re:Summary

[jonadab]

> Just FYI, I seen this guy bitching about it MONTHS ago.

Furthermore, the rant just posted on Slashdot is a verbatim copy of the one I read months ago (or, at least, the part that I re-read today is verbatim; I declined to re-read the whole thing, on the grounds that I remember it pretty well).

Re:Summary

[LordLimecat]

Summary: Once again, Bennet Haselton gets blocked from sending out automated email detailing how to bypass school and corporate IT policies, and wonders where he went wrong.

Here are a few protips, Bennett:
  * Offering services to get around various kinds filtering will eventually cause you problems on the internet, especially when said activities will make you an "undesirable" for IT stafff in general.
  * Sending out automated emails with automatically generated content likely to be on filters will cause you problems.
  * And biggest of all, you do noone any favors by teaching kids how to violate their school's computer-use agreement. If someone is a "victim" of filtering that they cannot simply uninstall, chances are 99% of the time they have no implicit / irrevocable right to the network / computer resources. Being a guest on the network means you play by their rules.

Re:Summary

[LordLimecat]

The domains he mentions in his automated emails are considered threats by a lot of filtering programs out there. Theyre literally about circumventing acceptable use policies.

Theres not much of a mystery here; I feel like this story comes up every few months, and he still doesnt get why hes so unpopular with IT departments.

Re:Summary

[eugene ts wong]

That guy is criticizing somebody for having a short attention span, but doesn't even have the time to spell and write properly.

yeah, spam blacklists are a poor solution

[Trepidity]

I could maybe see their necessity 10 or 15 years ago, but statistical classification techniques are good enough these days that a blunt tool like a domain blacklist doesn't really make much sense. Heck, Paul Graham was arguing that seven years ago, and it hasn't gotten less true.

Re:yeah, spam blacklists are a poor solution

[Anonymous Coward]

I wonder how many job opportunities I've missed or friends I've drifted apart from because of email dropped by statistical classification techniques. That's why everybody uses Facebook to keep in touch now.

Re:yeah, spam blacklists are a poor solution

[ColdWetDog]

I wonder how many job opportunities I've missed or friends I've drifted apart from because of email dropped by statistical classification techniques. That's why everybody uses Facebook to keep in touch now.

Friends? An AC on Slashdot?

Jobs? An AC on Slashdot?

Not to worry.

Re:yeah, spam blacklists are a poor solution

[niiler]

Mod up. This is a very good point. Closed systems like Facebook seem to work.

Re:yeah, spam blacklists are a poor solution

[pixelpusher220]

yes but maybe not for who think they work for...

Re:yeah, spam blacklists are a poor solution

[Anonymous Coward]

Pick your poison.

Facebook is 100% reliable in getting a message through.
Gmail and Yahoo are pretty good.

For everything else, losing a single email to a relative, friend, or employer is simply not acceptable. We're not talking about pictures of kittens, here.

I used to use my own domain for email, but now I'm lucky if messages end up in the receiver's spam box as opposed to just silently dropped. And even there, nobody checks their spam folder anymore.

So let Facebook and Google mine your messages, or wonder if anything you send ever makes it to the recipient.

Re:yeah, spam blacklists are a poor solution

[AlphaWolf_HK]

The spammers have found various ways around these. Often they throw a bunch of the "high target" key words (e.g. viagra, cialis, penis enlargement) in as images, or they'll use computer generated text that looks somewhat real enough to even fool some human readers in order to throw off those filters. This works because the more words you have, the less likely the small terms will be snagged.

Re:yeah, spam blacklists are a poor solution

[Anonymous Coward]

The problem is, 99% of connections to an SMTP server that hosts mailboxes for the general public are complete garbage. Running a statistical filter on those messages is a huge waste of resources when the assholes are already identified.

Re:yeah, spam blacklists are a poor solution

[Jstlook]

What about the slashdot article a few weeks back about the Dallas Cowboys complaining that Facebook wanted to charge him three grand to send out one update to his facebook friends?
Also, what about the fact that, at that level of users (100k+ish) Facebook *won't* post your update to each of your facebook friends? They just silently drop messages.
I don't know - just a thought.

Re:yeah, spam blacklists are a poor solution

[Onymous Coward]

Paul Graham and yourself are making the same error: blacklists are not all the same.

Don't conflate the mechanism (a list) with the method (how things get on the list).

Graham, showing his misunderstanding (emphasis mine):

Server blacklists tend to go bad, because the power they confer corrupts the people running them. They turn into vigilantes and start blacklisting innocent servers. ...
This is bad news, not just for the SBL but for the whole idea of blacklists. The SBL was started with the explicit aim of avoiding the kinds of abuses that had tainted other blacklists. So if even they are going the way of the MAPS RBL, one has to assume that every blacklist will, eventually.

So not only is Graham stuck on a single concept of what a blacklist is, but is trying to paint all list-providing organizations with the same brush he denigrated MAPS with at a particular point in time.

If you have ever shopped for blacklists with any degree of detail in your search you know that blacklists are not all generated the same way, they are not a homogeneous mass. One list might be composed of systems probed for open relays. Another might comprise only systems that have sent the list developer spams. Another might be of systems that have scanned the list maker's network. Heck, you could have a list that's just of IPs matching the birthdays of the top dozen R&B artists. Just look at the quantity of lists out there, as seen at (the now outdated)dnsbl.info. 80 blacklists by several dozen organizations — are they all going to be produced the same way, carry the same kinds of information?

So, yeah, the fact that blacklists haven't gotten less blunt as a tool for fighting spam has indeed not gotten less true, because mu, it wasn't true in the first place.

Re:yeah, spam blacklists are a poor solution

[Trepidity]

Well, sure, if you're not using good techniques, or training them well. Gmail is an example of doing it right, and has a very low false-positive rate, while not resorting to blanket domain blacklists. Why can't Microsoft and Yahoo match Google's performance there?

No Comparison To China and Iran

[Anonymous Coward]

The blacklists and censorship dealings in China and Iran are directly attributable to their respective governments, there is no similiar connection in hotmail and yahoo's blacklists.

Stop this, you look like fools.

Re:No Comparison To China and Iran

[rudy_wayne]

if you care about users in China and Iran

You had me up till there. At that point I realized you're an asshole and stopped reading.

Re:No Comparison To China and Iran

[Aristos Mazer]

His whole service is serving the dissidents in those countries. He isn't an asshole, he is actively trying to promote freedom in those countries. This isn't the "won't you care for the children" emotional plea that you were expecting. It is a description of the problem he actively works on, and if it is one that you care about, you should be helping here.

Re:No Comparison To China and Iran

[Anonymous Coward]

His whole service is serving the dissidents in those countries.

No, his whole service is for schoolkids to look at porn.

Spam is like cancer

[Anonymous Coward]

The only treatment is a deadly poison that you hope kills off the bad parts before the good suffers too much.

Re:Dude

[sexconker]

You're a spammer... Hotmail and Yahoo are doing us good... Get lost!

Yup!
Can't believe this kid has the audacity to complain about it on Slashdot with a wall of text to hide the fact that he's a fucking spammer.

People still use Yahoo mail?

[Anonymous Coward]

I think you just wanted to go on a political rant there. Seriously, you spend the post talking about the failings of two companies, ignoring the fact that there are other companies out there (well, you do mention GMail once, but you don't give any supporting evidence for it not being "open"), and act like two companies doing particular things is some kind of "failure of the free market."

So what's your solution? What's to stop a government-owned email provider from using this SmartScreen thing "as a matter of policy?"

Yes, Yahoo is appropriately named...

[msauve]

as you've discovered, it's made up of a bunch of yahoos.

You're a bleeding moron

[Anonymous Coward]

Seriously? It's fucking news that there might be domain blacklists that aren't public knowledge?

Re:You're a bleeding moron

[Anonymous Coward]

The news is that the domain blacklist applies to the content and not the sender domain.

"Free market" scare quotes

[Freddybear]

What's with the gratuitous complaints about the "free market" not giving some mythical "optimal solution" that lets you send your "100% guaranteed opt-in" spam without interference? I call bullshit. If Hotmail isn't accepting your "really honest it's not spam" mailing list stuff, maybe you should try contacting them about it. The "free market" doesn't magically solve problems without people doing what it takes to address the problems.

Re:"Free market" scare quotes

[ADRA]

No, the free market 'says' that if Hotmail is an inferior product then people will find a different product to use. You shouldn't give one hair arse if you're being blocked. Its those using inferior products that should feel sad about it. Email is about as far from monopoly terratory as you can get.

Re:"Free market" scare quotes

[Freddybear]

That's just silly. If you can't be arsed to do something about your "honest it's not spam" emails getting blocked, you don't have any business complaining about the people who do the blocking. Stop complaining about "the free market" as if you'd prefer an unfree one.

Re:"Free market" scare quotes

[PlusFiveTroll]

The problem with most be free email providers IS contacting them. You're not paying them, so they don't give a shit. Hell Google is hard enough to get a hold of when you are paying them.

The second problem is spammers lie about everything. This has turned server operators on to the line of thought that 'everyone is a liar'. If you weren't a spammer you wouldn't have been blocked in the first place. Needless to say this causes a number of race conditions.

And yes, I do run outbound and inbound SMTP services for a good number of customers at a small ISP.

Re:"Free market" scare quotes

[c_sd_m]

He did contact Hotmail about it. He got a form letter response, replied again and hasn't heard anything. Yahoo keeps just sending him generic, irrelevant articles from their knowledge base. Neither company is "contact-able" in any useful way.

Re:"Free market" scare quotes

[Freddybear]

Maybe Hotmail blew him off because he acts just like any other spammer. Changing domains and using remailer proxies isn't exactly the behavior of the usual legitimate bulk emailer. And yes, I do subscribe to a few of those, and I use ATT's Yahoo email account and I get my subscribed stuff just fine.

Re:"Free market" scare quotes

[PNutts]

Wish I had mod points. You hit the nail on the head.

Re:"Free market" scare quotes

[MaxToTheMax]

In fact, the author has become part of the free-market solution, by inadvertently "auditing" the quality of Yahoo and Hotmail's email service, and motivating their customers to demand better.

Re:"Free market" scare quotes

[chfriley]

And if he is really concerned about the "free market" not doing enough, there is always the option of starting a competing service that is more "free market" and responsive. Yes, that costs something, but that is the nature of the "free market" - it isn't free to everyone, someone's time costs something, always.

Server load

[betterunixthanunix]

Blacklists are nice because they reduce server loads. Sure, running a statistical classifier for one user is not so hard, but if you have to process hundreds of millions of messages per day, that is a lot of CPU time spent on spam.

Now, I agree that blacklists are bad, but we do need some system that doesn't require large amounts of CPU time or other resources. Hashcash is interesting here, in that the CPU time is mostly spent by clients; one might be able to slow spam down enough to let a combination of statistical filtering and greylisting take over.

Re:Server load

[nabsltd]

Blacklists are nice because they reduce server loads. Sure, running a statistical classifier for one user is not so hard, but if you have to process hundreds of millions of messages per day, that is a lot of CPU time spent on spam.

The CPU time spent on running something like SpamAssassin is insignificant compared to the bandwidth, disk writes, etc., caused by spam. Keeping the incoming e-mail in a RAM disk until you have truly accepted it for delivery (which isn't dangerous even if the server crashes hard) is the #1 thing that speeds up e-mail intake. At that point, scanning takes almost no time.

As you mention, though, greylisting does the best job of keeping your overall load down, since you don't even need to use network bandwidth on the body unless the sending server is known to retry, which basically eliminates every botnet member. Maybe this solution would work now that other more "instant" messaging systems are readily available, but 15 years ago when IM wasn't really a corporate thing, I couldn't use greylisting because "it slowed down e-mail too much", even though it didn't slow it down at all for the "important" clients, as their servers got whitelisted anyway.

Re:Server load

[gamanimatron]

but we do need some system that doesn't require large amounts of CPU time or other resources.

Why? CPU time is dirt cheap if you can concentrate your task. The bandwidth (a much scarcer resource) is already being spent, and better decisions will just tend to reduce your costs there. To me this smacks of laziness, not efficiency.

Re:Server load

[dodobh]

Nope. You can't acknowledge receipt until the message is written to durable storage.

And you have never run a really large email system. Bandwidth isn't really a limitation, disk io is.

Re:Server load

[dodobh]

Bandwidth isn't a scarce resource with email. It's disk io.

CPU time gets really, really expensive when dealing with email at a very large scale.

Bayesian scanners are nice for individuals, but do not work for groups of people with different tastes. False positives are even worse than false negatives.

Re:Server load

[sirsnork]

Not only that, but if your software is smart it only greylists server that it hasn't ever sent an email to anyway. So after the first week of being installed all your important customers are automatically whitelisted

Re:Server load

[nabsltd]

You can't acknowledge receipt until the message is written to durable storage.

Which is exactly what I said. If the message is rejected, everything has taken place in the RAM disk, and you don't care. If the message is accepted, then sendmail has written it to the queue at the very least. As long as the queue is not on the RAM disk, you're fine.

And you have never run a really large email system. Bandwidth isn't really a limitation, disk io is.

Again, I said this too: "CPU time....is insignificant compared to the bandwidth, disk writes". I guess Slashdot has gotten to the point where nobody reads anything before posting. But, having talked with people who run e-mail systems for millions of users, not having to receive the spam in the first place (unfortunately usually through blacklists, but often using greylisting) is the biggest win, as bandwidth is in reality far more of a limitation. It's quite easy to put together a disk array that gives you 500MB/sec throughput, while it's not easy to pay for a 4Gbps inbound line (which it what it would take to saturate those disks). Even assuming 8 bytes written per incoming byte, it's still pretty easy to spread load to effectively 2-4GB/sec worth of disk, while 2-4Gbps of Internet connectivity is pricey.

Re:Server load

[nabsltd]

Not only that, but if your software is smart it only greylists server that it hasn't ever sent an email to anyway.

Unfortunately, inbound and outbound SMTP servers often don't have the same IP address, so this doesn't work in practice.

So after the first week of being installed all your important customers are automatically whitelisted

But this is still very true. Only the first e-mail message is delayed, and that delay is mostly controlled by the retry time set on the sending machine. There are some really annoying ones that retry once a minute for a couple tries, then back off to an hour or more. This is the worst as far as greylisting is concerned.

Once a server is whitelisted, then my implementation allows 40 days of no activity from that server before it drops off the whitelist. This means that even a once-a-month mailing list from an obscure server doesn't see any delay after the first time. About 1/4 of the 380,000 IPs that have ever contacted my e-mail servers are currently whitelisted.

Re:Server load

[dodobh]

You are speaking of sequential io when you say 500MB/s. Email is random io. On spinning rust, your bottleneck is seek time.

You have talked with people who have run systems for millions of users. I *have* run systems for millions of users. Not having to receive spam saves you from spending CPU cycles on email, disk seeks, fancy RAMdisk based architectures (I know how to do this, but there's a good reason I recommend against it).

Simple summary

[Pollux]

He's saying that Hotmail, Yahoo, and GMail are running a cartel of free online webmail services.

He's trying to get opt-in email to accounts on these systems, and it's not going through. He has evidence indicating these services operate a common hidden blacklist service keeping those emails from getting to the accounts. He cannot reach people within these organizations to open up emails coming from his domains, as he does not have an inside contact to "assist" him with this problem. This leads him to speculate that Hotmail, Yahoo, and GMail are operating like a cartel, where only "approved" email list hosting service companies with inside contacts are able to do business with these services.

Better?

Re:Simple summary

[niiler]

Bingo. Good summary. I gave up using my own server to send email a couple of years ago for precisely these reasons. It wasn't worth trying to get de-blacklisted every few weeks because my server had an obscure domain name. If I recall, when I sent out more than 10 emails in a batch (we're talking maybe as many as 30) to members of a class, this triggered the anti-spam bots. When I did it from gmail or from other major providers, things worked beautifully. I had too many irons in the fire to deal with this, and while I would love to use my own server's email capability, it's not worth it anymore.

Re:Simple summary

[sorensenbill]

Much better, thank you!

Re:Simple summary

[kelemvor4]

Yes, now let's see if someone at dice can replace the article with this actual summary!

Re:Simple summary

[marcello_dl]

> I gave up using my own server to send email a couple of years ago for precisely these reasons

In fact, that's probably what the cartel wants, ultimately.

Re:Simple summary

[Anonymous Coward]

Or, you could just keep using your server as before. People who use providers which block your server could wise up and use something else, rather than let Google harvest all their email for marketing purposes while sometimes letting them see an email they want to see.

When you switch to Google, you become part of the problem.

Re:Simple summary

[SomePgmr]

This is weird... I don't think Google was mentioned in the summary at all.

But regardless, they're not operating with a list of approved senders. I build my own systems and send mail through them all the time. Sometimes just regular mail service, some for mass emailing (legally and legitimately). You'll have to take my word that I don't have super-secret inside contacts at Google, Yahoo and Microsoft to make sure this works.

Now if you meant to say they have anti-spam filters that occasionally throw false-positives and block mass emails from some domains I'd say, "Well no shit, welcome to email on the internet since the 90's".

Re:Simple summary

[Anonymous Coward]

operate a common hidden blacklist

I speculate there is also a whitelist. Normally flagging a message in gmail once means I'd never hear from sender again, but I don't know how many fracking times I've flagged "soap.com" for naught.

Re:Simple summary

[Anonymous Coward]

He didn't mention gMail in the 'Shares a Sooper Sekrit Blacklist' portion, AFAICT?

Re:Simple summary

[BitZtream]

No, you left out the part where it has nothing to do with his list that causing him to get blacklisted and the fact that the list contains sites themselves that are used for spamming.

Yes, open proxies get blacklisted, no shit, its a true story.

Re:Simple summary

[Anonymous Coward]

Never had that problem. Running your own mail server is a good learning expeirence and it gives you the freedom to blacklist 200/8 211/8 and anything written in Cyrillic or Arabic.

Google is still letting trought yonks of french spam

Re:Simple summary

[jibjibjib]

It's ok to blacklist email received from open proxies. It's not ok to block legitimate email for just *mentioning* them.

Re:Simple summary

[Anonymous Coward]

Ah, but he wasn't using his own server. He was using gmail but simply mentioning his server address in the body of the message got it bounced. So it's not so much a blacklist as a filter.

>> After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo)

Re:Simple summary

[Anonymous Coward]

It's ok to blacklist email received from open proxies. It's not ok to block legitimate email for just *mentioning* them.

You're not understanding how this works. A spammer will source a particular message from multiple places, open proxies, compromised systems, newly generated domains, etc. If all you rely on is a blacklist they'll always be one step ahead, so the messages themselves get fingerprinted. Then any message matching the fingerprint will start getting blocked regardless of the source.
By suddenly sourcing a shitload of emails, all at once, from several completely new domains, those emails get fingerprinted automatically as Spam.

As a similar example, if you always take your ATM card to a specific ATM, once a month, and withdraw $500, you've established a pattern of behavior. If you suddenly, all in a short period of time, try taking $20 out of ATM's all over the city, most likely your bank will flag your card and block further transactions, from any ATM, until you call them.

The submitter is matching the same pattern of activity as a high-volume spam operation, because he's basically trying to do the same thing the spammers are. Namely, get around blocks and filters which stop his email. The only difference is that his emails are wanted, while those of the spammers are not. If the only thing mail servers relied on was user reports of "spam/not spam" then we'd still get shitloads of spam- all you have to do is create enough dummy accounts to report "not spam" to offset the real people reporting "spam".

The reason any mention of those sites is tripping the filters is because they have been developed to deal with spammers who have messages which differ widely in content, save for a single address or URL. The spammers vary the content in the emails in an attempt to generate differing fingerprints for their mail, so when a huge volume of mail all shows up with nothing in common save a single URL, the filters will start flagging any email with that commonality.

It's a hard situation, and there's no easy solution for anybody. But the point of the entire story is not that he's getting blocked, but that Yahoo and Hotmail are either relying on a common service or directly working together behind the scenes. Anybody who runs mail servers for a living is actually already aware of this- if your mail starts getting delayed or you start getting backscatter from one of them, you'll see the other one start doing the same a short while later.

Re:Simple summary

[Anonymous Coward]

People who use blocking providers will not wise up, because they will not get any information, and go on living in blissfull ignorance...
Probably using Windows or MacOSX also...

Re:Simple summary

[Gr8Apes]

Having problems with this myself at the moment. It seems that entire blocks (as in Class B blocks) are being listed partly as an effort to remove the ability of "normal" people to run mail servers and force people to services such as Hotmail, Yahoo, and Google. Or, you can pony up the extra money and buy your static business IP(s) and for 10 times the cost for the same service, be "approved". You have to pay to play, it appears.

Re:Simple summary

[drinkypoo]

He's trying to get opt-in email to accounts on these systems, and it's not going through. [...] This leads him to speculate that Hotmail, Yahoo, and GMail are operating like a cartel, where only "approved" email list hosting service companies with inside contacts are able to do business with these services.

This leads me to speculate that it's no longer worth it to try to send mail through legitimate channels, when I can pay someone to just go ahead and spam on my behalf, and they can go through the pain of working out the details of getting my ads to my potential customer base.

It's too bad there's not more awareness of RSS, because it's potentially a much better way to stay connected with a customer base than spamming them.

Re:Simple summary

[LordLimecat]

This is the most ridiculous idea ever. Ive done IT consulting for over 7 years now, and I have dealt with hundreds of tiny companies with bizarre and obscure names who host Exchange on Windows SBS; somehow their emails manage not to get blocked.

Perhaps filtering outbound mail and taking action when your public IP starts sourcing spam has something to do with it. Perhaps making sure that the sent mail was legitimate also contributed.

What you suggest would mean that gmail and yahoo would be completely useless for a lot of small businesses (a HUGE part of business traffic, since larger companies are going to email more internally and less externally).

Re:Simple summary

[LordLimecat]

You skipped the part where the emails are sent in an automated fashion and that their content is very likely to violate computer acceptable use policy in a LOT of places, which I strongly suspect is related to the reason they are blocked. He also mentioned starting up a number of domains to get around the problem, which behavior is quite remeniscent of how spammers rapidly register obscure domains to get around blacklists.

Theres not a story here. People use blacklists, and have for years, and some of those blacklists are shared. If you want to stay off the blacklist, there are simple steps you can take:
  * Dont run an open relay
  * Probably a good idea to have a static ip, especially so that...
  * You have an accurate reverse DNS entry
  * Block outgoing SMTP from network except from trusted sources to prevent spamviruses from getting you blacklisted
  * SPF records wouldnt hurt
  * If you dont use SPF, it might help if an MX record actually exists for the domain you are sending from
  * Make sure your server can handle greylisting (ie, that it will retry after some period if it receives a "server busy" response)

Question that was never answered last time...

[Anonymous Coward]

Are the proxy servers you are sending out on these lists capable of relaying mail onwards on port 25? If so this is probably a significant factor in these blacklistings. If you block outbound connections to port 25 when you set up these proxies, you'll probably find your blacklist problems are significantly reduced.

5 second summary

[IamTheRealMike]

Blah blah blah ...... I sent craptons of mail to people who I'm sure want to receive it ..... but the system is telling me people don't .... blah blah ..... free markets suck.

I have worked on spam filters before. I've heard this story a million times. In case the article poster reads this, here's the blunt reality:

Those half-million people you think really really want new proxy sites all the time? Guess what, many of them don't. They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it. Expire addresses that signed up a long time ago - some people won't unsubscribe when it's no longer useful for them. Make sure it's a simple, obvious one click operation to unsubscribe, and I mean really one click - not "click, log in, go to preferences" etc. Being able to unsubscribe should be the easiest thing in the world.

If SpamHaus is blacklisting you, they probably think you're sending mail to their spamtraps. Hence the "zero false positives" claim. Are you sure every single address on your list replied to a confirmation mail? All 400,000+ of them? Because it sounds unlikely.

Re:5 second summary

[DRJlaw]

Those half-million people you think really really want new proxy sites all the time? Guess what, many of them don't. They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it.

You assume that this is case, yet the poster provides a link to management data which at least appears to show that your assumption is incorrect. Did you read the post where it mentions that "[it] showed a 'complaint rate' of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list)," or are you simply going to deny any version of reality that doesn't align with your assumptions.

Expire addresses that signed up a long time ago - some people won't unsubscribe when it's no longer useful for them.

Apparently, deny any version of reality that doesn't align with your assumptions.

BAD 'EXPERT'!

If I sign up to a mailing list, I expect to receive the output of that mailing list until I unsubscribe. I certainly don't want the mailing list silently dropping me, and I'm not very interested in the ISP offloading its mailing list problem onto me by making me affirmatively renew my subscription. Especially when you offer no evidence that 'addresses that signed up a long time ago' make up a disproportionate fraction of the alleged 0.1% spam report rate.

Pushing the problem onto the 400,000+ individual users instead of dealiing with it at the ISP level is exactly the sort of free market failure tha the poster complains of.

If SpamHaus is blacklisting you, they probably think you're sending mail to their spamtraps. Hence the "zero false positives" claim. Are you sure every single address on your list replied to a confirmation mail? All 400,000+ of them? Because it sounds unlikely.

Again, deny any version of reality that doesn't align with your assumptions. He isn't being blocked by SpamHaus. He's being blocked by Hotmail and Yahoo. Just admit that you haven't actually read the post, that you're spouting off about your own personal bugbear, and that your advice has almost no bearing on the actual problem. It'll make you feel better, honest.

Re:5 second summary

[Pope]

Why does he need to send 400,000+ emails in the first place? If it's just a list of proxy domains, why not just have an RSS feed that people can subscribe to? No emails needed.

Re:5 second summary

[Kergan]

+1. TD;DR the article, but the parts I did made this whole story reek of "your unsubscription method isn't braindead obvious enough to end-users, so they're unsubscribing by hitting the Spam button until your emails go away for good."

Re:5 second summary

[amicusNYCL]

They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it.

That's one possibility, and may even be likely considering his subject material. In this example he says he sent a total of 7 new proxy domains to 420,000 addresses, but only sent 1 domain to each person. So each domain got sent to a random 60,000 people, his reasoning being so that a censor could not subscribe and get a list of all new proxies, they would only get one (per address, at least).

But, instead of them getting those emails and blocking the proxies, it may be more effective for the censors to always report his emails as spam, thereby getting them blocked, and then no one gets any of the 7 new proxies. So the people reporting spam aren't doing it because they don't want the mail, they're doing it to stop other people from getting it.

Obviously, this is 100% speculation.

Re:5 second summary

[magic maverick]

Because then someone from the censorship companies or the censorship departments could easily get all the latest domains and block them automatically. By creating multiple domains and emailing them to a section of his subscriber list, he makes it that much harder to block all of them.

Re:5 second summary

[Anonymous Coward]

I certainly don't want the mailing list silently dropping me, and I'm not very interested in the ISP offloading its mailing list problem onto me by making me affirmatively renew my subscription.

Too bad, so sad. The fraction of the 400,000 email address owners who care about these emails matter far less than the tens of millions of us who don't want spam.
Receiving these emails should be much harder than not receiving them.

Re:5 second summary

[Anonymous Coward]

uh.. he said that "report as spam" was only seeing a 0.1% rate.
how does this differ from any other large mailing list?

Re:5 second summary

[magic maverick]

At the top of the emails:

[You are receiving this because you subscribed to the Circumventor distribution list.
To unsubscribe from this list, click here:
http://www.peacefire.org/circumventor/cv-unsub.html
or reply with the word "unsubscribe" in the subject.]

Seems pretty easy to me...

Re:5 second summary

[IamTheRealMike]

You assume that this is case, yet the poster provides a link to management data which at least appears to show that your assumption is incorrect

I assume this is the case because, like I said, having actually worked on a large spam filter I've seen this kind of story many times before. These people are always amazed to discover that people are pressing report spam on their wonderful bulk mail. Yet the fact remained that people were doing exactly that. They didn't want the mail.

Look at it this way. This guys screenshot shows Hotmail themselves saying he hit some of their spamtraps. From the SNDS FAQ we can see that "trap hits" means he mailed accounts that don't solicit mail - ever - so we already know his claim that every account is opt in isn't true. What else isn't true?

Pushing the problem onto the 400,000+ individual users instead of dealiing with it at the ISP level is exactly the sort of free market failure tha the poster complains of.

It's not a free market failure at all, these sorts of big webmail spam filters are very effective. If users are seeing false positives they can go and unmark the mail as spam, the system will learn that the user wants that mail and the problem is solved.

Again, deny any version of reality that doesn't align with your assumptions.

My assumption is that this story is much like all the other such stories I've come across - the guy is a spammer and doesn't realize it. This assumption is very, very likely to line up with reality.

Re:5 second summary

[Methuseus]

Because the RSS feed's server will likely get blocked, but the emails are less likely.

Re:5 second summary

[czth]

Um... wow, it's sorta sad that I have to explain this.

Imagine you're the Chinese Minister of Censorship, or the flunky that manages the Great Firewall. You learn about a website with an RSS feed with a continually updated list of anti-censorship proxies. What do you do?

(On the other hand, you haven't blocked Hotmail or Yahoo! or other email providers, because, well, riots are bad for business.)

Re:5 second summary

[Anonymous Coward]

Clearly not :P

Re: 5 second summary

[Urza9814]

RTFS. Hotmail confirmed that the portion of users marking it as spam was extremely small.

Furthermore, do you realize how many users will click the 'spam' button when they fully know it's something they subscribed to simply because they can't be bothered to take half a second to click the prominent unsubscribe link or send a reply? These people are trashing spam filters. And I know they're out there, because I got it all the time in college. Ran a student club with a mailing list of around 400 users (out of 40k+ students)...Email address were collected and added by hand, one at a time. Each message was typed and sent by hand from an officer's personal email account. And each one had an unsubscribe link highlighted in standard font at the bottom of the page. We'd still get a handful of 'this message reported as spam' emails with every single message we sent out. Yea, obviously some users didn't want our mailings, but they definitely opted in and we couldn't have made it easier to opt out...instead they chose to try to have our messages blocked from all users of that email service.

Point being, just because some people report it as spam doesn't mean it is. Also, the percentage reporting it for us would have been orders of magnitude higher than in this case and we still never got blacklisted.

Re:5 second summary

[afidel]

Because then the blocking companies would just subscribe to the RSS and the proxies would be blocked as soon as they were posted.

Re:5 second summary

[Anonymous Coward]

Mod parent up!

Re:5 second summary

[Anonymous Coward]

Why does it matter how he runs his business between willing parties?

The rub is that Hotmail and Yahoo told their users they'd accept email for them, but then go and do the opposite.

Re:5 second summary

[nabsltd]

Why does he need to send 400,000+ emails in the first place? If it's just a list of proxy domains, why not just have an RSS feed that people can subscribe to? No emails needed.

Because the people who want the proxy list would need to use a proxy to be able to read the RSS feed, as censorship in their country would block access once it was learned what the RSS feed contained.

On the other hand, incoming e-mail can be blocked as spam, but you can't decide to block an e-mail as spam without knowing something about it, and if it isn't coming from the same domain or IP address, and doesn't have the same content as previous spam, it's pretty tough to block. So, once the e-mail gets through, the "damage" is done (as far as the censors are concerned).

Re:5 second summary

[Anonymous Coward]

100% agreed

Re:5 second summary

[Kergan]

Fair enough. But you know, in my (admittedly convoluted) experience in this kind of stuff, a separate issue is user trust and laziness.

It's one thing to write the above notice to paying customers who you're sending news and updates to; even if users got tricked to subscribed, they trusted the business enough to purchase something, and they're likely to unsubscribe -- even though more than a few actually mark it as spam. It's an entirely different thing to write the above notice to an opt-in list, unless it wasn't double opt-in -- which, insofar as I can tell from TFS/TFA, it isn't. When not, for all you know, you mistakenly gave your email or your email got sold, and you've no clue whatsoever what may happen if you click the link. Will you get even more spam? Will you end up on new lists? And even when it's double opt-in, actually.

Here's a thought: instead of simple or double opt-in, I'd suggest that businesses running mailing lists implement a recurring re-subscription opt-in. By this, I mean send the whole list a quarterly reminder of the lists they're subscribed to. Unless readers click a link to confirm that they wish to continue to receive emails, they automatically get unsubscribed. Never mind that most lists would instantly get trimmed to a notch about zero -- if that -- because the recipients might fail to read that email: the fact of the matter is that you're getting filed under Spam precisely because you're not getting read and you're annoying your audience.

Re:5 second summary

[Anonymous Coward]

Because the censorship companies would *never* think of just signing up for your email, they certainly aren't that clever.

Re:5 second summary

[Anonymous Coward]

And why doesn't the Chinese Minister of Censorship just sign up on the email list? Heck using procmail you could just as easily injest the list and block from it...

Re:5 second summary

[DRJlaw]

Look at it this way. This guys screenshot shows Hotmail themselves saying he hit some of their spamtraps. From the SNDS FAQ we can see that "trap hits" means he mailed accounts that don't solicit mail - ever

Yes, you've noted three trap hits out of 68,000 messages. Do you want to bet that those three trap hits are signup confirmation emails resulting from (i) typographical errors in the email address submitted by someone attempting to sign up or (ii) 'drive by' sign ups by a third party who has an axe to grind against the list?

There is essentially nothing to prevent someone from signing mike@plan99.net up to a dozen mailing lists in the signup process. It doesn't matter if they're mile@plan99.net or simply a jerk -- the fact that it happens (at 0.0044% frequency) doesn't transform the mailing list operator into a spammer. Even Hotmail notes that "[w]ell-behaved senders will hit very few such accounts because they're generally sending to people who give them their address and because they collect and process their NDRs." They don't expect a zero rate.

so we already know his claim that every account is opt in isn't true. What else isn't true?

That you're not bothering to think through the signup and confirmation process, for one... that your putting claims in his mouth that he never made, for another... "The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing." There's simply no basis for you to say that those accounts were falsely opted-in.

FYI in the complaint rate section the SNDS FAW states that "more than 30% of the IPs sending mail to Windows Live Hotmail keep their complaint rate at less than 0.3% and this represents a good bar to shoot for." He's allegedly at 0.1%. Your expectations are simply unrealistic, and yet again show that you're not willing to deal with the reality of the situation rather than attributing anything other then perfection as being evidence that "the guy is a spammer." Frankly, you're a perfect example of the problem at hand.

If users are seeing false positives they can go and unmark the mail as spam, the system will learn that the user wants that mail and the problem is solved.

Again, deny any version of reality that doesn't align with your assumptions. Reread the actual problem -- the users do not see these emails when sent from this account, and therefore cannot unmark the mail as spam. The problem is not solved. The problem isn't even remotely what you concieve it to be.

Re:5 second summary

[magic maverick]

It is double opt-in. As in, I signed up for the emails. I get an email asking me to confirm. I click reply and send. I get the emails. Text from the confirmation email (with email addresses changed):

Dear webmaster@yahoo.com

We have received a request to subscribe this address to our mailing list, where send out the locations of new 'Circumventor' servers to help bypass Internet censorship.

To confirm that you want to subscribe this address to our mailing list, you MUST REPLY TO THIS MESSAGE without changing the subject line. (The subject line has a 14-digit number in parentheses on the end, and you have to leave that in the subject when you reply.) Just hit 'Reply' and hit 'Send', and that should be enough. This is to prevent people from signing up other people without their permission.

Please do not write any message to us when you reply, since the replies are processed automatically and your message will not be read. If you have any questions, please send a separate message to webmaster@hotmail.com

Once you reply to this message, you will be added to the Circumventor list.

Thank you.

Re:5 second summary

[czth]

They randomize which proxies get sent to which random parts of the list (see what RTFA gets you?). Granted, the Minister signed up enough accounts hed' probably get them all (unless they got suspicious at all the requests from evil-commies.cn). Still, harder than one central website.

Re:5 second summary

[Anonymous Coward]

And why don't those same companies just procmail injest his email list? Would be no harder, one long bash pipe....

Re:5 second summary

[IamTheRealMike]

Once again, what I'm saying is, you're accepting everything the poster says on the assumption it's absolutely true. Spamtrap accounts don't reply to confirmation emails or click on confirmation links - ever. That's the whole point of them. Even if you're a malicious troll who got a list of Hotmail trap accounts from somewhere, how do you get control over them to confirm signup?

The screenshot says more anyway. Judging from what he says the sizes of the mailshots are, it's a fresh IP that hasn't been used before. So the screenshot could have been taken before the reputation degrades. That by itself probably won't help, a new IP that sends links to newly registered domains which have no reputation to huge numbers of users and hits spamtraps is exactly the sort of thing spammers actually do.

Look. It's possible that this guy has done everything totally by the book and somehow has just got unlucky that his behaviour happens to closely match that of actual spammers. Or it's possible that we don't have the full story. Having been on the other side of such stories and investigated cases like these, I think "sender is not following standard mail etiquette" is far more likely than some enormous conspiracy theory against him. After all, plenty of bulk mail senders do just fine.

Re:5 second summary

[BitZtream]

Wrong.

If it requires anymore than clicking a link in the email, its failed. Going to a page, doing more crap, blah blah blah, I just hit 'spam' and move on, so does everyone else. If I don't want it, its spam, period. You as the sender need to make it so A) I want it and B) I don't get bored/annoyed trying to get rid of it after I'm done wanting it.

He also hasn't bothered to setup feedback loops with Yahoo and Hotmail, which would solve his problem and show that he had a clue.

He's also sending a list of open proxies which can be used to ... login to yahoo/hotmail with fake accounts and send spam.

There is nothing about what he is doing that makes him wanted by anyone.

I personally have several accounts subscribed to his list. I use his list to block domains at my mail server, he provides me an up to date set of lists every few days so I can block him.

He's really not that good at what he's doing.

Re:5 second summary

[psmears]

Spamtrap accounts don't reply to confirmation emails or click on confirmation links - ever. That's the whole point of them. Even if you're a malicious troll who got a list of Hotmail trap accounts from somewhere, how do you get control over them to confirm signup?

The malicious troll doesn't need to confirm signup - only to request it, at which point the list server will send an email to the spamtrap, and boom, your reputation takes a hit. All while you're conforming 100% to best practice.

Look. It's possible that this guy has done everything totally by the book and somehow has just got unlucky that his behaviour happens to closely match that of actual spammers. Or it's possible that we don't have the full story. Having been on the other side of such stories and investigated cases like these, I think "sender is not following standard mail etiquette" is far more likely than some enormous conspiracy theory against him. After all, plenty of bulk mail senders do just fine.

I see what you're saying, but he's not actually having his IP blocked in this case. The blocking is taking place based on the content of the message, specifically whether it mentions certain domains set up as relays. The interesting question (from his point of view and ours) is exactly how those domains become flagged as "spammy". For instance, I'd be interested to know (as others have asked) whether the relays allow traffic on port 25, and whether this is a factor.

Re:5 second summary

[DRJlaw]

I see what you're saying, but he's not actually having his IP blocked in this case. The blocking is taking place based on the content of the message, specifically whether it mentions certain domains set up as relays. The interesting question (from his point of view and ours) is exactly how those domains become flagged as "spammy". For instance, I'd be interested to know (as others have asked) whether the relays allow traffic on port 25, and whether this is a factor.

Actually, as I read it, it's the combination of mentioning certain domain names and the fact that the message originates from the mailing list IPs. It seems that other messages from the same IP would be received rather than blocked (not specifically discussed, but implied in the three not-banned domain names) and that messages containing the same domains sent from other IPs and email addresses would be received rather than blocked (specifically discussed in his gmail example, paragraph after the session transcript).

It's a far more specific block, and one that I suspect whitelisting the mailing list email address does not overcome (I'm not a member of the mailing list) -- which would be the ultimate issue here.

Re:5 second summary

[IamTheRealMike]

All spam filters do domain blacklisting. The reason is that the textual content can be randomized for free, but spammers typically want to sell something, which means providing links to their stores. It's much harder to avoid having links in your mail, so it makes sense to measure their spammyness and blacklist. Or at least it used to. The prevalence of link shorteners and hacked websites means it doesn't work as well as it once did.

I suspect there's a rule in Hotmail and Yahoos filters that say something like "if a mail contains a link to a young domain that has never been seen before, and it goes to lots of people, and some of them are marking it as spam, and it hits spamtraps, then it's spam". The act of distributing deliberately fresh domains as censorship evaders would then hit such a rule, especially if you do it at enormous scale via email.

Re: spamtraps, you're still assuming that some malicious entity knows where to find lists of spamtrap addresses. They aren't actually listed anywhere, right? Just scattered around the web waiting for crawlers to find them. So at some point Occams Razor applies.

Anyway, my point is simple. There are lots of safeguards in the big 3s spam filters. Those filters aren't perfect, but 99% of the time people complain, they're actually sending mail people don't want. It's possible this guy has found the perfect storm of edge cases that cause widespread failure - or it's possible that there aren't actually 400k people who want proxies emailed to them.

Re:5 second summary

[DRJlaw]

Once again, what I'm saying is, you're accepting everything the poster says on the assumption it's absolutely true. Spamtrap accounts don't reply to confirmation emails or click on confirmation links - ever. That's the whole point of them. Even if you're a malicious troll who got a list of Hotmail trap accounts from somewhere, how do you get control over them to confirm signup?

The confirmation email sent to the spamtrap account is itself the trap event. If the spamtrap does not confirm and the list does not send anything other than the confirmation email, then both the Hotmail management screen and his statements are still fully consistent. Do you want me to sign you up to his list to prove the point, or are you content with merely being lead to this very obvious conclusion through multiple Slashdot postings?

And yes, I am accepting it as true. It's trivial to follow the list signup procedure, respond to the confirmation message, and note that Hotmail even automatically categorizes the email as one from a newsletter. I unfortunately have to wait for the next mailing to confirm the unsubscribe link, unsubscribe, and then wait to not receive more messages, but it is consistent with everything that has been written, whereas you are merely guessing. And ignoring every other opinion to the contrary, e.g.:

http://features.slashdot.org/comments.pl?sid=3314491&cid=42276705
http://features.slashdot.org/comments.pl?sid=3314491&cid=42276435

The screenshot says more anyway. Judging from what he says the sizes of the mailshots are, it's a fresh IP that hasn't been used before. So the screenshot could have been taken before the reputation degrades.

You really insist on not reading the source material, don't you. "Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list)." The screenshot states that it is for a 24 hour period. With a subscribership of 420,000, he's not going to be emailing 420,000 Hotmail users over 24 hours.

It's possible that this guy has done everything totally by the book and somehow has just got unlucky that his behaviour happens to closely match that of actual spammers.

In that case, why isn't the spam-identified content blocked when sent from other IPs/email accounts? "This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself."

I think "sender is not following standard mail etiquette" is far more likely than some enormous conspiracy theory against him. After all, plenty of bulk mail senders do just fine.

In that case, why is the mailing list not blocked, but only certain content? "It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not."

What you keep saying is that you simply will not read what is going on, and will not address the actual problem, but by God you'll fight tooth and nail against anyone who dares to point that out. Bravo. You'll notice that others have picked up on it too. I replied because you were +5 Informative yet clearly wrong. That seems to have resolved itself now, so I'm done with you.

Re:5 second summary

[AvitarX]

But they could randomize the domain of the proxies, and email each one to a subset of customers, then they wouldn't all get blocked.

There are evil forces out there

[SpaceLifeForm]

Read my sig.

Really?

[Anonymous Coward]

Oh, someone is still using Hotmail and Yahoo?

Wouldn't it be just easier to do vice versa, block both and it would be a favor to us all.

gold standard for responsible mailing

[joostje]

Yes, verified opt-in is one requirement. But if you don't want to be marked as sender of SPAM, you should also make it *very* simple to unsubscribe. I know I've subscribed to a few lists, and at first read the emails, then ignored them, and eventually thought "should unsubscribe". But if that unsubscribing is difficult, I'll just hit "spam" in gmail (or whatever). I don't see the emails and more, and the sender gets blocked as spammer.

Re:gold standard for responsible mailing

[Anonymous Coward]

I work for a company that sends 2 email blasts per week to confirmed opt-in subscribers. We have a 1-click unsubscribe. We maybe 3-5 unsubscribes per week ( for a blast about 10,000 email addresses strong ). Believe it or not, most of those are our members CALLING us to have them removed. Our unsubscribe link is at the top & bottom of every email we send yet they call us for it. Granted, unlike the OP's business, we have a lot of non-tech savvy subscribers. I just wanted to point out that one-click solutions still don't mean anything.

I have had an email bill that we sent out get marked as spam by MSN & AOL. The reasoning had nothing to do with the content, but was actually because we sent X emails to their server in a short period of time and tripped their spam filters. It took an email and a phone call the first time. Since then, we were added to their respective whitelists and haven't had an issue since.

Re:gold standard for responsible mailing

[Anonymous Coward]

Even with a really simple unsubscribe, you'll have idiots that can't be bothered to understand how it works. On hlds, Valve's mailing list for server operators, there was this dude that wrote to the list to demand to be removed from this list or he would spam everyone in it. The unsubscribe URL is below every single message sent to the list. It's a list for server operators, it's opt-in only with a verification email. You would think he would be able to handle something as a simple unsubscribe. The point is, no matter how easy it is to unsubscribe, you'll have stupid people that will mark a message as spam even when they asked to receive it. There is just no way around that.

Re:gold standard for responsible mailing

[magic maverick]

Here's the latest email I got from Mr Haselton (with the email addresses changed though).
It's apparently very easy to subscribe. (Though it's not one click as you do need to enter your email address if you use the webpage option.) Is that good enough for you?

From: Bennett Haselton at Peacefire.org <webmaster@yahoo.com>
Reply-to: "Bennett Haselton at Peacefire.org" <webmaster@yahoo.com>
To: webmaster@hotmail.com
Subject: new Circumventor, in a new format
Date: Fri, 07 Dec 2012 04:00:02 -0500 (07/12/12 10:00:02)
Envelope-To: webmaster@hotmail.com

[You are receiving this because you subscribed to the Circumventor distribution list.
To unsubscribe from this list, click here:
http://www.peacefire.org/circumventor/cv-unsub.html
or reply with the word "unsubscribe" in the subject.]

Happy Holidays everybody -- your early Christmas gift enclosed:

https://www.kitepuddle.com/smart/

This Circumventor site is in a different format but it should work as well as the others. You *must* access this one with 'https' at the beginning of the Web address; it won't work with 'http'.

You can attempt to access the "regular" Facebook through this one, for example, but it might not work correctly; the most reliable way is to enter http://m.facebook.com/ on this Circumventor site, which will take you to mobile Facebook. Unfortunately Youtube still isn't accessible yet but we're working on it.

Don't waste too much time on those school computers - Santa's watching!

Bennett

***

"When I was in high school these twins got mono. They got stereo." -Demetri Martin

Peacefire.org
14615 NE 30th PL #10D, Bellevue WA 98007/blockquote.

Re:gold standard for responsible mailing

[Revotron]

You do realize you're talking about Valve servers, and this person who threw the temper tantrum on the mailing list is probably 12 years old and bought his server with daddy's credit card? It's no surprise, really. If you're looking for foolish, overdramatic, hot-headed people, look no further than Counter-Strike players.

Re: gold standard for responsible mailing

[Urza9814]

I'm on this particular mailing list, so I can confirm that he makes unsubscribing quite easier. Easier than any other list I've ever been on in fact. Every email has the following text as the first paragraph:

[You are receiving this because you subscribed to the Circumventor distribution list.
To unsubscribe from this list, click here:
http://www.peacefire.org/circumventor/cv-unsub.html
or reply with the word "unsubscribe" in the subject.]

Re:gold standard for responsible mailing

[Solandri]

The problem with designing something to be completely foolproof is that people underestimate the ingenuity of complete fools. Compare having to read the first four lines of that message vs. clicking a "spam" button on your email program. A not insignificant number of people are going to click the spam button, at which point the mail hosts start to classify it as spam.

I suspect what's needed is a "verified legitimate mass mailer" list. Sort of an inverse-spamhaus list. Legitimate mass mail services can somehow prove to the satisfaction of the major mail hosts that they're completely opt-in. Then the hosts know that if a user clicks "spam" for one of the mails sent from these services, that the message isn't really spam and the user is an idiot.

Re: gold standard for responsible mailing

[ortholattice]

This is still inconvenient, because the unsubscribe link requires you to enter your email address. This requires a redundant step by the user (who may make a typo).

I have several email addresses forwarded to one place, and when an email says it was sent to "undisclosed recipients" I have no idea which one I need to unsubscribe without a tedious analysis of the header. I don't think an average user could do such a header analysis.

Finally, I'm just plain suspicious of any site asking me to type in my email address. If they already know it, why are they asking?

Re: gold standard for responsible mailing

[Anonymous Coward]

Oh Christ. I __DESPISE__ mailing lists that do that! If I want to forward, copy/paste, or otherwise share something from a mass mailing, I shouldn't have to analyze every link in the message to be sure I'm not also sharing a bunch of personal information. Not the absolute worst if it's just an email address, but I've gotten some where they've got a dozen links spread through the email that all have name, address, phone number and email within the url string, and all encoded so that you can't even tell unless you're really looking for it....

Re:gold standard for responsible mailing

[Anonymous Coward]

It's apparently very easy to subscribe.

Spam is always easy to subscribe to. The question is: How easy is it to unsubscribe?

Hint: You should say what you mean, rather than the exact opposite, if you want to be taken seriously.

Re:gold standard for responsible mailing

[Anonymous Coward]

I'll just hit "spam" in gmail (or whatever)

just on that topic....

People report as junk/spam direct mail from friends, family, business contacts. They use it as a delete button. If you have internal mail users that forward mail out to a Hotmail address, or in fact if anyone at your organisation emails friends at Hotmail or AOL addresses you'll see this happening.

I know this because we signed up to Microsoft and AOLs feedback programs which email you (postmaster) an alert whenever mail from your server is marked as junk, and it includes a copy of the mail that was marked as junk, which would be helpful if it really was a problem mail but is more likely a copy of your users personal mail to loved ones that you would never normally see.

Remember that once the number of junk reports in a given time period go over a certain threshold they block your server and you won't be able to talk to a sane human to fix it. So wanting to be a responsible postmaster I've tried chasing up the reporters to find out why they junked legitimate mail but every single one denies they reported you for spam - I think they don't realise that's what the button does.

Hence you (as a postmaster) can't use the feedback systems as a method improving your mail service _unless_ you are solely a commercial bulk message sending company, in which case you've no legitimate users to worry about and can delist anyone that reports spam a message from your server as spam but If you have legitimate users at a large site you're going to get hundreds of false positives a day, you might see no genuine mail list issues in a week - you just can't act on the information at all. I've tried and just got burnt out within a day or two from all the false positives.

If (due to users getting phished) you do get a compromised internal account that manages to send mail out without your postmaster noticing (a case where you actually do want to receive this alerts), you'll see the number of reports increase which is an indicator but a fairly slow reactive one and not as good as reporting from your own mail relay. On top of this you'll normally see a report from spamcop.net anyway from which I've only seen one false positive in 5 years and they don't bombard you with hundreds of false positives a day (reading this back, no I don't work for the later organisation).

I hate that spam button, or rather I hate the fact 99% of users have no idea what it actually does.

[I also failed to prove to slashot I am a human, twice...]

Is this a repeat?

[rudy_wayne]

I could swear this same guy was complaining about problems with his "I swear it's not spam" mailing list several months ago.

Re:Is this a repeat?

[ADRA]

Sounds like the same guy. At least the exact same scenario...

Re:Is this a repeat?

[Anonymous Coward]

Not enough people bought it last time, so he figures he needs to tell the world again.

Re:Is this a repeat?

[Bill Dimm]

Yep, 2 months ago

Re:Is this a repeat?

[Cmdr-Absurd]

I was thinking the exact same thing. I'm SURE I've seen this exact story before.

Re:Is this a repeat?

[PRMan]

Before, he complained about the problem. Now, he is sharing what he found out.

Re:Is this a repeat?

[czth]

No - that was about Spamhaus (which is mentioned again in this current writeup but not the main point); this scenario is about Hotmail/Yahoo! having (he believes) a shared secret domain blacklist that applies to the content of emails.

Re: Is this a repeat?

[Urza9814]

Hence the words "frequent contributor" at the top.

I've been using his service for at least six years. It's as far from spam as you can get. Certainly far less spammy than the emails from newegg or Amazon (which is among the worst!) or any of the others that have no problem at all getting through spam filters. Multiple ways to unsubscribe right at the top of every message, verified opt-in, low volume, no embedded tracking features (all plain text), and legitimate content.

So what the hell else do you want? Should he start collecting phone numbers and personally call each subscriber to confirm before sending each message???

Distribute the load

[betterunixthanunix]

Part of the problem with spam fighting is that we are not distributing the spam fighting load. Hashcash distributes the load somewhat, in that it forces spammers to use more resources to send out their message and can slow them down somewhat. A distributed filtering system that allowed people to volunteer CPU time and bandwidth to filter spam (with some system of gaining the trust of an email server) might also work; imagine if hundreds of millions of people were relaying / filtering 100 messages per day.

Re:Distribute the load

[afidel]

There are several distributed reputation filter systems but they are all commercial AFAIK.

Re:Distribute the load

[nabsltd]

Hashcash distributes the load somewhat, in that it forces spammers to use more resources to send out their message and can slow them down somewhat.

Unfortunately, until you get to a significant number of bits, hashcash doesn't take all that long to compute, and you can pre-compute them.

I use 23-bit hashcash on all my outgoing e-mails, but if the address has been sent to before, there is likely a pre-computed 25-bit hashcash waiting. I use idle server time to pre-compute for any address that has been sent to from my servers. Since the hashcash expires in 25 days, I don't have to do this very often unless the recipient is a frequent one. Then, to keep the database small, I remove addresses from the "sent to" table unless they have been recently active, where "recent" depends on the total amount of activity to that address.

Re:Dude

[jellomizer]

I hate to use the if you were legit then you wouldn't need a proxy argument. However If he was using email the way most services want you to use it, he wouldn't have a problem.

Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.

Yahoo outgoing mail filter

[Anonymous Coward]

Yahoo has various keywords blacklisted too. Try sending an email from Yahoo containing the words "Western Union" or "Bitcoin" and it throws up a captcha.

Independent verification of verified/double opt-in

[bersl2]

I used to work security at a major hosting provider. If we got complaints about your mailing list, the first thing we'd do is ask you about how you got your list, to see if it complied with our requirement for verified opt-in lists only. We'd also sign up ourselves or check logs and code, because customers always lie (except when they don't).

Right now, I'd apply the same standard of skepticism. I understand that revealing such things would make your proported aim of censorship circumvention hard, but I'd still like to hear independent verification from someone who can reasonably demonstrate the depth of their commitment to opting in.

apple has a "secret list" too it seems

[crisper]

Apple has a "secret list" too it seems, I had one case of this with one domain. When I called I explained to normal tech support the issue, they had me escalated where I explained the issue in a bit more detail. Within an hour or two I had a call back from Apple support telling me that the domain had been removed, I didn't pry any more I just figured since they have the right to deny email for whatever reason then have the right to do this. This came after looking over logs, and some packet captures, to make sure it was being delivered to their servers before making the call to Apple. Nothing indicated any type of failure/deferred/blocked from looking at those logs/captures.

Great!

[Anonymous Coward]

I think it is a great feature!
So much spam is sent these days with only a short cryptic text and a URL that it is a necessity to block on domain names mentioned in messages.
Apparently it works fine.

Re:You are a spammer

[Anonymous Coward]

I love how you obviously do not understand the purpose of the emails this guy is sending...

Re:You are a spammer

[Anonymous Coward]

All 420,000 signed up for it, and confirmed their email addresses. How can it be unwanted? If they didn't want it, they could click the unsubscribe link.

How can you think that there aren't 420,000 people in the world who may have a common interest, and want to receive the same newsletter?

Not a hard problem to solve for PGP.

[DamnStupidElf]

Even S/MIME might meet your needs in this case. Encryption is cheap enough even for mailing lists now.

Re:Not a hard problem to solve for PGP.

[Anonymous Coward]

Wait, how can you encrypt an email for a mailing list? You would have to encrypt the session key with each recipient's public key, otherwise how on earth would they access it? If you tried to do it the other way around, then that means literally anyone can read the message, so what is the point of encrypting it?

Re:Not a hard problem to solve for PGP.

[Anonymous Coward]

Yeah, CPU Power is not going to be an issue with 420'000 receipients, the two who actuall have a key/certificate are encrypted very fast...

Re:Not a hard problem to solve for PGP.

[Megane]

The information is already being sent in clear text. The only reason to encrypt it would be to avoid automated blocking.

Since all the domains are composed of two English words, they could be sent as two words, with a space in between, and possibly another codeword to indicate .com, .org, or some other TLD. That would remove the "scan the message for anything that looks like a domain name I haven't seen before and scan it for open proxies" angle.

Perhaps he could use rot13, or a substitution cipher with the key as the first line. or some similar encoding that is easily decoded both manually and with a 10-line C program. It might even be possible for every message to use a different key, and the automation necessary to do that would also let you give everybody a different random subset of the domain names. These are meant to be good for 1-4 weeks, so taking 5 minutes to decode them should be insignificant.

Getting even more tough about using low-impact steganography, the text could be converted into small 1-bit .gif encodings of the domain names and inserted as MIME data. This would be hard to reject automatically by means other than the fact that they are encoded, but a decent GUI mail reader would be able to display them. If the font is kept consistent, they can even be converted back to plain text.

It's a balance between something easy enough to decode, yet difficult enough to be annoying for automated detection. Remember, true spammers need to have their message visible as plain text because their recipients aren't motivated to decode it, but someone who really wants the data can be expected to take five minutes to decode it.

Re:Not a hard problem to solve for PGP.

[Anonymous Coward]

Same AC here. While I appreciate the insight, the question remains, does any of the encryption software allow you to encrypt mailing list emails? An easy and automated solution in this case would be to encrypt with the mailing list's private key, that way anyone could read it. But that would not work with a lot of email lists for a multitude of reasons. Likewise, encrypting it with each recipient's private key would get unwieldy for larger lists; so, do they have mailing list keys? If I were the PGP guys, this is something I would have attempted to take care of so I can't believe there isn't a solution that preserves the confidentiality of the message.

Re:Not a hard problem to solve for PGP.

[DamnStupidElf]

Elliptic Curve ElGamal encryption is pretty fast.

Re:Dude

[lister king of smeg]

Its not Spam if you opt in. Spam is unsolicited. For this you have to request. Now is it possible the guy is bull shitting that part sure, however if we accept that the articles are bull why bother to read them?

Re:You are a spammer

[Anonymous Coward]

420k emails is a lot? tha tis nothing, grupon sends 10m plus everyday, but wait they are own but same people as gmail, hotmail,yahoo etc...

Re:You are a spammer

[glaurungn]

He sends proxy address to people that requested that information. He send it weekly because the proxys are blocked.

You missed the point.

[CaptainNerdCave]

The issue is that no one on the list of recipients got the chance to refuse the message.

How can you be certain he is not part of an internet forum dedicated to anonymity? What if he were sending an email with updates on domains that are security risks to a long list of subscribers to his IPsec newsletter?

There is a very long list of possibilities for what he could have been doing that was perfectly legitimate. Basically, USPS, UPS, FedEx, DHL, $common-carrier should not read your text-only message to determine if there is any information they don't like, and refuse to deliver it based on that alone.

Re:Dude

[Anonymous Coward]

> some connection too.

I'm sorry, your sentence abruptly ended. The connection also what?

Sigh

[junkgoof]

Blocklists are not a bad thing. I dealt with a number of them when I inherited an SMTP open relay 10 years ago or so. People tend to hate them because they rant at the (generally unpaid) people running the blocklist instead of taking steps to show they are mailing sanely. I configured my SMTP server and got the IP removed from all (and there are a lot of them) blocklists including a number with a reputation for being unreasonable. Politeness goes a lot further than ranting.

This guy may say what he's doing is normal and reasonable but it sounds as though he's blatantly spamming. If the guy does not want his stuff flagged as spam he should try sending e-mails with the same address people opted in for.

Re:Sigh

[psmears]

If the guy does not want his stuff flagged as spam he should try sending e-mails with the same address people opted in for.

He is doing that - but the mail is being blocked purely because it mentions certain domains in the body of the message.

Re:Dude

[Anonymous Coward]

Do you people not understand the concept of an email newsletter? For instance, I am subscribed to NASA Tech Briefs 's email newsletter, which purports to have an audience of over 77,000. Being a newsletter, of course those emails all have "the same web address in them" -- they're the same bloody content. This has been going on for decades (they've been a big thing since home users who never heard of usenet started getting internet access...), and as long as it ONLY GOES TO PEOPLE WHO VOLUNTARILY SUBSCRIBED, it's NOT MOTHERFUCKING SPAM! If your spam filter flags this, your spam filter is broken. Spam= UNSOLICITED bulk email, not all bulk email.

This looks familiar

[Anonymous Coward]

I got bored the last time you complained about your mass emails getting caught.

Re:Dude

[pixelpusher220]

if we accept that the articles are bull why bother to read them?

You must be new here. Welcome!

Re:Dude

[magic maverick]

After the last article I signed up for the service of getting emailed the proxy sites. Guess what, I've had no problem. I've not recieved any spam to the email address I used. I've only received emails that I specifically requested.

So, ah.

Dude, you're a fucking idiot. Hotmail and Yahoo are not doing anyone good... Get lost!

If someone is running an incredibly popular opt-in email list, that doesn't automatically make them a spammer. In fact, because it's all opt-in it makes them the opposite. It's solicited, not unsolicited. Mr Haselton is one of the good guys, and you are a moron if you can't see that.

Re:Dude

[crypticedge]

I have to use a mail proxy, not because I spam (we send about 20 emails a month) but because verizon blocks port 25 outbound, and won't let me get a static IP at home for my mail server.

I pay 20/year for my mail proxy, gives me 200/mo that we never hit.

Not a spammer!

[Anonymous Coward]

Please keep in mind that Bennett has a legitimate purpose for his email patterns- he is trying to distribute proxy domains to people living in parts of the world where the internet is censored.

Re:Not a spammer!

[Anonymous Coward]

School computer labs?

Seriously. That's one of his markets.

did you think of...

[sithlord2]

- Implementing DKIM?
- Implementing SPF?
- Make sure the sender address doesn't bounce?
- Make sure you don't open thousands of connections to the receiving party for each recipient ? (in case of yahoo, hotmail, gmail, ...)
- The contents of the e-mail is not considered spam? (provide unsubscibe link, no big images included, etc...)


Setting up a mass-mail infrastructure is not to be taken lightly. There are lots of reasons why you could be listed as a spammer. That's why most companies outsource their their mass-mailing to 3rd parties like MailJet, MailChimp, SendGrid...

Re:did you think of...

[nabsltd]

That's why most companies outsource their their mass-mailing to 3rd parties like MailJet, MailChimp, SendGrid...

As an e-mail server operator, I'm glad they do, as it makes it easy to block all the spam from those companies at the server level.

I've been added to e-mail lists without my permission quite often because I had to provide an e-mail address for the company to send me a bill or other actual important e-mail. They pass the e-mail address on to these third party companies without any confirmation on my part. Then, when you do go through the unsubscribe process, those companies claim that it might take 4-5 days to remove you from the list, when in reality we know it should happen instantly. This is so that when you keep getting the spam for the next week or so, you might not report it, or maybe if you are truly stupid, you might buy some of the junk that is advertised.

In addition, once you have any dealing with a company like MailChimp, they can play the "previous business relationship" card when sending you "something you might be interested in". And, since none of the companies you list are confirmed opt-in (which requires that they send you a first e-mail when you supposedly subscribe, and unless you click a link or reply to that e-mail, you do not get added to the list), they can play fast and loose with claims that you signed up for something that you did not.

Re:Dude

[Anonymous Coward]

Parent calls him a spammer, gets +5 Informative.
I call him a spammer a ways upthread, get -1 Troll.

Re:You are a spammer

[niiler]

His behaviors are _similar_ to those of a spammer in number only. Having visited his site: http://www.peacefire.org/ it seems that he gets his email list from people subscribing to it on his site. If I understand it correctly, people who sign up for this list are looking for regular updates to proxies so that they can avoid censorship. As proxies are discovered by governments or certain companies , they are blacklisted, and new proxies must be created and sent out to the interested masses:

"Of course, employees of blocking software companies have gotten on this list as well, so they add our sites to their blocked-site database as soon as we mail them out, but in most places it takes 3-4 days for the blocked-site list to be updated. So the latest one that we mail out, should usually still work. "

Now it could be that there is a better way of doing this, but it seems to me that no matter how this game is played, constant updates to users should be the norm...

Now that I think of it, perhaps a Firefox extension could do the trick. Signed extensions can be updated automatically. The extension could have obfuscated URLs that are decrypted with something like this: https://addons.mozilla.org/en-US/firefox/addon/domcrypt/ and then wired in to automatically select an available proxy from the current batch. Not perfect by any stretch of the imagination, but it solves the "spam" problem. Also, it maybe easier for users and harder for censors? Crap... now I'm not going to get any work done...

Hotmail and Yahoo

[Anonymous Coward]

Seems to me the problem is people are still using these for e-mail accounts.

Re:Hotmail and Yahoo

[Megane]

Their very lameness and ubiquity is what makes them perfect for people living under oppressive regimes. When "everybody" uses a mail service, it becomes harder to block it without a lot of people noticing and getting pissed off. When they are so ubiquitous that even members of the regime use them, it's even better.

I don't understnd the animosity here

[Anonymous Coward]

Early on (before I quit reading) the OP said:

  It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body.

It seems to be treating his email as spam even when he sends one email to a single address.That isn't spam.

Re:I don't understnd the animosity here

[Anonymous Coward]

So, if I forward to YOU and YOU ONLY all of my "Cheap Viagra Cialis Horny Housewives Want You Tonight, Legitimate Business Venture from Nigerian Prince" emails, will you gladly take and read them because they're not spam?

My point is, just because it's sent to a single address doesn't mean it's not spam. Each glossy flyer you get in the mailbox is sent to only ONE address - the catch is, they send lots of them to one address each. It's still utter crap regardless.

Re:I don't understnd the animosity here

[coofercat]

But it is spam if you repeat that same process 10000 times (or however many). Just about all abuse detection works this way - it's a matter of counting how many times you do something per unit time (even if the thing you're doing is supposedly legitimate). Once you trip the threshold, you get banned for a set period of time. It's possible that once you trip the email threshold you're banned for weeks, months, years or permanently (unless a human revokes the ban). Heck, I do something similar on my website to get rid of the vast swathes of Chinese botnets that seem to want to tell me all about Ugg boots and the like.

It's entirely possible also that this guy could have completely successfully emailed the entire list of 7 proxies to his friend in a single email from his 'dodgy' domain, including the words "viagra" and "gold dust" and "nigeria" when the domain was first set up. He might even have been able to send the same list to 50 of his friends. However, when he sent it to the 51st, he tripped some automated checks, which presumably he failed, and so got banned.

It's possible he'd reduce his spamminess to these providers if he spent a week sending his weekly email. That is, if he's sending to 700,000 people, send to 100,000 people each day, and spend 5 seconds or so sleeping between sending each one. Even that's probably enough to trip the thresholds, but you get the idea.

Use DKIM

[pr0gr3sR]

Had a similar problem with Yahoo... Implemented domain keys and signed all my outbound mail and it fixed the problem.

Re:You are a spammer

[Anonymous Coward]

I know that these days it's usually too much to expect people to RTFA. So instead I tell you to RTFS.

If you had actually read and understood the summary, then you would realize that, if the story in the summary is true, this person's actions were very legitimate and not the actions of a spammer.

Have a good day sir. Don't let the door hit you on the way out.

Re:Dude

[Anonymous Coward]

Why not use Verizon's mail server?

If you behave like a spammer...

[Anonymous Coward]

Spammers rotate domains to avoid filters all the time. Legitimate senders of mail don't.

Bennett is behaving like a typical spammer, rather than like a legitimate user of the internet. Rather than changing his behaviour, he wants to whine about it. None of this is new behaviour, he always thinks that his goals are so noble that he can do whatever he likes and not be called on it - which dates back at least a decade, when he was promoting the idea of abusing insecure servers so that you could hide your activity by channelling your traffic through them.

Re: Dude

[Urza9814]

I've been on his list for around six years, and as far as I can tell, everything he says in the article is 100% accurate.

Also worth noting that he submits articles about these things to Slashdot quite regularly. I recall one a few months back where he was first considering this exact experiment. I'd go find it, but I'm posting from my phone.

Web page

[stabiesoft]

With 420K users, why would you not have a web page that gets updated with the same info. Users could check it at any time for the latest version. A "pull" instead of a "push" approach.

Re:Web page

[Anonymous Coward]

Because it will be blocked by the people trying to censor (i.e defeat the entire purpose).

Slashdot does really have dumb fsck's these days.

Re:Web page

[Ksevio]

Read the summary and you'll find out!

Re:Web page

[c_sd_m]

Because (1) the webpage would get blocked for the people who need to use proxies and (2) you don't want to give everyone the same proxies.

Re:Web page

[Anonymous Coward]

Precisely. The main peacefire.org site is categorized by Websense (and others) as a Proxy Avoidance site. Many places automatically block such sites.

In contrast, I can subscribe once when I do have access to his site, and continue to get emails.

Re:Web page

[Anonymous Coward]

They do have a web page. I will put money on the fact that many people that sign up for the email are blocked from accessing his sight as it is on the country's blacklist. While what you say is true, he is in a somewhat distinct position.

Re:Web page

[Anonymous Coward]

Yeah, I'm not sure why that isn't mentioned sooner. Also, he rants about the 'free market' but his recipients are free to use a different service (free or paid) or setup their own.

Given that he's pushing out proxy info, I somehow doubt any government is going to have great interest in him NOT being marked as spam. I believe proxies are good because I'm not a school system, jail, big corporation, and I believe in 10000% unfettered free speech. Few here believe in it - they love to look for exceptions and use the edge cases to support swaths of regulations. Not me, it is better to tolerte some bad speech, expression, data than to have it filtered.

Also of note is that once he identifies the offending domains, he can retailer his list to them. That's a pain but...

Optimal != Perfect

[Anonymous Coward]

The "Optimal solution" isn't "perfect".

There are always tradeoffs, and the power of the free market is that it is relatively effective at weighing different options.
It basically brute forces the answer to any question. It's messy, ugly, often inefficient, but it works.

It's op-in

[Anonymous Coward]

I wouldn't be surprised if some large companies' customer email lists have that many subscribers, all of whom EXPECT their incoming mail to be delivered without errors.

tell me more,

[Anonymous Coward]

so i can feed the whois results straight into iptables.
i note that this spammer has a psychopathic sense of entitlement to leech off other people's work.
how about a proper day's work, ideally something not involving networked computers ?

Re: Dude

[Urza9814]

FWIW, I'm on that list. And if I was using hotmail or Yahoo I would be PISSED about missing those messages. Been on it since highschool where I used them to bypass the school's web filters (occasionally teachers would even promote these sites because we literally couldn't do our work without them); today I still use them for testing and occasionally at work if, for example, I need a document from scribd (why that is blocked I'll never understand...)

Re: Dude

[Urza9814]

[Quote]Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.[/quote]

[Citation needed]

Email is Electronic Mail. You have large mailing lists like these with physical mail; you'd have to be an idiot to have thought something similar wouldn't be developed with email.

Re:Dude

[Rakarra]

I hate to use the if you were legit then you wouldn't need a proxy argument. However If he was using email the way most services want you to use it, he wouldn't have a problem.

Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.

Then how do you send a message to a large group of subscribers (let's ignore the spam angle for now and say these people want the updates) notifying them of site updates, special offers, alerts, or whatnot. I don't think it's enough to say "well they should just go to the site and check it when they want to." First, I don't want to call up every web site I might have signed up with every day. I just don't want to go through that hassle. I would end up not doing it. Email is perfect for me, I can scan it quickly for things that interest, delete it all when done... no hassle for either myself or the services. What would you suggest to replace this that would work better?

Re:Dude

[wendyg]

Bennett Haselton is no spammer. He's been involved in anti-censorship for nearly 20 years; he began in high school by investigating the block lists operated by the filtering software installed in many schools and libraries.

Not a spammer.

wg

Top secret :Google gmail has faulty BLACKLIST too!

[Anonymous Coward]

Top secret fact :Google gmail has faulty blacklist too!

I have has a website I have had for over 12 years at niftyspot.com

Its mostly ARM CPU assembler link info and has been for ages

Gmail blocks mail from its server from google group lists I subscribe to among friends! Some messages delayed for 5 hours, some sp*m blocked

why? because the ONLY domain I know not listed in the first 1000 results is my website!

Google has a vendetta against me... my domain!!. I used to be number one search result (www.niftyspot.com) on EVERY search engine I know, and various listings sites, for the phrase "niftyspot" and also for phrase "nifty spot"

Now I am BANNED and have been for months, but chinese google has me number one result. As do 4 other search engines. But google DELISTED MY SITE ENTIRELY FOR MYSTERIOUS REASONS!!!

Google has a blacklist. And it has NOTHING to do with anything logical. The domain has been a good citizen and compliant for over 12 years, and the emails sent are a handful a week to a handful of people.

TRY IT YOURSELF IN BING; Then try finding my site in google’s crappy search engine. Or try other search engines besides bing

My www.niftyspot.com is delisted as destination search ENTIRELY off google, but for years and today ALWAYS number **ONE** in other search engines (http://bing.com, http://yahoo.com, http://duckduckgo.com, http://www.baidu.com)

Oddly enough, I stopped being number one and was delisted on just Google and blocked in google gmail after a stranger possibly in India wanted to negotiate to but niftyspot.com from me. I don't accuse him, but I sold it to him this morning because I do not know how to fight google.

I am enemy number one to google

even islamic terrorist websites are listed, and asian drug distributers are on google, and competitors, but not www.niftyspot.com

check the other searach engines and chinese google if you doubt me.

All i say is 100% fact and true. I am and was number one everywhere , but BANNED and DELISTED off of googles faulty or evil technologies.

I am the only known domain delisted off google AND harrassed in gmail as well

So quit picking on Yahoo and Hotmail. Google does evil too!

You may not be a spammer but... your 'customers'?

[Anonymous Coward]

You are operating an open proxy. Spammers will use it to obfuscate their landing pages. I'll bet this is the 99% use case for your service, regardless of your goals or ideology. Yahoo and hotmail filters are most likely doing the right thing...

Re:Dude

[mcl630]

He has even sued spammers.

His emails simplify the blacklister's job

[Goldenhawk]

Ironic. Almost all blacklist providers keep proxy sites on their default "bad sites" list. Were I running URLBlacklist or similar, I would simply sign up for his email service and make a point of adding every web domain spotted in his emails. Almost an instant kill for the blacklist provider; by the time email recipients can act on the information, it's already been blacklisted.

Re:You are a spammer

[kimvette]

Want to know who sends mass email in batches like that?

Apple, Microsoft, NewEgg, Amazon, Zappos (an amazon company), Woot (another Amazon company), ZD Net, and so on.

Not every large volume emailer is a spammer.

Re:Top secret :Google gmail has faulty BLACKLIST t

[Anonymous Coward]

Put on your big girl pants and sue.

Re:Dude

[PlusFiveTroll]

Why not use Verizon's mail server?

Most likely, because it sucks great big donkey balls. Now, that said I don't use version so I don't know for sure. What I do know from working for one of the top 10 ISPs (size wise) in the country is, most big ISP mail servers suck. Send any attachments of any size and they're apt to be blocked, get stuck in the queue, or just go in to the blackhole. Other issues are that the ISP might flag or block as your messages as spam because you want to send 200 messages on friday. And you have to put up with their filtering and blocking choices, that may not meet your needs.

SmartScreen most likely

[Anonymous Coward]

I have run into issues with SmartScreen Filters before. They operate on a threshold of "suspiciousness." If the domain sending the mail is not verified with them, thats a point off, if you mention selling or prices, thats a point off. If you have certain settings set on your mail server, those are points off. The IT department using SmartScreen can then set what they want their threshold to be.

I have had legitimate reciept emails from online retailers be bounced back before, because they mentioned that i got a GREAT DEAL and they had CHEAP PRICES while coming from an untrusted domain. Quite irritating when set up too strictly.

Re:Dude

[Anonymous Coward]

I think your scenario is exactly what RSS is for, right?

Except the summary is probably wrong.

[Medievalist]

I wouldn't be suprised if it's just Bayes. The majority of messages with links leading to those registrars' domains were categorized by human readers as spam, so automated bayesian analysis picked it up.

As long as you have Internet governance that is primarily concerned with eliminating certain forms of political speech (Great FireWall of [insert name of nation here]) rather than ensuring a free market and fair trade, you're going to have this problem. The same low-rent registrars are going to be used for criminal spam as for legit filter avoidance technologies, because they are looking for the same service (temporary domain names at minimum price).

Re:Except the summary is probably wrong.

[AdamWill]

"I wouldn't be suprised if it's just Bayes. The majority of messages with links leading to those registrars' domains were categorized by human readers as spam, so automated bayesian analysis picked it up."

Try reading the longer summary.

"It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold."

Re:Except the summary is probably wrong.

[Anonymous Coward]

it is all my fault, when i get email in a language i don't understand i mark it spam, clearly other people are doing the same thing for their languages. i realize he said 0.1% spam rate, but in foreign countries one offense is enough as they are building out the way it was done here.

Re:Except the summary is probably wrong.

[SomePgmr]

That's not what he was suggesting. Reread the post you replied to carefully.

Re:Except the summary is probably wrong.

[Anonymous Coward]

I wouldn't be suprised if it's just Bayes. The majority of messages with links leading to those registrars' domains were categorized by human readers as spam, so automated bayesian analysis picked it up.

I'll admit my own laziness here, I read through the first 30 pages and skimmed the rest. But here are a few questions:

- Sending the same email from seven different domains and mail servers? If so, that probably looks bad.

- Some of these domains recently came off the spamhaus DBL? That probably looks bad, even if you're off it now.

- The emails contain lists of proxy service urls? That could look bad. Maybe you can obfuscate these or reference them in another way.

- Do we have PTR records for the sending servers, SPF entries, etc? If not, that might help.

Yes, it's possible that Microsoft and Yahoo share or use a similar backend antispam technology. No, I would not jump to conspiracy theories. Billions of people send mail in to these services every day, including mail servers not operated by large institutions.

It's likely that this situation is just right on the edge of what looks like spam.

Re:Except the summary is probably wrong.

[Anonymous Coward]

He registers new domains every week or something like that. These were new domains not listed on any blacklist.

Re:Except the summary is probably wrong.

[Anonymous Coward]

I know for a fact that Spamhaus and other blacklisting companies have some sort of "headhunter" service they use which are humans who "verify" message content. (In addition to the other mechanisms for landing on their list). You can't get any official information about it, but I have on a couple occasions had a Spamhaus service Rep accidentally let slip that something ended up on their list because of a single report from a "Trusted Human Source". Most of the time people end up on their list it's because of reverse DNS lookups not being properly in place prior to launching the mail server, or reports from their partners.

Services such as yahoo and hotmail use some automatic heuristic filtering. If you create a new mail server and start flooding messages, legit or not you will almost always trigger these filters. This is because you're matching the behavior of a spammer who is shifting his server from source to source in an attempt to stay one step ahead of the blacklist filters, and once this activity is spotted it'll fingerprint the messages and start trying to catch them from any source. The only way to prevent tripping such filters is to start the mailings at a slower, low-volume pace and only start increasing the outgoing messages in stages after it's been around long enough to be fingerprinted as a "normal" server.

But anybody who runs any sizeable mail system already knows yahoo and hotmail share something on the "back end". Start seeing your messages delaying or bouncing from one and the other will soon follow, even when other services like google aren't doing any blocking at all. Although I am a bit surprised to see the submitter was getting that much backscatter from them, usually they're better at blackholing 'spam' than that.

Re:Except the summary is probably wrong.

[Anonymous Coward]

He registers new domains every week or something like that.

Yeah, and that doesn't resemble criminal spammer behavior at all,, does it? That's exactly the kind of thing a good spam block is going to be looking for, because 99.99% of the people doing that are criminal spammers.

2 days

[asmkm22]

How in the world did it take you two days to figure out Spamhaus was blocking your stuff?

Save yourself some time down the road and just go to mxtoolbox.com. Enter the domain name it and can check all kinds of things for you. If a list is blocking it, you can get details as to why. In the past I've seen various reasons, but most are pretty detailed and provide quick access to the forms you need to get removed.

As for your idea of a secret shared blacklist between hotmail and yahoo, it sounds more like it's just a dynamic content filter that pulls data from spam lists to prevent the propagation of bad links. I doubt it's 100% realtime, so after getting Spamhaus to unlist you, yahoo and hotmail need to wait for their next content filter update to see the changes.

Anyway, I just thought it was a little weird that it took you 2 days to even get to step two of your problem.

Re:Dude

[crypticedge]

Because they won't let me relay my own domain through it, it sucks big fat donkey balls and it's subject to far tighter restrictions than what I use to send outbound.

I run all my outbound mail through a good spam filter (that forces all outbound to be scanned, regardless if it makes it through the mail server) and have a fairly open file size limitation (20MB, compared to last I tested Verizons 5 MB)

My outbound proxy has no size limitations, my outbound proxy handles all blacklist issues, and my outbound proxy handles all RFC compliance issues (it's dyn, they aren't a bit player)

In the end, filtering it through dyn just works, trying to send through verizon ends in frustration.

Re: Independent verification of verified/double op

[Urza9814]

Been on the list since late 2005 and I never delete an email, so I can confirm.

You subscribe at his website and you get a confirmation request email. You confirm, and it sends another message confirming that you've been added. The content is legitimate, the volume is fairly low, every email gives two unsubscribe methods in the first paragraph of the message (click a link or reply with unsubscribe) and all messages are plain text.

Dear Moron:

[Anonymous Coward]

You are using SingleHop which is a huge source of SPAM. They don't care and they are fucking worthless to work with if you are getting spammed like crazy from their here-one-minute-gone-the-next virtual "Servers"

Here's the real issue

[CanHasDIY]

Preface: I am not taking the side of the spammer here. You keep that shit out of my inbox, fucker.

That said, the real issue is the censorship of people's messages without their knowledge or consent. Granted, nobody wants to have to filter through millions of V1@gr@ ads just to read their mail, but on the same note, nobody wants someone else going through their mail and arbitrarily deciding what will and will not be delivered. I understand the purpose of the spam filter, and am glad it's there - but a secret spam filter? Not cool - as far as I know, those who administer said filter may decide, 'you know what? I vehemently disagree with the political philosophy of Grassfire/MoveOn/other political group, let's add them to the secret blacklist.'

Real world analogy - think of Yahoo/Hotmail as UPS - just because it's a private company doesn't mean they have the right to go through the shit you ship (or do they? I, personally, don't ship a lot of stuff...).

Nothing new

[wukka]

For years I've noticed that Yahoo email will bounce emails I try to send which contain certain URLs. Legit websites, no spam, no malware. Next year Ixquick.com/StartPage.com are supposed to offer StartMail.com as a webmail alternative. I am anxious to try it out.

Re: All I can say is

[Urza9814]

Double confirmation of opt-in, two methods to unsubscribe at the top of every message, legitimate content, low volume, and no tracking features (all messages are plain text).

I've been a subscriber since 2005 (just looked that up for a different comment; I never delete email) and this list is as far from spam as you can get. Shit, I've replied to those messages and gotten a response from him personally (Still have those too.)

If you still think this is spam, then apparently by your definition I have literally never received an email that wasn't. He provides a great service and runs a clean list. I'm a huge fan if you haven't noticed...

In the Instant Messengers too

[bobmajdakjr]

I noticed this behaviour in the instant messengers too last week. Snickers aside, I was sending a list of URLs to someone (one at a time during a conversation) and the URLs that were on the imagefap domain never arrived to their destination.

You are missing a fundamental point here

[larwe]

(If you want a tl;dr version of my post: Don't whine that a free service run by someone else isn't guaranteed to meet your business objectives). Actually a couple of points. The first one is, bulk email is bulk email and it really doesn't matter how "good" an actor you are, whether you are opt-in, etc. You are not sending individual hand-calligraphed invitations to a royal wedding, or any other kind of lovingly crafted correspondence; you are sending bulk flyers which most people won't read. It is possible, maybe, to classify CONTENT algorithmically. It is impossible to classify intent (CE vs UCE). The second one is, email is NOT a guaranteed service. Broken cable. Bad MX record. Squirrelly hard drive in a mail relay somewhere and your email is gone. Why are you complaining about one particular failure here? It seems that people who actually want to have a certain level of service guarantee (in this case, avoiding a legitimate antispam measure - regardless of how effective it is) - need to pay the piper. You resent this, and believe you should get an above-baseline level of service for free. Can you hear my boiling tears raining down on the volcanic desert of the Internet? The third one is: you paid nothing to transport the email, you only paid to squeeze it out the urethra of your computer into the public internet - from then on all the transport is ON SOMEONE ELSE'S DIME. If they choose not to forward it, that's your bad luck. You have no recourse. Stop whining. I have zero sympathy for any problems experienced by anyone distributing bulk email for any purpose. If you told me your correspondence with Aunt Franny was being swallowed by a demon at Gmail, or something of the kind, I'd be more sympathetic, because that's a realer problem (though my second and third points above still apply to the Aunt Franny case). It is specious, maybe arrogant, to pontificate that consumers "lose" or the "free market fails" when bulk email doesn't reach its (mostly uncaring, if not downright resentful) endpoints reliably. In summary: Grrrr.

Re:Dude

[sexconker]

Its not Spam if you opt in. Spam is unsolicited. For this you have to request. Now is it possible the guy is bull shitting that part sure, however if we accept that the articles are bull why bother to read them?

They're spam because people are clicking the "THIS IS SPAM" button.
They're clicking that button because they don't want the fucking emails and he keeps sending them.

Re: Dude

[Anonymous Coward]

"...you'd have to be an idiot..."
That's the reason you're having to type this reply.

WTF upvotes for baseless aspersions

[Onymous Coward]

This man is running a list (among many other activities) supporting individuals' rights to information freedom under repressive governments and you're implying he's either incompetent or, worse, underhanded?

This is inane.

And how much effort is required to fucking test this?

Thank you. A confirmation message has been sent to address redacted.
YOU MUST REPLY TO THAT MESSAGE, in order to be subscribed so that we can notify you when new Circumventors are set up. Almost 50% of our subscribers forget to reply, and as a result, do not get added to the list. If you do not reply to that message, then your address will not get added!

What causes rudy_wayne and those who upvoted his post to like the idea that Bennett Haselton is spamming and lying about it? And is their credulity what keeps them from performing such an easy test? Whatever the cause of the inanity, how can we discourage this problem in the future?

Re:WTF upvotes for baseless aspersions

[Stan92057]

Follow the can spam laws and ya have nothing to worry about.
And we are under no obligation to agree with his methods or his beliefs.

Re:WTF upvotes for baseless aspersions

[Anonymous Coward]

This test proves that if you reply to the opt in email you are added. This does not prove the list doesn't contain email accounts added via other means.

Re:WTF upvotes for baseless aspersions

[Onymous Coward]

You are correct. Thank you for your precision.

The primary method of subscription was presumably the thing in question, and so was the thing tested.

More detail: this test proves that the subscribe form on the peacefire.org site does require opt-in. I assume this is the primary, possibly only publicly accessible means by which persons can attempt to add addresses to the list. Your precise reckoning does highlight the possibility that there may be other means and that they may have been subverted. I believe this is unlikely, and I think it's likely Mr. Haselton has investigated the possibility.

Perhaps Mr. Haselton will do something like a binary (or 6-ary) search for which addresses may be reporting to the URIBL and trace how those addresses were added, should future domains listings happen.

Re:WTF upvotes for baseless aspersions

[theArtificial]

Follow the can spam laws and ya have nothing to worry about.

An associate of mine created an email delivery system which many companies use and one of their larger clients sends out about 25million messages a day to their opt-in lists (it's a newsletter). There are users who opt-in and then mark the message as spam (forgetting they signed up, or perhaps even mis-clicked). This does happen. What then? As a company you're following the rules...

Re:WTF upvotes for baseless aspersions

[Stan92057]

To prove they are sending opted in emails they have physical proof. Or they should have it. The email body will be evidence of a real opt out email address and the opted in emails themselves. its a shame they got to prove it but hay thank spammers its there fault

Re:WTF upvotes for baseless aspersions

[Anonymous Coward]

>inanity

You keep using this word. It does not mean what I think you think it means.

Re:WTF upvotes for baseless aspersions

[Onymous Coward]

>inanity

You keep using this word. It does not mean what I think you think it means.

Correct. Rather, it means what I actually think it means.

Yes there is an algorithm that ends SPAM

[Anonymous Coward]

"I don't think there's an easy algorithmic solution"
yes there is a solution to SPAM. Signed e-mail!

SPAM is between the sender and the recipient. Soem people might actually wait for the male enhancement offer from Canada. NO, I repeat NO ISP can decide if a message is SPAM to me or not.

But make it mandatory to cryptographically sign a message, I can filter on the original sender, which can't be falsified, unless the keys are stolen, and then even they can be revoked. In addition counter sign the keys for people and institutions you know (those of your friends). That way you can accept messages from people that you might not know personally but someone you know, knows them.

Now every single recipient can decide if:
* she continues to accept any mail from a sender that they don't want the mail anymore
* She continues to accept any unsigned mail
* she continues to accept any mail who's signature is far enough away from their circle of trust (the people who's keys they signed)

Remember that does not require any money changing hands for keys, as anybody can create their self signed keys and have them signed by their peers. However, some trust organizations can verify identities on a commercial basis and that is fine with me.

You distribute a list of and run open proxies

[BitZtream]

Of course you're on the list. You're fucking retarded to not understand why.

Guess what, I have the same sort of list myself.

Free Market

[Anonymous Coward]

I think there's some confusion between idea of free market and freedom. What is happening is exactly free market economics.

Hotmail gets revenue from users.
Hotmail can block whatever it wants.
If your users don't like it, they should leave Hotmail.
Hotmail's revenue is harmed.

If the harm from losing revenue is greater than the benefit of blocking domains, Hotmail will stop doing it.

The invisible hand at work!

The poster's problem is that the benefit of blocking his domain for Hotmail is probably greater than the cost. More users will flock away if their spam filter is lessened even slightly than will walk away because they aren't getting this distribution list's email.

Re:You are a spammer

[mrbene]

Now it could be that there is a better way of doing this, but it seems to me that no matter how this game is played, constant updates to users should be the norm...

Now that I think of it, perhaps a Firefox extension could do the trick. Signed extensions can be updated automatically. The extension could have obfuscated URLs that are decrypted with something like this: https://addons.mozilla.org/en-US/firefox/addon/domcrypt/ and then wired in to automatically select an available proxy from the current batch. Not perfect by any stretch of the imagination, but it solves the "spam" problem. Also, it maybe easier for users and harder for censors? Crap... now I'm not going to get any work done...

There are multiple benefits of email delivery that aren't present in the Firefox Addon model:

• It's push notification - the updates only go out once. Firefox Addons are a pull - a server has to handle all the clients requesting updates (and sending the appropriate subset!).
• It's more difficult for the people that this list is supposed to enable to bypassing of to automate the immediate blocking of the new set of domains.
• It natively enables two-way communication at a human level.

If I were the OP, I'd consider moving to an encrypted blog method of delivery (still via email), but doing it while being very conscious of the level of technical know-how of the target recipients.

Stupid Rant

[Anonymous Coward]

I was enjoying this until I ran into the rant about the free market. One controlling entity creates a problem, so you want a new controlling entity to fix the problem. Oh sure. That will end well.

It's spam, of course.

[Animats]

If you want to distribute a "newsletter" to real subscribers, set up an RSS feed, or a Twitter feed for little stuff. Readers can then subscribe if they want, and they can unsubscribe without having to beg to be taken off the list.

This clown sent 420,000 emails. Of course he's a spammer.

Re:It's spam, of course.

[Anonymous Coward]

Then so is Scott Adams with his Dogbert newsletter. According to Wiki, he used to send out 500,000+ mails at times.

The bastard!

Re:It's spam, of course.

[Anonymous Coward]

Wait, so you're saying that the quantity of emails if the differentiating factor between email and spam? Because I always operated under the assumption that spam was 1) unsolicited and 2) shotgunned. This was a solicited communication (affirmative opt-in registration) and targeted (sent to those who signed up AND replied to the initial opt-in email). Sounds pretty non-spammy to me. If we are going just based off of volume, then Amazon, Google, Microsoft, Ebay, [insert other commercial interest] are all orders of magnitude larger spammers than this guy (I get loads of "oh, you looked at X so maybe you would like Y" offers from Amazon and I'd guess they have more than 420,000 unique customers). I don't care if someone is sending email, snail mail, or properly addressed physical containers of Hormel canned meat... if it is requested, it's not spam!

Re:It's spam, of course.

[Anonymous Coward]

Sending 420,000 emails to opted in subscribers does not make him a spammer.

If you follow the Can Spam Laws

[Stan92057]

If you follow the Can Spam Laws then you shouldn't have any problems.

Re:If you follow the Can Spam Laws

[BitZtream]

Its impossible for him his lists content to follow CAN-SPAM laws ... he lists open proxies so people can not be censored ... do you think the spammers don't use them to get around blocks as well?

He facilitates spamming (not intentionally, but does none the less) so he's being blocked as such.

Re:If you follow the Can Spam Laws

[Stan92057]

He has Zero authority to skirt and break laws for what he is trying to do period end of story.

Re:Dude

[Anonymous Coward]

Do you people not understand the concept of an email newsletter? For instance, I am subscribed to NASA Tech Briefs 's email newsletter, which purports to have an audience of over 77,000. Being a newsletter, of course those emails all have "the same web address in them" -- they're the same bloody content. This has been going on for decades (they've been a big thing since home users who never heard of usenet started getting internet access...), and as long as it ONLY GOES TO PEOPLE WHO VOLUNTARILY SUBSCRIBED, it's NOT MOTHERFUCKING SPAM! If your spam filter flags this, your spam filter is broken. Spam= UNSOLICITED bulk email, not all bulk email.

Obviously a lot more goes into it than a link and a filter doesn't know subscribed or not. Filters pass legit bulk and use a number of metrics to filter out the trash. OP doesn't have enough trusted qualities in his e-mail to get through the filters he's complaining about.

What I've figured out with Hotmail

[Predius]

I admin a few mail servers. I've run into trouble with Hotmail. Here's what I've learned:

First, there are a ton of url / domain blacklists available out there, no need to suspect a conspiracy within Hotmail and Yahoo. That said, I know they also maintain in house IP and domain based blacklists, along with full url blacklists. No idea if they share but I actually doubt it as that potentially weakens their competitiveness with the other email providers. Hotmail also uses a paid whitelist service too via an 'independent third party', although certain blacklist levels can even override that paid service.

Second, Hotmail splits mail up into three categories now, legit mail and spam which we're all familiar with, plus what they've dubbed 'graymail'. In short, graymail is legit opt-in mail that the user just never bothers to read. Thats right, your quadruple opt in email can be treated like spam by Hotmail if your users never bother to look at it. Generate too much, you're treated as a spammer. Can SPAM compliance or not, they don't care.

Third, if you manage to get on Hotmail's IP blacklist, there is no recourse that I can find. Their policy is tough expletive, move your mail server to a new IP or go away.

As far as the complaint level stats you can view through their Postmaster tools, they only show two of the three stats their system works on at the IP level, the complaint rate (people flagging mail, I *think* VIRI mail also counts in this column) and filter hits percentage, although this one is obfuscated to try and defeat spammers trying to tune around it. The missing stat is IP reputation, based on those first two stats over time along with external and internal RBL data. So when you DO setup on a new IP, it'll take awhile for their system to actually accept mail from you. You can subscribe to a feedback loop program, but that shows another issue with Hotmail:

They have no concept of traditional mail relays, they expect all individuals to be sending via Hotmail, Gmail, Yahoo, etc. All other port 25 traffic destined to them must be from commercial list serves. At least that's the impression I've gotten from going through all their postmaster policies and dealing with their ticket system. If you try to explain the idea of an ISP relay for use by people within that IP block, they just ignore it and resume pestering about opt-out notices, etc.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Screw that

[pavon]

And should the HFH and ACLU and all the other newsletters I subscribe to be blocked as spam as well? They send far more than 400k emails a month. Email is more convenient than RSS or worse Twitter, and is newsletters are a perfectly legitimate use of the medium.

Re:Screw that

[BitZtream]

And yet they seem to be able to do all the right things to not get blocked.

Perhaps you should consider that before you say anything else.

who cares about blacklists?

[someones]

they have a very long history of false positives and actually noone should be using them today anyway.

Re:Seriously...

[RKThoadan]

I've never wanted mod points more than I do right now to mod you down as well as the many other idiots like you who have made similar comments.

Read his entire article and you'll understand why he's doing that. I'm not going to tell you because your brain obviously needs the exercise. If you're too lazy to read then don't bother commenting. I'd also recommend actually reading the rest of the comments for the many people here who have subscribed to his list for years and verified that he's 100% legit.

There is nothing commercial or unsolicited about his service and it's a vital and important tool if value minor things like freedom. A single one of his e-mails is probably more valuable to the world than your entire pathetic existence.

Perform listwashing, just like spammers do

[Khopesh]

Ironically enough, you can isolate the "moles" by listwashing, just like spammers do for spam traps.

You've already started the process: you know that three sevenths of your subscriber base is probably safe. In your next run, make sure each of the remaining four groups is subdivided again. Each time you find a group that isn't a mole, you've reduced the potential mole list. Eventually, you'll have just a few accounts and you can silently drop them from your service (or confront them, your call).

There was also an earlier comment on spammer abuse of your proxies that I'd like to expand upon. While it asks you about proxying port 25, there's also the potential for abuse with respect to port 80/443: 419ers are increasing their use of proxies to hide their identity from free webmail providers so they can get free passes on sending spam. If you're better at cracking down on them (by e.g. blocking access to yahoo and hotmail on your proxies), you'll probably have better luck overall.

Maybe you can combine the above two ideas: groups of subscribers known to contribute to getting blocked will get domains whose proxies can't use freemail.

Wrong.

[kjs3]

> Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes.

To quote Pauli, this is not even wrong. The central fallacy to this entire anti-capitalist rant is that there's some nearly perfect solution to spam that the "market" participants are conspiring to deprive the consumer of. This contention is, not to put too fine a point on it, as deliberately dishonest as similar claims about running cars on water or perpetual motion machines. Spam is an arms race, not a problem with a "solution" that we've just been too lazy to find. Once you dispense with that fallacious premise, this entire screed can be summarized as "I'm butthurt because Spamhaus/Yahoo/Hotmail blocked my spammy-but-not-spam-because-I-said-so emails and they won't take my call" all wrapped up in a "won't someone think of the children...err...dissidents!" bow.

Email account providers have as many automated, heuristic-based blocking techniques as blacklist based. Have you considered that you might have tripped one? Like...a domain that was registered less than a week ago, first mailing we got from them was a carpet-bomb, content we've previously spotted and identified as spam? I mean, it's a lot less sexy than claiming there's a villainous corporate cabal in the back room twirling their mustaches as they condemn some hapless dissident to a life of Internet ignorance, but it is possible.

Re:Wrong.

[jibjibjib]

RTFS. He's not claiming that there's an almost perfect spam filter being suppressed by a conspiracy.

He's making the very plausible claim that spam filters naturally err on the side of false positives, to the detriment of the users, because false positives are a less visible problem than false negatives.

Re:Wrong.

[kjs3]

I did read the article, thanks. He's claiming that there's a solution to that problem that isn't being pursued. That's false. Visibility is a red herring, which you nicely went after.

Give up on Hotmail

[Anonymous Coward]

You're probably blocked by ReturnPath, which is used by both Hotmail and Yahoo. You won't get any help from Hotmail, at all. I work for a very large web hosting company and I email Hotmail about half a dozen times a day for delists and other info. They delist less than 1% of the time, the other 99% I get a pre-defined form letter basically saying they won't do anything about the issue. Any time you email Hotmail support, if something isn't fixed in THREE replies, the tickets auto-close and they will no longer reply. I run into this almost every week. I gave up getting help from Hotmail because not only are they incompetent, they purposefully tell their support techs to do as little as possible because if they have to escalate an issue, it costs them more money to resolve, so it's cheaper for them to do nothing at all than to do something.

Re:Dude

[Anonymous Coward]

So they need to unsubscribe properly. If you subscribe to a list then use the "Spam" button to "unsubscribe" then you are a jackass who needs to stop using the internet.

Re:Dude

[BitZtream]

The newsletter isn't the issue.

The content of the newsletter is.

He sends out a list of open proxies which can be used for spamming, THAT is why he is blocked.

Guess what? I subscribe to his 'newsletter' just so I can get his list of domains so I can block them myself.

Re: Dude

[BitZtream]

So you fail to understand why running open proxies gets you black listed?

Hell they don't have to have a 'secret domain blacklist' he freaking mails them the domains to ban.

Re:Dude

[BitZtream]

Not a spammer != Not facilitating spammers.

What do you think spammers do with a nice list of open proxies? THEY SPAM FROM THEM. He's distributing a list of spam producing sites and you're shocked that the list gets blocked?

What he said!

[Anonymous Coward]

And similar situation with Barracuda, as well.

There is a reason the big senders pay

[Anonymous Coward]

They get advice on all of the latest spam prevention techniques and how to act as a responsible mailer to avoid them; some email service providers also provide sample delivery statistics for individual email campaigns or newsletters, provide monitoring for your IP addresses and domains... The IP address serving your e-mail list is in pretty good shape (see senderscore.org), so your problem is elsewhere. It might have something to do with the newness of your domains, or perhaps there's something their filtering software doesn't appreciate about the site content...

There's not really a pay-to-play scenario going on here. More like the free mail providers have gone from being too spam-friendly to being really spam-paranoid.

Math

[Cow Jones]

I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

So somebody who wants to block all the proxies would have to subscribe several times in order to get the full list (it's not like multiple subscriptions would be noticed on a list with 420k recipients). I was wondering how effective this method was. Here are my results, in case anybody else was wondering:

With 20 subscribed addresses, the chance of getting the full list is 70%.
With 30 subscribed addresses, the chance of getting the full list is 93%.
With 40 subscribed addresses, the chance of getting the full list is 98.5%.
With 50 subscribed addresses, the chance of getting the full list is 99.7%.
With 100 subscribed addresses, the chance of getting the full list is 99.9999%.

Seems like this method of evading the censors is only effective if they're not smart enough to write a couple of simple scripts.

CJ

Re: Dude

[RR]

I need a document from scribd (why that is blocked I'll never understand...)

The McAfee block thing tells me that Scribd is a piracy website. Scribd hosts user-uploaded documents, and some of those documents are copyrighted by various companies.

Half of these people

[future assassin]

ragging on the article OP probably haven't got a clue that people actually use other methods than Twitter/Facebook to update users about their service. Why is everything have to be in the open social communication. If someone was tracking his twitter it would take a few seconds to block all the proxy domains.

Re: Dude

[Urza9814]

These aren't regular proxy servers. They're web proxies. Just a regular web server running one of the standard web proxy scripts -- used to use cgiproxy, then phpproxy, now it looks like he's switched to Glype proxy. It's just a generic web server running a generic PHP script -- I fail to see how that is cause for a ban.

Long-Lime Anti-Blocklist Crusader Still At It

[thenick58]

Good grief. Is Bennet Haselton still at it? I first crossed swords with him over a decade ago when I was Executive Director of Mail Abuse Prevention Systems, the famous or infamous, you pick, maintainers of the Realtime Blackhole List. Haselton should stick to what he knows best, and that is blocking of *websites*. I see his knowledge of blocking technology by email service providers is as dismal as ever. Hotmail and Yahoo have every right to block email they perceive to be spam. If they did not do so, their servers would crash under the barrage of email arriving every second. They spend countless expensive CPU cycles just *blocking* the spam from their networks. Is the system perfect? No. That is why both organizations have staff to deal with the "false positives" -- another needless expense for which you can thank the spammers. I know the anti-spam staff at both Hotmail and Yahoo. Members of their staff spend their entire days reviewing and responding to complaints about false positives, as well as tweaking the anti-spam filters. Forgive me if I don't shed any tears if the staff doesn't respond to Mr. Haselton's demands just because he stamps his feet. They're dancing as fast as they can. And you know what? Both Yahoo and Hotmail are *free* services! Imagine that! Do you know what else? There are many, many mailing list owners who will not accept subscribers with Hotmail or Yahoo email addresses *because* of deliverability problems at these two services. Both services are very well known by savvy mailing list managers for delivery problems. An entire industry of deliverability consultants has emerged to deal with email delivery problems at Hotmail and Yahoo. No, resolving email delivery problems at Hotmail and Yahoo is not for the faint of heart. I also see that Mr. Haselton has not lost his fondness of conspiracy theories. I seriously doubt that Yahoo and Hotmail are sharing their blocklists. They are competitors, after all. And suppose they did share information about their blocklists. What of it? It is entirely within their prerogative to do so. I would even say it represents efficiency for the two organizations to share their blocklists. But I'm reasonably confident that they don't. I believe Mr. Haselton has a fundamental problem with blocklists, period. I believe he has taken his philosophy at Peacefire about blocking web sites and is naively attempting to apply it to email. I base this belief on email exchanges and conversations I have had with Mr. Haselton. He does good work with Peacefire. I think he should stick to doing what he knows best and stop his crusade against blocklists. Until spam is eliminated -- which will never happen -- they are here to stay. And that is a Good Thing because email would be unusable otherwise. Nick Nicholas

IP Blocking

[Anonymous Coward]

I had a clients IP address blocked by MSN. We worked through all the SpamHaus etc and even the ISP requested that MSN remove the block, all the rest cleared their blocks. They refused and would not provide any details as to what the problem was. The site was a legitamate mail order company of some 20 years which sent no spam or large email blocks and only sent an email to confirm order receipt. We even showed them that we had a site limiter for restriction of sending emails above 200 per day. The site was SSH and complied to all of MSN requirements but they still refused to tell us what we had to still clear to undo the restrictions. Customer care just bounced the emails saying that they could not comment.

How MSN have the authority to block sending emails from a private domain is surely against the law. There is of course no appeal process so they are a law unto themselves.

In the end we had to migrate the whole server to a new IP address at our cost.

How do you say unsubscribe in Turkish?

[tepples]

We're talking about properly configured mail servers, no open relays, no backscatter, appropriate DNS, with opt-in recipients only and working, simple unsub options right in every email.

In how many languages are these unsubscribe options presented? I can't find the unsubscribe in a lot of Turkish mailing lists that I have ended up signed up to.

Spammers have access to bigger machines

[tepples]

Hashcash is interesting here, in that the CPU time is mostly spent by clients

So how do you allow legitimate mail sent from pocket-sized, battery-powered mobile devices without allowing mail from spammers who have access to bots running on compromised always-on PCs capable of running hashcash on beefy GPUs? I was under the impression that Bitcoin mining was a form of hashcash, and GPU-accelerated Bitcoin mining trojans were spotted over a year ago.

How do you say "remove me" in Turkish?

[tepples]

Seems pretty easy to me...

If you speak English. I have ended up subscribed to plenty of mailing lists in Turkish, and I can't read Turkish to find their unsubscribe processes. Besides, some spammers have long been known to see a "reply with the word 'unsubscribe' in the subject" as a request to sign up for all the spammer's other lists.

RSS with authentication

[tepples]

How many RSS readers support authenticated feeds? And how would a site requiring authentication let a web-based RSS reader log in to retrieve the feed without letting the RSS reader impersonate the user in other ways?

Address != person

[tepples]

as long as it ONLY GOES TO PEOPLE WHO VOLUNTARILY SUBSCRIBED

Addresses that voluntarily subscribe != people who voluntarily subscribe. Say someone cancels his Hotmail account, and later, someone else registers the same name. Who subscribed, and who continues to receive mail?

Hacky workaround

[halcyon1234]

1) Before sending out the mass email, send a test email to your own gmail, yahoo and hotmail account. See what bounces back.
2) Either drop that domain and register a new one OR
3) Base64 encode the URL in the email, with the provisio "Run this through base64decoder.com to get the address"

Re:Dude

[Anonymous Coward]

Can't believe this kid has the audacity to complain about it on Slashdot with a wall of text to hide the fact that he's a fucking spammer.

He probably wasn't expecting replies from fucking jackasses who can't read (like yourself, for example). That's the mistake, because a lot of people are idiots. I know you have reading comprehension problems, so I'll again point out that I'm referring to you.

Second DKIM, SPF, RFC-compliant

[nullchar]

Using SPF is a bare-minimum for even a self-run SMTP server that only has a handful of users with no mailing lists. Anything larger needs DKIM.

One other thing Bennett can try is to have a pipeline of registered domains ready to be used. Each domain in each URL is scanned by spam filter, and young domains (via registration age in WHOIS or the daily registry-published zone-file which yahoo/hotmail/gmail all have) are more "spammy" than older domains, simply because spammers do the same thing here - register new domains and mass-email them out as URLs.

Re:Dude

[Anonymous Coward]

wow. this may be the single most ignorant comment I've ever seen on a technical site. I would question if it was a troll, but somehow he's getting upvoted?

pro tip: SMTP expects address formatted as a list -- that is, an email from one person to one person is actually a degenerate case. and yes, there was email before SMTP -- except that was largely listserv-based, which is even more explicitly to-many.

Advertisements

[NewYork]

They call it spam. I call it advertisements.

Re:Seriously...

[PortHaven]

Well, gee, than MAYBE the summary should post clearly what he is doing. Instead of saying

"Well, I am not doing anything illegal and really really it's not spam"

There is NOT a single thing in that WHOLE post that says he's doing this for freedom. Nearly every spammer claims their list is double opt-in. So how the !@#$% is anyone to know. There is one link to Circumventor List - sorry, that's blocked by work. So it gave no info.

Don't frickin flame me....flame the idiots who post a slashdot entry without giving an iota of background info.

Re:Dude

[Anonymous Coward]

The guy is demanding that Hotmail and Yahoo serves as his free bulk mailer. This isn't like the old days with the post office making millions with junk mail. Fuck him. Why should they risk getting black listed themselves? He can buy a service for what he needs.