Slashdot Mirror

An anonymous reader writes: Spamhaus, an international non-profit organization that hunts down spammers, is accusing Verizon of indifference and facilitation of cybercrime because it failed for the past six months to take down stolen IP routes hosted on its network from where spam emails originated. Spamhaus detected over 4 million IP addresses, mainly stolen from China and Korea, and routed on Verizon's servers with forged paperwork. Spamhaus says, "For a start, it seems very strange that a large US-based ISP can be so easily convinced by abusers to route huge IP address blocks assigned to entities in the Asian-Pacific area. Such blocks are not something that can go unnoticed in the noise of everyday activity. They are very anomalous, and should call for an immediate accurate verification of the customer. Internal vetting processes at large ISPs should easily catch situations so far from normality."

Barence writes "Anti-spam outfit Spamhaus has called on the UK government to fine those who are running Internet infrastructure that could be exploited by criminals. Those who leave open Domain Name Server resolvers vulnerable to attack should be fined, if they have previously received a warning, said chief information officer of Spamhaus, Richard Cox. When Spamhaus was hit by a massive distributed DDoS possibly the biggest ever recorded at more than 300Gbits/sec — open DNS resolvers were used to amplify the hit, which was aimed at one of the organization's upstream partners. 'Once they know it can be used for attacks and fraud, that should be an offense,' Cox said. 'You should be subject to something like a parking ticket... where the fine is greater than the cost of fixing it."

Slashdot contributor Bennett Haselton writes "A recent webinar for newsletter publishers suggested that if you want your emails not to be blocked as 'spam,' you paradoxically have to engage in some practices that contribute to the erosion of users' privacy, including some tactics similar to what many spammers are doing. The consequences aren't disastrous, but besides being a loss for privacy, it's another piece of evidence that free-market forces do not necessarily lead to spam filters that are optimal for end users." Read on for the rest of Bennett's thoughts.

Lest you think that spam filters only rarely make mistakes any more, recall the instance in which after I mailed out a group of 10 proxy websites to my own mailing list, the British "anti-spam" outfit Spamhaus blacklisted two of the domains, which caused the registrar (Afilias) to disable all 10 of the domains en masse, so that the sites simply disappeared from the Web. (This happened even though our mailing list is 100% closed-loop confirmed-opt-in; users have to reply to a confirmation message in order to join the list, so the actual emails were not "spam.") It took several days to find out what happened and restore the domains, during which Spamhaus and Afilias refused to answer any of my inquiries, and have to this day not reached out or explained what they're doing to avoid similar screw-ups in the future. And this was just the latest in a long line of headaches caused by spam filters including filters at Hotmail, AOL, Yahoo, and Gmail, which had regularly categorized our emails as "spam" and caused users to miss them.

So when the email deliverability company WhatCounts announced their October 16th webinar on how to avoid having your mails blocked as spam, I watched in real time with some interest. The webinar (which you can view here), was presented by Brad Gurley, the "Director of Deliverability" for WhatCounts, who has worked in the email "deliverability" industry for 10 years. While email deliverability services is one of the products that WhatCounts charges for, the presentation didn't contain any blatant plugs for their own services, so I'm taking the contents at face value. Even if any statements in the webinar happened to be incorrect, it's still safe to assume that the presentation represents mainstream thinking in the email deliverability industry, which will determine what recommendations are made to email senders.

I hasten to add that WhatCounts should not be blamed for any of the recommendations that they made that I'm counting as "eroding privacy"; their job was to answer the question, "What is the best way to make sure my emails don't get blocked as spam?", and they answered it. The fault, if any, should lie with the spam filters which encourage these practices. Furthermore, I'm only saying that the practices encouraged in the webinar are eroding user privacy, not violating it. (If you ask every new subscriber for their name and geographic location, I would call that an "erosion" of privacy if it normalizes the practice of collecting more user data than you need, but it's not a privacy violation as long as the user willingly gives it to you.)

The webinar begins with some recommendations that are actually good netiquette, such as cleaning subscriber lists regularly (removing bouncing addresses), and displaying a prominent "unsubscribe" link for users who want to leave. If you run a newsletter, and good netiquette isn't a compelling enough reason to put an "unsubscribe" link near the top, here is a direct quote from the webinar:

"The Unsubscribe link should be prominently placed within the message body. Unsubscribe links that are hidden or hard-to-find will generate spam complaints from unhappy users who want to unsubscribe. Placing the link in the preheader has been shown to reduce spam complaints in many cases."

That's one reason that every message that I send to my own newsletter, contains this text at the top:
[You are receiving this because you subscribed to the Circumventor distribution list. To unsubscribe from this list, click here: http://www.peacefire.org/circumventor/cv-unsub.html or reply with the word "unsubscribe" in the subject.] (I give people the option of replying with the word "unsubscribe", even though that creates some hassle for me to process those requests manually, because many of our users are on censored networks and cannot access the unsubscribe link on the peacefire.org website.)

But, on to the less-stellar news: the presentation also says that the key to getting users to keep opening your emails -- and hence to signal to the email providers like Hotmail and Yahoo that your mails are not "spam" — is "engagement." Gurley suggests that senders "tailor mailings to segments of subscribers based on demographic data," including segmenting users based on city or zip code. Nothing sounds wrong with that, except that to "tailor" the mailings based on demographic data, you have to have that demographic data -- i.e. ask users for their age, sex, location, income bracket, or other information at the time that they join the list.

As I said, I don't consider this a violation of privacy if the user gives their information voluntarily, it's just an erosion of privacy, because it normalizes the process of asking users for extra data when there's no clear reason why it's necessary. In the late 1990s, you could join most companies' email lists without providing any more information than an email address; if you were asked for more information, it was for an obvious reason (such as filling out a profile on match.com, or ordering a product to be shipped). The less information about users was stored all in one place, the less opportunity there would be for the company to abuse it, or to be bought out by some other company that would abuse it, or for someone to hack into their servers and steal the information outright.

Our mailing list in particular serves a segment of the population who are particularly privacy-conscious -- they're using our proxy sites to circumvent Internet blocking software, so in almost all cases, just the simple act of being our mailing list could get them in some amount of trouble with somebody (although the severity would vary). So by design, we collect the minimum amount of information -- the email address -- necessary to send new proxy sites to the users. The more information that we asked for, the less likely the user might be to sign up in the first place.

Again, companies are within their right to ask for this information, but I don't think the rest of us newsletter publishers should be penalized for not asking for it.

The presentation goes on to say that email providers such as Hotmail and Yahoo judge whether an email is "spam" based on what proportion of the time users open an email from that sender. As Gurley says, "Give people a reason to open your email and keep opening it." The trouble is that this penalizes email notifications where you can fit all of the relevant content into the subject line -- many of my emails say something like "new Circumventor: badbadger.info", and for most users, that's all they need to see. Some subscribers have specifically said that they always want to see the new proxy site name in the subject line, because they're on a network where they are blocked from accessing their full email inbox, but they can use other webpages to see the subject lines of recently received emails. (For example, Yahoo Mail users might be on network where Yahoo Mail is blocked, but if you're signed in to yahoo.com you can see the subject lines of your last few emails on the www.yahoo.com front page.) If I'm being penalized by spam filters because user's don't open my emails, then obviously that's incentivizing me to do the users a disservice, by putting the proxy site name only in the message body.

(This might be an issue that is highly specific to my particular mailing list, because most people don't run email newsletters where they can fit all of the relevant content into the subject lines. However it's easy to think of other web applications that have a need for subject-only notifications -- Google Calendar sends me an email whenever one of my calendar events is coming up -- and those shouldn't be penalized just because the user never opens them.)

Finally, the presentation suggests that senders unsubscribe any user who hasn't opened the last 50 emails you sent them. This might set off mild alarm bells with tech-savvy readers, who know that the only way to tell if a reader has opened your message, is to embed images into the messages -- and if your newsletter content doesn't lend itself to images, you have to plant a surreptitious "web bug" image into the email, a tiny image that serves no purpose except that if you open the message and the image loads, it tells the sender that the message has been read. (For this reason, if you open an email message that does contain images, most email clients will not display them unless you click "Show images" or something similar -- because otherwise, if images always loaded automatically, spammers could use web bugs to tell who was opening their emails. So in fact, if a user opens your message and doesn't click "Show images", you generally can't tell that they opened your email.)

Again, I would consider web bugs to be an erosion of privacy more than a violation of it, on the order of asking for the user's zip code at the time they join their newsletter -- in both cases, the reason being that you are collecting more information than is strictly necessary for the operation of your mailing list. (In the case of web bugs, the "information" you're collecting is whether the user opened your message or not.)

Some people feel more strongly about it. A recent message posted on MIT's "liberationtech" mailing list had this to say about "web bugs", to a person who was asking about why his newsletter was being blocked:

You do not appear to use web bugs in your mailing list messages. A wise choice: web bugs are malware, they're invasive and abusive, and they actively degrade the security of recipients...which is a pretty crappy way to treat one's audience.

I think this is over the top -- all that a web bug does, is tell the sender whether you opened their message -- but, whether this opinion is valid or not, some people out there feel that way, and using web bugs in your email might piss them off.

Although before you cut loose the users who haven't opened your last 50 emails, Gurley's presentation also suggests trying to win them back with one last message with a "teaser" subject line like "We're saying goodbye...", or "Are we not going to talk to you any more?", or "Are we breaking up?". I hate subject lines like that, whether from spammers or from people I've signed up to get mail from. (Although now that I think about it, I doubt I'm really that mad about the 1 second of my time that they wasted; I think I just resent the fact that even just for that 1 second, they actually had me fooled, and I thought it really was a message from a friend.)

But again, we can't kill the messenger: Brad Gurley's job was to do a presentation on how to get your emails past the spam filters at the major email providers, and if using "come-on" subject lines works, because it gets more users to open your messages, then that's part of the answer. (Remember, this presentation was aimed at opt-in email senders, not spammers.)

So, I don't know that I can do anything differently with my list as a result of the presentation. I think it would be too off-putting to users to ask for their age and zip code, and in any case it wouldn't do any good for all the users who have already signed up. I probably couldn't use web bugs even if I wanted to, because the web bugs would have to load the image from a website, and if the user opened the email from a network where Web access was censored, the network's filter might block the website that the web bug loaded the image from. And for a list with many members who are still in high school, and whose parents might read their email over their shoulder, I don't feel like trying to get their attention by sending them an email with the subject "Are we breaking up?"

The more important takeaway here, though, is that there's no reason to expect the free market to deliver spam filters that are optimal from the user's point of view. In a world where users had perfect information, if Hotmail told their users, "We're going to start flagging the newsletters in your inbox as 'junk mail' unless the sender asks for your zip code when you sign up, and uses teasing subject lines to get you to open the message, and uses web bugs to verify whether you've opened it," their users would likely say, "Screw you, I'm going to Gmail!" (Which many of their users have apparently said anyway.) If this doesn't happen, it's because the vast majority of users don't have enough information for the market in spam filters to function effectively. And thus there's nothing to stop Hotmail and Yahoo from imposing arbitrary conditions on senders through their spam filters, which will lead to more legitimate senders resorting to "come-on" subject lines and web bugs -- ironically, looking more like the spammers they're trying to differentiate themselves from.

An anonymous reader writes "The NYT is reporting that the Largest DDoS in history reached 300 Gbps. The dispute started when the spam-fighting group Spamhaus added the Dutch company Cyberbunker to its blacklist, which is used by e-mail providers to weed out spam. Millions of ordinary Internet users have experienced delays in services like Netflix or could not reach a particular Web site for a short time. Dutch authorities and the police have made several attempts to enter the bunker by force but failed to do so. The attacks were first mentioned publicly last week by Cloudflare, an Internet security firm in Silicon Valley that was trying to defend against the attacks and as a result became a target."

Frequent contributor Bennett Haselton writes: "Hotmail and Yahoo Mail are apparently sharing a secret blacklist of domain names such that any mention of these domains will cause a message to be bounced back to the sender as spam. I found out about this because — surprise! — some of my new proxy site domains ended up on the blacklist. Hotmail and Yahoo are stonewalling, but here's what I've dug up so far — and why you should care." Read on for much more on how Bennett figured out what's going on, and why it's a hard problem to solve.

On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.

After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.

At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:

----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable

After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)

But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)

I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.

I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).

So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.

Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.

I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.

So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.

But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.

In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.

This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)

Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)

On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.

I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.

Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.

Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.

Frequent contributor Bennett Haselton writes: "Hotmail and Yahoo Mail are apparently sharing a secret blacklist of domain names such that any mention of these domains will cause a message to be bounced back to the sender as spam. I found out about this because — surprise! — some of my new proxy site domains ended up on the blacklist. Hotmail and Yahoo are stonewalling, but here's what I've dug up so far — and why you should care." Read on for much more on how Bennett figured out what's going on, and why it's a hard problem to solve.

On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.

After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.

At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:

----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable

After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)

But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)

I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.

I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).

So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.

Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.

I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.

So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.

But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.

In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.

This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)

Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)

On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.

I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.

Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.

Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.

Frequent contributor Bennett Haselton writes: "Hotmail and Yahoo Mail are apparently sharing a secret blacklist of domain names such that any mention of these domains will cause a message to be bounced back to the sender as spam. I found out about this because — surprise! — some of my new proxy site domains ended up on the blacklist. Hotmail and Yahoo are stonewalling, but here's what I've dug up so far — and why you should care." Read on for much more on how Bennett figured out what's going on, and why it's a hard problem to solve.

On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.

After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.

At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:

----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable

After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)

But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)

I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.

I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).

So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.

Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.

I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.

So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.

But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.

In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.

This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)

Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)

On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.

I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.

Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.

Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

Frequent contributor Bennett Haselton writes: After I sent 10 new proxy sites to my (confirmed-opt-in) mailing list, two of them ended up on one of Spamhaus's blacklists, and as a result, all 10 domains were disabled by the domain registrar, so the sites disappeared from the Web. Did you even know this could happen?"

Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.

Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.

On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)

So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.

A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.

Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.

Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:

DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.

Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.

Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.

Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.

So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.

Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."

achowe writes "Steve Linford of Spamhaus sent this to a private anti-spam list and asked that the message get out far and wide: 'For speaking out about the crime gangs located at the wikileaks.info mirror IP, Spamhaus is now under ddos by AnonOps. As our site cannot be reached now [actually sporadic], we can not continue to warn Wikileaks users not to load things from the Heihachi IP. ... AnonOps did not like our article update, here is what we said and what brought the ddos on us.'" At the conclusion of this message: "Spamhaus continues to warn Wikileaks readers to make sure they are viewing and downloading documents only from an official Wikileaks mirror site. We’re not saying 'don’t go to Wikileaks' we’re saying 'Use the wikileaks.ch server instead.'" Here is Spamhaus's full warning.

Today Spamhaus announced they are releasing a new list of IP addresses from which they've been receiving "snowshoe" spam — unsolicited email distributed across many IPs and domains in order to avoid triggering volume-based filters. "This spam is sent from many small IP ranges on many Internet Service Providers (ISPs), using many different domains, and the IPs and domains change rapidly, making it difficult for people and places to detect and block this spam. Most importantly, while each host/IP usually sends a modest volume of bulk email, collectively these anonymous IP ranges send a great deal of spam, and the quantities of this type of spam have been increasing rapidly over the past few months." A post at the Enemies List anti-spam blog wonders at the impact this will have on email service providers and their customers. The author references a conversation he had with an employee from one of these providers: "... I replied that I expected it to mean the more legitimate clients of the sneakier gray- and black-hat spammers would migrate to more legitimate ESPs — suggesting that it was, in the long run, a good thing, because ESPs with transparency and a reputation to protect will educate their new clients. His reply was essentially that this would be a problem for them in the short run, because it would swamp their new customer vetting processes and so on."

Hugh Pickens writes "The Russian Business Network, an ISP and Web hosting provider based in St. Petersburg, whose client list amounts to a laundry list of organized cybercrime operations appears to have closed shop after a number of its main upstream Internet providers severed ties with the group. The disappearance of RBN comes less than a month after Brian Krebs of the Washington Post wrote a series of stories detailing the organization and history of the shadowy ISP. However, experts at anti-spam group Spamhaus say there are strong indications that a huge swath of Internet space recently established in China may soon emerge as the next incarnation of the Russian Business Network. In related news FBI Director Robert S. Mueller, III gave a speech on cybercrime earlier this week where he said that the FBI has 60 Legal Attaché offices around the world working with partners in Russia, Romania,Poland, Hungary, Italy, and Estonia, among others, to investigate international cyber threats."

Slashdot contributor Bennett Haselton writes "A recurring theme in editorials about Net Neutrality -- broadly defined as the principle that ISPs may not block or degrade access to sites based on their content or ownership (with exceptions for clearly delineated services like parental controls) -- is that it is a "solution in search of a problem", that ISPs in the free world have never actually blocked legal content on purpose. True, the movement is mostly motivated by statements by some ISPs about what they might do in the future, such as slow down customers' access to sites if the sites haven't paid a fast-lane "toll". But there was also an oft-forgotten episode in 2000 when it was revealed that two backbone providers, AboveNet and TeleGlobe, had been blocking users' access to certain Web sites for over a year -- not due to a configuration error, but by the choice of management within those companies. Maybe I'm biased, since one of the Web sites being blocked was mine. But I think this incident is more relevant than ever now -- not just because it shows that prolonged violations of Net Neutrality can happen, but because some of the people who organized or supported AboveNet's Web filtering, are people in fairly influential positions today, including the head of the Internet Systems Consortium, the head of the IRTF's Anti-Spam Research Group, and the operator of Spamhaus. Which begs the question: If they really believe that backbone companies have the right to silently block Web sites, are some of them headed for a rift with Net Neutrality supporters?" Read on for the rest of his story.

In the aforementioned instance, AboveNet and TeleGlobe were not selling "parental filters" or other common types of filtered Internet access; the users being blocked from our Web sites were adults paying for what they thought were unfiltered Internet connections. What had happened was that AboveNet and TeleGlobe signed up to block Web sites on the Realtime Blackhole List, a list which was widely (but inaccurately) thought to be a list of "spammers", put out by a group called the Mail Abuse Prevention System. (MAPS and the RBL still exist, but under new management and in a form that bears little resemblance to their late-90's forerunners.) Most ISPs that used the RBL used it to filter only incoming e-mail, but AboveNet went all-out and blocked users from even viewing RBL'ed web sites, presumably because two of MAPS's founders, Paul Vixie and Dave Rand, were on the AboveNet board of directors. And it turned out that the RBL not only included spammers, but also Web sites that were not sending mail at all but were blocked because of their content -- in our case, our ISP got blocked because some other customers were selling mailing list software that MAPS believed could be too easily abused by spammers.

These two distinctions -- (1) the distinction between blocking incoming e-mail from spammers, versus blocking Web sites; and (2) the distinction between blocking traffic due to spam activity, versus blocking sites because of their content -- both go to the heart of what Net Neutrality is, and isn't, about. Net Neutrality is about user preferences -- not meaning that as a buzzword, but as an actual guiding principle to figure out what is and is not covered by the cause. If an ISP filters incoming mail from known spammers, that generally improves the user experience, and is something many users would expect an ISP to do anyway. But if an ISP blocks users from reaching Web sites (even, for the sake of argument, the Web sites of actual spammers), then that's generally counteracting the user's wishes -- if the user didn't want to go there, they wouldn't have typed it in. (After all, I visit spammers' Web sites all the time, usually right before I sue them.) Similarly, if an ISP blocks traffic from sites because of spam or other network abuse, that serves to protect their own users. But if an ISP blocks users from viewing sites because of their content, that's generally not expected by users, unless they've specifically signed up for something like parental controls. The Snowe Net Neutrality amendment proposed last year recognized both of these distinctions, and stated that nothing in the amendment would be interpreted to prohibit spam filtering, parental control services, or measures to protect network security.

The MAPS incident thus shaped most of my opinions about Net Neutrality 6 years before the debate even had a name. When I first found out in August 2000 that our ISP was blacklisted, like most people I believed that the RBL really was a list of spammers; after all the MAPS web page said that the RBL was a list of networks that "originate or relay spam". So I called my ISP screaming at them for being incompetent spam-enablers (the culmination of many frustrating issues with them), and saying that if they really were letting customers send spam, or running an insecure server that spammers were hijacking, I would leave on principle, if the cretins managing our server didn't drop it in the lake first. The ISP owner then told me what happened: that the ISP was not blacklisted for spamming customers, but because of the content of the other sites. (Buried in the list of RBL criteria on MAPS's site was the statement that sites could be blacklisted for providing "spam software", although the criteria did not define how they distinguished between spam software and regular mailing list software, which is how our ISP got caught in the net. And the criteria did not disclose anywhere the most controversial feature of the RBL, which is that if an ISP didn't comply, MAPS would start blacklisting other unrelated sites at the same ISP to put more pressure on them.) I agreed that this seemed to be absurd, and said I wouldn't leave the ISP if they were being blackballed just because of the content of hosted pages.

I don't know exactly what the mail software in question did or where MAPS thought the line should be drawn, but I am a purist about content -- it's a long-standing principle among the Internet security community that if a tool exists which exploits a security hole, you don't try to make the software disappear, you fix the hole. And besides, since MAPS and their supporters wanted to blackball ISPs that hosted spamming software (however you defined that), but the same people had never advocated blackballing ISPs that hosted network break-in tools and other cracking programs, for example, then what were they really saying? That spamming someone more unethical than breaking into their network?

But by far the most common objection to my complaint about AboveNet blocking Web sites was, "Hey, if a private company blocks things, as long as they're being honest to their users about it, who cares?" Well, true, but the fact that AboveNet blocked Web sites was not widely known even within the company; when I once called AboveNet feigning ignorance and asking them if they blocked RBL'ed Web sites, the technician who spoke to me said, "No, that wouldn't make any sense." (Well, half right.) Their AUP mentioned "protecting users from spam" but said nothing about blocking Web sites. In fact, other than "family-filtered" ISPs and similar services, I've never heard of any company blocking Web sites that actually did try to make their users aware of it. (On the other hand, even if AboveNet had fully disclosed their filtering, they were still a backbone company selling connectivity mainly to ISPs -- and I think if you sell something wholesale that can only be re-sold to the public by fraudulent means, then you're at least partly complicit in that fraud as well.)

If you're tempted to argue that backbone providers should be allowed to block whatever they want as long as they bury it in their AUP (although AboveNet and TeleGlobe didn't even do that much), just consider: When you access Google from your home computer, have you read the AUP of every network that the packets pass through, to check whether they reserve the right to block or even modify your traffic? Without doing a traceroute, could you even name all the networks that the traffic passes through? Do you really want the burden to be on you to check with all of them every time there's a problem reaching a Web site? Or do you feel like there's an understanding that as long as you pay your bill, they should let you go wherever you want?

Some have argued that if an ISP blocks the user from reaching a Web site, then even if the ISP is defrauding the user, that's still strictly an issue between the user and the ISP. But if a user is trying to reach your Web site, the user is trying to give you something of value: their attention, their eyeballs on your advertisements, sometimes even their money (with the expectation that you will provide them with something in return, of course, like some content worth reading). If the ISP steps in and blocks that, then the ISP has taken something of value that the user was attempting to give to you, and diverted it to serve their own interests. To me that doesn't seem ethically much different from the FedEx driver swiping the chocolates that someone tried to send you for Valentine's Day. Is that just between the sender and FedEx? Or do you have a beef because you didn't get the present that was intended for you, and you had to eat last week's chocolates to cheer up?

The modern-day threats to Net Neutrality are different: slowing access to Web sites unless the site owners pay a "toll", instead of blocking access to sites because of the content of other sites hosted at the same ISP. But they both boil down to the same thing: not giving end users what they have already paid for. If a user buys Internet access, they almost always buy it with the understanding that if they access a site, the content will download as quickly as their connection allows.

Thus the most common misconception about Net Neutrality is that the proponents are fighting against "capitalism" -- ISPs just charging more for different delivery speeds. But ISPs are already charging users for those delivery lines -- including different tiers for different prices. That's capitalism, and it works, with prices falling all the time in a fairly competitive market. But charging publishers for those higher delivery speeds to the user's house, is really more like double-billing, because the user has already been charged once for the lines that the content is coming over, so the ISP is trying to charge the content publisher again for the same service. Of course, if you charge party A for doing X, and then you try to charge party B for the same instance of doing X, and party B doesn't pay up so you don't do X, you're also breaking your deal with A. Brad Templeton of the EFF stated as much on his blog in 2006:

The pipes start off belonging to the ISPs but they sell them to their customers. The customers are buying their line to the middle, where they meet the line from the other user or site they want to talk to. The problem is generated because the carriers all price the lines at lower than they might have to charge if they were all fully saturated, since most users only make limited, partial use of the lines. When new apps increase the amount a typical user needs, it alters the economics of the ISP. They could deal with that by raising prices and really delivering the service they only pretend to sell, or by charging the other end, and breaking the cost contract. They've rattled sabres about doing the latter.

And I think the same is clearly true if, instead of trying to extract money from the content publisher, the ISP tries to extract something else, like an agreement to shut down certain Web sites before the ISP will let their users view other sites hosted at the same company. You can talk all day about how evil those Web sites are, but the ISP has already sold the user a connection with the implied ability to access them.

Anyway, this all came out in 2000 when a Slashdot article revealed that AboveNet had been blocking Web sites, and AboveNet stopped doing it two hours after the article came out. (TeleGlobe stuck with it for a few more months.) But from the hostility of the reaction, you'd think that we had published cartoons in a Danish newspaper showing Paul Vixie with a bomb in his turban. I got more e-mails than I could count arguing that AboveNet had the right to block whatever Web sites they felt like, regardless of whether the end users knew it was happening. To those people, I'd be sincerely interested in their answer to this question: Does that mean they've have no problem if they found out their ISP was silently blocking sites for political reasons? There is a clear line between following user preferences by blocking spam, and countermanding user preferences by blocking sites because of their content -- and once you've crossed that line, where's the logical stopping point? Seriously, I would have liked to have known how they would answer that, if I could have gotten any meaningful dialog going with them, which most of the time I couldn't. At the time, I'd just spent four years telling people that kids looking at porn was a non-issue, and that by the way if their kids came to my Web site I'd even help them get around their blocking software, and I still got more angry e-mails for disclosing the fact that AboveNet blocked Web sites based on their content, than I'd gotten in all the previous four years combined. (A few even accused us of moving into a blacklisted address block on purpose. This was because the actual move happened after the blacklisting was in place, even though I told them all that our ISP had announced the coming move two months before -- repeat, before -- they ever heard from MAPS. Some people were so in love with that "smoking gun" that they didn't believe me; that's their prerogative. But don't take my word for it -- when one supporter wrote to MAPS to ask about un-blocking our site, MAPS officer Kelly Thompson replied:

>Would it be possible to
>selectively unblock peacefire.org (209.211.253.169)?
Technically? Yes, it is. It's a violation of our policy, though, so I can't do so.

I would be willing to help you find other free or reduced cost hosting, however.

It was MAPS's decision, not ours or our ISP's, to have our site blocked. That should settle that once and for all, just as soon as there is peace in the Middle East and a black lesbian in the White House.)

But what do all these people think about Net Neutrality, 6 years later? I tried to track down the influential people who had spoken out supporting AboveNet's blocking of Web sites, or at least their right to block Web sites. My position was, we can agree to disagree on that, but if they really feel that way, why haven't they been speaking out against Net Neutrality? The proposed Snowe amendment was pretty clear:

SEC. 12. INTERNET NEUTRALITY
(a) Duty of Broadband Service Providers- With respect to any broadband service offered to the public, each broadband service provider shall--
(1) not block, interfere with, discriminate against, impair, or degrade the ability of any person to use a broadband service to access, use, send, post, receive, or offer any lawful content, application, or service made available via the Internet.

John Levine, webmaster of Abuse.Net, head of the IRTF's Anti-Spam Research Group, and one of the most vocal critics of Peacefire's campaign against AboveNet's Web filtering, said that he would have opposed the bill but didn't bother because it didn't have much chance of passing. Well, it didn't, but the bill was significant not because of its likelihood of passage, but because it articulated the principles that the Net Neutrality coalition had rallied around, and with the momentum behind the movement, it's likely to achieve at least some of its goals, by legislation or otherwise.

Paul Vixie, Dave Rand, and Steve Linford did not respond to requests for comment on Net Neutrality. But Paul Vixie wrote something very interesting in a May 2006 blog post:

Second, there's network neutrality. In telephone service, the government mandates that all companies providing voice-grade telephony interconnect with eachother at preset rates, thus ensuring that any phone can call any other phone and that new phone companies can enter the field to help ensure competition. In Internet service, the government mandates nothing. Recently SBC (I mean AT&T, I think, is it Wednesday?) rattled its sabre and said that Google and other content supplying companies should be paying for the use of SBC's backbone to reach SBC's eyeballs. Most of us said, uh, what? "Aren't SBC's own customers paying SBC to carry that traffic?" Some of us even said "I am not an eyeball, I am a person!" But anyway, from time to time these Internet companies shut down interconnects in hopes of creating new cash flows among eachother, and until the government regulates this, we're all at risk of higher prices or lower service with zero notice. Some well meaning democrats are trying to challenge this with "network neutrality" legislation, but this probably isn't their year. Or their decade.

San Francisco has a government, though. And if San Francisco owned and operated its own wireless Internet plant, we could mandate that any Internet company wishing to do business in this city interconnect at fair and reasonable cost to all other Internet companies wishing to do business in this city.

"Until the government regulates this"? "Government mandates"? "Fair and reasonable cost"? Quick, call the anti-socialist intervention squad! How long does it take those San Francisco hippies to suck the new arrivals' brains out anyway? Of course, I agree with everything he said. It's just that if you replace "create new cash flows" with "try to get ISPs to remove content from their servers", this describes exactly what Vixie and AboveNet were doing a few years earlier. He's a smart guy, and I'm sure this didn't escape his sense of irony, so perhaps this confirms something I'd suspected all along, which is that Vixie understood the subtleties of the issue better than most of his cheerleaders, and may be having second thoughts about AboveNet's Web-blocking misadventure. From the beginning, in a 1997 interview with Sun World, he sounded like someone trying to at least keep an open mind:

Concentration of power into a single individual: It's very true that power has corrupted every individual in whom it has ever been concentrated in the history of mankind. I do not feel that I am necessarily above whatever elements of human nature give rise to that. I worry about it. Probably other people worry about it more than I do.

Although, he didn't get to making any such frank statements during the controversy over AboveNet's Web site blocking. (Perhaps MAPS's lawyers were worried that he was a little too unfiltered and advised him not to comment; at the time, the MAPS Web site had a "How to sue MAPS" link on the front page.)

Speaking of which, Anne Mitchell, Director of Legal and Public Affairs for MAPS during the time when AboveNet was blocking Web sites, was the only MAPS adherent from the era that I could find who has since clearly and publicly come out against Net Neutrality. In May 2006 she wrote:

Here's the thing that the 3Ns (Net Neutrality Nuts) don't get: bandwidth costs money. And if you can't charge those who use the majority of it accordingly, then you are going to have to amortize it across everybody.

So, if a net neutrality law passes, don't be surprised when your costs to have an Internet account skyrocket.

Because somebody has to pay those bills, and if the law says that the ISPs can't charge the big guys - the big users - differently, it means that they have to charge them the same rate that they charge everyone else. And that means not that their rate will go down, but that everybody else's rate will go up.

And then again in February 2007 in another blog post titled "Towards A Nanny Internet", she wrote, "Network neutrality is the idea that ISPs should be forced to charge everybody the same for their Internet use", grouping it together with proposed anti-bullying and anti-anonymity laws.

Well, points to Anne for being consistent, and for publicly declaring her views in no uncertain terms, which is all I'm asking of the other supporters of AboveNet's website blocking policy. (Although she's coming at it from a different angle this time, "How do we work out who pays for the traffic" rather than "ISPs should be allowed to block whatever they want without telling anybody".) But this is also a textbook example of what I think are the three major fallacies of opposition to Net Neutrality:

First, lumping it together with other examples of unpopular regulation and calling it one more example of Big Government -- an argument also tried in other editorials ("Politicians and public figures alike should realize the absurdity of advocating more red tape to keep the Internet free"). This meme has never really caught on, possibly because groups like the ACLU and the EFF that have traditionally opposed true Internet censorship, have lined up in favor of Net Neutrality. All the proposed "red tape" and "regulation" really says is that if a user attempts to access a Web site over a connection that they've paid for, the ISP may not block or slow down their access, a law which most people would hardly consider tyrannical.

Second, asserting that "Network neutrality is the idea that ISPs should be forced to charge everybody the same for their Internet use." I've never actually heard anyone advocate anything close to that, but a common question among skeptics is why different "tiers" for Internet traffic are really any different from different-tiered pricing for dial-up vs. DSL, or for different levels of Web hosting. The difference is that when users and Web site owners pay for those connections, they are paying for their respective connections to the rest of the Internet. But an ISP charging a Web site owner to carry their traffic the last mile to the user's house, is not charging for a product or service, but really charging a fee not to break a service that they've already agreed to provide to the user.

Which leads to the third misconception: "Here's the thing that the 3Ns (Net Neutrality Nuts) don't get: bandwidth costs money... So, if a net neutrality law passes, don't be surprised when your costs to have an Internet account skyrocket." But it's not about how much a service costs, but about the ethics of double-billing for it. We know that ISP pricing models can already support the total traffic that people consume today, and ISPs do already follow net neutrality principles most of the time, so nobody's costs will "skyrocket" just because a neutrality law passes. If vastly more people start trying to stream CNN over the Internet 24/7, and fully using the services that ISPs have "only been pretending to sell" as Brad Templeton put it, then ISPs may have to charge more for users who consume too much bandwidth, encouraging people to stay at today's average levels by rationing themselves and perhaps watching 24 on their $5,000 TV sets sometimes instead of downloading it off of BitTorrent to their laptop every week because it makes them feel like a haX0r. Much as we all love our unmetered connections, it wouldn't be a violation of Net Neutrality for ISPs to charge users for bandwidth hogging, to keep everyone from going too far above today's levels. What ISPs should not do is charge users for implied full-throttle connections, and then turn around to charge publishers for moving bits over those same lines, or block the connection for any other reason.

So, yes, Virginia, blocking of Web sites does happen -- and by "Virginia", I mean FTC Chairman Deborah Platt Majoras, who said in a speech in August 2006: "I have to say, thus far, proponents of net neutrality regulation have not come to us to explain where the market is failing or what anticompetitive conduct we should challenge; we are open to hearing from them." This was echoed in an editorial later that month from Sonia Arrison of the Pacific Research Institute:

Internet service providers have voluntarily upheld content-neutral practices without the need for government intervention, and consumers would never stand for blocked Web sites... If the loss of net neutrality principles was really a problem, advocates wouldn't need to scare Americans in order to win their support. Using government regulation preemptively to shortchange business partners is a reckless abuse of the public policy process. New laws should be based on facts and reality, not fear and hypothetical situations.

I guess both of those ladies' ISPs must be blocking access to the SaveTheInternet.com Web site, so I e-mailed both of them the coalition's list of examples, and added a note about the AboveNet/TeleGlobe incident as well. No personal response from either of them yet, but I'm sure they just got lost in the shuffle while they were so busy sending out corrections. (On the other hand, I did get a courteous response from Randolph J. May of the Free State Foundation, when I wrote to him about an editorial he penned which also argued that violations have not happened: "It is generally agreed that except for a few isolated and quickly remedied incidents, neither the cable operators nor the telephone companies providing broadband Internet services have blocked, impaired or otherwise restricted subscriber access to the content of unaffiliated entities." He said he hadn't known about the AboveNet/TeleGlobe incident either.)

Another theme in some anti-Net-Neutrality editorials is that existing laws are enough to deal with the problem. In Majoras's speech, she said, "We should not forget that we already have in place an existing law enforcement and regulatory structure." Arrison's echoed that "Numerous federal agencies already have set a basic legal framework in place to preserve fair competition and business practices on the Internet". Well, as Yogi Berra says, in theory, there is no difference between theory and practice, but in practice, there is. After I found out AboveNet and TeleGlobe were blocking my Web site, I called about twenty lawyers in the Bellevue phone book, figuring: I wasn't greedy, but surely there would be financial damages for deceiving users and blocking our site, enough to pay a lawyer in return for handling the case? I think about two lawyers called me back, and they both said that even though what the backbone companies were doing clearly looked like fraud, it would take tens of thousands of dollars just to get started, and even if we ever got to court, the judge could call it however they wanted. Whatever laws exist now, they may help the slightly smaller big guy against the bigger big guy, but are not much use to the little or medium-sized guy.

So, any informed debate about Net Neutrality has to include the fact that, yes, some providers have blocked Web sites on purpose, for long periods of time, and no, the free market didn't fix it by itself. Even if something on that scale never happens again, if the free market and the anti-trust laws didn't automatically correct a case where Web sites were being blocked outright, then it's wishful thinking to think that those forces will prevent ISPs from merely slowing down Web access to sites that haven't paid a "toll", as they have made noises about doing. One AboveNet customer, Sam Knutson, said when he found out about the Web site blocking, "This type of behavior on the part of an ISP is reprehensible. I pay for a pipe and don't expect this type of monkey business." Well, I agree that it's reprehensible; whether we should "expect" more of it or not, depends on how much the Net Neutrality movement achieves its goals.

6031769 writes, "As reported on CNet, Spamhaus is choosing to ignore a judgement of $11.7M against them in an uncontested trial in an Illinois court. According to Spamhaus, the judgement has no impact on them, since they are a British organization." From the Spamhaus reply to the judgment: "Default judgments obtained in US county, state or federal courts have no validity in the UK and can not be enforced under the British legal system... As spamming is illegal in the UK, an Illinois court ordering a British organization to stop blocking incoming Illinois spam in Britain goes contrary to UK law which orders all spammers to cease sending spam in the first place."

Iphtashu Fitz writes "25 year old Christopher William Smith, considered one of the worlds biggest spammers by the Spamhaus Project, is now sitting in a jail without bond. Smith allegedly had a doctor issue 72,000 prescriptions in the space of one year in conjunction with orders obtained through spamming. The doctor, Philip Mach, had a license to practice medicine in New Jersey but he provided prescriptions to people throughout the United States without ever evaluating them, both of which are big no-no's. Federal authorities have already seized over $3 million in cash, luxury cars, and houses."

Iphtashu Fitz writes "25 year old Christopher William Smith, considered one of the worlds biggest spammers by the Spamhaus Project, is now sitting in a jail without bond. Smith allegedly had a doctor issue 72,000 prescriptions in the space of one year in conjunction with orders obtained through spamming. The doctor, Philip Mach, had a license to practice medicine in New Jersey but he provided prescriptions to people throughout the United States without ever evaluating them, both of which are big no-no's. Federal authorities have already seized over $3 million in cash, luxury cars, and houses."

Iphtashu Fitz writes "The Chinese government has joined an international anti-spam effort started by the U.S. and UK. Over the weekend China stated that it would join international enforcement efforts against spam by adopting the London Action Plan on Spam Enforcement Collaboration. The London Action Plan was launched after a conference on spam enforcement hosted jointly by the UK Office of Fair Trading and the US Federal Trade Commission in London in October 2004. It was the first international forum to focus exclusively on spam enforcement. China is well known for being one of the biggest origins of spam, with as much as 20% of all junk e-mail originating from within its borders."

ermita2 writes "The South Florida Sun-Sentinel published this weekend a story about how South Florida is home to more spammers than any other place in the world according to Spamhaus. The reason for that seems to be Florida's long history and friendliness with dirty business from real estate speculation to the penny stock scammers. Among the interesting tidbits is anecdotal evidence that the amount of spam worldwide fell for a while after last year's hurricanes in Florida."

Iphtashu Fitz writes "Jeremy Jaynes of Raleigh, NC now has the dubious honor of being the first spammer sentenced to jail for the felony of spamming. Virginia judge Thomas Horne sentenced Jaynes to 9 years in prison based on a jury recommendation after he was convicted of sending out 10 million e-mails a day. Jaynes, who sent out much of his spam using the name "Gaven Stubberfield", has held a position on the SpamHaus Registry of Known Spam Operations for a long time. Unfortunately the sentence has been postponed while the case is being appealed." Commentary on the sentence available at Forbes as well.

Iphtashu Fitz writes "Jeremy Jaynes of Raleigh, NC now has the dubious honor of being the first spammer sentenced to jail for the felony of spamming. Virginia judge Thomas Horne sentenced Jaynes to 9 years in prison based on a jury recommendation after he was convicted of sending out 10 million e-mails a day. Jaynes, who sent out much of his spam using the name "Gaven Stubberfield", has held a position on the SpamHaus Registry of Known Spam Operations for a long time. Unfortunately the sentence has been postponed while the case is being appealed." Commentary on the sentence available at Forbes as well.

www.sorehands.com writes "The well known spammer Scott ("Snotty Scotty") Richter has filed for bankruptcy protection. In a Denver Post article Richter claims to have less than $10 million in assets but more than $50 million in debts including the $49 million that Microsoft is seeking. Microsoft is not the only lawsuit that Richter is defending, as a law suit filed by anti-spammer Dan Balsam and being handled by anti-spam attorney Timothy Walton is still pending. Hopefully, Microsoft will have the automatic stay from the bankruptcy court dissolved so that they can stop Richter from spamming and gather more evidence."

An anonymous reader submits "According to a new Spamhaus report, MCI makes $5 million a year hosting spammers and illegal spamware. MCI/UUNET has long topped the Spamhaus spam supporting ISPs list, with nearly 200 active SBL entries. MCI even took on spammers such as iMedia, when they were terminated by Savvis in their half-hearted response to leaked pro-spam memos."

An anonymous reader submits "According to a new Spamhaus report, MCI makes $5 million a year hosting spammers and illegal spamware. MCI/UUNET has long topped the Spamhaus spam supporting ISPs list, with nearly 200 active SBL entries. MCI even took on spammers such as iMedia, when they were terminated by Savvis in their half-hearted response to leaked pro-spam memos."

A number of people wrote in with this story: "Count Texas in the growing list of states fighting spammers with CAN-SPAM. Texas Attorney General Greg Abbott filed the lawsuits today, charging a University of Texas student (and a cohort in California) with sending out millions of unsolicited commercial emails under the pseudonyms PayPerAction and Leadplex, among others. Spamhaus rates PayPerAction the #4 spammers in the world."

Cpyder writes "ICANN has decided to go forward with the implementation of two new top level domains, namely .mobi (for mobile use, sponsored by Nokia and T-Mobile) and .jobs (for job sites). The ICANN Board meetings regarding the approval are available. It is not yet known when these domains will be available for registration, as this decision merely starts the technical and business negotiations for terms under which these domains will be registered. Normally the domains should become active somewhere next year. Several other new TLDs are still up for discussion. These include .asia, .mail, .tel and .xxx. Last October, ICANN approved .travel and .post. More on these new TLDs at PCWorld and Google News."

kaip writes "Internetnews.com has a story of a spammer. The individual sends 60 million spam emails for four days worth of work and claims that one in 19 of AOL users clicks the links in his mortgage spam (this number should however be taken with a grain of salt, see rules 1 and 2). Maybe not everybody has heard of the Boulder Pledge... The article also tells how the CAN-SPAM Act, which legalises spamming, is turning the US into the spam haven of the world. Currently, 86 percent of the total spam volume is coming from the States."

smash writes "According to AustralianIT, recent spam laws, including fines of up to 1.1 million dollars per day has seen a reduction in local spam production. This has been confirmed by SpamHaus. Surely people in other countries are just as ticked off with the problem - perhaps this is an example to follow?"

dankinit writes "The FTC is considering a proposal made popular by Lawrence Lessig which would offer a bounty to people who help catch spammers. The proposal looks to harness the power of volunteers online who might want a piece of the multimillion dollar fines spammers could incur. Spamhaus founder Steve Linford doesn't like the idea though, explaining '...the FTC already has so much information on their identities that to get anymore would be useless.'"

An anonymous reader writes "According to this ZDNet article, The Spamhaus Project has warned that organised cirminal gangs in Russia are supplying U.S.-based spammers with details of compromised PCs that can be manipulated to send junk mail. According to Spamhaus director Steve Linford, the Russian gangs aren't constrained by any anti-spam or cybercrime laws in their home country and have no respect for legislation implemented in other countries. Also, apparently 70 percent of spam is sent from China by American spam outfits who in turn have hosting arrangements with Chinese ISPs."

BMcWilliams writes "Spamhaus director Steve Linford announced a new funding plan Tuesday. According to Linford's announcement, large ISPs and big corporate users of the Spamhaus zone transfer service (renamed the Spamhaus Data Feed Service) will be required to pay an annual subscription fee ranging between $190 and $14,500.(The free public-query mirrors will continue to exist.) The point of the new plan is to ensure that 'the millions of users who rely on our anti-spam systems can be assured we'll be here for as long as spammers plague the Internet'."

BMcWilliams writes "Spamhaus director Steve Linford announced a new funding plan Tuesday. According to Linford's announcement, large ISPs and big corporate users of the Spamhaus zone transfer service (renamed the Spamhaus Data Feed Service) will be required to pay an annual subscription fee ranging between $190 and $14,500.(The free public-query mirrors will continue to exist.) The point of the new plan is to ensure that 'the millions of users who rely on our anti-spam systems can be assured we'll be here for as long as spammers plague the Internet'."

BMcWilliams writes "Spamhaus director Steve Linford announced a new funding plan Tuesday. According to Linford's announcement, large ISPs and big corporate users of the Spamhaus zone transfer service (renamed the Spamhaus Data Feed Service) will be required to pay an annual subscription fee ranging between $190 and $14,500.(The free public-query mirrors will continue to exist.) The point of the new plan is to ensure that 'the millions of users who rely on our anti-spam systems can be assured we'll be here for as long as spammers plague the Internet'."

yet another coward writes "The New York Times reports on new measures against spam. (Sperm sample required, sorry ladies) Microsoft has increased efforts to track and prosecute spammers. Hotmail receives 2 billion (2 * 10^9) spam messages per day. In a twist of weirdness, the Direct Marketing Association is funding investigators who cooperate with the FBI on spam investigations. Spamhaus also gets a mention."

Eggplant62 writes "ChinaTechNews.com is reporting that Steve Linford's Spamhaus.org will open operations with the help of Chinese government officials and ISP's in order to remove spammers operating servers on China's portion of the Internet. For years, China's unwitting ignorance of the spam issues they have with the rest of the world has been a major stumbling block in the fight to control spammers who operate from the netblocks of foreign nations. Seeing China take steps to help the world curb the scourge of junk email has me cheering all the way. Go Steve!"

securitas writes "The SpamHaus Project is the group pushing ICANN to create a new trusted-sender system and the .mail top-level domain. SpamHaus proposes that registrants under the .mail TLD would pay at least $2000 per year to and 'agree to abide by certain anti-spam mailing practices.' The interesting twist is that companies that comply with the US CAN-SPAM act - which SpamHaus opposed due to the legalization of bulk unsolicited commercial e-mail - would not be eligibile to register a .mail address. The .mail TLD proposal was recently discussed on Slashdot."

An anonymous reader submits "Statistics for February have UUnet leading the Spamhaus top 10 worst Spam ISPs chart. The Register point out that ISPs like UUnet and Abovenet continue to host spammers despite advertising anti-spam AUPs." And the competition is probably wishing they had as much luck.

ThePretender writes "From the InfoWorld article: 'A spammer's claim to his clients that he had an agreement with anti-spam technology vendor Brightmail to not block his traffic was contradicted by Brightmail officials today.' From the sounds of it, Scott Richter (apparently a notorious spammer) might just be looking for some media attention, he even goes as far saying he has similar agreements with some major ISPs. Ouch! May the drama unfold..."

Iphtashu Fitz writes "Microsoft and the Attorney General of New York have announced multiple lawsuits against what they term as a spam ring operating throughout New York and responsible for sending billions of illegal junk e-mail. According to articles at ABCNews.com, CNet News.com and elsewhere the state of New York has filed 6 lawsuits against alleged notoriuous spammer Scotty Richter and accomplices. Richter is well known among the anti-spam community, holding the dubious distinction of being ranked number 3 on the Spamhaus Registry of Known Spam Offenders. Microsoft has seperately filed 5 other lawsuits."

BenLev writes "The New York Times has an article profiling Spamhaus Project director Steve Linford. The feature goes behind the scenes at Spamhaus, 'one of the leading groups that is trying to make the world safe from junk e-mail', showing that it operates from Linford's houseboat on the Thames near London, spammers don't like him, and his volunteer corps likens itself to the X-Men."

McDutchie writes "Steve Linford of Spamhaus announced in a press release that the latest Wintel virus, W32/Mimail-E, was created by spammers for the specific purpose of DDoS'ing Spamhaus, Spamcop, and SPEWS. It's becoming more and more clear that the spambags are the ones behind the recent mess with the Windows viruses. They must really be getting desperate."

McDutchie writes "Steve Linford of Spamhaus announced in a press release that the latest Wintel virus, W32/Mimail-E, was created by spammers for the specific purpose of DDoS'ing Spamhaus, Spamcop, and SPEWS. It's becoming more and more clear that the spambags are the ones behind the recent mess with the Windows viruses. They must really be getting desperate."

Spamhunter asksL "Everyone has their favorite tools to stop spam at the inbox, whether it's using a scoring tool like SpamAssassin, bayesian filters, or something as extreme as challenge/response whitelists (which creates a few problems itself). What I'd like to know is, what are your tools for actively investigating and shutting down spammers? I've found information sites like SPEWS and Spamhaus to be invaluable in tracking down spam gangs and spam-friendly ISP's in order to put pressure where it belongs. Sometimes just chasing the chain of ownership in WHOIS is helpful. What tools, approaches, and resources do you find helpful?"

Ubi_NL writes "According to the Register there is a close correlation between the DDOS attacks on a number of anti-spam lists and the presence of the Sobig virus. Now that Monkeys.com is gone, and spamhaus.org is taking heavy blows, are the spammers actually winning the battle by using viruses?"

ShaiHulud-23 writes "A suit was recently filed by EMarketersAmerica.org, a fledgling secret organization of spammers, against the Spamhaus Project, (and other anti-spam sites) seeking to prevent the publication of the anonymous plaintiffs' IP addresses in the Spamhaus Block List (SBL). The suit requested a response from the named defendants, and Spamhaus director Steve Linford has provided one, dismantling the spammers' case point by point."

ShaiHulud-23 writes "A suit was recently filed by EMarketersAmerica.org, a fledgling secret organization of spammers, against the Spamhaus Project, (and other anti-spam sites) seeking to prevent the publication of the anonymous plaintiffs' IP addresses in the Spamhaus Block List (SBL). The suit requested a response from the named defendants, and Spamhaus director Steve Linford has provided one, dismantling the spammers' case point by point."

SuperBanana writes "The Register reports in a story today that spammers have banded together under the name EmarketersAmerica.org to sue various anti-spam groups- days before a large conference on spam hosted by the FTC(which will be attended by many spammers). Anti-spam groups think the timing is not by coincidence, but believe the move may backfire because they will be able to countersue and get access to spammer's internal documents. By the way, if you're wondering who these guys are, check out Spamhaus's directory of top spammers."