It may be that your apache is statically linked. Or it may be that apache records the version of OpenSSL at compile time rather than at run time (dumb). Recompile anyway; you need the practice.
What do you mean HTML is a hack? What do you mean I can't change it? That's nonesense. I can change it.
A <font> tag does provide information. The most useful information is relative size. The tag <font size=+1> says the text following is larger than the previous.
I've seen the sliced up images. I think they are doing that because they are too lame to do image maps. It's certainly not needed to do layouts even with tables, unless what they want is a layout with a non-rectangular image.
But let's not dwell on your lack of ability or understanding of HTML. I will give you the fact that CSS most certainly is an improvement, and is needed now. The problem is it is useless if it is tied to browsers that not everyone can use. The issue is not about getting people to upgrade browsers; it is about getting enough variety of browsers so people have choices that fit their needs. Then we don't have to have these obese monsters (no pun intended... but it is fitting) for browsers, and can find something better that works the way we need it to work, and conforms to standards correctly as well.
Forward compatibility is pointless if you don't have a path into it for everyone.
Since I don't do things that involve those kinds of higher level languages, my first encounter with "UML" actually was "User Mode Linux". So when I see "UML" that's what I think of. In fact it took a while for me to figure out why it all those developers seemed to be interested in using User Mode Linux.
I propose we adopt a new meaning for "UML" as "Ubiquitous Mysterious Logic" or "Ugly Men Lurking".
Tell it to Zeldman. I'm not the one who is doing CSS. I've put that off until at least 98% of users are using a browser that does CSS correctly so I don't have to do hacks.
There's nothing wrong with making it look good, too. In fact I'd like that, too. The problem is developers like Zeldman prefer to make it look good only if you are using their preferred browsers, and look bad on others.
Web sites can be made to look good w/o CSS. Those who promote CSS point out that one advantage of CSS is not having to put in all those HTML tags. That's why they make web sites that have CSS and no more HTML than is needed with CSS. The result is a web site that looks ugly on NS 4. It's not me that wants the feature as I can make web sites that don't need CSS. But developers like Zeldman are the ones demanding the feature, and insisting that people upgrade their browser for his benefit, disregarding other issues that make upgrading not practical for many. I just want web sites to look as good as HTML can do it.
And I don't use HTML hacks, as my HTML comes out looking good and it's the same exact HTML for every different browser. The problem sites are ones that depend on CSS (or in other cases, depend on Javascript, Java, Flash, etc). Those that are done in HTML look fine.
CSS didn't even exist way back then (it was being talked about as a great solution). Even today, CSS is broken in Netscape 4, and not everyone can, or is willing, to upgrade beyond that, yet. As soon as some leaner browser shows up, I might. As soon as 99% of people have upgraded, then I will switch to using CSS.
I have P3 800, so shouldn't it be even faster for me? You'd think. Maybe it's just the rapidity of my vision. Even on this P3 800 I can see a difference in the speed between NS 3 doing rendering and NS 4. It's just that on my previous Celeron 333 the NS 4 speed was not tolerable, so I used NS 3. I remember doing some timing tests way back then between NS 3 and NS 4 on some complex HTML tables, and NS 4 was 22.7 times slower than NS 3 at rendering them (the page was made sufficiently complex that it took NS 3 a total of 3 seconds to render, while NS 4 took 68 seconds. That was just quantifying what I was actually seeing visually on much less complex pages.
NS 4 is still slow at rendering multi-block GIFs. NS 3 did them nearly instantly. Maybe this is because NS 4 has too much per-block startup overhead. Or maybe it is because NS 4 mistakenly thinks a multi-block GIF is an animated GIF and artificially inserts a delay. Download this image [184,565 bytes] to disk first, so you aren't seeing any network delay, then test some browsers on it referencing the local file and see how quickly it renders (BTW, certain older builds of Moz and Konq won't even render it at all). You'll need to do this on X running in 24 bit or more per pixel, with a video card that can do that (what can't these days).
If people come to the site for just content, then why are the designers jerking around with the gloss and flashiness? I do get colors on www.alistapart.com, but the layout is horrible. I don't have that kind of trouble doing layout that works on a wide variety of browsers, and I don't even check the browser type to customize. If you want to know what features are only available in the newer browsers, maybe you should ask the developers of www.alistapart.com what it is they use that won't work on anything but the newer browsers. The features I use seem to be working fine on older browsers.
I just see too many web sites that appear to be more intended to show off the designer's l33t w36 sk1llz that an ability to stick to information and readability. That turns me off from many sites quickly. Things like pointless use of flash will do that. Things like using Javascript to implement a hyperlink will do that.
Oh, and yeah, I am guilty of sticking in some of my own l33t HTML sk1llz on linuxhomepage.com, but at least it works everywhere.
His "dump all the crappy hack and bloated code" is still making a web site that won't work in NS 4. Take a look at www.alistapart.com in NS4. It displays, but it's crap. It could be made to work and even do so in the same output without having to distinguish the browser type. Yes, it would take more bytes transmitted to do that, possibly twice as many in some cases. But I already debated the issue last year on that site I just pointed you to, and that was indeed their purpose, to get me to upgrade my browser to the very newest so the features they wanted to use would work.
We disagree on which browser is the current best. Mozilla is not working out for me. It is noticeably slower than NS 4. And NS 4 was too slow for me compared to NS 3 until I upgraded CPU and RAM. Mozilla will have to wait until my next hardware upgrade. Mozilla also has some bugs whereby it won't even function correctly in my desktop system at all. NS 4 had problems but I found ways to work around it. Those ways don't work with Mozilla, and until I have the time to deal with it, it won't be my default browser.
Yes, times do change, and standards do change. But everything is according to the standard. The site is PHP driven, and PHP doesn't seem to handle DTDs. But it works now because the default HTML still works.
When will features in HTML be obsoleted from the standard and removed from browsers? I don't know yet. That time has not come. Hopefully there will be an adequate transition period to a decent browser that can do the new replacement features before expecting everyone to get off the old browsers. Unfortunately, the browser situation is getting worse, not better. What browser no longer accepts HTML 4.0 or even HTML 3.2? If you find one, then that browser has jumped the gun because HTML is not yet depricated. But those sites you mention using XHTML and CSS come up a total mess in NS 4 because unlike the goals for XHTML and CSS, they don't work in NS 4 (CSS is there, but it's horribly broken). When a new browser can do some of the things (including X standards) that NS 4 can do, then let's talk about browser upgrades. But you have to help try to get decent browsers implemented and deployed if you don't want to have to face people who are having to make hard choices to find the least-broken browser. I don't deny NS 4 is broken. It just happens to be the least broken right now. Konquerer appears to be poised to edge it out soon. Maybe the next version will. One can only hope. Still, Konquerer is more obese than I would like.
If you at least admit that browser programmers are doing a bad job of getting standards out to users, then we'll at least have something to agree on.
The same point remains. His server based authentication on the forgeable domain name. Some spammer did discover his server worked that way. I see spammers frequently testing my servers exactly that way, by trying to use local domains, including the name of the server itself, to see if any of those have been blindly (and stupidly) configured to allow relaying. They don't get through mine because I don't configure it to trust forgeable information. And it is easy for a spammer to try this because he already has lots of email addresses from which he can do MX lookups to find mail servers. If that server is stupidly configured, that email address stands a good chance at being allowed to relay by merely being the MAIL FROM or the From: address. If that doesn't work, they try the reverse DNS of the mail server IP address as the hostname part. Some spammer did that on Bret Fausett's mail server, and at least one of those spams went into a spamtrap mailbox, resulting in the initial listing. Then when he discovered the situation and read about how to get it unlisted by being tested, he asked for that, probably without yet knowing the mechanisms the spammer used, and therefore that the relay test would have to use. When he discovered that the test mail forged his own domain name, he got into a tizzy and started writing his article.
It doesn't matter what the mechanisms actually are. There are many others that some mail servers will be fooled by to allow relaying. If a spammer can use it to relay spam, then a relay tester must include that mechanism in the testing to determine if that particular mechanism has now been closed.
They need to use the right tools for the job. If the job says make it look exactly like this paper brochure, then they have to choose something like a giant image or PDF. Sure, it can be approximated in HTML, but they need to not whine when it fails because a different environment with a different look and feel ends up showing it differently. Do you know how many pixels wide an 18pt font is on my display?
As for XHTML and CSS, they might not be there. Sure, feel free to use them where they are, but don't expect me to have to deal with the problems of some new browser just for your benefit. What if the CEO's cousin's computer... the one that is so old that it only has 256 colors... can't even run the new browser at all, and therefore has no XHTML or CSS?
If the designer loses his job working for a PHB, at least now he has a chance to get a decent one. Sure, I can understand he would rather keep the job and make a real honest attempt to accomplish the task in a way that keeps everyone in the company and West BFE happy. But who is to blame when someone reports to the company that the site shows up lousy on some other computer?
You can design for the middle or you can design for the CEO's cousin. If management makes the decision, then the web designer is just doing what he can with what he's allowed to do. The real culprits are the ones that don't allow the web designer to design it as content.
Do you see any XHTML or CSS on http://linuxhomepage.com/? Does it look on your browser today like it looked in Netscape 3 back when I originally designed it?
They won't know, unless they were expecting it in the first place, or the sender found a means to tell them, or they discovered it in the list I make available to them (a scan of server logs filtered on their email address for the past 90 days). I do want to make that work better.
The consistency should be between different web sites when viewing content on the same browser with the same OS under control of the same theme. Don't assume that <h1> gets you 18 point times new roman, even on Windows in Explorer, because as you mention it being the default, it is certainly changable, and may well not be the default when you change browser or even OS (though such changes should only happen when you change whatever is appropriate for a look and feel change).
Many web designers come from the graphical arts field, which is full of brochure-on-paper production. They get to pick (or are told specifically) the paper size, font theme, color theme, etc. That's a fixed environment and quite easy to work in. Now along comes the web, and it isn't that way anymore. Some people have smaller browser windows, or are limited to 640x480, or to 256 colors. Others have 1600x1200 and 16 million or even 1 to 4 billion colors. Some have 28.8k or worse download speeds and others have T1 and better. Some have plenty of CPU power and RAM to run the latest browser, and some are stuck with an older browser on a hand-me-down computer. The world is full of diversity and these brochureware designers still can't cope with it. They could build the whole site in a big image file and piss off the 28.8k dialup users and be scrunched over on the left side of the 1600x1200 users. Even with advances in new protocols and formats, so many problems remain that their assumptions still don't work. They might be better off with PDF, which was designed more for the kind of thing they are doing.
I make it known to my customers that inbound mail is subject to spam filtering. I even make available to them a list of all the spam (what server it came from, and what MAIL FROM had in the SMTP) attempts that got blocked. If they discover something they want is getting blocked, I can whitelist it. And I have done so already in a couple of cases. Being small, I can do this myself. Eventually I'll have to automate this so I can grow. The plan is to give each user a choice which blacklists to use, and give them their own private blacklist and whitelist, and the ability to automatically allow inbound mail from anyone they send outbound mail to. Is that reasonable enough for you?
You're speaking of SPEWS. And the whole point is that because spammers move around to evade blocking, the front line of defense is the ISPs. Blocking just the spammer alone is a futile effort. Getting spammers disconnected doesn't work at too many of the larger ISPs because they would rather take spammer money than keep non-customers happy. So of course the logical way to go is to block ISPs. And SPEWS doesn't immediately block a whole ISP (unless they are so small they have less than a class C network or something). They raise the pressure gradually so the ISP gets the message before too many customers are impacted.
The only way these ISPs are going to respond to dealing with spammers is when they are forced to decide between the revenues of spammers vs. the revenues of non-spammers. Absent that force, they just keep spammers online and the internet suffers. With that force, most will eventually see the financial issue and make the decision (and yes, some have decided to go with spammers and have gotten 100% blocked... and rightly so).
As long as you stay with an ISP that supports spammers, then you are, every time you make a payment to them, saying "It's OK for you to keep spammers online because I'll keep paying you even though it causes me grief". If it's so costly to change ISPs for you, then maybe you should have done due diligence in the first place to discover what their real intentions are with regard to spam.
I already ignore spammers. They have not gone away. That idea is stupid because there will always be some small percentage that don't ignore them, and it only takes that small percentage for them to get more money out of spamming than they put in. Then while they spam, they force us to do deal with all the junk. Even if you just count the 3 seconds it takes to delete each piece of spam at a typical low end wage of an office worker, spam costs over a billion dollars a year (at its current rate) just in lost worker productivity in the USA. Now that Europe has caught up and edged past the USA on internet users, I'm sure the figure is nearly as high there, and will soon be higher. And this doesn't count the time it takes for staff to manage the situation and clean it up.
What's a little guy to do? For starters, try convincing your ISP to stop supporting spammers. But if you say to them "because it costs me so much money to move to another ISP, I will stay with you no matter what you do", then why would they give up the revenue from the spammers just for you? Maybe what you should do is figure out why and how you got yourself into a mess where you can't change to another ISP?
One thing web designers need to do is get off this "consistent look and feel" drug. It doesn't work that way because it is not supposed to work that way. For example in HTML, the <h1> tag starts a header at level 1. That doesn't mean the enclosed text has to be rendered in some particular size or some particular font or some particular color. It just means it is a header at level 1. The look and feel is up to the browser or the windowing system of the user. Even applications that get ported between different systems (for example on both Microsoft Windows and on Macintosh) are supposed to have the look and feel of the respective system so the user can access it in their own familiar way. When I go from one web site to another and find the buttons are always different, that is where consistent look and feel has failed.
I've heard this rant so many times. Most ISPs, such as Earthlink, allow users to turn off the spam filtering. Got one that doesn't, and won't whitelist for you? Then move on.
One thing we do not need to be doing on the internet is encouraging incompetence by continuing to pay them. If they don't, or can't, run spam filtering for you the way you want it done, then yes, it is time for you to move on. That's why we have competition and free trade.
If webmasters want to see XHTML become popular, they they should (encourage someone to) write a lean-and-mean web browser that does XHTML correctly and without the obesity and sluggishness of most of the recent browsers. If XHTML is as good as they say it is, then a lot of other junk like Javascript, Java, and Flash won't be needed, so leave those out of the browser to keep it lean-and-mean... and secure, too.
Older browsers are a problem. But newer browsers are a problem, too. Featuritis (also known as creaping featurism) has grown to be a plague. Browsers are just too stuffed with features that in effect try to make them be a platform unto themselves. The ideal browser will be a lightweight "traffic cop" that manages the various pluggable rendering components (document interpreters) to create a seamless appearance, and won't be all that much different than a windowing system. Then I can remove that Javascript plague with a simple "rm" command in the render library directory.
99.9% of new web browsers are obese. What this guy essentially wants is for me to upgrade my browser so he can use his fancy tricks. But he doesn't want to help put pressure on browser makers to get their browser under 8MB.
As vile as spam is, the ends don't justify the means. Regardless of whether my mail server used to be 'open' or not, I stand by the legal analysis that placed fault on the blackhole operators who forged their identity.
If the list operator who tested your mail server did not test it by using the proper practices, which includes doing everything that spammers are known to be doing, or known to be capable of doing, then it would be the list operator who had failed to properly and correctly test your server. If it had been marked as closed, because of that, when in fact it was still open, then it would be the list operator who would have been negligent.
Security practices, and spam prevention is a form of security practice, do include performing tests that mimic what the security prevention is supposed to prevent. Your mail server is supposed to prevent relaying of forged addresses. So you have to do forged addresses to test that facility.
The only thing the list operators did wrong that I can see is they failed to get your signature in writing on a piece of paper that explained it to you. Had they done so, that piece of paper would have stated that they would be performing a test that adheres to current best practices in security testing, and that test would include every form of forgery and trickery known.
The ends not only do justify the means, they are also absolutely required!
Also, some mail server software is defective in ways that certain types of attempts, which spammers might try, and therefore have to be tested in a thorough test, could cause that defective software to fail, and may result in damage to your mail server. If that happens, your remedy should be with the maker of the defective software, unless the defects were documented and avoidable by proper configuration.
And if you want to have a private dialog about this, I am willing to explain it in more detail if you need that. I am not a lawyer, so I can't give it to you in purely legal terms, but I can certainly give you some real life analogies. You can find my email address a number of ways, such as the domain registration of one of my web sites.
Your concern about failing to circulate blacklist removals is misplaced with regard to DNS based blacklists. The data expires in a finite amount of time from the cache, and removal processes are working pretty damned good. I've watched a number of notices posted on news.admin.net-abuse.email asking to be removed from the SPEWS list, and I check out whether they have fixed the problem or not. In most cases I find that the data had already been removed from SPEWS by the time I checked that (so now I check SPEWS first before checking to see if the problem is fixed).
Private blacklists are a problem because there's virtually no way to track them all down and get removed from everywhere (once you fix the problem). That's why we need central DNS based blacklists. But what we also need is to shield these central lists from stupid lawsuits from people who refuse to fix their problems or simply don't have a clue. Those who even so much as threaten to sue the list operators instantly get their IP addresses and domain names put in thousands of private blacklists where no one even looks to see if anything is ever fixed. And when they end up shutting down the central lists, they make things worse due to all the private lists. That's the primary reason SPEWS is so secret. Sure, it comes across to people who didn't know about it as a "Star Chamber" thing. And I didn't use it for several months until I verified it actually works to list what needs to be listed, and removes things when fixed.
Here is what I wrote to this guy back on July 25 when the article had just come out. I never received a response from him. Was he totally embarassed by his idiocy once it was explained to him? I guess so.
<lettertext>
I just read the article you wrote on New Architect Magazine entitled "Blind Vigilantes; Blackhole lists offer dark prospects". I feel you have missed certain points in your analysis, and as a result, you misunderstand what is going on. That's OK, because the majority of network administrators still do, too. As a lawyer you would not be expected to know this kind of stuff. You clearly know a lot more about it than the average lawyer. I'm writing in hopes of filling in the gaps. I sincerely hope you have the time to read this. It's long, but I think this is important.
First of all, I use these blackhole lists myself, so it is possible that your reply to me could bounce back. I can override it if I know the IP address of your mail server. But I won't know it until there is a server log telling me about it bouncing. What I'll do is get your IP address at that time, add it to the exception database, and you can repeat the reply later on. Or you can send me mail from Hotmail, which I believe is not blocked anymore.
I want to fast forward to the point in your article where I think the main misunderstanding is:
How had it gained access to my mail server? Simple. It had forged the headers on its email to convince my mail server that the email it sent was from a permitted user. You see, my mail servers were set up to pass mail only from a domain name of which I am the only user. It blocks everything else. That's not an open relay. Unless you're a user in my domain, you can't use it.
One of the methods spammers use to send their mail through a mail server configured like yours is to do exactly what you are complaining about. I see upwards of 10,000 of these a day on my servers. The spammers have these massive lists of email addresses, quite many of which are valid. What they do is look up which mail server those users would use, which is not hard because that's exactly what the whole system is designed to be able to do. Every delivered piece of email had to do that. Once they have this information, then they forge that user in their FROM line and start sending mail to the user's server. In the case of a server set up to test only the domain name in the FROM line, it works, and the spam message gets sent on its way.
That's why your mail server is considered to be an open relay, because it is possible for a spammer to use it, despite the fact that they are doing something illegal such as forging your domain name. If it lets a spammer forward mail, it's an open relay.
The group based in Denmark had pretended to be me, forged an email as though it had come from an address that only I am authorized to use, passed it through the mail server in my house, and then placed me on a list of people who should be blocked from sending mail. They circulated that list around the world. ISPs used by my friends and family here the United States subscribed to this list. Now, through no fault of my own and in fact because of the trickery of Danish email activists I was no longer able to send email to many people in my address book.
It is standard practice for every program (there are several available) which does the open relay tests to try dozens of different ways to fool a mail server into forwarding mail. Forging the domain name of the users of that server is one of the simpler tricks. There are some that are more complicated. These programs are simply doing exactly the same thing that a spammer would do. It's the same principle used by security test programs which test whether or not a computer can be broken into. They have to pull all the punches a hacker might try. Otherwise such programs will fail to detect a flaw and the program itself will be worthless.
I periodically run tests on all my mail servers to make sure I have not accidentally configured out the relay controls. I watch these tests take place, and they do this forgery exactly as expected.
It's hard to describe how angry this made me. The Danish consortium had lied about their identity, and I was paying for it.
The worst thing about being blacklisted, however, wasn't that I could no longer send email, but that spammers began actively trying to use my mail server to send their spam. You see, blackhole lists work both ways. ISPs use it to block traffic, but as I've recently discovered, the spammers themselves use the lists as a kind of directory of servers to use for sending their mail.
Actually, that is not true. Read on and this will be explained.
If you look at my mail server logs, you'll see that every few seconds or so, someone, somewhere tries to access my mail server and use it to send mail. Each time, without fail, my mail server declines the request and refuses to relay the requested message. It isn't an open relay. It's just doing its job. But my machine is bombarded with requests from all over the world from spammers seeking to use its minimal capabilities to send their penis enlarging, breast enhancing, get-rich-quick messages.
Last year, one of my client companies, a local web hosting business, had a case of one of their customers running a spamming operation right from the server they were paying my client to use, in violation of their AUP. The customer got cut off, and my client asked me to help him clean up the mess. In so doing, I obtained a copy of not only the spamming software (a special version intended for running from web servers), but also a copy of a big list of about 1.5 million addresses.
There was something very interesting in this list. The first 1000 or so entries were email address that were familiar to me. They were OTHER SPAMMERS. That's right, other spammers have their own names in these lists. What that means is if any spammer discovers an open relay, the others find out about it fairly quickly. The "spammer network" as I might call it is very well connected. They all see the successes of the others. And much like wild animals on the African Savannah when one makes a kill, the others circle around to take their own bite out of the carcass. That's what is happening to your server.
The anti-spam group have some of their addresses on these lists, too. That's how they first find out if your mail server is an open relay. They get spam that some spammer who found it relayed through. That's how you were first put on the list.
The blackhole lists are run through a distributed database called DNS. This is the same thing that allows looking up a domain name to get the numeric IP address which the routers use to send packets to the correct destination. But the point about it is that DNS works as a general distributed database, and unless someone runs the DNS server wrongly, there is no mechanism to get a list of these addresses. All that can be done is to pick and address and do a lookup. Unlike a regular database, there is no means to do a query lookup like "give me all the IP addresses which are open relays".
In reality, there are sometimes some breakdowns in that security and the blocked addresses can get out. I've acquired one such list myself. But for the most part, spammers do one of two things. They scan the net at high speeds looking for open relays, and they scan through their mailbox which is on the lists to check for good pickings in recent spam they received.
But, hey, I'm a lawyer, right? I'm supposed to be able to solve this kind of dilemma. And there are a few things I could do.
For one, the Danish antispam organization falsified an email header to gain access to my mail server. Illegal access to a computer system is, if not a criminal violation, then a trespass on my private property. As I've discussed previously in this space, one of the novel legal theories now catching on for these kinds of unacceptable accesses to computer systems is a centuries-old tort called "trespass to chattels." At a minimum, I ought to be able to sue the Danish company for the damage it caused me from its illegal access.
They have a legal defense. You actually gave them permission to do the scan. Although you did not know the scan involved the address forgery, their defense is that the practice is the only way to test to see if a mail server is an open relay (that is, if it could be used by a spammer who would forge the address). As mentioned above, this and many other tests like it are standard practice in security testing (and testing for an open relay is simply one form of security test).
This is why when an open relay listing is in the database they will not remove it by periodically testing on their own accord. That would truly be illegal. They require you to consent to the test before they will do it. And again, the standard for these tests is to do exactly every know trick a spammer would try.
Granted, the damage caused by my inability to send an email is likely not terribly significant. You can always pick up the phone, print the message out, and fax it or mail it or just use a different mail server. But in spite of all that, I could probably get an injunction, or least a dollar or two to compensate me for my injuries and establish that I have been wronged.
It is not their test that put you in the list in the first place. It was the fact that they received a copy of spam that some spammer relayed through your server first. It is that spammer that trespassed on your server and caused you the real harm.
The problem, of course, is that the loose organization of individuals who compiled the blackhole list is based in Denmark. Who knows whether the organization is a real legal entity or just some name cooked up by a group of self righteous individuals. However, they do have a domain name, and an IP address, and they circulate their work to ISPs around the world. In other words, there is a group for me to sue. But taking legal action on foreign entities is difficult. I would have to translate my legal documents into Danish. I would have to hire someone in Denmark to personally deliver these translated documents to the entity that I would be suing. That costs time and money.
Those who compile the database are just the messengers. But your real problem is that these guys are just the little fish. The big ones are even harder to reach. They are rumored to be in Bulgaria, an Eastern Europe country formerly behind the infamous Iron Curtain.
But I could sue them here in Los Angeles, California, that much I know. By sending their forged email through my mail server, which is located in my den in Los Angeles, they fulfilled certain California legal requirements that would let me sue them here. The connection to Los Angeles is also bolstered by the fact that I live here and my injury was suffered here.
Of course, all of this is starting to sound like the kind of hypothetical legal conundrum that you might find on a law school exam. Problems like mine often remain hypothetical because the expense of bringing them to trial is so great, and the ability to gain any monetary relief from lawsuits is minimal. That's why the black hole providers have been able to get away with their vigilante justice for so long. For any individual user wronged by their efforts and from what I understand, there are a lot of people in similar situations the costs of pursuing these organizations, which are often located overseas, is too great. These groups of volunteer organizations have no assets to speak of they are volunteers after all and plaintiffs' lawyers are hesitant to take a case without the prospect of a lucrative damages judgment.
And there is the risk that they would win if they were present to defend their practice. They would certainly bring up the point that the original listing was due to a spammer discovering your open relay, and that they received permission from you to test their server.
Before you think that this is all just about me and the fact that my father no longer receives any email from me, there are bigger policy implications for private individuals and companies that take steps to block connectivity. Much bigger.
I've long championed the idea that the Internet should remain largely unregulated by governments. But at the same time, any private operator at an end point in the Internet's architecture can restrict the flow of content to a user. What's wonderful about the Internet is that it enables end-to-end communication from anywhere in the world to anywhere in the world. For all of the problems caused by spam, email is still the most widely used application on the Internet. So the idea that private parties could get ISPs to block some people from talking to other people should be deeply troublesome.
The choice to use the information from blacklists to reject delivery of email in a mail server is something the owner of the mail server would do. This becomes a private property issue. I have the right to refuse any mail into my mail server I wish (except on the basis of the few parameters law now prohibits, like gender, race, religion, etc). I have the right to get my list of IP addresses to block from anywhere I like. If Joe down the street tells me he blocked email using his private little list of IP addresses and it cut out 90% of his spam, then of course I'd like for him to share it with me.
Could there be an issue of libel here? Sure, there could. But it's a clear line between saying "You are a spammer" and saying "Your mail server allowed a spammer (who uses forgery) to send spam to me, and when you gave me permission to test it, I found that by mimicking just what the spammer would do, it was still allowing it."
The Danish blackhole list operators want to block access to computers that might be used for spam, but it's easy to imagine blacklists used for less noble purposes. For example, imagine that the RIAA compiled a list of IP addresses which, it contended, had at some time used peer-to-peer file sharing programs. Because these peer-to-peer systems could transmit copyrighted materials in a way that infringes on the copyright owner's rights, the RIAA could argue, those IP addresses should be blocked. It isn't difficult to imagine that the RIAA could pressure a sufficient number of ISPs into subscribing to this copyright blackhole list and blocking access to their users, or to any traffic emanating from them.
I do worry that the techniques used to reduce and prevent spam could be put to less noble uses. I also worry that facilities that exist on the internet to allow anonymous communications (which some people sometimes need to have) are abused by spammers (there are techniques to reduce that abuse) and in turn blocked by anti-spammers.
Personally, I don't consider the anti-spam movement to be less noble than peer-to-peer file sharing. The vast majority of what is shared on those networks is copyrighted material being shared well beyond the rights of the copyright owners. While I'm not advocating that those file sharing programs be outlawed, or the networks they use be shutdown, I do consider it to be less noble a thing that the effors of the anti-spam community to help keep mailboxes cleaner.
Breaking end-to-end connectivity for any application, whether email or peer-to-peer or the Web, threatens the very thing that makes the Internet valuable. These are matters of principle. Which reminds me I have a lawsuit to file.
It depends on who is doing the breaking. If I break connectivity in my own server, even if I use information from someone else that I choose to use, who offers that information to me freely (I didn't illegally copy it), then what law have I broken? What tort have I committed? Who have I harmed? If it involves my customers in a service I provide to them, then it's a matter of the business relationship between me and that customer. In practice, my customers want the spam blocking since it proves to be very effective against spam.
As to your mail server. It is an open relay, and it needs to be closed.
If a thief enters a building by opening an unlocked door, it is breaking and entering. Merely opening the closed door was breaking, as opposed to the door being wide open. It does not matter if there was a lock on the door or not. It does not matter if the lock was left unlocked. It is still breaking.
Your mail server has a closed door, but it has no lock. You are making the assumption that spammers won't do the "breaking in" thing with address forgery. But they do. What you need is the equivalent of a lock on your mail server. Instead of just checking the FROM line to see if it has your domain name on it, it needs to check something that a spammer simply cannot forge at all. Usually this is an IP address. If you want to be able to use your mail server from other locations, then the IP address is not good enough. There is another method that is used which requires you to log in to READ your mail first. The way that works is when the mail reading login is done, the server notes what the IP address is from which the successful login came, and puts that IP address in a list which is valid for sending mail for some period of time, say maybe 30 minutes to an hour. Thousands of people use this technique successfully. It's typically called "SMTP after POP" (in reference to the POP protocol used to read mail in most cases).
The following has a number of useful links to help in testing and closing an open relay:
Oh, and one more thing, telemarketing does not add to the GDP at all. If anything it takes away from it. There is no creation of value in telemarketing (or any marketing for that matter). If we don't spend the money on something the telemarketers are hawking on us, what do they think we're going to be doing with it? Burn it?
I won't be crying at all the day (not likely to ever happen for the very reasons you describe) they make telemarketing illegal. I will definitely be celebrating. Then I'll be worrying about the consequences of that stupid law.
Slashdot Mirror
It may be that your apache is statically linked. Or it may be that apache records the version of OpenSSL at compile time rather than at run time (dumb). Recompile anyway; you need the practice.
What do you mean HTML is a hack? What do you mean I can't change it? That's nonesense. I can change it.
A <font> tag does provide information. The most useful information is relative size. The tag <font size=+1> says the text following is larger than the previous.
I've seen the sliced up images. I think they are doing that because they are too lame to do image maps. It's certainly not needed to do layouts even with tables, unless what they want is a layout with a non-rectangular image.
But let's not dwell on your lack of ability or understanding of HTML. I will give you the fact that CSS most certainly is an improvement, and is needed now. The problem is it is useless if it is tied to browsers that not everyone can use. The issue is not about getting people to upgrade browsers; it is about getting enough variety of browsers so people have choices that fit their needs. Then we don't have to have these obese monsters (no pun intended ... but it is fitting) for browsers, and can find something better that works the way we need it to work, and conforms to standards correctly as well.
Forward compatibility is pointless if you don't have a path into it for everyone.
Since I don't do things that involve those kinds of higher level languages, my first encounter with "UML" actually was "User Mode Linux". So when I see "UML" that's what I think of. In fact it took a while for me to figure out why it all those developers seemed to be interested in using User Mode Linux.
I propose we adopt a new meaning for "UML" as "Ubiquitous Mysterious Logic" or "Ugly Men Lurking".
Tell it to Zeldman. I'm not the one who is doing CSS. I've put that off until at least 98% of users are using a browser that does CSS correctly so I don't have to do hacks.
There's nothing wrong with making it look good, too. In fact I'd like that, too. The problem is developers like Zeldman prefer to make it look good only if you are using their preferred browsers, and look bad on others.
Web sites can be made to look good w/o CSS. Those who promote CSS point out that one advantage of CSS is not having to put in all those HTML tags. That's why they make web sites that have CSS and no more HTML than is needed with CSS. The result is a web site that looks ugly on NS 4. It's not me that wants the feature as I can make web sites that don't need CSS. But developers like Zeldman are the ones demanding the feature, and insisting that people upgrade their browser for his benefit, disregarding other issues that make upgrading not practical for many. I just want web sites to look as good as HTML can do it.
And I don't use HTML hacks, as my HTML comes out looking good and it's the same exact HTML for every different browser. The problem sites are ones that depend on CSS (or in other cases, depend on Javascript, Java, Flash, etc). Those that are done in HTML look fine.
CSS didn't even exist way back then (it was being talked about as a great solution). Even today, CSS is broken in Netscape 4, and not everyone can, or is willing, to upgrade beyond that, yet. As soon as some leaner browser shows up, I might. As soon as 99% of people have upgraded, then I will switch to using CSS.
I have P3 800, so shouldn't it be even faster for me? You'd think. Maybe it's just the rapidity of my vision. Even on this P3 800 I can see a difference in the speed between NS 3 doing rendering and NS 4. It's just that on my previous Celeron 333 the NS 4 speed was not tolerable, so I used NS 3. I remember doing some timing tests way back then between NS 3 and NS 4 on some complex HTML tables, and NS 4 was 22.7 times slower than NS 3 at rendering them (the page was made sufficiently complex that it took NS 3 a total of 3 seconds to render, while NS 4 took 68 seconds. That was just quantifying what I was actually seeing visually on much less complex pages.
NS 4 is still slow at rendering multi-block GIFs. NS 3 did them nearly instantly. Maybe this is because NS 4 has too much per-block startup overhead. Or maybe it is because NS 4 mistakenly thinks a multi-block GIF is an animated GIF and artificially inserts a delay. Download this image [184,565 bytes] to disk first, so you aren't seeing any network delay, then test some browsers on it referencing the local file and see how quickly it renders (BTW, certain older builds of Moz and Konq won't even render it at all). You'll need to do this on X running in 24 bit or more per pixel, with a video card that can do that (what can't these days).
If people come to the site for just content, then why are the designers jerking around with the gloss and flashiness? I do get colors on www.alistapart.com, but the layout is horrible. I don't have that kind of trouble doing layout that works on a wide variety of browsers, and I don't even check the browser type to customize. If you want to know what features are only available in the newer browsers, maybe you should ask the developers of www.alistapart.com what it is they use that won't work on anything but the newer browsers. The features I use seem to be working fine on older browsers.
I just see too many web sites that appear to be more intended to show off the designer's l33t w36 sk1llz that an ability to stick to information and readability. That turns me off from many sites quickly. Things like pointless use of flash will do that. Things like using Javascript to implement a hyperlink will do that.
Oh, and yeah, I am guilty of sticking in some of my own l33t HTML sk1llz on linuxhomepage.com, but at least it works everywhere.
His "dump all the crappy hack and bloated code" is still making a web site that won't work in NS 4. Take a look at www.alistapart.com in NS4. It displays, but it's crap. It could be made to work and even do so in the same output without having to distinguish the browser type. Yes, it would take more bytes transmitted to do that, possibly twice as many in some cases. But I already debated the issue last year on that site I just pointed you to, and that was indeed their purpose, to get me to upgrade my browser to the very newest so the features they wanted to use would work.
We disagree on which browser is the current best. Mozilla is not working out for me. It is noticeably slower than NS 4. And NS 4 was too slow for me compared to NS 3 until I upgraded CPU and RAM. Mozilla will have to wait until my next hardware upgrade. Mozilla also has some bugs whereby it won't even function correctly in my desktop system at all. NS 4 had problems but I found ways to work around it. Those ways don't work with Mozilla, and until I have the time to deal with it, it won't be my default browser.
Yes, times do change, and standards do change. But everything is according to the standard. The site is PHP driven, and PHP doesn't seem to handle DTDs. But it works now because the default HTML still works.
When will features in HTML be obsoleted from the standard and removed from browsers? I don't know yet. That time has not come. Hopefully there will be an adequate transition period to a decent browser that can do the new replacement features before expecting everyone to get off the old browsers. Unfortunately, the browser situation is getting worse, not better. What browser no longer accepts HTML 4.0 or even HTML 3.2? If you find one, then that browser has jumped the gun because HTML is not yet depricated. But those sites you mention using XHTML and CSS come up a total mess in NS 4 because unlike the goals for XHTML and CSS, they don't work in NS 4 (CSS is there, but it's horribly broken). When a new browser can do some of the things (including X standards) that NS 4 can do, then let's talk about browser upgrades. But you have to help try to get decent browsers implemented and deployed if you don't want to have to face people who are having to make hard choices to find the least-broken browser. I don't deny NS 4 is broken. It just happens to be the least broken right now. Konquerer appears to be poised to edge it out soon. Maybe the next version will. One can only hope. Still, Konquerer is more obese than I would like.
If you at least admit that browser programmers are doing a bad job of getting standards out to users, then we'll at least have something to agree on.
The same point remains. His server based authentication on the forgeable domain name. Some spammer did discover his server worked that way. I see spammers frequently testing my servers exactly that way, by trying to use local domains, including the name of the server itself, to see if any of those have been blindly (and stupidly) configured to allow relaying. They don't get through mine because I don't configure it to trust forgeable information. And it is easy for a spammer to try this because he already has lots of email addresses from which he can do MX lookups to find mail servers. If that server is stupidly configured, that email address stands a good chance at being allowed to relay by merely being the MAIL FROM or the From: address. If that doesn't work, they try the reverse DNS of the mail server IP address as the hostname part. Some spammer did that on Bret Fausett's mail server, and at least one of those spams went into a spamtrap mailbox, resulting in the initial listing. Then when he discovered the situation and read about how to get it unlisted by being tested, he asked for that, probably without yet knowing the mechanisms the spammer used, and therefore that the relay test would have to use. When he discovered that the test mail forged his own domain name, he got into a tizzy and started writing his article.
It doesn't matter what the mechanisms actually are. There are many others that some mail servers will be fooled by to allow relaying. If a spammer can use it to relay spam, then a relay tester must include that mechanism in the testing to determine if that particular mechanism has now been closed.
They need to use the right tools for the job. If the job says make it look exactly like this paper brochure, then they have to choose something like a giant image or PDF. Sure, it can be approximated in HTML, but they need to not whine when it fails because a different environment with a different look and feel ends up showing it differently. Do you know how many pixels wide an 18pt font is on my display?
As for XHTML and CSS, they might not be there. Sure, feel free to use them where they are, but don't expect me to have to deal with the problems of some new browser just for your benefit. What if the CEO's cousin's computer ... the one that is so old that it only has 256 colors ... can't even run the new browser at all, and therefore has no XHTML or CSS?
If the designer loses his job working for a PHB, at least now he has a chance to get a decent one. Sure, I can understand he would rather keep the job and make a real honest attempt to accomplish the task in a way that keeps everyone in the company and West BFE happy. But who is to blame when someone reports to the company that the site shows up lousy on some other computer?
You can design for the middle or you can design for the CEO's cousin. If management makes the decision, then the web designer is just doing what he can with what he's allowed to do. The real culprits are the ones that don't allow the web designer to design it as content.
Do you see any XHTML or CSS on http://linuxhomepage.com/? Does it look on your browser today like it looked in Netscape 3 back when I originally designed it?
They won't know, unless they were expecting it in the first place, or the sender found a means to tell them, or they discovered it in the list I make available to them (a scan of server logs filtered on their email address for the past 90 days). I do want to make that work better.
The consistency should be between different web sites when viewing content on the same browser with the same OS under control of the same theme. Don't assume that <h1> gets you 18 point times new roman, even on Windows in Explorer, because as you mention it being the default, it is certainly changable, and may well not be the default when you change browser or even OS (though such changes should only happen when you change whatever is appropriate for a look and feel change).
Many web designers come from the graphical arts field, which is full of brochure-on-paper production. They get to pick (or are told specifically) the paper size, font theme, color theme, etc. That's a fixed environment and quite easy to work in. Now along comes the web, and it isn't that way anymore. Some people have smaller browser windows, or are limited to 640x480, or to 256 colors. Others have 1600x1200 and 16 million or even 1 to 4 billion colors. Some have 28.8k or worse download speeds and others have T1 and better. Some have plenty of CPU power and RAM to run the latest browser, and some are stuck with an older browser on a hand-me-down computer. The world is full of diversity and these brochureware designers still can't cope with it. They could build the whole site in a big image file and piss off the 28.8k dialup users and be scrunched over on the left side of the 1600x1200 users. Even with advances in new protocols and formats, so many problems remain that their assumptions still don't work. They might be better off with PDF, which was designed more for the kind of thing they are doing.
I make it known to my customers that inbound mail is subject to spam filtering. I even make available to them a list of all the spam (what server it came from, and what MAIL FROM had in the SMTP) attempts that got blocked. If they discover something they want is getting blocked, I can whitelist it. And I have done so already in a couple of cases. Being small, I can do this myself. Eventually I'll have to automate this so I can grow. The plan is to give each user a choice which blacklists to use, and give them their own private blacklist and whitelist, and the ability to automatically allow inbound mail from anyone they send outbound mail to. Is that reasonable enough for you?
You're speaking of SPEWS. And the whole point is that because spammers move around to evade blocking, the front line of defense is the ISPs. Blocking just the spammer alone is a futile effort. Getting spammers disconnected doesn't work at too many of the larger ISPs because they would rather take spammer money than keep non-customers happy. So of course the logical way to go is to block ISPs. And SPEWS doesn't immediately block a whole ISP (unless they are so small they have less than a class C network or something). They raise the pressure gradually so the ISP gets the message before too many customers are impacted.
The only way these ISPs are going to respond to dealing with spammers is when they are forced to decide between the revenues of spammers vs. the revenues of non-spammers. Absent that force, they just keep spammers online and the internet suffers. With that force, most will eventually see the financial issue and make the decision (and yes, some have decided to go with spammers and have gotten 100% blocked ... and rightly so).
As long as you stay with an ISP that supports spammers, then you are, every time you make a payment to them, saying "It's OK for you to keep spammers online because I'll keep paying you even though it causes me grief". If it's so costly to change ISPs for you, then maybe you should have done due diligence in the first place to discover what their real intentions are with regard to spam.
I already ignore spammers. They have not gone away. That idea is stupid because there will always be some small percentage that don't ignore them, and it only takes that small percentage for them to get more money out of spamming than they put in. Then while they spam, they force us to do deal with all the junk. Even if you just count the 3 seconds it takes to delete each piece of spam at a typical low end wage of an office worker, spam costs over a billion dollars a year (at its current rate) just in lost worker productivity in the USA. Now that Europe has caught up and edged past the USA on internet users, I'm sure the figure is nearly as high there, and will soon be higher. And this doesn't count the time it takes for staff to manage the situation and clean it up.
What's a little guy to do? For starters, try convincing your ISP to stop supporting spammers. But if you say to them "because it costs me so much money to move to another ISP, I will stay with you no matter what you do", then why would they give up the revenue from the spammers just for you? Maybe what you should do is figure out why and how you got yourself into a mess where you can't change to another ISP?
One thing web designers need to do is get off this "consistent look and feel" drug. It doesn't work that way because it is not supposed to work that way. For example in HTML, the <h1> tag starts a header at level 1. That doesn't mean the enclosed text has to be rendered in some particular size or some particular font or some particular color. It just means it is a header at level 1. The look and feel is up to the browser or the windowing system of the user. Even applications that get ported between different systems (for example on both Microsoft Windows and on Macintosh) are supposed to have the look and feel of the respective system so the user can access it in their own familiar way. When I go from one web site to another and find the buttons are always different, that is where consistent look and feel has failed.
I've heard this rant so many times. Most ISPs, such as Earthlink, allow users to turn off the spam filtering. Got one that doesn't, and won't whitelist for you? Then move on.
One thing we do not need to be doing on the internet is encouraging incompetence by continuing to pay them. If they don't, or can't, run spam filtering for you the way you want it done, then yes, it is time for you to move on. That's why we have competition and free trade.
If webmasters want to see XHTML become popular, they they should (encourage someone to) write a lean-and-mean web browser that does XHTML correctly and without the obesity and sluggishness of most of the recent browsers. If XHTML is as good as they say it is, then a lot of other junk like Javascript, Java, and Flash won't be needed, so leave those out of the browser to keep it lean-and-mean ... and secure, too.
Older browsers are a problem. But newer browsers are a problem, too. Featuritis (also known as creaping featurism) has grown to be a plague. Browsers are just too stuffed with features that in effect try to make them be a platform unto themselves. The ideal browser will be a lightweight "traffic cop" that manages the various pluggable rendering components (document interpreters) to create a seamless appearance, and won't be all that much different than a windowing system. Then I can remove that Javascript plague with a simple "rm" command in the render library directory.
99.9% of new web browsers are obese. What this guy essentially wants is for me to upgrade my browser so he can use his fancy tricks. But he doesn't want to help put pressure on browser makers to get their browser under 8MB.
If the list operator who tested your mail server did not test it by using the proper practices, which includes doing everything that spammers are known to be doing, or known to be capable of doing, then it would be the list operator who had failed to properly and correctly test your server. If it had been marked as closed, because of that, when in fact it was still open, then it would be the list operator who would have been negligent.
Security practices, and spam prevention is a form of security practice, do include performing tests that mimic what the security prevention is supposed to prevent. Your mail server is supposed to prevent relaying of forged addresses. So you have to do forged addresses to test that facility.
The only thing the list operators did wrong that I can see is they failed to get your signature in writing on a piece of paper that explained it to you. Had they done so, that piece of paper would have stated that they would be performing a test that adheres to current best practices in security testing, and that test would include every form of forgery and trickery known.
The ends not only do justify the means, they are also absolutely required!
Also, some mail server software is defective in ways that certain types of attempts, which spammers might try, and therefore have to be tested in a thorough test, could cause that defective software to fail, and may result in damage to your mail server. If that happens, your remedy should be with the maker of the defective software, unless the defects were documented and avoidable by proper configuration.
And if you want to have a private dialog about this, I am willing to explain it in more detail if you need that. I am not a lawyer, so I can't give it to you in purely legal terms, but I can certainly give you some real life analogies. You can find my email address a number of ways, such as the domain registration of one of my web sites.
Your concern about failing to circulate blacklist removals is misplaced with regard to DNS based blacklists. The data expires in a finite amount of time from the cache, and removal processes are working pretty damned good. I've watched a number of notices posted on news.admin.net-abuse.email asking to be removed from the SPEWS list, and I check out whether they have fixed the problem or not. In most cases I find that the data had already been removed from SPEWS by the time I checked that (so now I check SPEWS first before checking to see if the problem is fixed).
Private blacklists are a problem because there's virtually no way to track them all down and get removed from everywhere (once you fix the problem). That's why we need central DNS based blacklists. But what we also need is to shield these central lists from stupid lawsuits from people who refuse to fix their problems or simply don't have a clue. Those who even so much as threaten to sue the list operators instantly get their IP addresses and domain names put in thousands of private blacklists where no one even looks to see if anything is ever fixed. And when they end up shutting down the central lists, they make things worse due to all the private lists. That's the primary reason SPEWS is so secret. Sure, it comes across to people who didn't know about it as a "Star Chamber" thing. And I didn't use it for several months until I verified it actually works to list what needs to be listed, and removes things when fixed.
Here is what I wrote to this guy back on July 25 when the article had just come out. I never received a response from him. Was he totally embarassed by his idiocy once it was explained to him? I guess so.
<lettertext>
I just read the article you wrote on New Architect Magazine entitled "Blind Vigilantes; Blackhole lists offer dark prospects". I feel you have missed certain points in your analysis, and as a result, you misunderstand what is going on. That's OK, because the majority of network administrators still do, too. As a lawyer you would not be expected to know this kind of stuff. You clearly know a lot more about it than the average lawyer. I'm writing in hopes of filling in the gaps. I sincerely hope you have the time to read this. It's long, but I think this is important.
First of all, I use these blackhole lists myself, so it is possible that your reply to me could bounce back. I can override it if I know the IP address of your mail server. But I won't know it until there is a server log telling me about it bouncing. What I'll do is get your IP address at that time, add it to the exception database, and you can repeat the reply later on. Or you can send me mail from Hotmail, which I believe is not blocked anymore.
I want to fast forward to the point in your article where I think the main misunderstanding is:
One of the methods spammers use to send their mail through a mail server configured like yours is to do exactly what you are complaining about. I see upwards of 10,000 of these a day on my servers. The spammers have these massive lists of email addresses, quite many of which are valid. What they do is look up which mail server those users would use, which is not hard because that's exactly what the whole system is designed to be able to do. Every delivered piece of email had to do that. Once they have this information, then they forge that user in their FROM line and start sending mail to the user's server. In the case of a server set up to test only the domain name in the FROM line, it works, and the spam message gets sent on its way.
That's why your mail server is considered to be an open relay, because it is possible for a spammer to use it, despite the fact that they are doing something illegal such as forging your domain name. If it lets a spammer forward mail, it's an open relay.
It is standard practice for every program (there are several available) which does the open relay tests to try dozens of different ways to fool a mail server into forwarding mail. Forging the domain name of the users of that server is one of the simpler tricks. There are some that are more complicated. These programs are simply doing exactly the same thing that a spammer would do. It's the same principle used by security test programs which test whether or not a computer can be broken into. They have to pull all the punches a hacker might try. Otherwise such programs will fail to detect a flaw and the program itself will be worthless.
I periodically run tests on all my mail servers to make sure I have not accidentally configured out the relay controls. I watch these tests take place, and they do this forgery exactly as expected.
Actually, that is not true. Read on and this will be explained.
Last year, one of my client companies, a local web hosting business, had a case of one of their customers running a spamming operation right from the server they were paying my client to use, in violation of their AUP. The customer got cut off, and my client asked me to help him clean up the mess. In so doing, I obtained a copy of not only the spamming software (a special version intended for running from web servers), but also a copy of a big list of about 1.5 million addresses.
There was something very interesting in this list. The first 1000 or so entries were email address that were familiar to me. They were OTHER SPAMMERS. That's right, other spammers have their own names in these lists. What that means is if any spammer discovers an open relay, the others find out about it fairly quickly. The "spammer network" as I might call it is very well connected. They all see the successes of the others. And much like wild animals on the African Savannah when one makes a kill, the others circle around to take their own bite out of the carcass. That's what is happening to your server.
The anti-spam group have some of their addresses on these lists, too. That's how they first find out if your mail server is an open relay. They get spam that some spammer who found it relayed through. That's how you were first put on the list.
The blackhole lists are run through a distributed database called DNS. This is the same thing that allows looking up a domain name to get the numeric IP address which the routers use to send packets to the correct destination. But the point about it is that DNS works as a general distributed database, and unless someone runs the DNS server wrongly, there is no mechanism to get a list of these addresses. All that can be done is to pick and address and do a lookup. Unlike a regular database, there is no means to do a query lookup like "give me all the IP addresses which are open relays".
In reality, there are sometimes some breakdowns in that security and the blocked addresses can get out. I've acquired one such list myself. But for the most part, spammers do one of two things. They scan the net at high speeds looking for open relays, and they scan through their mailbox which is on the lists to check for good pickings in recent spam they received.
They have a legal defense. You actually gave them permission to do the scan. Although you did not know the scan involved the address forgery, their defense is that the practice is the only way to test to see if a mail server is an open relay (that is, if it could be used by a spammer who would forge the address). As mentioned above, this and many other tests like it are standard practice in security testing (and testing for an open relay is simply one form of security test).
This is why when an open relay listing is in the database they will not remove it by periodically testing on their own accord. That would truly be illegal. They require you to consent to the test before they will do it. And again, the standard for these tests is to do exactly every know trick a spammer would try.
It is not their test that put you in the list in the first place. It was the fact that they received a copy of spam that some spammer relayed through your server first. It is that spammer that trespassed on your server and caused you the real harm.
Those who compile the database are just the messengers. But your real problem is that these guys are just the little fish. The big ones are even harder to reach. They are rumored to be in Bulgaria, an Eastern Europe country formerly behind the infamous Iron Curtain.
And there is the risk that they would win if they were present to defend their practice. They would certainly bring up the point that the original listing was due to a spammer discovering your open relay, and that they received permission from you to test their server.
The choice to use the information from blacklists to reject delivery of email in a mail server is something the owner of the mail server would do. This becomes a private property issue. I have the right to refuse any mail into my mail server I wish (except on the basis of the few parameters law now prohibits, like gender, race, religion, etc). I have the right to get my list of IP addresses to block from anywhere I like. If Joe down the street tells me he blocked email using his private little list of IP addresses and it cut out 90% of his spam, then of course I'd like for him to share it with me.
Could there be an issue of libel here? Sure, there could. But it's a clear line between saying "You are a spammer" and saying "Your mail server allowed a spammer (who uses forgery) to send spam to me, and when you gave me permission to test it, I found that by mimicking just what the spammer would do, it was still allowing it."
I do worry that the techniques used to reduce and prevent spam could be put to less noble uses. I also worry that facilities that exist on the internet to allow anonymous communications (which some people sometimes need to have) are abused by spammers (there are techniques to reduce that abuse) and in turn blocked by anti-spammers.
Personally, I don't consider the anti-spam movement to be less noble than peer-to-peer file sharing. The vast majority of what is shared on those networks is copyrighted material being shared well beyond the rights of the copyright owners. While I'm not advocating that those file sharing programs be outlawed, or the networks they use be shutdown, I do consider it to be less noble a thing that the effors of the anti-spam community to help keep mailboxes cleaner.
It depends on who is doing the breaking. If I break connectivity in my own server, even if I use information from someone else that I choose to use, who offers that information to me freely (I didn't illegally copy it), then what law have I broken? What tort have I committed? Who have I harmed? If it involves my customers in a service I provide to them, then it's a matter of the business relationship between me and that customer. In practice, my customers want the spam blocking since it proves to be very effective against spam.
As to your mail server. It is an open relay, and it needs to be closed.
If a thief enters a building by opening an unlocked door, it is breaking and entering. Merely opening the closed door was breaking, as opposed to the door being wide open. It does not matter if there was a lock on the door or not. It does not matter if the lock was left unlocked. It is still breaking.
Your mail server has a closed door, but it has no lock. You are making the assumption that spammers won't do the "breaking in" thing with address forgery. But they do. What you need is the equivalent of a lock on your mail server. Instead of just checking the FROM line to see if it has your domain name on it, it needs to check something that a spammer simply cannot forge at all. Usually this is an IP address. If you want to be able to use your mail server from other locations, then the IP address is not good enough. There is another method that is used which requires you to log in to READ your mail first. The way that works is when the mail reading login is done, the server notes what the IP address is from which the successful login came, and puts that IP address in a list which is valid for sending mail for some period of time, say maybe 30 minutes to an hour. Thousands of people use this technique successfully. It's typically called "SMTP after POP" (in reference to the POP protocol used to read mail in most cases).
The following has a number of useful links to help in testing and closing an open relay:
</lettertext>
Oh, and one more thing, telemarketing does not add to the GDP at all. If anything it takes away from it. There is no creation of value in telemarketing (or any marketing for that matter). If we don't spend the money on something the telemarketers are hawking on us, what do they think we're going to be doing with it? Burn it?
I won't be crying at all the day (not likely to ever happen for the very reasons you describe) they make telemarketing illegal. I will definitely be celebrating. Then I'll be worrying about the consequences of that stupid law.