Slashdot Mirror
Ask Jonathan Zdziarski
You may recognize the name Jonathan Zdziarski from a recent Slashdot book review of his book Ending Spam. Aside from his DSPAM spam filter Jonathan has also contributed several other projects to the open source community under the GNU General Public License. These projects include Verizon-Compatible SMIL Multimedia Gateway, The Reactive Automated Blackhole List Server, Apache DoS Evasive Maneuvers Module, and several others. Want to know how to effectively contribute projects to the open source community? Curious to ask another programmer about his history? Now is the time to ask. Moderators will select the top few questions that we will forward on to Jonathan sometime tomorrow. The answers to the questions will be displayed next Tuesday when we will encourage Jonathan to participate in the discussion as time permits.
Seeing how Johnathan has put much of his time and effort into Open Source projects over the years, it would seem he is a good canadate for this question: What do you think about the proposed change to the GPL with the upcoming GPL 3? Is it a welcomed breath of fresh air to the Open Source Community, or will it just be a reiteration of the previous GPL? What are your thoughts and comments on the GPL 3?
--
Do you get those pesky Nigerian 419 emails? Post them here, and watch the database grow! : http://urgentmessage.org/
Most antispam software seems to be fairly reactionary - wither it is based on keyword patters, urls, sender, ip, or the checksum of the message a certain amount of spam has to first be sent and identified before additional messages will be tagged and blocked. Spf, domainkeys, etc... requires a certain percentage of the Internet to adopt before they will be truely effective.
What do you see on the horizon as the next big technique to battle spam? How will this affect legitimate users on the Internet?
"The similarities of sysadmins and drug dealers: both measure stuff in K's, and both have users."
Mr. Zdziarski, it appears as if you are a supporter of use of statistical methods to filter out spam. But these filtering methods have limitations, in that there are ways of getting around these filters. Since human beings can recognize spam better than any software filter, do you not believe that more emphasis should be put on developing software that facilitates DIY spam filtering?
Have you noticed any decrease in the amount of spam since a few of the hardcore spammers have finally been prosecuted? I always wonder if scare tactics will work against these guys, or if they will just move their colo to some small country offshore where it becomes harder to press charges.
I guess the more serious version of this question is the tradeoff of precision and false negatives vs. overkill and false-positives. For instance, my email provider lets me pick country-blacklists, so I reject all email from China, Korea, and Nigeria, where I don't know anybody, and Japan gets accepted with extra filtering, because I know a couple people there who normally don't send me mail - it's not quite a nuke-Asia-from-orbit approach, because people who actually do want mail from people in China can accept it, but people who don't can reject it all and lose the occasional message from a friend at a cybercafe.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Jon, your acheivements thus far are impressive. I am personally most impressed by your adherence to Open Source Solutions in a corporate environment.
I myself have had numerous interactions with less-than-technically-savvy management-types. Any time I bring up solutions that are quite obviously a better technical and financial choice over software-giant-type solutions; conversation seems to hit a brick wall. The ignorance of these people on such topics is astounding, and I find many approaches I have tried seem to yield no results in the short term. "Well, yes, your example proves that we would save $500,000 per year using that Open Source solution. But We've decided to go the Microsoft (or what-have-you) route."
With your track record, I can only assume you have found some ways to overcome this closed-mindedness.
I would greatly appreciate any input you have on this; from the perspective of someone who has overcome this obstacle before.
A couple fans told me that my last journal entry was mint; give it a shot. Hope you like.
How do you deal with spam checking software causing a delay at the point where you do the spam filtering? As communication backup becomes more important in the business place you have some companys dealing with literally millions if not billions of emails a day. Even an efficent filter will take to go through that many emails, How do you deal with this?
I have two questions:
1. In your new book, you basically state that Bogofilter is not a bayesian filter, which was news to some of the Bogofilter people I have spoken to. Can you explain why you feel that Bogofilter is not a bayesian filter?
2. Bayesian filters have been around for some time now but there still seems to be no standardized testing methods for determining how well filters work in comparison to one another. Do you think that comparitive testing would be useful and if so, how should it be performed?
Thanks Jonathan.
Jonathan,
I develop and manage a lightweight Open Source Application that's used to send announce only and discussion mailing lists, similar to the Mailman and Majordomo projects. It's very popular and has a loyal following.
What advice do you have as a developer of this program to:
* Help my users send legitimate messages (either by education (specifically) or by programming techniques)
* Help Spam Filtering Software check the messages my program sends out for possible abuse
* Be a part of the solution to sending legitimate messages to many people, rather than perhaps be part of the problem.
I understand that any tool can be circumvented and abused and I do believe context always plays a part in how to judge something as Good or Bad. I'm sure like many different types of software, Spammers are a problem for my business as well.
I find myself in an interesting position, where I can change how many email messages are sent out. If I can send "better" email messages that are not filtered as spam if they are legitimate and can stop possible abuse of my program, I can help in a solution to people who would like to send out announce only and discussion email messages.
Thanks for your time.
Dada Mail - Program, Art Project or Absurdity?
The SMTP standard that we use for mail transfer was developed in the late 70's - early 80's and has, for the most part, never been updated. In that time period, the idea of hordes of spam flowing through the net wasn't even considered.
It has always been the most obvious solution to me that what we really need is SMTP 2.0, where a server only accepts mail from a user that can authenticate themselves with a name and password. A server can also accept mail from another server, but only for mail directed at legitimate users on it's system. Mail servers would have to register with a central authority, and must include their active IP address in that registration. Any attempt to deliver mail from an unregistered server is bounced.
Wouldn't this simple fix stop 99% of spammers in their tracks? Isn't it about time we updated the SMTP standard?
Life, the Universe, and Everything... in my image.
For example, certain spam blacklists would censor more than was strictly necessary (a subjective opinion, I realize) to block a spammer -- sometimes blocking a whole Class C to get one individual. This would cause other innocent users in that netspace to have their e-mail to hosts using the blacklists silently dropped without any option of fixing the problem besides switching ISPs.
This is an extreme example, but most anti-spam approaches have the following characteristics:
Recently I had to fix an installation where daily messages from a particular host stopped appearing in a mailbox. This system was connecting with an ISP that had offered no spam filtering and had been using a client-based Bayesian classifier with great success, but suddenly the mail coming into the system had scaled back by a factor of ten. Sure enough, the ISP installed a server-based spam filter which took out most of the spam and a good deal of the legitimate mail -- they had a (not well publicized) means of accessing the account settings and turning off the filter, and a holding tank for mail classified as spam, but beyond the last two weeks everything was thrown out.
I'm curious about what you think about server-based approaches vs. client-based approaches to spam classification and filtering and if, maybe, the cure is worse than the disease.
Try not. Do or do not, there is no try.
-- Dr. Spock, stardate 2822-3.
This is arguably out of scope for this interview, but I still feel it's something many Slashdotters would be interested in hearing about.
On your webpage you have an essay describing your Christian beliefs and why you have them. You say many things that most Slashdotters (and nerds and scientist in general) regard as utterly ridiculous. You think the earth is no more than 10,000 years old, you think Christianity is logical, you regard the Bible as a historial document, etc.
No doubt you are aware of the fact that most nerds disagree with you on these things. Indeed, they might even consider you "crazy" for holding them.
Without going into the truths of the beliefs in question, which I'm sure will be debated enough in the Slashdot thread anyway (and I hope you'll join in), what do you think the reason is that so many scientists, nerds and people otherwise rather similar to you think your beliefs are obviously incorrect? Do you think they are all deluded? Do you agree that there might be a possibility that your beliefs are not rational (again, without going into whether or not they are so)?
Best regards,
an AC