Mailing List Managers?

greyrax asks: "So I'm trying to convince someone to go open source for their list manager. They're about to upgrade to an expensive proprietary solution. With about 400,000 subscribers to this newsletter, a database back-end would certainly be helpful. Bounced address management and easy unsubscribes is important (I've read Smartmail vs. MailMan comparison here). Virtual host support and a web interface are desirable. Any thoughts from the /.ers since this thread last year would be appreciated."

Comments

web interface...

[3-State Bit]

one thing i hate is having to "e-mail with subject \"help\" to receive a list of possible commands", each of which, I gather, includes sending an e-mail with a certain subject, then receiving an e-mail in confirmation of it. Blech!

Whatever you use, make sure it has a clean web interface, and configure it to include a link at the end of each e-mail sent to the group to the effect of "Click here to change or configure your mailing list membership."

I really like Mailman...

[drdink]

Although the largest list I've ever served with Mailman was 116 members, I really think Mailman has potential to do what you want. It has all the features you mentioned, though the database backend isn't SQL and I'm assuming that is what you meant.

Re:I really like Mailman...

[Fweeky]

Can you turn off it's silly password authentication stuff and make it use email like every other MLM in existance?

That monthly subscription "reminder" is the second most braindead feature of any MLM I've seen - the first being some Win32 one which stripped Message-Id. If every list I was on sent them, once a month I'd get a flood of 51 "reminders" all containing a plaintext password.

Re:I really like Mailman...

[KyleCordes]

Mailman does not qualify for the criteria in the initial posting, with its silly unsubscribe process.

It also has limited support for announce-only lists, a kind of list I happen to need.

I also prefer web-based tools for subscribe/unsubscribe; in the modern era of users with widely varying technical ability, relying on people to figure out how to send email commands is unwise (for a large list).

Re:I really like Mailman...

[drdink]

Yes. All you have to do is comment out the crontab entry that runs at the start of each month which sends those out.

Re:I really like Mailman...

[Fweeky]

Yes, but can you remove the need for a password entirely?

Re:I really like Mailman...

[drdink]

The password is necessary for the web interface. It is what allows you to login and modify your settings via the webpage. It is what allows you to access private list archives. The password is the authentication replacement for sending a "CONFIRM" e-mail for everything. You wouldn't want to have to wait for a confirmation e-mail to modify your settings on the webpage or to access private list archives.

Re:I really like Mailman...

[Fweeky]

No. The password is only required if you want to use the web interface regularly enough that getting a cookie through email is impractical.

I extremely rarely, if ever, use or even bother looking at the web interfaces for any of my mailing lists. I don't want nor need passwords for them. My one exception is the password for my own Ecartis setup, and it doesn't pointlessly remind me of it every month.

You wouldn't want to have to wait for a confirmation e-mail to modify your settings on the webpage or to access private list archives.

Yes, I would. If I'm unsubscribing, it's a once only thing and I do not need a password; a couple of emails is perfectly sufficient. If I'm going on vacation, that's once or twice a year (well, decade in my case, but nm) and it's also perfectly sufficient.

Should I decide I'm going to be using authenticated sessions with a MLM enough to warrant getting a password, fine; Ecartis lets me do that. There's a reason I don't bother in all cases but that of my own Ecartis managed lists, though; MLM's are not high maintainence for most users and passwords are therefore an unnecessary irritant.

Qmail

[m0rph3us0]

I think you'd want something like EZMLM and the manager software for it, it's not the prettiest thing but you can make an interface for it pretty easy. as long as the program can create directories and add to files you're interface is done. gotta love Qmail's config system.

Re:Qmail

[Christopher Cashell]

I would have to agree with this.

ezmlm has the advantage of tight integration with QMail, and extremely impressive speed. One suggestion I have with it, though, is to make sure you include the ezmlm-idx patch. This adds a few very important features, and greatly enhances ezmlm's usability.

I've run a couple of very large mailing lists with it and even under heavy traffic, it held up like a champ.

ezmlm supports a couple of different database backends for storage, although even without them it works remarkably well. I don't remember the name of it, as I don't really use it, but there is a web based mailing list management program available, as well.

And before anyone complains about the license of QMail/ezmlm, yes, that sucks. The license is a royal pain in the butt, as it doesn't allow direct distribution of modifications, only patches. It still works though, and works really well.

Re:Qmail

[Electrum]

And before anyone complains about the license of QMail/ezmlm, yes, that sucks. The license is a royal pain in the butt, as it doesn't allow direct distribution of modifications, only patches. It still works though, and works really well.

qmail does not have a license and does not need one:

http://cr.yp.to/softwarelaw.html

``What does all this mean for the free software world? Once you've legally downloaded a program, you can compile it. You can run it. You can modify it. You can distribute your patches for other people to use. If you think you need a license from the copyright holder, you've been bamboozled by Microsoft. As long as you're not distributing the software, you have nothing to worry about.''

Re:Qmail

[Anonymous Coward]

I would completely disagree with this. ezmlm and i completely did not get along. ended up switching to postfix and mailman ... works like a dream ever since .. and i get far fewer questions about list management

Re:Qmail

[embo]

I would add a hearty vote of support for this as well. ezmlm on qmail should have no problems scaling to whatever you need for capacity. There are web interfaces available, and it automatically handles bounces, unsubscribing people after X number of bounces.

qmail's config system may seem different from what you're used to at first, but it's really quite intuitive when you understand the concepts behind it.

Re:Qmail

[swillden]

I've run a couple of very large mailing lists with it and even under heavy traffic, it held up like a champ.

And if it starts to bog down on whatever machine you've got it on, ezmlm also provides support for "sub-lists", allow you to distribute the load over multiple systems.

I haven't used it for any large lists, but it looks like it's really designed to scale.

Re:Qmail

[dasunt]

So what is http://cr.yp.to/qmail/dist.html then?

That refrence you give talks about licenses with regards to restricting rights of a consumer - basically, shrink wrap licenses that limit (perhaps illegally) my ability to resell a product or modify it for my means. That's wrong. However, even though I may have the right to resell my copy of Windows XP Retail, I don't have the right to make 100 copies and sell them.

Qmail, along with DJBDNS (and a lot of other DJB's software) has a copyright that restricts distribution of modified binaries. Which means, if DJB get's hit by a truck tomarrow and dies, Qmail's developement is legally frozen.

Re:Qmail

[dubl-u]

And before anyone complains about the license of QMail/ezmlm, yes, that sucks.

qmail does not have a license and does not need one[...]

Please don't be obtuse. The legal meaning is as you say, of course. But the common meaning is, "the terms under which the program is available." In the common sense, qmail has a license. And the OP is right, it's a pain.

DJB's insistence on keeping an iron grip on qmail and keeping it at v1.0 for five years is what made me finally change over to Postfix. It's in the same league as qmail regarding security, reliability, and having a good architecture. But unlike qmail, you don't have to apply patches to get needed features. And the configuration files are actually readable and helpful to admins.

Three months after conversion, my only regret is that I didn't switch from qmail to Postfix sooner.

Re:Qmail

[raju1kabir]

Qmail, along with DJBDNS (and a lot of other DJB's software) has a copyright that restricts distribution of modified binaries. Which means, if DJB get's hit by a truck tomarrow and dies, Qmail's developement is legally frozen.

Only if the truck driver throws it into reverse and makes sure to run over every copy of diff and patch in existence.

Re:Qmail

[Electrum]

That refrence you give talks about licenses with regards to restricting rights of a consumer - basically, shrink wrap licenses that limit (perhaps illegally) my ability to resell a product or modify it for my means.

Yes, which is precisely the issue being discussed here. Are you trying to distribute a modified version of qmail? No? Then why does it concern you? If you want to modify your own version of qmail, then you are perfectly free to do so. If you want other people to use your modified version, then either install it for them or give them patches.

Not being able to distribute modified binaries is a Good Thing. Read the qmail or vchkpw mailing list for a while. Users are stupid. Very stupid in many cases. It is amazing how badly people can screw things up when installing from the original, unaltered source. Now you want to let just anyone distribute screwed up binaries? Think of the support issues involved with that. If people want to install modified versions, they can patch the source themselves.

``Why can't we rename your files? Compatibility is essential. Files must be accessible by the same names on all systems.''

However, even though I may have the right to resell my copy of Windows XP Retail, I don't have the right to make 100 copies and sell them.

Yes, but because of copyright law, not because of the license. You can buy a book and resell it. You can't make copies and sell them. The same goes for software. You can buy a copy at Best Buy and then sell it. You can't make copies and sell them.

Re:Qmail

[Electrum]

But unlike qmail, you don't have to apply patches to get needed features.

What features do you need that are not available in stock qmail?

And the configuration files are actually readable and helpful to admins.

How are qmail control files not readable? Files are named according to their function and only contain the necessary values that you place into them. They are fully documented in the man pages. The values of all the control files can be shown easily by running qmail-showctl. How is this difficult?

Re:Qmail

[Russ Nelson]

If he's dead, who's going to enforce his copyright, and to what end?
-russ

Re:Qmail

[Russ Nelson]

You do not need a license to use qmail. Period. End of sentence. And your problem with this is?

Clue: qmail is at version 1.03. qmail has never had a security flaw. Now, if Dan was to change qmail in any way, do you think that would 1) increase qmail's security, 2) decrease qmail's security, or 3) have no effect on qmail's security?

Configuration files are easy to read. cat /var/qmail/control/me. Works for me.

What features do you need that qmail does not have?
-russ

Re:Qmail

[dubl-u]

Howdy, Russ. You've been a very helpful person on the qmail lists and int the qmail community and I appreciate that. I'm going to try to take your apparent excess of attitude in that context.

You do not need a license to use qmail. Period. End of sentence. And your problem with this is?

My problem is that whatever you call it, Dan Bernstein has been careful to keep legal control of qmail in a way that prevents others from forking. I was very excited when it came out; qmail had a fresh approach that showed a lot of promise. And there it stopped. I decided not to.

Clue: qmail is at version 1.03. qmail has never had a security flaw. Now, if Dan was to change qmail in any way, do you think that would 1) increase qmail's security, 2) decrease qmail's security, or 3) have no effect on qmail's security?

If security were my only concern, I'd just unplug everything from the net and be done with it. Alas, I have other concerns, which I balance with security.

But let me ask you a related question: If people other than DJB are patching qmail to get features they want, do you think that would 1) increase qmail's security, 2) decrease qmail's security, or 3) have no effect on qmail's security versus funneling all the changes through an expert coder who knows the system better than anybody?

Configuration files are easy to read. cat /var/qmail/control/me. Works for me.

That's swell! You should probably use qmail, then. I find Postfix much more easily managed. Many other people do, too.

That, of course, doesn't mean that qmail should change a whit. I gather that DJB has consciously decided that qmail won't be all things to everyone. In which case, you guys should presumably be happy that other packages address other audiences, yes?

What features do you need that qmail does not have?

Honestly, will it do any good to say? If you really don't know of any feature differences between the two and would like to add some things to a qmail 2.0, I'm glad to make suggestions.

But what I suspect will happen is that you will tell me that I can get everything I want with add-on programs A, B, C, D, and E, along with patches X, Y, and Z, plus some jiggery-pokery with various dot files and magically named files. That's been my experience with qmail, and others can verify that by looking at the qmail home page for spam prevention, high-volume servers, and other add-ons.

Then I'll tell you that I'd rather just download the postfix RPM, set a few lines in a config file, and go. At which point I'll receive some more attitude for not being sufficiently smart, elite, tough, or whatever quality it is that makes people willing to deal with a mail server on its terms rather than their own.

But if you're really looking for help understanding the difference between the two mailers, drop by the postfix mailing lists. There are a number of former qmail users there, and I'm sure you can collect a wealth of feature suggestions there.

Re:Qmail

[dubl-u]

How are qmail control files not readable? Files are named according to their function and only contain the necessary values that you place into them. They are fully documented in the man pages. The values of all the control files can be shown easily by running qmail-showctl. How is this difficult?

Have you run a Postfix system? I've run both Postfix and qmail systems. I used qmail for four or five years. I find Postfix easier to manage. You may like the qmail style better. That's fine: my getting chocolate ice cream is not some sort of implicit assault on your preference for strawberry ice cream.

For postfix, there is a main configuration file, called main.cf. The stock main.cf has extensive commenting, describing many of the common options. When I want to do something with Postfix, I just edit the config file and rummage around a bit. Then I type "sudo postfix reload".

With qmail, I always have to read a bunch of different man pages, tracking down the particular program that happens to handle whatever I need to change. Then I echo some value into some magically named file. Then I have to read the man pages again to figure out exactly which signal I send to which program and/or programs.

As I said, maybe this is just me. Maybe most other people find it easier to think like Dan Bernstein. Maybe I just don't have enough practice adminstering Unix boxes; I've only been doing it for 15 years or so.

What features do you need that are not available in stock qmail?

Go browse the qmail web site. Note the dozens and dozens of links to patches, add-on programs, and other cruft.

Personally, I wanted db-backed virtual domains, and pretty complex spam rules with exceptions for certain senders, certain recipients, and certain mail servers. For qmail, I just never bothered because it was too much of a pain.

Re:Qmail

[Electrum]

I used qmail for four or five years. [snip] With qmail, I always have to read a bunch of different man pages, tracking down the particular program that happens to handle whatever I need to change. Then I echo some value into some magically named file. Then I have to read the man pages again to figure out exactly which signal I send to which program and/or programs.

You claim four or five years of qmail experience and still don't understand how qmail works? Wow. If you changed something related to delivery, then you restart qmail-send. If you changed something for qmail-smtpd, then you restart it. I don't see how that is complicated.

Maybe I just don't have enough practice adminstering Unix boxes; I've only been doing it for 15 years or so.

If this is anything like your four to five years of qmail administration, then I wouldn't count it for much.

Go browse the qmail web site. Note the dozens and dozens of links to patches, add-on programs, and other cruft.

Yes. Different people have different needs. Should the program be bloated with every possible feature that someone might want? I guess the Postfix author thinks so. But you didn't answer my question. Which of those features do _you_ need? For at least 95% of the people installing qmail, the stock distribution works just fine.

Personally, I wanted db-backed virtual domains, and pretty complex spam rules with exceptions for certain senders, certain recipients, and certain mail servers. For qmail, I just never bothered because it was too much of a pain.

Pure FUD. db-backed virtual domains? Use vpopmail or vmailmgr. Filtering? Use your favorite delivery agent, such as procmail or maildrop, as the default delivery method. Override this for specific recipients in their .qmail files. Other exceptions are handled by the delivery agent.

qmail follows the UNIX tradition: do one thing and do it well. That is why qmail is so extensible. You can easily add programs into any part of the mail process to do things how you want. You can also replace diffent components if you like. This is why qmail is so powerful.

Re:Qmail

[Anonymous Coward]

DJB's insistence on keeping an iron grip on qmail is what keeps me using it. I have enough software that has versionitis. Postfix is an accident waiting to happen. Want to add a feature? Recompile. With qmail, I haven't had to add any features..ur...bug fixes, it is all there.

Postfix configuration files? Blah, postfix is to sendmail like. And you know how sendmail sucks. DJB's design makes sense.

I wish more software was of DJB's quality.

Re:Qmail

[dubl-u]

You claim four or five years of qmail experience and still don't understand how qmail works? [...] If this is anything like your four to five years of qmail administration, then I wouldn't count it for much.

Cutely phrased, but in your eagerness to be snide, you missed my point.

Long ago, I was a full-time sysadmin, running large fancy networks of what then passed for cutting-edge unix systems. Eventually, I got sick of being a sysadmin for money; it was turning me into a grumpy BOFH. Today, I do other things for a living; I just maintain a few boxes for my own entertainment.

When I installed qmail five years ago, I figured out how it worked and knew it just fine. But I have to touch my mail server maybe every few months, which is plenty of time to forget all of the details.

Qmail is easy to use in the same way that a trumpet is easy to play: if you learn a lot about it and keep in practice, it's easy. But a novice will have little luck just picking it up and getting some music to come put.

I know a lot of geeks don't get this point about usability, but try reading Don Norman's "The Design of Everyday Things" and maybe that will help you get what I'm saying.

Personally, I wanted db-backed virtual domains, and pretty complex spam rules with exceptions for certain senders, certain recipients, and certain mail servers. For qmail, I just never bothered because it was too much of a pain.

Pure FUD. db-backed virtual domains? Use vpopmail or vmailmgr. Filtering? Use your favorite delivery agent, [...]

Actually, I wanted to do a lot of the rejection at the SMTP layer. Which I could also do for qmail with yet another program or patch, I know. But the point isn't whether it's possible, it's how easy it is. With postfix, it was all in the config files. And even better, the postfix author keeps improving things in response to user needs.

If somebody whose only job is maintaining a big ol' mail server asks me what to run, I'll certainly say "check out qmail". But when a non-expert asks me what they should run, I'll suggest postfix.

qmail follows the UNIX tradition: do one thing and do it well.

Great! I'm happy for it. I'm happy for you. It sounds like you should use qmail. But recognize that circa 99% of the computer-using population doesn't ever touch anything that follows that workbench-style approach.

For most purposes, people just want appliances. A master chef may have a fantastic commercial-grade kitchen but only use an iMac. A master Unix geek may know which four lowercase letters aren't valid arguments to ls, but max out his cooking skills heating frozen burritos in a microwave.

This doesn't mean that the geek or the cook (or the iMac or the microwave) are somehow intrinsically lame; having chosen one thing to master, they just want other parts of their lives to be as easy as possible.

Personally, I needed a mail server that was more an appliance than a workbench. Postfix fills that need for me.

email interface...

[Fweeky]

One thing I hate is having to "log in with the plaintext password I store and mail you every month without fail to some random mailing list name to do anything because this MLM is too braindead to understand listname-requests, or even listname-(command)".

Whatever you use, make sure it has a clean email interface, and configure it to include rfc2369/rfc2919 List-(Subscribe|Unsubscribe|Post|Help|Owner|Id) headers so I can filter and automate control of it.

Ecartis is a great example of a MLM with support for both email and web-based manglement. Email is the standard double-opt-(in|out) stuff, with various other methods of authentication to make sending batched/automated commands easy for admins. Web emails you a "cookie" (effectively a temporary password), and lets you set up a (secure) password once logged in; if you forget your password, you just don't include it on login and get another cookie.

No monthly spam with one of your passwords going out for all to see to some random location in your filters (mine end up in lists/(test|news|announcements)), and an extensive but by no means required web interface, without the need for a monthly insecure irritating to filter spam.

Mailman under high useage

[Halvard]

Check out this link at the Mailman website. It details who's using it. Names like Apple Computer. My recollection from about a year a go was that the had a 1 million user list. I use Mailman as well for all client list serves. Works great.

I use something called sympa

[Yarn]

It has a nice web interface for my non-techie friends but behaves as a proper MLM for everyone else. It can use a mysql backend too

Sympa

[kiowa]

The company I am working for is currently in the process of selecting a mailing list manager as well. And after a bit of wandering about we are about to settle on Sympa. It seem to have it all. Web-interfaces, sql-backends and fairly good documentation.

We also took a peek on Mailman, but came to the conclusion that it would take too much work on having to integrate it into our design. No stylesheets to edit, and if you want it to integrate into a search engine other than pipermail (which can't search at all, only list threads) you have to get down dirty with the source. And having an extra thread of yet another project isn't a viable solution atm.

Re:Sympa

[Baloo Ursidae]

I'm 100% with you on that one. If anybody wants to take a peek at the optional web interface for it, I've got it here.

Re:Sympa

[Anonymous Coward]

Getting mailman to do searching for its lists using ht://dig is actually very simple, even to the level where you enable/disable searching in the mailman web interface. Doing this require NO mailman code hacking, 2 lines in the config file and a simple 20 line (with comments!) bash wrapper script.

no problem.

ListProc

[chad_r]

The Listproc MLM was closed source for a long time, but has recently gone open source. The source can be found on SourceForge, and appears to be using the Netscape Public License.

It doesn't use a relational database. It uses text files for configuration and subscriber lists, and then builds dbm files for quicker lookups. It does have a web front end. It has a crude bounce management--you can set a per list option to auto-delete any address that bounces multiple times. But it's not foolproof, and it doesn't have something more reliable like address probing.

Ecartis?

[dacarr]

Listar has morphed into Ecartis, which is an open source ML manager. I'm on a list that has about 1500 members, and from a user standpoint, it seems to handle a lot better than, say, Majordomo.

Click on the "About" link for more details. I've not taken a look at what's under the hood, but if you want something that's largely text based, this could work.

Re:Ask Slashdot

[greyrax]

I wondered how long it would be until people were on to my little plan! :)
NOT
I did wonder, as I posted this, if people would think that it was for spamming. This is an opt-in newsletter for a talk radio show. Believe it or not, there can actually be legitimate ways of building a large subscriber base. :)

Re:So how much money do you spammers make?

[greyrax]

See this reply.
I'm sure that spammers have much, much larger lists. Heck, if you can spend $99 for a CD with 500 million (or whatever) e-mail addresses, why stop at 400,000? :)

A complete solution - the FreeBSD toaster

[gregwbrooks]

If you go here you'll find a great recipe (including install scripts) for a FreeBSD-based solution that includes Qmail and EZMLM-idx, but also includes a webmail interface and wicked-cool support for virtual domains. The setup is supported by an active e-mail list and it's been rock-solid on my moderate-sized lists, as well as much larger lists used by others.

Probably more of a solution than you need, but it's a very good way to take a generic boxen and quickly turn it into a slam-dunk mail server and listserver.

The guy who developed it (Matt Simerson) is uber-conscious of security issues and a helluva nice guy to boot; he's put together an exceptionally tight package from a security standpoint. If nothing else, you may want to look at what he's got and do a modified or stripped-down install.

Re:A complete solution - the FreeBSD toaster

[drshannon]

For a Linux based toaster, check out Bill Shupp's Toaster which is based on Matt Simersons. The virtual domain support is indeed wicked cool, using Vpopmail.

ezmlm

[smerritt]

BugTraq uses ezmlm, and it's certainly a large mailing list.

Mailman

[roly]

I use Mailman and its great!

ezmlm's backend with Mailman's frontend?

[ptomblin]

I have a bunch of mailman lists, and my users like the web based interface. Some of them like the monthly password mailing, some hate it. I'm not to thrilled about how badly it handles bounces, though, and I wish it could be configured to just reject email from non-subscribers (which is usually spam) rather than presenting it to me to have to make a decision on.

My web hosting company (Gradwell.net) has ezmlm, and from what I've been reading, it seems to be much better on the back end, giving me the option to reject all email from non-subscribers and better bounce handling. The only problem is that a lot of my users are not very email savvy, and they need the crutch of a web interface. Before I write my own, it would be nice to find out if there is one available. Does anybody know of one?