Slashdot Mirror

MHO also says that if you are looking at setting up a mail server, you should check out Postfix by Wietse Venema, or qmail first. I have been using postfix instead of sendmail for quite some time now, and have not had a single problem. Of course, I only have 600-1000 users, so my system is certainly not a true test of its capabilities.

I'm not sure about other mail servers, but there's a patch available for qmail which implements SMTP tarpitting.
The way it works, is that the SMTP server keeps track of the number of RCPT TO addresses entered by the spammer. After a predefined number (e.g. 10 or 15), the SMTP server delays its responses by sleeping for a few seconds first). This shouldn't affect most normal use, depending on what number you choose.

This slows the spammer down significantly, and has the appearance of a stalled connection--hopefully, causes the spammer to give up and move on.

I have no idea how it works in practice, but it sounds good on paper. It's far from bulletproof, but it sounds like it would help, without impacting legitimate mail traffic. Can anybody comment?

Why should I be forced to use several meg of HD space for an admin tool (ie. Linuxconf) that I will never use ? Why should it (or any other UNIX for that matter) always install sendmail .. I never use it, I always use qmail instead. Shell is different, a computer without a shell is like a car without an ignition. Perl is a programming language. Not an esential program..

I am told that there are patches available to improve qmail's performance on very-high-volume sites.

The correct URL for patches is, of course, http://www.qmail.org/, not http://www.quaker.org. I blame the Scotch.

Just wondering, does everybody use Sendmail?

We use qmail, and we've been very pleased with the results.

Are there reasons why people would use Sendmail over qmail? Are there reasons why people would use qmail over Sendmail?

Just interested :>

Thanks.

I'm not familiar with Exim, but aren't there more efficient solutions?

Although my experiences have been with much smaller configurations, qmail reportedly handles loads of this magnitude on lesser hardware.

Mutt seems to me to have the nicest of the text interfaces; it is somewhat unfortunate that it doesn't have huge support for the multiplicity of folders that a MH user grows to. (I've got 350 mail folders and 179MB of archived email, for instance.) For managing that, the user interface of EXMH combined with a variety of shell scripts are pretty much necessary.

Mutt is still the nicest way of reading mail on a console...

Ever try to do mail via BITNET or UUCP with qmail?

Weird. I'd bet noone mentioning BITNET *knows* anything about it :) If I knew it, I would try it with qmail yes! I've run uucp over ssh with qmail, I've used FidoNET with qmail. No problem. My only problem with sendmail was that you can *fix* it to do something you want it, but you'll never understand it completely. On the other hand, if you take a few hours, and read all (included) qmail docs, and you know Unix, you can do *anything*, and you understood qmail totally.

If I let myself summoned with qmail bashing, let me share a few resources on it:

• www.qmail.org
• Life with qmail
• Dan J. Bernstein's site (funny, it's down, slashdotted?)

BTW, let me mention, Dan (the author of qmail) is the Math Professor on University of Illinois at Chicago, who sued the government to not restrict him distributing his crypto source at his web page (for his students!).

• Qmail's maildir format means that you the machine wouldn't bog down as soon as the users started getting large mailboxes - each mail is stored individually, and the pop daemon doesn't have to read through huge files to find out how many messages are there
• The fact that users' maildirs are stored in their home directory also means you can spread them across filesystems easily.
• the configuration and management of qmail scales a lot easier than sendmail's - much more sensible config files / config file names, seperate config files for different things
• global aliases are stored as seperate files (although you can use a hack to use the /etc/aliases format if you like) so managing aliases is easier
• Qmail is more secure ( http://web.infoave.net/~dsill/qmail-challenge.html ll/qm ail-challenge.html)

I say FreeBSD because I know that it's reputation of stability and fast networking are true from experience, but I guess Linux or another open source operating system (OS OS, heh) would do the job fine. I've seen a system at a commercial ISP running with about 15K users on FreeBSD & Qmail, which is why I'm recommending it. They switched from sendmail when it started becoming too slow because of massive mailboxes; every time a user with a 50Mb mailbox logged on the mail server would chug until it had gone through the whole file. If someone gets sent one 50Mb attachment, that means that the pop3d gone through 50Mb of data to say "1 New Message" - it's not an issue with the maildir format.

I guess you could achieve the same effect with a clustering solution, but I think that's probably unnecessary.

You might also want to check out postfix.

URLS:

• http://www.qmail.org/ - Qmail
• http://www.postfix.org/ - Postfix (by Wietse Venema)
• http://www.freebsd.org/

There are multiple solutions listed on www.qmail.org for a single-uid mailstore which scales to millions of users. I have a customer who needed a system which could handle a million users on a pair of HA Sun Enterprise boxes. I delivered it.
-russ

Note: this is not for the faint of heart, and probably involves overhauling your mail setup.

I've also looked into setting up something like that. It came down to using something like LDAP which is scaleable, standard, and OS-nonspecific for mail users. Then I had Qmail and Cyrus on the backend. I applied the Qmail LDAP patch, allowing Qmail to use LDAP for its user list. For Cyrus, there is another pwcheck file here. It adds the ability for it to authenticate against an LDAP server. Finally (yay!) we need the Qmail/Cyrus glue (as Qmail uses a slow mail format compared to Cyrus, and has no IMAP support [yick!]).

Incoming mail goes to Qmail, which uses scripts to deliver to Cyrus (users LDAP listed). User logins go through Cyrus server programs (which use LDAP auth), and can get their messages that way. This should also support virtual domains.

All in all, it sounds good ;-) I have yet to implement it (I'm going slowly and testing every step. Right now I'm converting my user base to LDAP, then I'll update Cyrus, etc).

Have fun.

qmail can do what you want. It's open source, free and very well done.

Depending on the hardware and OS, you should have no trouble running >25k users. There are POP and IMAP daemons available (some by the author, some not) and it is fast, secure and customizable

It natively supports Maildir and mbox delivery. Maildir stores messages as separate files in a directory instead of the mbox structure which makes it the ideal solution for NFS mounted mailboxes, which is crucial for that many users.

I can't say enough good things about qmail. You should at the very least evaluate it.

Don't make the mistake of trying to use Exchange! It is a real PITA!


"You can twist perceptions, reality won't budge." --Rush

Sendmail can do it but it's fairly inefficient. You may want to look into PostFix. Qmail is also popular. I believe you can use db or dbm files with to route mail to other servers in at least the first two cases.

The Cyrus server at CMU is probably your best bet. You'll find it at at this link.

It's worth noting that this project is currently supporting all of CMU's e-mail needs. It's also my understanding that it forms the basis for Netscape's Message Server and Post.Office. This should satisfy any concerns about it's scalability. It has lots of handy features like kerberos authentication, a database style message repository, support for ACAP, etc.

Alternatively try QMail. Personally, while I think it provides better SMTP performance than Sendmail, I'd rather use the Cyrus IMAP server than the UW one (the only one supported by QMail). You could go with using a combo of sendmail|postfix + Cyrus for incomming mail (i.e. what your MX records point to) and QMail for outgoing mail. It depends on your performance needs

Exchange Server is NOTORIOUS for being both difficult and expensive when you need it to scale to a large number of users, although I understand it's improved substancially since the 4.x days when it was just impossible.

To give a touch of context here, that prof is Dan J. Bernstein, the ever-popular author of Qmail

www.qmail.org

If you a truly bent on migrating to Linux, please do let us try to stop you first. :) If you're actually running BSD3.1, then I recommend you upgrade it, cause that is really old. Otherwise, I'll assume thats its FreeBSD3.1, which is new a good.

FreeBSD's TCP stack is more robust than Linux's, and it handles disk I/O well, making it ideal for things like mail servers. I havn't had any problems with lack of support for hardware, and like everyone else mentioned, it can run all kinds of apps.

As far as your want for POP3-only users go, I have a solution for you. First thing to do is install qmail. Its more stable, more modular, more easily configurable, less crackable, and leaves a smaller footprint than sendmail. Then, follow these instructions and set yourself up with enought POP accounts for all the users you could want.

Have fun.

--Ed Bardsley

• Plain old bugs. These will occur in any piece of software, no matter how many times it has been proven (example: TeX, as released, actually had a few bugs and logic errors).
  
• Incomplete/Incorrect implementations. Implementations of drivers that have assumptions about hardware peculiarities and/or many variations on similar hardware. Example: The video driver for XYZ video card with the ABC chipset works, but PDQ video card had the same ABC chipset and it doesn't work. Or maybe rev D of the XYZ card fails intermittently while rev C works fine. This sort of problem gets fixed with time, provided that people pay attention (submitting bug reports always helps). This is where Windows "shines"; since the companies selling the hardware write all the drivers they also test it with all the hardware.
  
• Design flaws. The biggest one of these I can think of is the C language's array handling and pointers. As long as Linux applications are written in C, array handling will be a problem. Developers that pay close attention to the problem typically produce safer code (example: Dan Bernstein's qmail MTA, while not strictly a Linux application, has not to my knowledge had any buffer overflows since it was released, exploitable or not). I'm not advocating rewriting the kernel in Java or SML; I'm merely pointing out that C, for all of its expressiveness and flexibility, carries some weighty problems with it.
  

What really scares me is how nobody has yet pointed out the way Fairfax describes open source in their article:

Open Source is an increasingly valuable marketing position that has grown from a philosophy that software and its programming code should be shared among all programmers.

(Emphasis added.)

That really is the way quite a few companies view it -- not as a philosophy or even a "new business model", but as a marketing tactic.

It seems that, where we had to do some education last year on the meaning of the word "free" in "free software", we now have to do some education about the meaning of the word "open" in "open source".

An earlier post under the subject line "Poor Us" stated:

>I say, if you can see the source code, its open source.

I don't mean to jump down that person's throat, but that is only one small part of what's necessary for something to be open source. I use an alternative MTA, Qmail, which comes in source form, so you can definitely see the source, but the author keeps tight control over what actually goes into it -- nobody else can check things into the source tree, even suggestions for improvements or alterations are generally rejected. There are lots and lots of patches, but nobody but the author gets to fuck with Qmail's source.

There have been discussi ons on the Qmail mailing list about just how free (or open) Qmail is, based partly on that requirement. (Follow Vern Hart's reply from the link.)

Apparently, the proliferation of patches to Qmail (and, more importantly, the fact that the author places no restrictions on such patches!) allows it to meet the OSD, but it's a close shave.

I use this as an example, and especially refer readers to Dave Sill and Vern Hart's "more free/less free" phrasing in the referenced thread on the Qmail mailing list. Freedom or openness is a continuum, and supplying source is only one step along it.

If we let corporations get away with promoting the idea that "if you can see the source, it's open source software" -- and especially if we start believing it ourselves -- then the movement is finished as a significant force in computing. That's the first step toward letting it become the mere "marketing position" that Fairfax claims.

On many of the modern Unix variants, /etc/passwd is only a textual representation of a database file which holds the real user information. The getpw*(3) routines use this database file to access passwd data. This makes things way faster than they used to be, for example, on SunOS4, where ls(1) was written so stupidly that it scanned the (sequential) passwd file for every single uid lookup it needed to make. Type "ls -l /home" on a SunOS system with like a thousand registered users, sit back and relax.

Speaking of today: FreeBSD, for example, uses a Berkeley DB database to store passwd information. In fact, it uses two databases, one with and one without passwords, for "security". This speeds up lookups quite a lot, but beware: The DB files are still generated from text files, so adding users with huge user databases is a lengthy process.

The question is whether you actually want to create that many Unix user accounts. For mail servers, you can often get away better with creating mail accounts only. This requires some hackery with your friendly MTA (postfix or qmail), but it is quite doable and also has positive security side-effects.

Look into Cyrus imapd if you need a message store implementation which is able to handle mailboxes for users who don't have a unix login. Beware, Cyrus comes with a ugly^H^H^H^Hpretty tcl-based administration interface which you can replace by a bunch of home-grown perl scripts to automate administration. Cyrus makes it fairly easy to integrate your own authentication mechanisms through a seperate process, although the performance of such a mechanism would have to be determined.

In a nutshell: Unix in itself is not prepared to handle very large user populations. If you need to serve a lot of users with shell accounts, look into NIS+ or Kerberos and distribute the load onto a bunch of machines served by central (and well-hardened) user-database-servers. If you need to support only mail, you might be well off with one fast machine and a special purpose mailer configuration.

On many of the modern Unix variants, /etc/passwd is only a textual representation of a database file which holds the real user information.
getpw*(3) uses this database file to access passwd data. This makes things way faster than it used to be, for example, on SunOS4, where ls(1) was written so stupidly that it scanned the (sequential) passwd file for every single uid lookup it needed to make. Typing "ls -l /home" on a SunOS system with like a thousand registered users was an invitation to get ahold of some (some!) coffee.

Speaking of today, FreeBSD uses a DB database to store passwd information (in fact, it has two databases, one with and one without passwords, for "security"). This speeds up lookups quite a lot, but beware: The DB files are still generated text files, so adding users with such huge user databases is a real pain.

The question is whether you actually want to create that many Unix user accounts. For mail servers, you can often get away better with creating mail accounts only. This requires some hackery with your friendly MTA (postfix, qmail, sendmail, exim or even smail), but it is quite doable and also has positive security side-effects.

Look into Cyrus imapd you need message store implementation which is able to handle mailboxes for users who don't have a unix login. Beware, Cyrus comes with a pretty tcl-based administration interface which you almost certainly want to replace by a bunch of home-grown perl scripts to automate administration.