Slashdot Mirror
Good POP3 Server for Huge Mailboxes?
brainchill asks: "I've got about 10,000 users split between a couple of quad 550 xeon machines. The machines have 2GB of ram. The problem is that the UW POP3 server takes a huge hit in both cpu and memory utilization when a 40+MB mail spool is requested via POP3. Sometimes it's bad enough to drag the monster boxes to their knees. What other POP3 daemons do you guys have experience with and how do they perform with large mailboxes"
Read the various qmail + whatever guides. Also, remember that system tuning can make a difference as well.
MS Exchange.
You know you want to.
You know, I think it's funny that SMTP (SIMPLE Mail transfer protocol) turned into an FTP replacement. If people want to send 40 meg files around the internet, maybe they should use something designed to transfer files, like ftp or scp.
You won't get good performance with mbox, period. You need to switch to Maildir. qmail-pop3d works great with Maildir. Maildir scales far better than mbox since it doesn't have to parse out the individual messages. It also doesn't have to use locking. This also makes Maildir inherently more reliable than mbox. There are many tools available to convert between mbox and Maildir.
I use mbx personally (NOT mbox) and it scales wonderfully. The mailbox is fully indexed to speed up searching and can be accessed simultaneously by various processes. See here for more information.
Try Qpopper
"With Microsoft, you get Windows. With Linux, you get the full house" - unknown
I can only second that. qmail runs like a charm and scales.
Check out cr.yp.to/qmail.html and www.qmail.org
Have you looked at Cyrus? It is probably best known as an IMAP server, but it has very nice pop3 support as well.
Cyrus stores messages in a variation of the maildir format - it maintain a database of the flags, headers, etc for the messages in a folder to speed up access.
Notable features include shared mail folders (with independent views), quotas, multiple mail partitions (with the ability to move users across partitions on the fly), duplicate email checking, and a server side filtering language (sieve).
Most of this would probably be most useful if you were using IMAP, but it should scale quite well as a POP server.
If you want your users to love you, check out Dartmouth's
Anyone who has ever been to Dartmouth or any other school using a BlitzMail installation will vouch for the strength, ease of use, and plain usefullness of the system.
I would say: try popa3d.
We just went from cucipop to popa3d on a sendmail box supporting ~10000+ users. The load dropped from ~8 to ~1.5 during peak hours.
Before cucipop we were using qpopper, and the switch from qpopper to cucipop made a similar drop in load. Remember that this were with the "old" versions of qpopper, before the remote-root vulnerabilities were found. Don't know about the performance of today's qpopper.
http://www.dbmail.org/ I was looking thru their site, havent actually used it but it sounds like it might reduce the load on you're servers massively. tho i'd want to use postgres and not mysql for the back end.
CommuniGate Pro is the best Email Svr I'Ve ever used.
Its easy to configure, very feature rich, performant and features cluster config.
The only problem is it costs you an arm an a leg for the amout of users you have.
I'm running it with only a couple of hundred users using mostly IMAP but I never had any problems with it.
Before you email me, remember: "There is no god!"
Could your problem be caused partly by hardware? Hevanet.com, the best ISP in Portland, Oregon, USA, uses a special SCSI system run with a special version of NetBSD supplied by a company in Arizona.
Retrieval of mail stresses the filesystem; Hevanet's system is a combination of OS and hardware meant to take the load.
Good christ, you'd think that by the time you outgrew a QUAD XEON mailserver with only 5000 users, you'd have been reevaluating performance before plunking down what must have been close to 10-15 grand or more at the time on a second one!
Your mailbox format is all wrong. Storing all messages in a single file is pretty much the worst way to do anything useful. You want to explore some alternative storage format such as mbx or maildir. I personally use maildir on ReiserFS on Linux and have good luck. (The filesystem is VERY important for maildirs. ReiserFS's block tail support and directory indexing give it major disk space and speed advantages for a maildir mailserver application, while running something like maildirs on XFS would instantly kill your server. I hear mbx is pretty good too, if you're stuck on some sort of standard filesystem since it uses indexing and fewer files than maildir. The downside is that it's not as immediately parseable as maildir or mbox... Ie you couldnt write a script to say... delete extremely high scoring spam messages from any user who hasn't checked their mail in over 3 months, or other things ISP's might routinely do to maintain their servers.
Finally, if you plan to scale way up there (60,000+), you need to start looking at better cluster systems than just a couple machines. Specialize the tasks of several machines to do mail storage or talk POP3. Look at something like POPular for specialized POP3 server clustering software.
~GoRK
extracted source is 84k. highly secure, and tiny. default pop3 daemon in openbsd.
/etc/shadow authentication processes.
from their DESIGN doc: http://www.openwall.com/popa3d/DESIGN
Here's some real performance data that I've collected (popa3d running
via inetd; larger sites would use the standalone mode instead):
24864 295.50re 16.92cp popa3d*
12749 4578.88re 15.50cp popa3d
That is, 12749 POP3 sessions took 32.42 minutes of CPU time (on a 350
MHz Pentium II); of those, more than a half was spent in the temporary
child processes. It's not that bad though, as this system was running
an (intentionally) expensive crypt(3) that got accounted to the child
Before upgrading to popa3d, the same machine was running qpopper (out
of inetd, too):
12025 3169.38re 35.56cp popper
It used to take a bit more CPU for less POP3 sessions.
The key to getting better performance is Maildir. Instead of using a single machine, use a "cluster" of machines and share the Maildir of NFS (or some other network sharing means). Of course, a maildir format mailbox would probably lower your overall utilization as well.
I chose Cyrus for a customer that needed a MySQL backend for his server. But I quickly ran into a problem : the minimal timeout for unlocking the mailbox in the Cyrus POP3 server is 10 minutes (yes, that's right. 10 _minutes_ !). As people with buggy mailers (*cough* Outlook Express *cough*) are very common nowadays, I was forced to go patch the sources to weed out that stupid limitation. What's sad is that I found lots of messages on their mailing-list talking about this problem since a long time, and that one-liner patch never made it to the tree, which would lead me to think the authors are unconcerned about the needs of their users (I hope I'm mistaken here)...
:-) Or else, you could try using LARTs on your unruly, mailbox-filling users... Good luck to you, anyway !
BTW, there is a fine POP3 server that we've used without problems for a year (and we've customers that *never* empty their mboxes, so we've huge 300 MB horrors lying on the primary MX hard disk). It has no frills but works like a charm. It's called Solid-POP3, it's Polish, made by the same people who brought you the PLD Linux distro, and you can download it here (alternatively, just do an `apt-get install solid-pop3d' if you run that good ole' Debian
Xenu brings order!
Samsung Contact should deal with that on a single server without breaking a sweat.
You think admins care about how big the email is. not really, until it trashes the server when you try to send a 450 MB powerpoint slide show home so you can "work" on it. When the server bombs, and they will, I garantee you will be the first one to gripe when you can't read you joke of the day with your coffee, or check out the pr0n you frat buddy sent you.
you know it wants you to ;-)
Ok, we call it Openwave Email Mx now, but it still scales like a mother. I run it. But then I work for Openwave on Intermail :-)
e /e mail_mx/index.html
http://www.openwave.com/products/messaging_suit
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/