E-Mail Server Setup Advice?

dhammala asks: "I am responsible for setting up and maintaining a mail server for small web-hosting type business. We currently host about 75 domains, around 100 mailboxes and due to the efforts of our sales team, we are wanting to get ready for some great increases in those numbers. I am worried about my current configuration and ease of administration. More importantly (well, at least to the customers) is email deliverability -- it seems that messages delivered to some big players are being marked as SPAM or disappearing altogether. I am asking the Slashdot community for it's insight and advise on 1) if my current choice of software/configuration is a good match for this situation and 2) if there any additional measures I might take to ensure email deliverability?" "Here is an overview of our current setup:

• We lease servers at ev1servers.net.
• The servers are running RHEL ES3.
• We chose to use Postfix and have it configured to support virtual users and domains mapped in MySQL tables. The reference I used to configure this setup is located here. We initially chose Postfix over qmail because it was open and over sendmail because the config files are actually readable.
• I have added in SQLGrey grey-listing for Postfix to provide a simple level of SPAM detection for our users. We are not wanting to deal with the customer service and higher box loads of mail scanning at this time. We might choose to use a 3rd party vendor to do this as needed.
• Messages are delivered locally via maildrop in maildir format.
• Courier IMAP is running to support both IMAP and POP access to the mailboxes.
• Postfix Admin was setup for easy mailbox administration.

For deliverabilty, I have/am taking the following steps:

• I have verified that our reverse IP records are correct
• I have created SPF records for all of the domains
• I have verified that our server is not listed in any blacklists (great scanner at dnsstuff.com)
• I have started to install DomainKeys for Postfix

In doing all of that, I have found that our IP is listed in the BlarsBL. Do I need to be concerned about this rogue list? The IP was there before I even began to setup the box.

I have not yet been able to get DomainKeys to work with Postfix. It was during my configuration attempts that I started to question this setup and wondered if this was the best setup for our situation.. this inquiry has lead to this posting.

In a perfect world, I would have an email server that:

• is easy to administer,
• supports automated mailbox setup/removal (currently I can just insert rows into my tables and the mailbox setup is done)
• supports current technologies, like grey-listing, DomainKeys, etc
• is secure
• makes the best use of system resources -- I want to get the 'best bang for the buck'

So what do you think? If I stick with this setup will life be grand? I am open to something new AND even taking the time to learn a new setup. If I do need to switch to something different, my only concern would be the ability to migrate existing mailboxes and messages over to the new setup.

Are there any other technologies or configurations that I need to implement to support the best deliverabilty rates?"

Comments

Full out of box solution.

[Anonymous Coward]

It's not free, but great support and full everything right out of the box, including IMAP, POP, SMTP, HTTP, authentication, account management, quotas and everything else you could possibly want.

If your company can't afford it, that sucks, but I'd rather use that than try and get courier, postfix, pop3d and squirrellmail or whatever to all work together.

http://www.sun.com/software/products/messaging_srv r/home_messaging.xml

Re:Full out of box solution.

[BoomerSooner]

Don't use the Sun Messaging if you want to keep your clients happy. My university used to run QMail with Horde/IMP and it was kick ass (I subsequently moved my mail servers to that combo). They then for some unexplained reason (probably easier to admin or they got a deal since they are a university) they switched to this piece of shit software.

IT FUCKING SUCKS. You have about 1/10th the features and it runs significantly slower (I cannot verify the hardware is the same but the previous systems were IRIX on SGI machines, my best guess is this POS is running on Sun based systems).

Clients want more features not less. If you cannot administer the system yourself then hire someone who can. There are many consultants out there that are reasonably priced. If my email went from something great to something shitty I'd be pissed.

When Yahoo Mail and GMail are better than what you can offer for free are better than what you charge to offer, do you really deserve to have any customers?

Learn to properly admin a mail server, end of story. Hell you could even use Exchange. However that is more difficult than any other combo you can put together in my opinion.

http://www.clamav.net/

[kapplepc]

I recommend setting up ClamAv with FreshClam to filter out virus/worm type email. I have found it performs very well on my server. I have also found they have a very fast responce to new viri as they appear.

http://www.clamav.net/

Why Postfix/courier?

[Lenolium]

So, I have a mail system setup, it's running around 70 domains, and 500 email accounts.

I am using courier as the mta, and courier as the pop, and courier as the imap.

The courier makes a fine MTA, but you do have to tweak a few of it's settings to make it more wideopen to allow it to connect to misconfigured exchange servers.

Other than that it has been great. I have a email account management system that I wrote that lets each domain have admin users that can add and delete accounts as they please.

I have SpamAssassin setup for some users (most of them post their email addresses in plain text on their websites) and even with that load, there is still plenty more capability in this little server.

Now, if you want a system that scales to tens of thousands of users, you are going to need to get something a little bigger than this, you are going to need to get a mail system that can distribute the messages over a number of servers. That is something I have not researched.

Re:Why Postfix/courier?

[iamcadaver]

Now, if you want a system that scales to tens of thousands of users, you are going to need to get something a little bigger than this, you are going to need to get a mail system that can distribute the messages over a number of servers. That is something I have not researched.

It might be called Hula: http://www.hula-project.org/

Courier IMAP

[embo]

Courier IMAP is running to support both IMAP and POP access to the mailboxes.

I would switch to dovecot. I found the performance to be quite a bit better than Courier, and it seemed more stable as well.

CommuniGate - Not free but....

[DigitalReverend]

It is easy to use, can be expanded to cluster servers and is reliable.

Works with squirellmail, and a bunch of other cool features. Plus the name of the company is kind of cool.
http://www.stalker.com/content/solutions.htm

Suggestions

[Bruha]

Check out inter7.com

They use Qmail which is open source. Who told you it was not?

Qmail is highly scalable and I think www.qmailtoaster.com and a few other sites provide great setups that allow you to set quotas and such.

large game sites use qmail.

Hell hotmail.com uses qmail to send emails. Not sure about the rest of it.

Inter7 can get you setup properly and provide maintenance if you have problems but otherwise their setups are self manageable.

To create mail directories on fly.

[DavidTC]

Download this.

Then, assuming you know how to write PHP code, throw away the php. It's not that good. It can't handle fields being added to the database. But writing php for database manipulation is trivial, so I'll assume that's what you're already doing.

Anyway, what you need from 'vmail' is the 'maintain' perl script. It's fairly easy to understand. Basically, you want a 'new' table in your database with new email addresses, a 'deleted' one for deleted addresses, and a 'moved' one for moved emails. So in addition to editing your main mail table, you also put email addresses in those tables when those things need to happen.

The script pulls these out of each table and does the things. It doesn't mess with your main table at all.

Now, the perl script needs to turn that email address into a directory. It starts out as hostname.dom/u/user/ from user@hostname.dom, but you can change that however you want with a bit of perl study.

Or, instead of putting the email address in the tables, you can just put the directories, and tell it not to try to make that into a directory at all. That's probably simplier if you already have the rest of the system set up with other pathnames, and you don't know perl.

I experimented with cgi scripts and whatnot, but this was much easier. You can either put in cron to run every minute or so, or you if it is important updates happen instantly, you can make it suid and run it from a cgi script or from php.

As an added bonus, that script is so nicely written you can make other tables and make other things happen. It's a nice way to keep restrictive permissions on your webserver, but have nice, protected php pages that can make 'requests' that get executed at certain times.

I have a copy of it that lets people change users listed in .htaccess files, although I don't currently have an interface to it. And I have one that will create apache config files and empty directories with the right permissions, and then restarts the web server, for when domains get added.

Personally...

[Shads]

... after hosting using Exim3 and Exim4, Postfix, and Sendmail... if i were doing a "Large" config again (read 1000+ domains, 30k+ accounts) I wouldn't consider anything *but* sendmail. It's not the easiest, newest, or anything like that, but it does scale extremely well. The setup I'm currently using (about 10 domains, 70ish accounts) is:

Exim4 SMTP
Dovecott IMAP and POP3
Bogofilter
Spamassassin (SA-Exim)
Clam-AV

It's a rocking system, I'm currently having about 18000 messages a day tossed at me of which about ~17000 are spam. My personal accounts were getting about 2500 spam/day until I enabled all the anti-spam software and virus removal. I now get about 1-2 Spam a day and I've not had a single false positive.

For a small mid range setup I would probally use exim4. It's simple, has great features, and it's nice to have spamassassin at smtp time instead of having to process the entire message.

I don't recommend standard RBL's, however, the URI RBL's are *extremely* effective and an order of magnatude more sane in what they block (eg: if the message contains a link to viagraforyou.com it blocks the message, rather than blocking random dsl servers and /16 netblocks of ip addresses to catch a single spammer... some of the standard rbl's are nutzo.)

Theres a nice tutorial and informational link about using all the good features of sendmail and several additional ideas and theories on what is effective and what isn't at http://acme.com/mail_filtering/ the guy gets *insane* quantities of mail (mostly spam) and tells how he deals with it.

Synopsis: Large site- Sendmail, Medium/Small Site- Exim4.

Alot of people like qmail and postfix over sendmail and exim, but I just don't care for them having used them. Although if forced to choose between postfix and qmail it would be qmail.

QmailToaster

[T4D]

You might want to check out QmailToaster. It's free, supports multiple domains, has a web interface, and has SPF and ClamAV integration.

Postfix + Cyrus + SQLGrey +DSPAM + Mysql

[madstork2000]

I own a small hosting company. I have setup my business so that all accounts (except shell accounts) are stored and authenticated against MYSQL databases.

For that reason I chose Cyrus as the actual local mail system. It supports IMAP / POP3 can be scaled pretty easily. And despite reports that it is hard to configure, I have found that it really is not too bad if you keep things simple.

Currently I host about 3000 domains, and roughly 5000 email accounts, though most are nothing more than SPAM traps.

If you do go this route, the key is a reliable and robust MYSQL server(s).

The main advantage of MYSQL based virtual acounts is web-based management is trivial. ADD / UPDATE / DELETE can be done simply by updating a record.

The draw backs I have found are: a database/DB Server is an additional point of failure. Replication has been a bit tricky at times. Do not run DSPAM in the same database as your user / hosting accounts.

-MS2k

Another place to ask advice

[hadaso]

You might want to ask your question also at the forums at emaildiscussions.com. There is a subforum there for "setting up an email service" and there are several active participants that are email admins running operations like yours or bigger (or smaller) that can give you good advice.

Outsource It

[Vagary]

Another solution would be to contract with a third party. There are companies that do nothing but provide you with email solutions. They can do this based on very strong commercial products. These companies themselves will host and run the hardware for you. They will do all of the configuration and deployment and maintenance and administration for you. I'm not familiar with their prices, though - but do look into it. The upgrades and crashes and migrations are their responsibility. Meeting QOS is their responsibility. They will deal with the commercial mailserver vendor(s) for you. They already have support contracts with them. All you do is tell them how big of a deployment you want and you're set.

As someone who used to support an email server for a living, I'd have to strongly second the recommendation for outsourcing. Email is not your main business and you don't have enough customers to afford learning enough to give your customers the service they'll demand. (Although maybe it'd be cheaper to make those mistakes now than when you have so many customers you have no choice but to do it yourself?)

95% of your customers would probably be happiest with addresses forwarded to GMail accounts that are configured to put the forwarded address in the From: field. Think about it.