Slashdot Mirror
E-Mail Server Setup Advice?
dhammala asks: "I am responsible for setting up and maintaining a mail server for small web-hosting type business. We currently host about 75 domains, around 100 mailboxes and due to the efforts of our sales team, we are wanting to get ready for some great increases in those numbers. I am worried about my current configuration and ease of administration. More importantly (well, at least to the customers) is email deliverability -- it seems that messages delivered to some big players are being marked as SPAM or disappearing altogether. I am asking the Slashdot community for it's insight and advise on 1) if my current choice of software/configuration is a good match for this situation and 2) if there any additional measures I might take to ensure email deliverability?"
"Here is an overview of our current setup:
I have not yet been able to get DomainKeys to work with Postfix. It was during my configuration attempts that I started to question this setup and wondered if this was the best setup for our situation.. this inquiry has lead to this posting.
In a perfect world, I would have an email server that:
Are there any other technologies or configurations that I need to implement to support the best deliverabilty rates?"
- We lease servers at ev1servers.net.
- The servers are running RHEL ES3.
- We chose to use Postfix and have it configured to support virtual users and domains mapped in MySQL tables. The reference I used to configure this setup is located here. We initially chose Postfix over qmail because it was open and over sendmail because the config files are actually readable.
- I have added in SQLGrey grey-listing for Postfix to provide a simple level of SPAM detection for our users. We are not wanting to deal with the customer service and higher box loads of mail scanning at this time. We might choose to use a 3rd party vendor to do this as needed.
- Messages are delivered locally via maildrop in maildir format.
- Courier IMAP is running to support both IMAP and POP access to the mailboxes.
- Postfix Admin was setup for easy mailbox administration.
- I have verified that our reverse IP records are correct
- I have created SPF records for all of the domains
- I have verified that our server is not listed in any blacklists (great scanner at dnsstuff.com)
- I have started to install DomainKeys for Postfix
I have not yet been able to get DomainKeys to work with Postfix. It was during my configuration attempts that I started to question this setup and wondered if this was the best setup for our situation.. this inquiry has lead to this posting.
In a perfect world, I would have an email server that:
- is easy to administer,
- supports automated mailbox setup/removal (currently I can just insert rows into my tables and the mailbox setup is done)
- supports current technologies, like grey-listing, DomainKeys, etc
- is secure
- makes the best use of system resources -- I want to get the 'best bang for the buck'
Are there any other technologies or configurations that I need to implement to support the best deliverabilty rates?"
It's not free, but great support and full everything right out of the box, including IMAP, POP, SMTP, HTTP, authentication, account management, quotas and everything else you could possibly want.
v r/home_messaging.xml
If your company can't afford it, that sucks, but I'd rather use that than try and get courier, postfix, pop3d and squirrellmail or whatever to all work together.
http://www.sun.com/software/products/messaging_sr
I recommend setting up ClamAv with FreshClam to filter out virus/worm type email. I have found it performs very well on my server. I have also found they have a very fast responce to new viri as they appear.
http://www.clamav.net/
So, I have a mail system setup, it's running around 70 domains, and 500 email accounts.
I am using courier as the mta, and courier as the pop, and courier as the imap.
The courier makes a fine MTA, but you do have to tweak a few of it's settings to make it more wideopen to allow it to connect to misconfigured exchange servers.
Other than that it has been great. I have a email account management system that I wrote that lets each domain have admin users that can add and delete accounts as they please.
I have SpamAssassin setup for some users (most of them post their email addresses in plain text on their websites) and even with that load, there is still plenty more capability in this little server.
Now, if you want a system that scales to tens of thousands of users, you are going to need to get something a little bigger than this, you are going to need to get a mail system that can distribute the messages over a number of servers. That is something I have not researched.
Courier IMAP is running to support both IMAP and POP access to the mailboxes.
I would switch to dovecot. I found the performance to be quite a bit better than Courier, and it seemed more stable as well.
I support email servers for a living. I have for almost seven years - exclusively on Solaris, AIX and NT (though I do so on linux for my personal use).
While I think that your deployment is a reasonably sane one - as far as going the OSS/free route is concerned - I agree with the other poster here who said that having nobody to blame will be an issue in the future. When your job is on the line, it's good to have someone else who is supposed to know and fix everything for you when you are hard-up for solutions. Email administrators for the largest and biggest corporations in the world don't do it all in-house. Even they contract out for support for their enterprise level products. Because their customers and bosses expect great reliability and performance and features and they don't want to wait for several days (or longer) while you read some half-assed documentation on a website, chat up some gurus in IRC and post to some web forums and usenet groups hoping for help.
Also, there is nobody certifying that the products you are using will absolutely work together. And on whatever platform you're using. They may say they've tried it on it - but I doubt in many cases they will say it's been certified through a thorough internal QA process that weeds out a lot of bugs and such.
Also, when you really must have something fixed, you will either have to write the code yourself (laborious to do, without even talking about testing and implementing). If you have a commercial product and a contract, you can present a business case to get your issue some priority and have a fix. And you can always threaten to drop the product if they don't do what you want (it works more often than you'd think).
Even when full-fledged, thorough, all-encompassing high-capacity commercial servers - the position of email admin is a full time job for at least one or more people. Using a dozen different open source products and maintaining everything and keeping a constant sandbox environment to work in (you don't want to introduce upgrades or patches or changes on production, of course!) will consume all of your available time. If you are the full-time email admin here and that is your only responsibility - have at it. But if you have other responsibilities... I think the commercial path might be better for you.
Again - I'm an OSS advocate. Yet, I feel strongly that there are some cases in which commercial software and support is valuable. Depending on the specifics of your duties and position, this may or may not apply to you. But consider it. Especially if you're going to be fairly huge some day.
Another solution would be to contract with a third party. There are companies that do nothing but provide you with email solutions. They can do this based on very strong commercial products. These companies themselves will host and run the hardware for you. They will do all of the configuration and deployment and maintenance and administration for you. I'm not familiar with their prices, though - but do look into it. The upgrades and crashes and migrations are their responsibility. Meeting QOS is their responsibility. They will deal with the commercial mailserver vendor(s) for you. They already have support contracts with them. All you do is tell them how big of a deployment you want and you're set.
After working with commercial mailservers for several years, I was ready to setup a deployment of my own for my own personal project. Not having any funds, I decided I was going to go the OSS rout. Just figuring out what would work together and what wouldn't (you have to make sure your POP, IMAP and webmail servers all use the same mailbox formats. You have a gazillion options for accounting from LDAP to MySQL, countless authentication mechanisms, etc). It drove me nuts. It was at that point that I started to see the light and the real value in what I did with commercial products. Having an entire server that supports everything you could possibly need or want in an email solution through one install and one configur
Ah HA! caught an Exchange salesman in the Act! This is astroturfing at its best ;)
It is easy to use, can be expanded to cluster servers and is reliable.
Works with squirellmail, and a bunch of other cool features. Plus the name of the company is kind of cool.
http://www.stalker.com/content/solutions.htm
I read Slashdot for the headlines, because the headlines, unlike the articles, are usually original and never duplicated
If I were you, I would ask myself at least this:
"If the building the server lives in falls into the center of the earth, but my boss wants the mail back up (not necessarily with their data, just live again), would I be able to put Postfix, SQLGrey, LDAP auth and Courier back together in less than 4 hours except for user accounts?"
If you are sufficiently detailed enough to pull that off within 4 hours except for user accounts, you probably have the bits you need to wing all the rest of the bells and whistles (webmail, MAPI integration, upgrading one piece and making sure the other 11 don't die, etc.) which eat up your time.
The main problem with going commercial (which you should consider if you're sure it'd take you at least 8 or 9 hours of a day to put this together from zero) is that the extra features (Outlook integration via MAPI, etc.) tend to cost.
OTOH, if you go commercial the extra features are not yet another thing you need to spend time on because presumably you're paying for parts that already work.
Choosing a path starts with your honest appraisal of your own skills.
(Personally, I find Postfix + SQLGrey + Postfix virtual delivery agent to rock. But I don't mind going into the guts to add a new user or set a quota on an abuser or whatever. QMail I've found to be a tad dangerous. Often the slightest typo either puts qmail-send in a mad CPU loop (which is annoying) or delivers it in the wrong place (which is really annoying) or 5xx's it (bad). Adding extra steps into the qmail handling pipeline is a nightmare and should never be done on a production box first. But maybe I suck.)
"The Devil does not know a lot because He's the Devil, He knows a lot because he's old." -- unknown
Check out inter7.com
They use Qmail which is open source. Who told you it was not?
Qmail is highly scalable and I think www.qmailtoaster.com and a few other sites provide great setups that allow you to set quotas and such.
large game sites use qmail.
Hell hotmail.com uses qmail to send emails. Not sure about the rest of it.
Inter7 can get you setup properly and provide maintenance if you have problems but otherwise their setups are self manageable.
Then, assuming you know how to write PHP code, throw away the php. It's not that good. It can't handle fields being added to the database. But writing php for database manipulation is trivial, so I'll assume that's what you're already doing.
Anyway, what you need from 'vmail' is the 'maintain' perl script. It's fairly easy to understand. Basically, you want a 'new' table in your database with new email addresses, a 'deleted' one for deleted addresses, and a 'moved' one for moved emails. So in addition to editing your main mail table, you also put email addresses in those tables when those things need to happen.
The script pulls these out of each table and does the things. It doesn't mess with your main table at all.
Now, the perl script needs to turn that email address into a directory. It starts out as hostname.dom/u/user/ from user@hostname.dom, but you can change that however you want with a bit of perl study.
Or, instead of putting the email address in the tables, you can just put the directories, and tell it not to try to make that into a directory at all. That's probably simplier if you already have the rest of the system set up with other pathnames, and you don't know perl.
I experimented with cgi scripts and whatnot, but this was much easier. You can either put in cron to run every minute or so, or you if it is important updates happen instantly, you can make it suid and run it from a cgi script or from php.
As an added bonus, that script is so nicely written you can make other tables and make other things happen. It's a nice way to keep restrictive permissions on your webserver, but have nice, protected php pages that can make 'requests' that get executed at certain times.
I have a copy of it that lets people change users listed in .htaccess files, although I don't currently have an interface to it. And I have one that will create apache config files and empty directories with the right permissions, and then restarts the web server, for when domains get added.
If corporations are people, aren't stockholders guilty of slavery?
Agreed. Postini does a great job. Been using them for years in one capacity or another. I just wish they had a consumer level solution.
... after hosting using Exim3 and Exim4, Postfix, and Sendmail... if i were doing a "Large" config again (read 1000+ domains, 30k+ accounts) I wouldn't consider anything *but* sendmail. It's not the easiest, newest, or anything like that, but it does scale extremely well. The setup I'm currently using (about 10 domains, 70ish accounts) is:
/16 netblocks of ip addresses to catch a single spammer... some of the standard rbl's are nutzo.)
Exim4 SMTP
Dovecott IMAP and POP3
Bogofilter
Spamassassin (SA-Exim)
Clam-AV
It's a rocking system, I'm currently having about 18000 messages a day tossed at me of which about ~17000 are spam. My personal accounts were getting about 2500 spam/day until I enabled all the anti-spam software and virus removal. I now get about 1-2 Spam a day and I've not had a single false positive.
For a small mid range setup I would probally use exim4. It's simple, has great features, and it's nice to have spamassassin at smtp time instead of having to process the entire message.
I don't recommend standard RBL's, however, the URI RBL's are *extremely* effective and an order of magnatude more sane in what they block (eg: if the message contains a link to viagraforyou.com it blocks the message, rather than blocking random dsl servers and
Theres a nice tutorial and informational link about using all the good features of sendmail and several additional ideas and theories on what is effective and what isn't at http://acme.com/mail_filtering/ the guy gets *insane* quantities of mail (mostly spam) and tells how he deals with it.
Synopsis: Large site- Sendmail, Medium/Small Site- Exim4.
Alot of people like qmail and postfix over sendmail and exim, but I just don't care for them having used them. Although if forced to choose between postfix and qmail it would be qmail.
Shadus
You might want to check out QmailToaster. It's free, supports multiple domains, has a web interface, and has SPF and ClamAV integration.
I own a small hosting company. I have setup my business so that all accounts (except shell accounts) are stored and authenticated against MYSQL databases.
For that reason I chose Cyrus as the actual local mail system. It supports IMAP / POP3 can be scaled pretty easily. And despite reports that it is hard to configure, I have found that it really is not too bad if you keep things simple.
Currently I host about 3000 domains, and roughly 5000 email accounts, though most are nothing more than SPAM traps.
If you do go this route, the key is a reliable and robust MYSQL server(s).
The main advantage of MYSQL based virtual acounts is web-based management is trivial. ADD / UPDATE / DELETE can be done simply by updating a record.
The draw backs I have found are: a database/DB Server is an additional point of failure. Replication has been a bit tricky at times. Do not run DSPAM in the same database as your user / hosting accounts.
-MS2k
You might want to ask your question also at the forums at emaildiscussions.com. There is a subforum there for "setting up an email service" and there are several active participants that are email admins running operations like yours or bigger (or smaller) that can give you good advice.
95% of your customers would probably be happiest with addresses forwarded to GMail accounts that are configured to put the forwarded address in the From: field. Think about it.