Ask Slashdot: Self-Hosted Gmail Alternatives?

linkedlinked writes "I'm tired of building my sandcastles on Google's beachfront. I've moved off Docs, Plus, and Analytics, so now it's time to host my own email servers. What are the best self-host open-source email solutions available? I'm looking for 'the full stack' — including a Gmail-competitive web GUI — and don't mind getting my hands dirty to set it up. I leverage most of Gmail's features, including multi-domain support, and fetching from remote POP/IMAP servers. Bonus points: Since I'm a hobbyist, not a sysadmin, and I normally outsource my mail servers, what new security considerations do I need to make in managing these services?"

Comments

Spam filtering

[josgeluk]

Well, for starters, you want a damn good spam filter.

Re:Spam filtering

[wagnerrp]

The problem with CRM114 is that it let's anything past if you just put 'POE' in the title.

Re:Spam filtering

[wagnerrp]

The C.R.M. 114 was a radio transmission discriminator in the movie Dr. Strangelove. The spam filter was named as a reference to that movie. The discriminator would only allow radio transmission prefixed by a three character code phrase dialed into the unit. It was intended to prevent unauthorized messages from being received by nuclear bombers on their terminal attack. In the movie, the passcode used was 'POE', standing for Purity Of Essence, a phrase repeated by a base commander who drank only rainwater and grain alcohol, afraid the Russians were attacking by poisoning the drinking water and contaminating our natural bodily fluids.

Re:Spam filtering

[baptiste]

I think the whole exercise is short sighted. I've been there, done that. The amount of effort to keep everything running, updated, configured, etc is a PITA. Setting up a solid spam filter is a huge undertaking because it's a multi layered approach. SA or equiv, various milters, and more and you still won't come close to GMail. When I finally gave in and decided to switch to Google Apps I was floored by the improvement in Spam filtering. Are there quirks with Google' stuff. Sure. But they are improving it. I finally today got most of my stuff tied to my personal count migrated to my Apps account. The family enjoy using their apps accounts too compared to what we used to have. We've used IMP, Squirrel Mail, ROundcube, and others. Roundcube is the best in that group interface wise, but is still very buggy. Was Horde fun to play with way back before Google's services existed? Yup - because they were something not easily done elsewhere. But now? So good luck - it certainly can be done, but to be done right requires a lot of effort that's only worth it if you have nothign better to do or are a internet services admin at work and like to tinker at home. And even then... I can spend all that time spent screwing with my internet 'stack' and apply it to better things now that Google just handles the day to day stuff. Am I concerned about them 'owning' me - maybe a little. But so far, they've not done evil to me. Plus even if I wanted to migrate all my stuff back to a personal server again, Google Voice is the deal breaker for me. Can't live without it.

Re:Spam filtering

[leighklotz]

When I finally gave in and decided to switch to Google Apps...Can't live without it.

Just hope they don't delete your account.

Re:Spam filtering

[tm2b]

Can't live without it.

And that makes you Google's bitch.

Re:Spam filtering

[Firehed]

2 spam/day? I consider something very very broken with gmail if two spam messages per month hit my inbox - across all of my Google accounts.

Don't get me wrong - props for building something useful, and even more props for open sourcing it. But objectively, the collective knowledge of Google's engineers combined with the sheer volume of email they can analyze is going to produce a far more accurate tool than any single person could ever hope to produce. I also despise email interfaces that aren't gmail, although that's obviously a matter of taste (I'd put using Outlook just over fellating a goat on the list of things I want to avoid)

Re:Spam filtering

[RoFLKOPTr]

(I'd put using Outlook just over fellating a goat on the list of things I want to avoid)

Fellating a goat isn't that bad.

Don't you want it to just work?

[Anonymous Coward]

Especially with email, I like the fact that I'm not going to accidentally break something, miss an email and lose my job.

I also like that I'm not updating everything all the time with security updates. Google does all that for me.

I also like the integration between all the services.

I also like the two-factor authentication. (Good luck getting that set up on a self-hosted system, I suppose you could use X.509 on a USB drive or something).

Don't fix what ain't broke.

Re:Don't you want it to just work?

[DrgnDancer]

I've been a sysadmin for about 15 years now. I used to host all my own e-mail, my own website, all that stuff. I had a webmail interface (Squirrelmail), spam filtering, IMAP, blah, blah blah. Then about 6 years ago I got deployed to Iraq. I couldn't use SSH from the DoD network, so updates became a big issue, spam became an issue as I couldn't maintain my filters easily. After a couple of months I went hosted on my domain. Web based admin tools meant I could maintain stuff without SSH, they had a much less "hands on" backup procedure (at the time mine involved CDs), the service was down less often than my DSL used to be... Honestly at this point I can't see the value in maintaining all this stuff for myself. I pay less for hosting than I would have to pay for a "business class" DSL or cable line for the static IP, they handle most of the hard work, and what they don't handle, I manage from a web based dashboard.

There are tradeoffs and disadvantages, but for 80-90% of personal uses cases I can't see why you'd want to personally maintain a server these days. If you simply enjoy doing it, that's one thing. If you have a business of any size, again, there's a good argument for self hosting. For most people though, just pay someone to take care of the grunt work for you. You'll have less downtime, and spend a lot less of your free time fiddling with it.

zimbra

[lampsie]

Grab yourself a Zimbra appliance from http://www.turnkeylinux.org/email - up and running in a few minutes, and it should give you most of what you'd expect coming from Gmail.

Roundcube

[Compaqt]

SquirrelMail is awesome for being simple, fast, and non-JavaScript.

If you want something more JavaScripty, there's Roundcube.

It's not gmail, but the point is your data's yours.

Postfix/Dovecot setup tutorials

Email is public anyway.

[Xiph1980]

You do know that whatever email solution you choose, unless you use full encryption in all your email messages, outbound and inbound (good luck with that) it's still pretty much in the open, and anyone who knows what they're doing in the intermittant path, especially your internet provider, can intercept and read (parts of) those emails?
At least google has proven their worth with standing up to the US gov't in stead of just bending over and giving them all plus some extra as some others have.

Re:Email is public anyway.

[CastrTroy]

One wonders why it has to be so public though. You can easily set up secure login and data transfer to your own servers. I would wonder why email servers wouldn't be able to set up secure services for sending mail between them. Sure at one time encryption may have been too much extra work, but now it seems like it would be quite advantageous without having too much extra load on the systems. It would be really nice if you could request a message to be sent using encryption between your mail host and the destination of the mail. Sure, it may or may not be encrypted when it is stored (it could be) but at least nobody could read it as it goes over the wire. They could use the existing ssl infrastructure to easily accomodate secure communication between email servers.

Re:Email is public anyway.

[Anonymous Coward]

Uh, look at the headers in one of your mesages. Nowadays, most mail transport agents ("email servers") will use encrypted channels for talking to each other. In a typical 2011 setup, there will be an encrypted connection between the sender's mail user agent and his mail relay, then an encrypted connection between that one and the destination user's mail transport agent, then an encrypted connection between the destination's imap server and his mail user agent.

It's "open" in that this is transport security, not end-to-end encryption. The message is still present in an unencrypted form on each of the systems involved in transmitting it. And none of these steps are guaranteed to be encrypted -- normally, the MTA will just fall back to an unencrypted connection if the other host does not advertise that it can receive encrypted data. But it's not exactly public, either.

Re:Email is public anyway.

[PAjamian]

There is, it's called TLS (which is the same technology that modern SSL uses, so the same encryption used by https) and is implemented by STARTTLS. It establishes a secure connection between two email servers and sends the email off secure between them and it suffers from the following pitfalls:

1. It only encrypts the data stream between two email servers that support it, or between the email server and client.
2. The email is still decrypted and stored plaintext in the queue of any given email server, and is subject to reading by the admin of any server in the chain.
3. It relies on each email server in the chain supporting TLS (most do, but there are still old ones out there that do not and the ones that do will generally fall back to unencrypted email if need be to communicate with an older server).
4. While it is possible to purchase and verify certificates between two servers no one does because a lot of servers use self-signed or invalid certificates, so verifying them would simply cause a lot of email communications to fail, thus it is susceptible to a man in the middle attack.

The best way to secure your emails has been and still is to use PGP (and before someone says it, that includes GPG), which secures the email end-to-end and so it is not subject to any snooping or attack in between with the exception that the envelope sender, recipient, and email headers still have to be sent in plain text. Of course this requires that both the originator and recipient of the email both have PGP support installed on their email clients and it requires the maintenance of PGP keys be done by the end-user, so it is more complicated than the vast majority of email users are willing to commit to.

Try Zimbra!

[i_want_you_to_throw_]

My company uses Zimbra. It works pretty well for us.

Re:Try Zimbra!

[msobkow]

I'll second that. Zimbra has the best HTML interface of any web-based email system I've ever used, but still supports "traditional" email clients as well.

Re:Try Zimbra!

[ZorinLynx]

>betting on a Google server failure or data breach

Nah, it's not this.

Google has proven they can no longer be trusted. People have had their accounts suspended for "name violations" and other perceived ToS violations that have led to loss of access to Gmail. They CLAIM to have fixed the issues, but then I see more people suspended. Also, their appeal process is inefficient and unreliable (I know one person waiting over two weeks now and their account still isn't fixed)

Roundcube

[wolrahnaes]

The best webmail UI I've used other than Gmail is Roundcube. It's simple, clean, and works quite well.

Re:SquirrelMail?

[wolrahnaes]

You have to be blind if you consider Squirrelmail anywhere close to comparable to a modern interface like Gmail. It pretty much embodies the visual style of '90s Perl scripts, and that's certainly not a good thing.

Re:iRedMail

[tjhart85]

He can't use that! It's hosted on Google Code!

why?

[mr.dreadful]

As a guy who ran email servers for a small organization, let me say enjoy it while you can, because email admin is a never-ending pain in the butt. The spam management, the 24x7x365 server monitoring for security issues, the blacklisting and DNS issues, and that people get really bitchy when their email service is disturbed in any way.

That being said, I hear nice things about Zimbra.

Re:why?

[dbc]

Spot on. I ran my own full mail server for a while. It got old very fast. You really need at least two servers for fail-over and simply the ability to down one while you update the other. (And those two should be geographically separated so power outages don't take out both, etc.) *blech* So in the end what I've done is just have simple pop accounts, and then use fetchmail to pop the mail down to my own IMAP server. If my server goes down, I don't care, the mail just spools up at the ISPs (yes, multiple). If things go totally haywire, I can repoint the clients directly at the pop accounts and keep mail flowing -- of course I give up the convenience of IMAP in that case. Anyway, by outsourcing the core POP account you offload all the DNS issues, can get spam filtering if you want it, and get relief from the 24x7 server(s) health monitoring. I like the increased privacy over having a hosted IMAP service.

Re:why?

[1s44c]

As a guy who ran email servers for a small organization, let me say enjoy it while you can, because email admin is a never-ending pain in the butt. The spam management, the 24x7x365 server monitoring for security issues, the blacklisting and DNS issues, and that people get really bitchy when their email service is disturbed in any way.

I also manage such things. I don't know why you say it's a never-ending pain because that's just not my experience. I use BIND, Postfix+Postgrey, DNSBLs, Spamassassin, ClamAV, SPF, Cyrus, Roundcube, and Nagios monitoring everything. Every now and then I get someone panicking because he hasn't got mail for 4 hours, and every now and then I have to investigate where a specific mail went wrong. Every 2 years or so I rebuild the systems on a newer distro and in the mean time I apply updates as needed. I have learn/spam and learn/ham folders that all users can dump spam and ham in and spamassassin is trained from those. It is work to look after these things but I would not call it a never ending pain in the butt. Most of the time it just works.

I totally agree about people getting bitchy when their email is disrupted in any way. I did have to go to work on xmas day once to reboot a crashed mail server. Guess it serves me right for using an old dell server for a critical service.

Re:why?

[Vellmont]

I've hosted my own email for the past 15 years, and I simply don't see the problem you're describing at all. Spam is well handled by spamassassin. I've never had blacklisting or DNS issues. With just YOU controlling everything, and not multiple people, the change management problems are minimal. If you choose software with a proven track record, then the security problems become minimal. Install all your software from a linux distribution with multi-year support, turn on auto-updates, and the security problems mostly go away from all but the most dedicated and skilled attackers. You're a lot less juicy of a target than say Google, so the skilled attackers don't really care about you anyway. If it's just YOUR email, then the people getting bitchy is just you. I'd never host email for someone else. The only real issues are when the internet connection is down. Even then, you can get to any old mail, but new mail obviously doesn't come in. Even that you could fix with a low priority mx record pointed to a gmail account.

The one thing I would caution is you need to know what the hell you're doing. The OP said he was "a hobbiest and not a system admin". Well, if you want to host your own email, you'll soon learn the skills to be a real system administrator, (or give up and go back to hosting).

Re:why?

[Anonymous Coward]

Spot on. I ran my own full mail server for a while. It got old very fast. You really need at least two servers for fail-over and simply the ability to down one while you update the other.

No, you don't. If you are running a mail server just for yourself, you know it is going to be down, and you are probably trying to get it up again instead of reading mail. Other MTAs are required to hold on to mail they can't deliver for up to three days. If they don't, you probably didn't want that mail anyway.

You do not need two servers for anything else than the DNS entries.

Re:why?

[elbles]

I agree completely. I started hosting my own e-mail server when I was in college (~6 years ago now), and I've been running it ever since. I did a lot of learning as I went along, and the setup has been about as stable as you can possibly expect it to be running over a home connection. Just in case though, I threw in a VM from Linode earlier this year (initially acting as my primary MX and forwarding to my home server, but now acting only as my backup MX), which brings the reliability up to a pretty good standard for personal e-mail. Plus, it gives you a public IP with reverse DNS, which can easily cost you another $10-15/month with cable or DSL, if they even offer it on a residential package (and it's a huge boon for a sending mail server, beyond simply using your ISP's mail server as a smarthost).

That Linode VM is only about $30/month, and it comes in handy for lots of other things. If it's a hobby, it's well within the realm of affordability. Can't recommend them enough for something like this (their competitors are probably good too, but I only have personal experience with Linode).

All in all, if I spend 2 hours a month maintaining the setup (generally upgrading ClamAV), that'd be a lot. I use CentOS+Sendmail (been running Sendmail since the get-go, don't have much motivation to swap it out) out of the box, with custom compiled (latest-and-greatest) versions of SpamAssassin and ClamAV.

Re:why?

[darkmeridian]

You don't understand why it's a never-ending pain while you're detailing how you have to do all of that stuff, which sounds like a full-time job? Look, if you're a sys admin, then it's not a never-ending pain; it's your job. But if it's not your job, it's a never-ending pain.

For many other people, email is mission critical. If you lose your email or lose connectivity for a bit, you're fucked. So yes, you can go through all that effort to run your server on a Dell and risk losing all your data, but it's not as easy as you're saying.

Been doing this for a decade...

[emx]

... and I can safely write that there is no way you will ever achieve anything comparable to gmail.

You can try:
- squirrelmail, ugly and so last century
- openwebmail, old-fashioned Perl webmail, not maintained any longer
- zimbra mail, lots of functionalities and fancy features
- roundcube, decent but nowhere near what you're hoping for

Spam control on the server side is going to be an issue. You will have to use a combination of solutions (e.g. custom sendmail configs, RBL/XBL blacklists, spamassassin, greylisting, procmail rules, smf-spf, j-chkmail) and it will take quite some time and effort to get everything fine-tuned.

For anti-virus, clamav works well

For IMAP I found that dovecot does a decent job. If you want to fetch from remote servers into your own server then fetchmail can do the job.

Usual security considerations apply - patch early, patch often.

You will spend long hours maintaining this, highly recommend using a log colorizer to help watching logs e.g. ccze
In the end you will feel you got a half-baked solution that doesn't even come close to comparing to gmail in terms of functionality, ease of user interface, security and spam control. But hey, it'll be your own stuff.

Maybe not Zimbra

[jra]

I've run Zimbra for 3 years now, back to 5.0.9, which I installed for my then employer. The architectural people there have taken, right along, an attitude that I can characterize only as "RFCs? Who cares about those?"

It doesn't handle fixed-pitch well; its editor won't re-wrap (though they might have finally fixed that in 7), it doesn't uknow from RFC 2369 -- in fact, it handles mailing lists poorly in general; notably, you can't change the Reply-To in any way when replying, if you generally want HTML off (as I do), the only way to turn it on is to dive into the Preferences and switch it, then reload; same turning off...

Check for bugs filed on their bugzilla by jra@baylink.com if you want a full list of the ignominy. But in general, I would say: evaluate it pretty thoroughly to see if you can deal with its crap before deploying.

It's *very* pretty. I just don't know if it's worth the trouble.

Thats funny

[WindBourne]

For over 15 years, I spent my time doing my own servers. Figured out that I was spending too much time doing server admin and not enough building sand castles. Now, I am on Google.

Re:Thats funny

[Pieroxy]

Same here. I had an online agenda, mail, address book and all that running from my dsl box. Things were fine. Now I am full 100% Google. There is no way anyone is going to approach this level of polish with a 10 foot pole with open source stuff. You can get things done, sure, but it's going to take a heck of a lot of time and the result will be nothing compared to Google. But your data is yours. Can't beat that.

Re:Thats funny

[JAlexoi]

I don't have a target painted on my forehead

And that will be your undoing. As an admin you should feel like it, always. Because you are responsible for security. If you think small sites don't get hacked, then where are the spam relays coming from? A crapload of them are small MX'es hacked....

Re:Thats funny

[MojoRilla]

So you offer two factor identification? SSL webmail?

Gmail does.

Security through being small isn't security.

Re:Roundcube

[booch]

I'm also a big Roundcube fan, and use it on several sites. The nice thing about it is that you can just point it at an IMAP server, and it uses the IMAP server for authentication. It's quite easy to set up, and the GUI is a lot nicer than other competitors, like SquirrelMail.

Zimbra is nice too, but seems to lock you into a full stack of software. (There have been promises of a stand-alone version, but I've never been able to find it.) That might be the right answer for the original poster, but I found it too limiting and inflexible for my needs.

You don't want to do this.

[cshark]

The whole beauty of gmail isn't that you get a lot of neat features. It's the fact that your email almost always gets from point a to point b. This is because you have the luxury of being on a "big" mail server. Smaller mail servers, like one that you or I would set up do not get special treatment. The whole system right now is stacked against small mail servers. The minute you hit operation, you'll find that you might already be on spam lists, and that you have to fight to get yourself off of them. The minute you find that you're off the lists, you'll probably end up back on them because someone three ip addresses away has been sending welcome emails from his web site, and someone forgot that they asked for one.

If none of that scares you, the following list will get you close to what gmail can do.

So here is what you need first and foremost:

1. A dedicated server just for Zimbra with Domain Keys installed
2. A block of 24-32 ip numbers. (49 ip numbers would be ideal, but it's harder to buy odd blocks like that.) Put your mail server as close to the middle of that range as possible. It sounds like a lot, but most collocation facilities can hook you up with this for 300-500 usd a month.
3. Proactive attention to getting your ip block removed from all spam lists (especially Barracuda, their list is the most annoying for the high number of false positives) before the fact. Just let them know you exist.
4. Pray that all of the hundreds of moving pieces you've just put in place don't break, that bad hackers don't brute force their way into your server. Strong passwords don't really help as much as people tell you they do either. That's now something you have to worry about too.

So there you go.
It doesn't make sense to me that you would try to do this for something that only you would use.
The expense is too high, and the benefit just isn't there.

Over the last few years, I've been offloading my email to the social networks and blogs. Facebook, Linked In, personal Drupal installations, Twitter, etc.

They don't have a lot of the core problems that email has, and pretty much everyone I communicate with will use one or multiples of those.

For everything else, I use Gmail for domains because, even if I end up upgrading and paying per account... it's still less of a headache than the Dante inspired hell that is managing my own email server.

I hate running fucking email servers.
Hate them.
Hate.
Hate.
Hate.

There. I feel better now.

Re:You don't want to do this.

[rsborg]

Remember, telling a friend, relative, or business "*Your* system is rejecting my RFC-complaint emails. *You* should look into fixing that if you want to hear from me." is perfectly acceptable, even though a business telling you exactly the same thing isn't.

I don't know who your friends or relatives are, but if anyone told me that, I'd personally put them on my ignore list, as I have way too many things to do, and have about 0 control with any of my email "systems".

This kind of message is similar to "your government is not accepting my packages, *you* should look into fixing that if you want my deliveries"... it may be valid, but good luck with getting any meaningful response on that.

Why #2?

[theNAM666]

The previous "why" poster has it right. It's like you're complaining about success. You are never going to do it 50 percent as well as Google. -- don't try. Rolling your own is an academic exercise. Zimbra is ok-- if you can live in the 90s. Google is it. Just backup your data.

Re:Not Worth it unless you have a very good reason

[Junta]

The open source web mail interfaces are not even close to what gmail does.

On this point I have to disagree. gmaill is highly capable and all, but I actually prefer roundcube's interface over gmail's.

I also disagree that maintaining a mail server competently is that hard for a single domain with maybe a half-dozen users. If you stick to packages provided by a linux distribution, distribution updates will handle most security updates. Many ISPs have blessed relays for your use that alleviates the blacklist problem significantly.

That said, I have co confess current state of gmail makes it hard to find reason to do it yourself. The only reason I could think of is fear for what google could do in the future given the fact they really can hold your email address hostage. If you pay for your own domain (using any subdomain like offered by dyndns or co.cc is begging for them to hold your domain hostage down the road (as dyndns already has done to its users), landing you in the same place. Since so many free offerings from other companies have either evaporated or 'altered' in unacceptable ways, it's not unreasonable to be wary of Google's take on the perceived business value of free email with ads. If data suggests the cost is higher than the revenue sometime later, say goodbye to your email.

Re:roundcube

[digitalchinky]

I hope this doesn't sound too negative, roundcube is pretty good, but I've found it to be somewhat troublesome if you're running it over SSH - accessing it across a slow network, or any network with latency issues (wifi / 3g etc) tends to bog apache down to the point where it (apache) can require a restart. CPU load is pretty high when this happens too. The ajax thing isn't so good for trigger happy mouse clickers either - it often just stops responding, the only thing that gets it running again is a page reload.

For sure it looks a whole bunch nicer than squirrelmail, but it doesn't have nearly as many features - also configuring plugins for roundcube can be an exercise in frustration at times, not a whole lot of documentation - the forum while very helpful, tends to assume you know a great deal about the underlying mail server as well as systems administration in general.

How to properly run a small mail server

[toygeek]

1) Install Linux
2) Put all the software on it
3) Be happy with yourself for mail actually working
4) Get blocked by your friends email hosts because they have no idea who the hell your server is
5) Learn about reverse dns, all the fucking host entries that you have to add so that you don't get automagically blocked by half the populated world
6) Some asshole user sends email with no subject and an executable attachment, it comes back to them bounced and they scream at you.
7) Same asshole user bitches and moans 3 times a day about how much spam they get and what a piece of shit your server is

This ends up with the following consequences:

1) Give up your life as an actual person. You're now a mail server admin
2) You stop giving a shit about said asshole user.
3) You start to second guess your decision to run your own mail server after somebody exploits something (weak password from asshole customer?) and sends half a million spam messages, and 2/3 of them bounce back at you.
4) You start growing pale and have hideous dark bags under your eyes
5) You're "that guy" in your apartment complex ("he never leaves!")
6) Eventually you miss your life, the outside world, and what is left of your sanity.
7) You start prioritizing your life and you finally give up and.....go back to Gmail.

You're opening the door to a world of pain.

[veg]

It's ironic for me that you should post this on the day after I just abandoned my last home-maintained mail server in favour of Google.
For the past 15 years I've been a mail administrator in some capacity for a variety of mail systems ranging from my own personal colo to a vast multi-national corporation. Solving the technical problems of building and maintaining a functional and reliable system was fun for a number of years, especially when email was dominated by geeks. But nowadays, running your own server is a perpetual nightmare.

First, there's the problem of where to host it. It has to be accessible wherever you are, and it has to be able to send mail out. If you're planning on hosting it at home, on the end of a cable/DSL/fios connection, bear in mind that your IP address will almost certainly be blackhole listed. Also, your ISP may well be blocking outgoing mail to prevent spam. You will probably have to configure your system to route all out going mail via your ISP's SMTP server. Why are you hosting an SMTP server again?
If you're hosting it in a nice VM or in a colo, you're better off, but paying. Google costs you nothing.
Next, storage. Obviously that's no problem because you have a mirrored RAID eleventy-five array you built yourself. If that's in the colo then you can forget about it - except when a drive goes bad or it crashes unexpectedly. But then it's fine because you're paying for support aren't you. And backups. You are backing it up aren't you?
Next the server software. Personally I've had a lot of success with Sendmail/Cyrus IMAP/IMSP/Squirrelmail and friends, despite enduring jeers from other sysadmins who think they have a better combination. In the end, it doesn't matter. They all suck. They all need patching regularly. They all break. They all need tweaking on a regular basis.
Then the final turd in the swimming-pool: spam. It costs you so, so much; bandwidth, around 95% of all of the inbound traffic is spam; time, configuring and maintaining spamassassin and various blackhole lists that occasionally start rejecting mail indescriminately; pride, the only time your clients contact you will be to ask why the mail is so slow and why there's so much spam. "But my gmail doesn't get this much spam - can't you filter it" they say, while you bite chunks out of your tongue. Spam to a mail administrator is like the gopher in Caddyshack: it will keep you awake and turn you into a monster. And the day will come where you, spam-slayer and junk-mail terminator, get put on a blackhole list for being a spammer. That's really fucking harsh the first time.

I could go on. but we're already in the TL;DR territory.

Most people do not host their own mail server. They live longer and healthier lives as a result. Follow their example and let Google worry about all of that for you - and in return you just have to pay them...nothing.

Re:You're opening the door to a world of pain.

[discord5]

Personally I've had a lot of success with Sendmail/Cyrus IMAP/IMSP/Squirrelmail and friends, despite enduring jeers from other sysadmins who think they have a better combination. In the end, it doesn't matter. They all suck. They all need patching regularly. They all break. They all need tweaking on a regular basis.

This! Even on the software side of things, it's constant fiddling and tinkering. I spent about 7 or 8 years administrating qmail and postfix. If it wasn't Spamassassin or the anti-virus going haywire, there would be some other issue. Some braindead mope setting his password to something ridiculous resulting in a flurry of spam sent out a week later, some guy infecting his laptop with something nasty and sending out a fuckton of spam... A bug in all the shit that glues qmail, spamassassin and the anti-virus together that generates a veritable shitstorm of bounce messages to yourself, resulting in more bounce messages to yourself until finally the queue is stuffed with bounce messages...

Of course, nothing would be complete without the mail queue going corrupt. And once that happens you know you'll be making a tarball of that sucker and cleaning it as fast as possible to get it back online. After that you get do something fun, that's digging through the mailqueue with some obscure shell script from some guy who actually had this very rare thing happen to him too that one time, only with just a small difference, so it won't work out of the box of course. Oh, don't worry, at times like these there will be absolutely nobody breathing down your neck, especially not the person who told you to go F*** yourself when you suggested that it might be a good idea to not be so dependent on a single mail server.

Then the final turd in the swimming-pool: spam.

And the problem with spam is : once you've mitigated the issue you just KNOW that by this time next month you'll be at it again and again and... And then there's the problem of false positives. If someone so much as suspects having a false positive there's hell to pay. "You marked this as spam but this is an actual e-mail". Not "The mailserver marked this as..." but YOU.

various blackhole lists that occasionally start rejecting mail indescriminately

Oh, don't worry, the foam you have at the mouth that day can be reused in meetings about why the mailserver was rejecting all incoming e-mails.

the only time your clients contact you will be to ask why the mail is so slow and why there's so much spam

Or why they can't send out an attachment of 4GB, why their mailbox is full, why their mail from russianbrides.com isn't coming through, ... Oh don't worry, deep down you know by the sheer volume of mail you handle daily your users love you.

put on a blackhole list for being a spammer. That's really fucking harsh the first time.

That was the breaking point for me. I simply gave the mailserver an IP in a range that wasn't blacklisted and started looking for a new job. On my way out I congratulated the guy who was promoted to the new mail admin and whistled a merry tune as I shut the door behind me. I vowed never to touch mailservers again in my life and became a better person because of it.

Take this advice and heed it well : Unless you have a REALLY good reason to do your own e-mail, just fucking don't. I'm sure that a lot of people are going to say "Run qmail", "Run postfix" or "Run sendmail" or whatever and point you towards a lot of incredible HOWTOs, but the truth is that's just the beginning of it, and it will slowly devour more and more of your time until one day somewhere between 10PM and 1AM you're upgrading some part of the mailserver again and wondering to yourself : "What happened? I used to do so many cool and interesting things..."

If you don't want to deal with Google, find a reliable company you want to deal with and have them do it for you. Running a decent mailserver is just a pain in the ass.

Re:jklf;asdfj;a

[PNutts]

lkjdsafj;dklas

An excellent example of a self-hosted mail server with encodings issues.

SMTP is usually encrypted these days

[billstewart]

If you're running the SMTP server on your machine, and set it up to accept encrypted SMTP, most SMTP MTAs systems will encrypt mail to you and your ISP won't have access to it. The real issue is getting other people to accept SMTP from you, as opposed to deciding that any home internet connection that tries to send mail is a spam botnet zombie.

And gmail may not be proactively handing the Feds everything they want on a whim, but if the Feds hand them a subpoena and a "don't tell the customer" order, they'll hand over your mail, IP records, and anything else in the subpoena, and won't tell you, because they don't have a choice.

Re:SMTP is usually encrypted these days

[unencode200x]

A PhD student did some great research on how the process of state and federal authorities getting your info/email from various ISP/providers. It's very, very scary stuff. No subpoenas needed for a heck of a lot more than you'd think:

Here is the actual video (part 2 is really good); well-worth watching:

DEFCON 18: Your ISP and the Government: Best Friends Forever 1/3

If you have a NAS...

[AliasMarlowe]

Some NAS devices support a complete email server, even if it's not always installed or active by default (usually it's not). We have a Synology NAS, and use its email server to combine local email (for our dyndns "domain") and accounts on a number of external hosts. Since it's on the NAS, useful features such as automated backup to external disk include the email with little extra configuration.

In the land of the free...

[Requiem18th]

Even before opening this article I knew it would be overflowing with cries to drop this self-dependency stupidity and just surrender to the corporate gods.

What the fuck?

What is the purpose of free software if you are not supposed to use your freedom? You can build your system using open standards, install an open source OS with an open source mail server. But you will get blocked because you are not a business? More over, what is the purpose of freedom when you are not supposed to exercise it? It really has come to the point where "freedom" means "freedom to work for the system".

It should not be like this, it doesn't have to be like this. There's plenty of solutions, something like WoT can be build to prevent spam much better than a simple "block everything not from gmail yahoo or hotmail" that's just business whoring.

Re:In the land of the free...

[cgenman]

The guy is "a hobbyist, not a sysadmin" and is looking for a self-hosted alternative webmail. The thing is, unlike a lot of other parts of life, mail hosting is basically a sewer of pain. Potholes and pitfalls are absolutely everywhere. To make a bad analogy, the guy basically posted "I'd like to be more independent. So I've decided to learn to fix my car, start growing some vegetables in my backyard. And, oh yes, have a baby. Are babies hard?" All of those are valid goals, that people everywhere should aspire to. But, as the germans say, he needs to be aware of the commitment and Kindersheisse of maintaining a mail server.

And I've been on both sides of the "black-hole everyone's mail" problem. If a server is sending out spam, a single server can easily be sending out hundreds of pieces of spam to each and every one of your users per day. Chances are, that "server" is a hacked Windows XP box someone in their IP block left online (there really aren't anything other than hacked Windows XP boxes online these days). Or a server with inadequate protections that is being maliciously harnessed. Or someone put the address into a blacklist wrong. Either way, without these blacklists e-mail service as we know it would be over. And, unfortunately, there are people profiting from spam, fighting every bit as hard as the legitimate users to get off of the blacklists.

And that's without taking into account the basic technological issues, like needing redundancy and response significantly higher than take-it-or-leave-it services. If your docs server is down, you have to wait a bit to access your documentation. If your mail server is down for long enough, you lose all of those messages. Also, all of your clients get messages that your system is down, but you don't. You get hit constantly by volumes of spam, leading to waves of DDOSing. People don't back any mail up, but require it to be available forever. And, this may just be personal perception, but I swear that all mail servers are coded to be suicidal.

So yes, the effort put out to host one's own mail server is disproportional to the payoff in terms of personal information security. Because it's not building a server. It's committing to hosting an ongoing part of the mail ecosystem.

Re:In the land of the free...

[kwalker]

Speaking as a hobbyist here, I have done what you're asking about for the last eight years and I have very close to zero problems, however there was some ground work that had to be laid. Oh, disclaimer, I'm also a sys admin for a major hosting company (I won't tell you who) so my definition of "easy" may not match yours.

0) If you're hosting at home, make sure you have an ISP that doesn't suck (i.e. use a local ISP). I use a local ISP that has DSL/FTTC (If you happen to live in an area served by the FTTC) connections), so I pay the local ILEC for a DSL line and the ISP for the connection. If you're hosting at a VPS/colo, make sure you pick a good one that will help you out: sell you a dedicated IP AND either give you control of your reverse DNS or setup your reverse DNS for you to your specifications. This is actually a critical part. I have stayed on a 1.5Mbit DSL line for years because anything faster in my area removes my choice of ISP and that is unacceptable to me.

1) Setup your preferred e-mail infrastructure. A dedicated VM/box (I'm using a low-powered Via C7-based server. It draws 30W and handles more than just my e-mail) with whatever SMTP/IMAP server you want. I happen to use Postfix/Dovecot tied together with Procmail so I can do my own mail rules and interpose SpamAssassin in the chain to catch all the crap that comes in. You can use whatever MTA/MDA you want, but if you plan on using webmail, you should probably have an IMAP server for it to feed from.

a) Make your SMTP/IMAP servers secure. This involves setting up TLS/SSL, creating some certs (Self-signed is okay if any users of the system know to accept the "unverified" cert the first time they connect), and enabling SMTP AUTH. There are HOWTOs aplenty on how to do this.

b) Setup your filtering software, because spam handling is all on you. Personally I use SpamAssassin with whitelists/blacklists, some customized scoring for built-in rules, and a very old bayesian filter (Old meaning it has been in use for years and is very well trained, not that it has been neglected for years). I have

2) Contact your ISP or VPS/colo provider to make sure they don't filter ports and to request changes to your reverse DNS entries. I have a single IP address and I had problems with organizations like Comcast blocking me until I had my reverse DNS changed to my domain name.

3) Make sure you aren't part of the problem. I have a cron job that alerts me if I start picking up a lot of "undeliverable" bounce messages and I clean them out so I'm not annoying other mail admins by repeated attempts to deliver crap. You should also check various RBLs to make sure your IP isn't on any of their lists and if it is, contact them to get removed. I haven't had any problems with RBLs in years due to steps 0-2.

4) Happy mailing. Like I said in the beginning, I've used this setup for years and it has worked for me and several others. I routinely e-mail people on Yahoo, Gmail, Hotmail, Comcast, Cox, and other big ISPs and have not had any significant problems in years.

Notes
I've tried several webmail programs, squirrelmail, roundecube, even some "groupware" and "group office" ones, they all have their pros and cons, but they all talk IMAP on their back end, and you may end up having multiple devices accessing your e-mail like I do (desktop, laptop, phone, tablet, etc).

To keep your bayesian filter updated, make sure you have a "spam" and "not spam" folder. Personally I have three, a "really spam" folder (For anything scoring 20 or higher), a "probably spam" folder (For anything marked as spam), and a "not spam" folder for anything that was mistakenly marked as spam. I almost never have to think about the "really spam" folder because I have never found anything in there that shouldn't be. I do go through the "probably spam" folder weekly looking for false-positives, and there have been a few. Those get moved to the "not spam" folder. Weekly I have a script

Re:SquirrelMail?

[wish bot]

Isn't Squirrel just an interface? He's going to need something a little more than that - Postfix is the thing you need.

Now, having done exactly this for a long time (and having also moved everything over to Gmail for domains) I have a few observations:

- running your own email server gives you a warm inner glow and feeling of independence, but that's about it.
- check your logs daily, intrusion attempts happen constantly.
- dedicate the box to email only, that is - close down every port you don't need.
- don't run anything you don't need on that box.
- for the love of god don't run php (which might cut out squirrel mail).
- you'll need a set of good spam handlers. There's some good suggestions in posts below.

Personally, if you were really going to do this, I'd get a Mac mini. It comes with everything you need in terms of unix tools by default. It runs low power, it runs quiet. And there's slightly less chance of you getting owned. Always kep your patches up to date.

I eventually moved away from this because I got tired of being a paranoid sys-admin at home. Dealing with uptime issues also made me rethink what I was doing when email started to become critical to my finances - you'd be surprised how unreliable home dsl and power systems are when you really, really need them.

Re:SquirrelMail?

[ComaVN]

for the love of god don't run php

Out of curiosity, why not? Apart from attracting poorly educated programmers who use it to create insecure crap, what is inherently problematic about it?

Re:Repeat after me

[Firehed]

Do you know my password? No? Security by obscurity.

Almost all security* is based on someone not knowing something. Very very often, that something is either a password or very large random number. Or the physical pattern on a key. Or door/alarm code. Or something read via RFID. Or the algorithm that determines the number on my RSA fob. More commonly when making that claim, it's just a nonstandard port for a service, hidden URL, or combination of several.

If an attacker has the exact same set of information that I have, then that attacker has access to the same systems I do. The amount of information they need (or the level of obscurity, if you will) determines the level of security. Something where you need to be on my VPN to get access to a whitelisted IP and then SSH in to the system where password-only auth is disabled is going to be a hell of a lot harder than something where you just need to know to hit port 8080 instead. But ultimately, my passwords and private keys are just very obscure information.

And in terms of end results, not being a target absolutely makes me more secure than an equivalent system that is a target.

* As far as authentication and encryption is concerned, at least. SQL injection and XSS protection being the two best examples where it comes down to actual implementation details.

Full stack

[PAjamian]

Postfix 2.8.x for the MTA (2.8 has the new postscreen feature which is great to help with SPAM control)
Dovecot for IMAP POP3 as well as for SASL AUTH
Roudcube or Squirrlmail (take your pick) for webmail
PostgreSQL or MySQL for database backend
Spamassassin to catch what SPAM is missed by postscreen.
ClamAV to scan for viruses
Amavisd-new to interface psotfix to spamassassin and clamav
PostfixAdmin for managing your domains and accounts from the web.

Use virtual domains with postfix "virtual" for the delivery agent, use maildir format for your mailboxes (mailbox path needs to end in "/"). Make sure and use the submission port (587) for your outbound emails, not the SMTP port (25) which should only be for inbound emails. Don't use SMTPS (which works over port 465) unless you have to support a really old email client that doesn't support STARTTLS (which works over the submission and smtp ports). Stitch all the pieces together and if done right you'll have a great email system like all the pros use.

If you need help come into #postfix on freenode IRC network.

Re:That sword cuts both ways

[RoFLKOPTr]

How many teams of for-profit hackers will be targeting your personal server?

Thousands. Have you ever run a server and looked at access logs? There are thousands of bots running automated attempts to exploit any vulnerability they can find. There are no automated vuln bots that will ever make it into Google's servers. And skilled for-profit hackers don't even bother trying... there are better, smaller, more vulnerable fish that can be fried in much less time.

Re:Repeat after me

[dakameleon]

"Secret" is not the same thing as "obscure". Your password is not "obscure", it's a secret - the same goes for all your examples. Yes, security-through-obscurity is more often cited in the example of "but my port wasn't 8080!", but I mention that here because the claim that teams of for-profit hackers will most likely not be targeting your system.

Sure, that might be the case - but that shouldn't set your mind at ease when it comes to security. Chances are you're going to be using an off-the-shelf or open source mail handling software - and that means you're as vulnerable to attacks on that. If that particular program is cracked elsewhere, spammers looking for mail servers to cover their activities will find a way to identify systems running the software on default settings and exploit it. So while your particular instance might not be the primary target, you could still get hit by issues.

I'm not saying that you should stick with Google - just don't believe that you can slack on security just because you're small and so unlikely to be a target. In a fair percentage of cases, this means leaving it up to a larger entity with expertise - be it Google, Microsoft or some other provider - can be worth the trade-off.