Slashdot Mirror

https://www.roaringpenguin.com... they provide and support CANIT PRO, which is basically mimedefang and spamassassin on a debian base, with dynamically updated blacklists and filtering rules. It works really well. David is one of the guys behind behind mimedefang, so you are also helping open source by going with these guys. The pricing for us was really decent.

They usually work with appliances, but we managed to use our own configuration to do some sweet stuff: we put the mail filtering cluster in the DMZ, along with the DB. but we put the customization interface is on an internal network. That way there is no firewall exception for the DMZ (ok except SMTP... can't avoid that one.) and the DMZ gateway doesn't need access to internal credentials at all (Active Directory in our case) It just knows that the interface machine on the inside is trustworthy. Even though the DB has no access to authentication services, the users can still customize their filtering to their desire.

I think for big companies, one concern is that I have never heard anyone rave about spam filtering. In terms of brand-awareness it is a completely one way street, Either people are satisfied with it, in which case they shrug, or they get irrationally violently abusive of the service, and have un-realistic expectations. It is a risk for any major brand to operate spam filtering, with literally no upside (ok, aside form revenue, but if it is a small part of a business, the reputation risk might outweigh the revenues.) Touching people's email brings out all the consipacy buffs you can imagine, and for some small but vociferous group they always have their own solution, and whatever the email admin does is crap. That's a thing that was great about Roaring Penguin's CanIT PRO when we rolled it out, it gave each user the ability to turn off the filtering entirely, if that's what they wanted.

It worked like a charm. Whenever we got some idiot (the truth hurts!) who thought they could do better, we just said fine, here is how to turn it off. Out of 6000 boxes, we had about 200 opt-out right away, most of them turned it back on within a few days, after a year it was down to 60 or so, and then when there were some malware infection episodes, it came out that their 'custom' solutions were not actually working that well, and everyone came back into the fold. Being able to let people opt-out saved us literally months of pointless arguments while letting us deploy good service for the co-operative many.

This was for about 7000 mailboxes, which is small as far as mail installations go these days. The real clients for this stuff is hosting providers and outsourcing companies (cloud based) I think the reason for large companies exiting the business is the huge trend of small companies to cloud, there just isn't much of a market for small email installs anymore... People are using huge hosted configurations. It's gradually getting dismantled now because of some organization move to a single outsourced solution with many hundreds of thousands of mailboxes...

One more thing.. if you have the skills, a great way to become known is to write and give away Free Software. I wrote three GPL'd software packages: RP-PPPoE, Remind and MIMEDefang which got me far more business leads than $100,000 worth of ads.

One more thing.. if you have the skills, a great way to become known is to write and give away Free Software. I wrote three GPL'd software packages: RP-PPPoE, Remind and MIMEDefang which got me far more business leads than $100,000 worth of ads.

One more thing.. if you have the skills, a great way to become known is to write and give away Free Software. I wrote three GPL'd software packages: RP-PPPoE, Remind and MIMEDefang which got me far more business leads than $100,000 worth of ads.

I run a very profitable company that started out as a Linux consulting shop.

I started my company back in 1999 when Linux really wasn't on business's radar. The keys to success were:

• Promote Linux where it makes sense. I set up plenty of firewalls, file servers, mail servers, web servers, etc. for my small business clients.
• But don't be religious. I certainly didn't waste my breath trying to convert them away from Windows on the desktop.
• But on the third had, do have some religion. There's no way I would have installed a Windows server for anyone. I would have politely declined their business, stating that my specialty is Linux. No-one ever actually asked me to do that... I made it clear up front I was a Linux guy willing to coexist with Windows machines, but not actually work on them.
• Keep your ears open and figure out what your clients want. Back in 2000, one of my clients wanted mail filtering, from which was born MIMEDefang and eventually my commercial anti-spam company that has a dozen or so employees (and, btw, that runs completely on Linux, including servers, desktops, phone system, and even my Nokia N900.)

For me, it has been a terrific 14 year ride with a great future ahead. Not a losing proposition by a long shot.

So, OSS is a nice idea, but realistically, you cannot run a true business on it, period.

Hmm. I guess my 13-year-old small software company is not a true business, then.

We run entirely on OSS. Asterisk for our phones, SugarCRM for CRM and Ledger-SMB for accounting. Linux on the desktop for everyone, including non-technical staff.

That being said: Accounting is the weak point. We outsource payroll, and when it comes time to file taxes, I give a couple of boxes of paper to my accountant and he does his magic with whatever proprietary software he likes.

I have a slide deck about our software infrastructure, mostly concentrating on Asterisk but also mentioning the other software we use.

I wrote a program called Remind and I use it to track vacation days, who has the support pager, etc.

It's very old-school UNIX. You enter all your data in a text file and it renders the calendar. I use git for revision control so it's easy to see who booked time off and when.

I'm guessing Remind will appeal to about 0.001% of the target audience. :)

I've used sendmail + spamassassin and squid for years with IPv6 on a personal level. That's not the problem. The problem is the backend database support. While even Roaring Pengiun Software supports IPv6, where do they get their database from? No major database/lookup service supports IPv6 yet. The same is true for Squid - where are you going to get your block lists and filters for IPv6 traffic when no one is selling it?

I've just had a couple of days off work with a nasty virus, and even with my head full of cotton wool I had a play with setting my Netgear DG834 into "Modem only" mode (via the hidden page http://192.168.0.1/mode.htm) and running RP-PPPoE on my linux server. I managed to get it up running IPv4 pretty quickly. Now all I need to do is wait for my ISP to start supporting IPv6. Unlike Andrews and Arnold who have been running IPv6 for ages, they don't think it will be a concern for some considerable time. Don't they understand that some of us want to start seeing if things work and gaining experience right now?

(Blatant plug) Our product has had this for years, only we do it properly. Our feature is called "Locked Addresses" and it works like this:

• The system generates a random email address using a strong random-number generator. The address is unlikely to be guessed.
• Initially, the address is in the "unlocked" state.
• The very first time the system receives a message for the address, it locks to the sending address or domain (your choice.)
• If anyone else tries to use the address (ie, someone other than the locked-to domain or address), they get a "User Unknown" SMTP error.

So not only can you give out your locked address, but it can't be sold or given away.

I have a POTS line, but I still use Asterisk plus some home-brewed programming to implement some nifty features:

• Telemarketer Deterrence: Any call coming from a non-local area code (except for a few hand-whitelisted numbers) goes to a recording asking the caller to press "1" to prove he/she is not a telemarketer. This completely kills automatic diallers. Based on my logs, I see that it stops between 3 and 7 calls per week.
• Number Recall: All my call records are logged to a PostgreSQL database. If I forget someone's number, I dial a special extension and a little AGI app I wrote prompts me to enter as much of the person's name as I can remember. It then rummages through the call log looking for matches and reads them out to me using the Festival voice synthesizer.
• Blocking of 1-900 numbers. That's a no-brainer; I don't want people in my house to be able to call a 1-900 number.
• Automatic prefix for long-distance calls. I don't have to remember to dial the magic prefix to get cheaper long distance; Asterisk remembers for me.
• 7-digit dialling. Our area code now requires 10-digit dialling. I hate it. So if I dial a 7-digit number, Asterisk automatically prepends our area code.

These are small features, but I find them really handy. Our work Asterisk setup is a lot more sophisticated (PDF slides)

If you are going for cheap, you can get a USB thermometer for like 9 bucks. I'm sure you can find them locally for a bit more.
http://www.dealextreme.com/details.dx/sku.7003

The included software is win-only. Several people have coded some linux tools of varying usefulness. You'll probably want to do your own calibrations, mine is consistantly off, but I've seen others complain of nonlinear responses.

http://err.no/personal/blog/tech/2008-07-22-10-17_kernel_patches_TEMPer_thermometer.html
http://www.roaringpenguin.com/products/temper-tools

As for IR, um...
cheap
not so cheap

Well, I did this study and our results are here.

We in no way imply that Gmail's inbound spam filtering is bad. It's probably excellent. It's just difficult or impractical for Google to filter outbound mail without either human review or complaints because of false-positives.

What we're saying is that spammers are trying to evade IP reputation systems by hijacking organizations with good reputations or which would be impractical to block. There will be a CAPTCHA-cracking arms-race, but unfortunately I think the system will reach equilibrium with spammers quickly breaking CAPTCHAs and continuing to abuse free e-mail systems.

... and use it as a tool to monitor our Asterisk system.

I've been a big fan of the CanIt spam filter for years. It's underpinnings are OSS and you get full source code when you buy the product. Their support is excellent. At an ISP I run I installed it from source and it worked flawlessly. I would recommend the CanIt-SMB appliance for your needs unless you think you'll grow beyond 100 users soon. You won't be sorry.

I've been a big fan of the CanIt spam filter for years. It's underpinnings are OSS and you get full source code when you buy the product. Their support is excellent. At an ISP I run I installed it from source and it worked flawlessly. I would recommend the CanIt-SMB appliance for your needs unless you think you'll grow beyond 100 users soon. You won't be sorry.

This is exactly why you use spam filters like MIMEDefang (or his commercial big brother CanIt). They actually do all of the spam filtering *during* the actual SMTP dialog. Ie, DSNs are not sent to forged senders. The server sending the spam does not have the opportunity to get rid of its message before the message is identified as spam. RFC 2821 permits the issuing of 4xx or 5xx error codes right up until the final 221 QUIT message. A rejection before the QUIT forces the sending MTA to handle the bounce to the envelope from.

Asterisk is great! OK, its configuration language is pretty sucky, but we've done some amazing things with it -- too long to post in a /. article. Just look at the slides instead: (1.1MB PDF file)

I didn't know there was still spam out there? I got CanIt from Roaring Penguin and don't see spam anymore.

I wrote it so I'm biased, but Remind is the smallest (about 120kB) but by far the most flexible personal calendar tool I've seen.

I'd suggest you look into Canit-Pro from Roaring Penguin. It's from the author of MIMEDefang. Actually it's MD's commercial big brother. They make an appliance but I still run the app locally on Fedora boxes. They give you the full source code. It's extremely extensible. It makes Barracuda Networks' products look like child's play. Basically it will take the knowledge you already have and give you a platform to extend and build upon it. Canit-Pro is slick. The auto-tempfail by recipient and IP is great. The regex and user controls are worth their weight in gold. By far the most essential feature that is lacking in most other canned spam filters is the ability to scan incoming messages during the SMTP transaction. That way you can reject the message as spam before you actually accept it. This eliminates the need for DSNs. Give the demo a try sometime. You'll like it.

I used to "roll my own" with SpamAssassin and MimeDefang. Then I started using CanIt at work (I liked them initially because the author is the author of MimeDefang). They have a free version that works well for me at home now. We have been using it for about 4 years at works and it does a great job incorporating grey listing, SA, MimeDefang, ClamAV, etc. into an easy to install and maintain system with a nice web interface and a database backend. It can scale well when we need it to and the support is great (a MAJOR factor for my company).

Did I mention it is cheaper than the other commercial offering as well. OSS, great support, low cost!

It's really a great system from a company that is making a positive contribution to opensource projects. We have been using it for about 4 years with very few complaints. Most of the complaints have to do with good email not getting through because I've been a bit too draconian with the settings, but it's easy to whitelist any senders or domains that trigger the filters in error.

You got screwed on the Barracuda. It's a piece of shit. I've had 3 different ones under my thumb and I am less than impressed, even in the most recent beta code. The spam filtering is marginal at best. Their Bayes methodology is a joke at the least. Their support pricing is not good at all. You would have been much better buying a Can-It Pro appliance from Roaring Penguin. You've probably heard of their famous OSS cousin, MIMEDefang, the tool that glues Sendmail to AV and spam filtering tools via the Milter library. Can-It Pro kicks ass. It actually gives you, the admin, complete control over every single aspect of your spam filtering solution. It also gives the admin to empower the end-users (power users) to give them a significant amount of flexibility in their own spam scanning. What does Barracuda give the admin in way of spam scoring flexibility? Jack shit. What does Barracuda give users of their systems? Less than jack shit. You can whitelist though. Woo. For less than you can buy an overpriced Barracuda you could have bought a Can-It Pro appliance w/ full support. You also get control over the appliance too. You can't even log into the Barracuda appliance to do something trivial like hardcode the speed and duplex of the nic or set the NTP server to your internal corporate time server. Nice.

This works well with Exchange and is simple to maintain: CanIt Appliance

remind -- you'll regret you don't thave a life complicated enough to take full advantage of it ;-)

Need I remind you of... uh... remind?

Remind is a powerful open source program available on several platforms.

We are planning a Squid implementation to proxy web traffic and there are add-ons to scan for viruses, popups, etc. I can't say how well that works just yet, but I'm very confident it will do the job admirably.

If you're having serious problems with spammers rumplestiltskinning (rcpt dictionary attacks), sendmail-8.13.x allows you to limit the number of concurrent connections per IP address, limit the number of connections per minute per IP address, and slow down the flow of 'rcpt to:' commends by calling sleep(1); after a threshold number configured in your mc file.

If that's not enough, and it wasn't for me, this one line hack to sendmail posted to the mimedefang list will hang up on the fuckers after hitting your badrcptmax threshold. Totally out of RFC spec, but when did spammers play by the rules? You'll want a script to cull through your mail logs to firewall off or blackhole route IPs which attack you in this manner too, pretty trivial.

http://lists.roaringpenguin.com/pipermail/mimedefa ng/2003-January/012863.html

Finally, I'd like to sing the praises of milter-spamc combined with the spamassassin daemon. It's written in c, very lightweight, and it offers a configuration option to deny messages tagged as spam durring the smtp transation with a 551 notification (actually, it offers a series of 44x and 55x notifications; see the docs). You can also configure it to accept the message and just tag it with X-Spam headers per normal, but giving the spammers notice that the message was even accepted just makes me happy in so many ways. And don't forget to RBL block the fuckers too. --M

Enterprise solution?

Look at CAN-IT. http://www.roaringpenguin.com/anti_spam/centralize d.php.

It's a great product, based on mimedefang and spamassassin.

It kicks ass.

True that, but there are commerical products that use OSS stuff that they didn't mention, like Roaring Penguin Software

Remind is a flexible open source program
http://www.roaringpenguin.com/penguin/open_source_ remind.php

I would highly recommend MIMEDefang which allows you to write your milter in perl which is a good thing(tm)
http://www.roaringpenguin.com/penguin/open_source_ mimedefang.php

Yeah, maybe I'll write up a more complete description of my rules sometimes that makes this more explicit.

They can still send you an email that asks if their important document arrived. So long as that email doesn't contain the original virus payload, they will get through.

And if the original mail didn't contain a virus payload either? I'm worried about false positives, not sending potentially-infected files. Here's a good example. Since you mentioned MIMEDefang, this came up on their mailing lists:

Example: someone sends a business inquiry and attaches a vcard. With the default filter, if the vCard filename includes the email address and the domain is a .com - say "My Name (here at there.com).vcf" - it will trigger filter_bad_filename. Your server discards the message, but they never get a bounce notice, and of course they never hear back from you. If you're lucky, they'll try to reach you by phone. If you're not lucky, they'll figure "Well, these people have never responded to a single one of my emails, I guess I'll take my business elsewhere." If they get a bounce notice, at least they'll know you didn't get the message.

You should check out Remind. It was designed as a cal/clendar replacement. It has an optional GUI, postscript, and html output support. The postscript output is nice.

Remind is written in simple C. The GUI is in TK. It should compile and run on anything resembling *nix, including MacOS, and Cygwin. Remind is licenced under the GPL.

MimeDefang is free, but you need to know a bit about Linux, Sendmail, etc.

CanIt is commercial, but much cheaper than any other commercial product. Installation is easy and well documented. Does much more and has a great Web front end. It's a fantastic deal for anyone who doesn't like to fiddle with sendmail settings for a hobby.

We have been using Roaring Penguin's commercial Mimedefang & Spamassassin combination, called "CanIt" for around a month now. ( Interestingly enough, Roaring Penguin seems unreachable at the moment. Hmmmmmm... )

It's incredibly effective. You can set up custom rules for identifying spam ( regex supported ), and the whole thing has a nice PHP-based interface. It was pretty easy to convince management that we needed it - typically management get the most spam anyway. And it's good to support a company pushing open source software :)

Previously I was using blacklists and my own ip-address list with iptables, but it just became too much, and this has dropped our spam from ridiculous levels to basically nothing.

Well worth a look...

As I wrote earlier, we've been doing this for quite a while. You can see our statistics here.

Greylisting will not catch anwhere near 97% of spam. Our statistics show it catches anywhere from 15% to 30%. Nevertheless, the fact that it uses hardly any resources on your computer makes it worth doing.

You can also mitigate the delay problem by having a secondary MX record. Rather than waiting an hour, a legitimate SMTP host will retry the message on the seondary MX and it will get through almost immediately.

Maybe I'm a bit late to the show, but I noticed MIMEDefang version 2.34-BETA-5 includes a new "--enable-running-on-scummy-sco" option.

I wonder how many SCO admins will actually use this option. :)

I personally use a large number of DNS blacklists. I call them from Sendmail and reject mail with them. Many people don't like DNSBLs; of course I believe these people are ignorany fools who couldn't admin a mail system if their life depended on it. That's ok. At the very least you should be able to use the DNSBLs that list open relays, open proxies, open SOCKS boxes, and vulnerable formmail.cgi web servers. We can surely all agree that you don't want your mail server talking to another mail server that's known to be vulnerable. Most of these specific lists require that an open * be abused before they list them. I'd also contend that we can all justify using Spamhaus's Spamhaus Block List (SBL). It lists known spammers and it very specific about it. You can block roughly 75% of spam with that list alone. Where you use these DNSBLs is up to you. Like I said above, I call all of mine straight from Sendmail. You can configure SpamAssassin to call these DNSBLs for you and assign a score you define. It's pretty easy. This way you can still use lists like SPEWS that rely on collateral damage to score mail but not outright block it. I use SPEWS and love it but it does block some legit mail by design. If you only score off of SPEWS you can minimize the FPs while still maximizing your spam filtering efforts. I am preparing to score foreign countries and RFC-Ignorant domains off of this as well.

I do not recommend you use the DCC. I highly recommend you use Razor which IMHO addresses the shortcomings in DCC. Submissions to Razor have to be confirmed unlike in the DCC. This way other people confirm that the message someone submits is actually spam and not JCPenny's spring mailing list. SpamAssassin can make these calls as well.

The mail system you're describing is going to be fairly large. This isn't something you want a single box handling. Ideally you'd put the spam and AV checks on a mailhub ahead of the actual MTA or cluster of MTAs. These boxes act as a spam firewall of sorts and takes the CPU intensive tasks you mentioned off of the actual mail server. I'm not actually using this type of setup myself but I will be eventually. There was a Slashdot article a while back about a setup roughly your size and what I guy did to make it work. It was quite a nice setup. I can't find the link now. IIRC, he scored mail and then sent probable spam via a seperate mail queue to a seperate spool for each user. Then using IMAP the user could check their probable spam for FPs. It was a nice setup.

You also mentioned Bayesian filtering. Let me make something very clear. Bayesian filters must be applied on a user by user basis. You can't simply enable Bayes for all 50,000 as one lump sum. It will never be able to learn what is an isn't spam that way. You have to let it learn on a user by users basis. The existing Bayes abilities within SpamAssassin don't work well (or at least easily) when SA is called from MIMEDefang. There are supposedly hacks for this but I have yet to see a working one. Along those same lines user-defined preferences also don't work well (or at least easily) fro

Your management's point is valid, but it is made less so by the fact that telephone support for proprietary software is not all that it's cracked up to be, and also by the very strong support for free and open source offerings.

The people who provide support for proprietary products are often quite good, and they are familiar with the most common issues you'll encounter with their software. But I've found them to be rather bad with unexpected issues. They'll sometimes tell you to upgrade to the newest version, which is the last thing you want to hear.

On the other hand, if you deal with free or open source software you'll often get support from people who have an active role in developing the software - perhaps even the founder of the project. These people can identify problems, and even fix them and release a new version to address your needs. No joke. I use a program called MIMEDefang to add an annoying disclaimer to our company e-mail (a legal requirement, but a PITA nonetheless). I regularly get list e-mail about this program in which the core developers answer questions posted on the list. They do this numerous times a day.

Ever get a support-related reply from Bill Gates when you've had a Windows BSOD? It's a different world, and one I like a lot better.

Eventually, the spammer gave up - it must have noticed that I was firewalling the connections as soon as I detected them. MIMEDefang, combined with a modified filter script and ipchains or iptables, can do some neat tricks.

Most of the people using sendmail (Myself included) use it because its the only option for our needs.

Until qmail and/or postfix reach the feature set of sendmail (or come anywhere near it) it will remain useless to me.

Unless you are a serious user of milter features (an extensive MIMEDefang setup, for instance) I think you'll find a switch to postfix or qmail to be a net plus. The learning curve probably won't be as bad as you assume... just set up a test server someplace and see what's actually involved in getting it to do what you want.

As a longtime sendmail admin myself, I think I can understand your resistence: "Why screw with a working mail setup?" But when you have to keep sweating repeated vulnerabilities in a huge process that runs as root, you have to start wondering how elastic the definition of "working" has become. Anyway, stop reading config docs and give one of the modern MTAs a real test drive! ;-) You'll probably be pleased.

Broadband via cable TV is available, but cable modems are a fairly new thing over there and many apartments don't have the cable lines anyway. (Take a look at any apartment building in Japan and you'll see dozens of those mini satellite dishes perched on the balconies.) However, if you want cable TV and broadband Internet, you can get a pretty good deal by combining the two -- about 80 USD/month. You might need a local friend to help you, though, because most cable providers don't have English-speaking customer service.

If you just want the Internet access, a better option is ADSL, which has exploded in popularity over the last couple of years. Before ordering, you first need to decide whether you want land-line (as opposed to cellular) phone service. If you want a land line, get ADSL Type I, which includes phone service and Internet access. If you plan to get a cell phone in Japan, choose ADSL Type II, which provides Internet access only, but for a lower price.

The cheapest ADSL service is probably Yahoo! Japan BB, but they don't provide any English support, not even for sales. You're better off going with a company that has a dedicated English-speaking support line such as Global OnLine or eAccess. Unfortunately, these providers usually serve only the larger metropolitan areas, so if you're in a suburb or a smaller town, your only choice might be good old NTT. All you have to do is call the English-speaking sales line for NTT (the number depends on whether you live in the east or in the west) and tell them you want ADSL Type II. They'll be happy to hook you up for about 25 USD/month, and you can rent an ADSL modem from them for another 5 USD/month. Important tip: NTT will send you a CD-ROM containing PPPoE drivers that only work with the Japanese version of Windows, so you should download the freeware program RASPPPOE before you go and bring it along with you. It's compatible with NTT's ADSL modems.

There's another catch: Because NTT only provides the physical ADSL connection, you'll need to find an ISP that supports ADSL. I got mine through OCN for about 20 USD/month. They offer sales and support in English.

The Macintosh has about the same percentage of market share in Japan as in the U.S. (in other words, not much), so you can expect the same level of support and availability over there that you'll find here. I expect it's entirely possible to hook up your Mac to a Japanese ADSL modem, but don't expect much technical support if things go wrong. (I had no trouble connecting through my Linux laptop once I got the Roaring Penguin configuration set up right.) As for 802.11b, coverage is almost non-existent, although just about everyone over there does email wirelessly through their cell phone. Text messaging and services like DoCoMo are far more popular than the Internet in Japan, at least for now.

You should visit the ISP Japan FAQ for more details. You might also want to check out my Japan page for tips on living and working in Japan.

It always uses "john@some-randomly-selected-domain" as its From: address.

Fortunately, the targetted domain is one whose users never pick up mail, so I can use it as a honeypot, and feed systems not found in relays.osirusoft.com into a private DNS blacklist. However, I got tired of chasing this dirtball, and set up MIMEDefang to automatically add this cretin to the server's firewall rules when one of its attacks is detected.

You can have high-performance message scanning; you just have to be clever about how you implement it.

For example, someone was using MIMEDefang to scan almost 2 million messages/day on a single machine.

To scale up, just throw in another equally-preferred MX record.

You can have high-performance message scanning; you just have to be clever about how you implement it.

For example, someone was using MIMEDefang to scan almost 2 million messages/day on a single machine.

To scale up, just throw in another equally-preferred MX record.

You can setup SpamAssassin in a site-wide configuration. You could also put it together with MimeDefang and integrate it with Sendmail.

Dell uses MIMEDefang on its Internet bastion hosts.

Sshhh... don't tell M$ Dell runs Red Hat and Sendmail on its Internet mail servers....