Slashdot Mirror
A Programmatically Accessible Email Archive?
JohnnyConatus asks: "Does anyone know of a service that offers corporate email archiving and also provides a read-only interface for accessing the archived emails programmatically? Ideally this would be in the form of an database connection or a web service. My current employer is required by the SEC to archive all email communication with clients and we would like to incorporate the archived emails into our internal applications. I have called just about every email archive service I could find via Google, and while most offer a web application to search the emails, none so far have a solution for doing so programmatically. For various reasons, archiving the emails ourselves is considered the last resort. If we had to implement archiving locally, a program that archived by acting as a mail gateway would be the ideal since we'll be supporting a wide-range of mail servers."
How much mail does your company move per day? Thousand of messages? A gig in attachments per day?
You could very easily implement this as a simple forwarding daemon, or as an plugin to your existing MTA, just store all mail going anywhere in a separate, append-only mailbox, then use IMAP to access it remotely.
IMAP is an industry-proven protocol, there exist many open-source implementations, and has been specifically developed for situations where the mail will remain on the server. It provides you with searching and tagging, plus you can organize the mail store as you see fit (f.e. each years mails in a separate folder, while still able to search all of them at once) (sort known spam in a separate folder while keeping it around). Granted, I'm not aware of any IMAP server that uses an SQL back-end, so this may become a bottleneck for you.
The javamail api can do everything you need, and you can plug bouncycastle's api along with it so you have it PGP encrypted.