Slashdot Mirror
Improving Unix Mail Storage?
At first, there was mbox, then there was Maildir, and Bill begat Outlook and .mbx. CaraCalla wonders if there is a better way to store mail than the way we currently store it today. I admit, with the changes that email has undergone over the past 5 years (changes in what is being sent, not necessarily in how it is sent), it may be time to reinvent the mail format. Read on for CaraCalla's analysis of the current mail options, and his thoughts on where we may go in the future. If you were to design your own MUA, how would you design its mail storage?
CaraCalla asks: "Does anybody know a good, free solution for storing mail on unix hosts? The reason that I ask this question is my discontent with available techniques:
- mbox: There are problems with locking, corruption, access-times, and bloat.
- Maildir: Do you really want to clutter your system with millions of small files? That's waste of inodes, space (unless perhaps you use Linux/ReiserFS or SGi) and just try to open a Maildir with 1000+ mails and see how long it takes your favorite Mailprogram to only display the subjects.
- Cyrus: Basically the same as Maildir with database features.
- UW-Imap mbx: That's classical mbox with extensions allowing multiple access.
- Evolution: Basically mbox with database features.
- Windows clients: Typically some proprietary db-format. Pathetic.
But the thing that bugs me most is disk space. Typical inboxes are made of 5% to 10% of Text including Headers and HTML. The rest are BASE64- (or UU-) encoded pictures, word documents, zip archives and so on. The problem here is the encoding which wastes considerable amounts of space (at least one third).
Some ideas about the ideal mail-storage:
- One file per Mailbox-folder, allowing multiple folders per user. Should those files reside in one central location or in users Homedirs?
- Compression: Should messages be broken into pieces and the MIME-attachments stored separately (thus searching of the text parts would still be possible without decompressing the whole file)?
- File format: gdbm, Sleepycat db? Something new?
- Should the security model allow users to directly access their files, grep them, copy them around?
- Shared folders, virtual domains?
- Unicode support in folder names? Imap message-IDs, flags, useragent specific state-information?
- How would MTAs deliver mail? How would clients access? File-locking (NFS)?
- What about backwards-compatibility? Writing libmailstore (anyone)? adopting UW c-client?
Does my ideal mailstorage exist somewhere? Is somebody working on a project addressing this? Does anybody have some other hints? And please no mbox/Maildir flamewar!"
MS Exchange has been doing this for almost a decade now. Version 5 was the first decent one to get and 5.5 is great.
I've been joking for years about getting two shell accounts on opposite sides of the planet and setting each up with procmail to bounce all my mail between the two (always rewriting the header so as to avoid a loop.) I figure at any given time, my mail would be in both places and neither simultaneously.
.procmailrc for a few seconds and change it back. Plenty of mail storage without chewing-up precious file-system quota.
If I want to read some, I'd just chmod
Some people have a way with words, and some people, um, thingy.
No. This is Unix we're talking about, not Windows. Unix doesn't have any problems.
I have been pwned because my
Now I'll never get to sleep tonight.
If it ain't broke, you need more software.
At first, there was mbox, then there was Maildir, and Bill begat Outlook and .mbx.
How do you misspell "began"? The "t" key is up above on the first row and the "n" is on the third row.
So far I have blocked about 94% of the SPAM coming in through our mail server. It only misses a couple and is highly configurable! Download and install it!
OFFTOPIC!? With that great deal of spam reduction, the space required to store the emails is greatly reduced!
Cheers!
Tom
http://tomgould.com/
You're newfangled system better have DRM built in. I don't have any data, but it must be obvious that artists are losing billions in revenues every week due to mp3s being sent as attachments. This criminal behavior must be stopped or the practice of free expression will come to a screeching halt.
We have to have the remote hammer to pop out of the monitor to whack the end user. This is a must for any admin that works with more than 300 people. Hammer trigger from e-mail, pager, SMS, or telephone number.
Power mains must be connected to the user's chair, see above for trigger.
The MUA must forward all p0rn to the admin account. Likewise with credit card info.
The MUA must know when the user is about to do something to tick off the admin, like sending a "me too" to everyone in the office, or replying to a confidential e-mail to the whole office and prevent the user from reproducing. X-rays are fine for impromptu sterlizations. The side effect of loosing all your body hair is no problem, as it alerts others to a stupid co-worker.
The MUA must alert the admin when a coworker he has got the hots for changes her home phone number. Just to be fair, if the the admin is female, the reverse applies.
The MUA must analyze the admins e-mail and throw a bucket of cold water if (s)he attempts to send a really stupid e-mail.
Also, the MUA must be able to launch nuclear missles at spammers automatically. After that, it should refer the e-mail to the admin to see if a stronger response is warranted. Better yet, the MUA should employ a time machine to go back and choke the spamming creep when the spammer is still a baby, then use X-rays on the parents as above.
The MUA should have a hypnotic effect on the object of the Admins desire and cause that person to preform disgusting oral acts on the Admins body each time a new e-mail arrives. (HOORAY FOR KELZ!)
For the PHB, he should (by the same hypnotic effect) do a "Full Monty" when the big cheese walks in. Twice.
The MUA should be able to cause back dated confirmation messages from HR approving a 51 week paid vacation upon pressing a special key combination, unless it's the PHB pressing the keys, then it should cause an e-mail to be sent to HR from the PHB's account turning in notice.
Sorry, if you had a day like mine, you'd need a laugh about now...
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
Say no to software patents.
Are you trying to tell me that a 5MB empty mailbox is asking too much? A text message that says "Hi!" costing 1.2MB is somehow wasteful?
Lotus Notes Uber Alles!
This