Lot of traffic coming our way because of this story. Just wanted to pipe in and say that the live demo on our Zimbra site is running our older 3.0 M1 release. Also, the live demo is heavily throttled. Last week we posted some screen shots from a server running (to be released soon) 3.0 M2:
I am not familiar with the groupwise APIs. Zimbra web client makes SOAP requests in XML, and gets back answers in JSON. In our sources there is file called soap.txt which documents our APIs. I'll let you be the judge of the effort required to port.
We are working on getting bugzilla opened up. In the meantime, http://www.zimbra.com/forums/ is the place to report/discuss problems like this. You might also want to search and see if your problem has already been reported by others and if there is a known fix/workaround. IMO install issues should always be discussed/checked on the forums before being promoted to bugs.
We do not use tomcat sessions and we do not need them. We have our own session objects. We (carefully) cache mailbox change information in these session objects for notification purposes. Replicating these session objects will make it a fully distributed session object, and therefore expensive. We also like the locality of the session object wrt where the mailbox lives.
ZCS server logs all operations to a journal we call "redolog" - used for crash recovery and incremental backups - a safety net across the three different ways we store data (db, lucene indices, and plain rfc822 files). But redolog can also be used for master/slave replication. So while we have support for partitioning mailboxes across servers (any mailbox lives on only one of these active servers), you can, in addition, replicate each of these mailbox servers to standby servers if you want.
If you have a SAN, you will probably want to use a shared-disk failover/takeover?
With Zimbra, you can use Perdition to get cyrus-murder like proxying to the actual mailbox server, without users being aware of where their mailbox lives. We do not support the mupdate protocol which cyrus-murder needs and cyrus-imap implements.
We don't need tomcat load balancing or mysql replication or
clustering. We designed the system to scale without requiring
any of these.
The Zimbra architecture for scaling mailboxes across hardware boxes is
a lot closer to the cyrus-imap way of scaling - don't let the presence
of tomcat/mysql mislead you into thinking otherwise!
You can have M postfix boxes route mail to N mailbox server boxes.
Each of those N mailbox servers is/runs the { tomcat, mysql,
filesystem } triple. Instead of one huge database with all your
users' metadata, you partition your users across these each of these
mailbox servers' database instances. Postfix can route mail via LMTP
to one of those N mailbox servers where the mailbox actually lives (we
use transport_maps). The Zimbra web UI served up from each of these
mailbox servers can redirect the user to the mailbox server on which
their mailbox actually lives - eg, think of mail.yahoo.com which
redirects you to us.f300.mail.yahoo.com. In the future, don't rule
out us making this even more transparent. Someone we know, in a
multi-node install test, configured perdition (the IMAP proxy) to
consult user info from LDAP and proxy IMAP connection over to the
user's actual mailbox server.
Beyond using its own MTA (postfix), the problem for larger sites is that each box is expected to stand alone. There is no way to have webservers, mailservers and database on seperate boxes.
You can have multiple mailbox servers! See this discussion.
Yes, Zimbra is not just a webmail client you can slap on top of any IMAP/POP server. The reason for this is not malicious. It is just that a lot of the (compelling, if I may say so myself) functionality you see in ZCS - search, tags, conversations, group calendar, etc, etc - are only possible because of our own server.
On a related note, even though you have to use our server, and even though we love our UI dearly and we hope that you will too, we are client agnostic - we will support as many clients as we possibly can. Which is why we make your data available via IMAP/POP/RSS/iCal/etc.
Both the source and binary downloads contain pre-built binaries for the 40+ dependencies you refer to - however the binaries are only for FC and RHEL (for now).
Sometimes relational databases are overused in email systems (very silly to put your JPG attachments inside the DB). But databases are faster than doing readdir(3) and read(2) all the time, irrespective of how much the buffer cache in the kernel helps. Specially when you are designing to scale and trying to cram as many busy users as you can in a box. Even cyrus-imap maintains an index file, as does Gnus/nnml - both of which can be file per message. These systems (not just the two examples) can not afford to walk directories and open a lot of files, when you want to list your inbox. Some folks (including us) choose to put this index in a relational database - for various reasons - they're doing groupware and a database is so much easier/required, don't reinvent sorting and indexing, formalize your data structures and don't have to write new tools if schema has to change for some reason.
Leaving the message on the filesystem, but keeping message metadata in the database (of some kind, relational or otherwise) is the way to go. "Email don't need no databases" is not a feasible option when writing mail servers.
Slashdot Mirror
Lot of traffic coming our way because of this story. Just wanted to pipe in and say that the live demo on our Zimbra site is running our older 3.0 M1 release. Also, the live demo is heavily throttled. Last week we posted some screen shots from a server running (to be released soon) 3.0 M2:
d ar_candy.html
http://www.zimbra.com/blog/archives/2005/10/calen
Check out the new group calendaring screen shots that went on our blog yesterday:
d ar_candy.html
http://www.zimbra.com/blog/archives/2005/10/calen
We wanted tight integration for performance and robustness (no transactions spanning applications).
I am not familiar with the groupwise APIs. Zimbra web client makes SOAP requests in XML, and gets back answers in JSON. In our sources there is file called soap.txt which documents our APIs. I'll let you be the judge of the effort required to port.
We are working on getting bugzilla opened up. In the meantime, http://www.zimbra.com/forums/ is the place to report/discuss problems like this. You might also want to search and see if your problem has already been reported by others and if there is a known fix/workaround. IMO install issues should always be discussed/checked on the forums before being promoted to bugs.
We do not use tomcat sessions and we do not need them. We have our own session objects. We (carefully) cache mailbox change information in these session objects for notification purposes. Replicating these session objects will make it a fully distributed session object, and therefore expensive. We also like the locality of the session object wrt where the mailbox lives.
ZCS server logs all operations to a journal we call "redolog" - used for crash recovery and incremental backups - a safety net across the three different ways we store data (db, lucene indices, and plain rfc822 files). But redolog can also be used for master/slave replication. So while we have support for partitioning mailboxes across servers (any mailbox lives on only one of these active servers), you can, in addition, replicate each of these mailbox servers to standby servers if you want.
If you have a SAN, you will probably want to use a shared-disk failover/takeover?
With Zimbra, you can use Perdition to get cyrus-murder like proxying to the actual mailbox server, without users being aware of where their mailbox lives. We do not support the mupdate protocol which cyrus-murder needs and cyrus-imap implements.
The Zimbra architecture for scaling mailboxes across hardware boxes is a lot closer to the cyrus-imap way of scaling - don't let the presence of tomcat/mysql mislead you into thinking otherwise!
You can have M postfix boxes route mail to N mailbox server boxes. Each of those N mailbox servers is/runs the { tomcat, mysql, filesystem } triple. Instead of one huge database with all your users' metadata, you partition your users across these each of these mailbox servers' database instances. Postfix can route mail via LMTP to one of those N mailbox servers where the mailbox actually lives (we use transport_maps). The Zimbra web UI served up from each of these mailbox servers can redirect the user to the mailbox server on which their mailbox actually lives - eg, think of mail.yahoo.com which redirects you to us.f300.mail.yahoo.com. In the future, don't rule out us making this even more transparent. Someone we know, in a multi-node install test, configured perdition (the IMAP proxy) to consult user info from LDAP and proxy IMAP connection over to the user's actual mailbox server.
See also this thread on our forums.
Yes, Zimbra is not just a webmail client you can slap on top of any IMAP/POP server. The reason for this is not malicious. It is just that a lot of the (compelling, if I may say so myself) functionality you see in ZCS - search, tags, conversations, group calendar, etc, etc - are only possible because of our own server.
On a related note, even though you have to use our server, and even though we love our UI dearly and we hope that you will too, we are client agnostic - we will support as many clients as we possibly can. Which is why we make your data available via IMAP/POP/RSS/iCal/etc.
Both the source and binary downloads contain pre-built binaries for the 40+ dependencies you refer to - however the binaries are only for FC and RHEL (for now).
Sometimes relational databases are overused in email systems (very silly to put your JPG attachments inside the DB). But databases are faster than doing readdir(3) and read(2) all the time, irrespective of how much the buffer cache in the kernel helps. Specially when you are designing to scale and trying to cram as many busy users as you can in a box. Even cyrus-imap maintains an index file, as does Gnus/nnml - both of which can be file per message. These systems (not just the two examples) can not afford to walk directories and open a lot of files, when you want to list your inbox. Some folks (including us) choose to put this index in a relational database - for various reasons - they're doing groupware and a database is so much easier/required, don't reinvent sorting and indexing, formalize your data structures and don't have to write new tools if schema has to change for some reason.
Leaving the message on the filesystem, but keeping message metadata in the database (of some kind, relational or otherwise) is the way to go. "Email don't need no databases" is not a feasible option when writing mail servers.
If you've been to our website and you believe you're a fit, you know where to send your resume. :)
http://www.zimbra.com/careers.html