Slashdot Mirror

Rule #1.

NEVER plug a laser printer into a UPS. The power that the fuser draws is WAY too much.

Look at some of the cheap office units, they show little pictures on them, notice the printer icon is on the surge side, NOT battery/surge side.

If the power goes out, you should NOT be trying to print.

http://articles.techrepublic.com.com/5100-10878_11-6085460.html See #6

http://arstechnica.com/guides/other/ups.ars/3

http://www.jetcafe.org/npc/doc/ups-faq.html#0405 see 04.05

Would you put a space heater on a UPS? Shredder? Vacuum? Table Saw? If you put a laser printer on it, you may as well.

system..

http://www.jetcafe.org/npc/doc/mail_arch.html ..based on NFS of all things.

But having worked on proprietary scalable email systems myself I was highly doubtful of their NFS-based solution. It's hard to get it right and be fast enough, especially when you have as much email(most of it spam) to contend with. At the MTA level we made it so it first tries to send the message to the storage server in memory then returned a response back to the SMTP client. If that took too long or memory was running short, it saved it to disk and tried to deliver, in that case it would return a response back to the SMTP client. Then there were the dedicated queueing servers which could run in memory in the mta or on another server. Little things like these made a difference and it's still in use today, never ever mysteriously losing mail while working on a considerable amount of email. Some pretty smart people worked on it. The protocol used was proprietary and relied on acknowledgements.

Using NFS just begs for pain and suffering. Maybe they moved on from NFS but obviously something's not right, you shouldn't lose 1 email. Gmail has shown it is possible to create such a beast with open source software which is commendable.

Wouldn't a data centre have a number of UPS and a standby generator? I figure they did an 'upgrade' that didn't take.

How many remember the decline of usenet thanks to these assholes? You know, the ones that tried so fucking hard to equate SPAM with "free speech"?

Gee, where are they now...pity the FBI isn't breaking down their doors too, just for helping start this mess.

I wasn't planning to weigh in on this complex topic, but I can't let this bad advice pass without comment:

Maildir message store.

Store the mail in maildirs. [...]

Don't use Maildir! It does not scale well at all. Maildir's claimed greatest strength is the ability to use an NFS-mounted spool without locking. While this is true, Maildir has a fatal flaw that more than outweighs this benefit.

Because Maildir encodes flag information into the filename, constant rescanning of the mail spool directories is inevitable, and the network traffic for these directory scans will kill the performance. Worse yet, NFS caching can cause different servers to have different views of the same mailbox at the same time.

How do I know this? I once tried to implement a scalable mail architecture for a former employer using Linux servers and NetApp NFS servers. On paper, Maildir sounded like a great solution, since NFS locking would not be required. After implementation, we discovered how expensive all those directory scans are -- the performance was a fraction of what it might have been with a better mailbox format.

Now, a format similar to Maildir (one file per message, no NFS locking), which NEVER renames files once created, might have a lot of potential. Of course, this would require either an index file or a database for metadata, and perhaps some direct server-to-server notification of new email, but Maildir's promised advantages might be achievable without its Achille's heel of the massive overhead of constant directory scanning.

Another thing I learned from that project: Qmail sucks for this! It may work well on small systems with a few user accounts on a single server, but it is nearly impossible to adapt to a scalable multi-server environment where the users have no system accounts. The source code is so convoluted and obfuscated that it's incomprehensible, and it's so poorly modularized that customization is a serious nightmare.

Against my better judgment, I agreed to use qmail for the project after heavy lobbying by a new employee who left the company soon thereafter. I spent several frustrating weeks fighting with the horrible qmail codebase, trying to integrate it with an external database system to control delivery of the email -- it was ridiculous.

After the qmail evangelist quit, I immediately dumped qmail and started from scratch with sendmail. Integrating the database into sendmail was a snap -- I just created a new "map" type, which was a very cleanly modularized piece of code, then I made a few modifications to the config file and it Just Worked. In a day or two, I was able to accomplish the goal that wasn't even close after 2-3 weeks of dealing with the morass of qmail's source code.

I'm now convinced that qmail's vaunted "security" is merely "security through obscurity", and I just don't trust the software. But as long as you don't want to do something different than qmail expects, it seems to work fine for many people with small, simple systems. Forget about using it for an enterprise email system.

I don't want to try to describe the email system that I would design in this situation, but I would avoid "one big server" designs and focus more on multi-server designs where platform reliability depends on server redundancy. (Think Google!)

Ideally, you should be able to rip any given server out of the rack without notice (simulating a crash), rebuild a replacement from scratch and put it into service -- all without affecting the users in any noticable way. Designing "stateless" servers where the service data is not local can help here. (For inspiration here, read Earthlink's white papers: A Scalable News Architecture on a Single Spool and A Highly Scalable Electronic Mail Service Using Open Systems)

Good luck designing this system. It's not a trivial task, and what some people evangelize (because it works on a small scale) will be inappropriate and fall apart on a large scale...

I wasn't planning to weigh in on this complex topic, but I can't let this bad advice pass without comment:

Maildir message store.

Store the mail in maildirs. [...]

Don't use Maildir! It does not scale well at all. Maildir's claimed greatest strength is the ability to use an NFS-mounted spool without locking. While this is true, Maildir has a fatal flaw that more than outweighs this benefit.

Because Maildir encodes flag information into the filename, constant rescanning of the mail spool directories is inevitable, and the network traffic for these directory scans will kill the performance. Worse yet, NFS caching can cause different servers to have different views of the same mailbox at the same time.

How do I know this? I once tried to implement a scalable mail architecture for a former employer using Linux servers and NetApp NFS servers. On paper, Maildir sounded like a great solution, since NFS locking would not be required. After implementation, we discovered how expensive all those directory scans are -- the performance was a fraction of what it might have been with a better mailbox format.

Now, a format similar to Maildir (one file per message, no NFS locking), which NEVER renames files once created, might have a lot of potential. Of course, this would require either an index file or a database for metadata, and perhaps some direct server-to-server notification of new email, but Maildir's promised advantages might be achievable without its Achille's heel of the massive overhead of constant directory scanning.

Another thing I learned from that project: Qmail sucks for this! It may work well on small systems with a few user accounts on a single server, but it is nearly impossible to adapt to a scalable multi-server environment where the users have no system accounts. The source code is so convoluted and obfuscated that it's incomprehensible, and it's so poorly modularized that customization is a serious nightmare.

Against my better judgment, I agreed to use qmail for the project after heavy lobbying by a new employee who left the company soon thereafter. I spent several frustrating weeks fighting with the horrible qmail codebase, trying to integrate it with an external database system to control delivery of the email -- it was ridiculous.

After the qmail evangelist quit, I immediately dumped qmail and started from scratch with sendmail. Integrating the database into sendmail was a snap -- I just created a new "map" type, which was a very cleanly modularized piece of code, then I made a few modifications to the config file and it Just Worked. In a day or two, I was able to accomplish the goal that wasn't even close after 2-3 weeks of dealing with the morass of qmail's source code.

I'm now convinced that qmail's vaunted "security" is merely "security through obscurity", and I just don't trust the software. But as long as you don't want to do something different than qmail expects, it seems to work fine for many people with small, simple systems. Forget about using it for an enterprise email system.

I don't want to try to describe the email system that I would design in this situation, but I would avoid "one big server" designs and focus more on multi-server designs where platform reliability depends on server redundancy. (Think Google!)

Ideally, you should be able to rip any given server out of the rack without notice (simulating a crash), rebuild a replacement from scratch and put it into service -- all without affecting the users in any noticable way. Designing "stateless" servers where the service data is not local can help here. (For inspiration here, read Earthlink's white papers: A Scalable News Architecture on a Single Spool and A Highly Scalable Electronic Mail Service Using Open Systems)

Good luck designing this system. It's not a trivial task, and what some people evangelize (because it works on a small scale) will be inappropriate and fall apart on a large scale...

My slides relevant to this discussion can be found at http://www.shub-internet.org/brad/papers/dihses/ and http://www.shub-internet.org/brad/papers/sistpni/.

And yes, Nick Christenson has been a long-time friend and co-author of mine.

Feel free to contact me directly if you want some referrals.

http://www.jetcafe.org/~npc/book/sendmail/

A good book on sendmail performance tuning, although a lot of it covers the OS.

Then get The Practice of System and Network Administration.

http://www.everythingsysadmin.com/

earthlink's mail server complex has come up on freebsd-isp a few times

this guy used to work at both sendmail and earthlink and he has links to some good resources

AOL are the largest ISP in the US, in the world if you discount DoCoMo as a mobile ISP. Most businesses can't just disregard AOL's members.

"Calculus" is just a term for a formal system, usually equational in nature, which supports some form of deduction based on rewriting. There are easily hundreds, probably thousands, of calculi which appear in the programming language theory and related literature, and few of them if any have anything to do with the differential calculus you are familiar with from high school.

If you are interested in learning about lambda-calculus, you might want to start with untyped lambda-calculus, unless you have a background in logic. There are many books on the subject; I started with Chris Hankin's "Lambda Calculi: A Guide for Computer Scientists", which is IMO quite readable. You can also find many tutorials and lecture notes on the web, for example here, here and here. If you know Scheme, or have read SICP, then you already know mostly what lambda-calculus is, though you probably don't realize why it's so remarkable. (Hankin's book is good for that.)