Slashdot Mirror

I would vote for some high speed NAS heads covering the arrays. These tend to get a lot of use, though they are really just LSI engenios. http://www.bluearc.com/html/products/titan-3000.shtml

http://www.bluearc.com/ - Hands down the best I have seen. We use it here at $GOVFUNDEDLAB, pound the living snot out of it, and it keeps on going without issue.

We've had a couple Bluearc http://bluearc.com/ boxes for about a year and I really like them. It sounds like they have all the features you want. The backend drive array is all fiberchannel and the demo unit we looked at had a fix of FC and SATA disks (in seperate shelves of course.)I think we also looked at Netapp and a few others, but nobody else could come close in terms of price and features to what we wanted and what my boss was willing to spend. The service has been very good. Every time we've had a drive fail, I've had a message from their support people waiting for me when I came in and I usually get the replacement drive within a couple days (and that delay is more a matter of how our receiving department handles packages than anything else.)

>(Think one Bluearc machine is 3 times
>faster than a cluster of the highest end
>Netapps).

Their controller throughput seems to be 20Gbps - I'm lazy to check but I think I've seen one or two vendors with better performance.
http://www.bluearc.com/html/products/titan.shtml

>Bluearc actually treats all of these raid-5
>shelves as disks that it does raid-1
>across. This gives you redundancy and
>double-striping in hardware from end to
>end.

Don't all high end disk arrays work the same way? You create storage groups, then you create RAIDs of those.

> just check out specs.org to see how fast the thing is.

http://specs.org/? WTF?
Does this web site have a BlueArc NAS back-end?

Just thought I'd add a few details and address some of the questions here. My name is Thom O'Connor and work for CommuniGate Systems (CGS), and was the one who put together and ran these tests - you can (mostly) verify this by looking at the comments in the source on the results page.

First off, on the SPECmail test itself. SPECmail is a standardized test (the only one I'm aware of for email) that attempts to closely regulate a level playing field for measuring email performance. It is critical to understand that this is not just measuring SMTP. The 30 million message a day text is a little vague, but it is important that this includes a distribution of delivery, relayed, and retrieved email. Sure, anyone can just relay many millions of messages an hour.

SPECmail does POP and SMTP, so the test measures not just MTA behaviour but also local delivery and then retrieval of the messages. The SPECmail test also uses Quality of Service (QOS) measurements such that a message injected via SMTP to the system MTAs (the CommuniGate Pro Frontend servers in this diagram) must then be delivered locally into the users' account, then be retrieved within 60 seconds. Satisfying the QOS criteria during the benchmark is often the most difficult part.

So, SPECmail itself just does POP and SMTP, which is a little 1990s I agree, but SPEC is coming out with a SPECimap test in the near future, and CGS is also very interested in seeing a SPEC VoIP/SIP test for measuring CommuniGate Pro's Real-Time capabilities.

A few others questions I've seen raised here:

1. The CommuniGate Pro Dynamic Cluster described in this test is fully and completely appropriate for production use in all aspects. In fact, if you're running a 2+ million user ISP on a CommuniGate Pro Dynamic Cluster, we'd recommend you to use these results as a guide for your architecture (although load balancers should be added to the gateway point for all inbound connections). In fact, CGS has ISP customers running architectures which match the layout of the described system almost exactly. All systems in the Cluster service all accounts - you could lose 4 Frontend Servers and 3 Backend Servers, and all users could still access their email (albeit with decreased capacity).

2. HyperThreading was disabled in the BIOS because the downloadable Solaris 10 x86 operating system would not (yet?) support the Intel x86_64 Potomoc chipset properly. That said, on top of the recent security vulnerabilities on the topic, we have also discovered miscellaneous threading and even NFS issues related to having HyperThreading enabled on Linux 2.6, FreeBSD 5.4, and Solaris 10 x86 systems.

3. On NFS...NFS is used safely and securely in this test. The integrity of data storage is one of the major criteria that the SPEC organization closely evaluates when reviewing a SPECmail submittal. Obviously, there are many ways to cheat and/or cut corners using Solid State Disks, unsafe RAM for message queueing, and other techniques that you would never want to use on your production message system. However, the test described here was performed using a standard (albeit excellent) BlueArc Titan Storage System with write caching only in NRRAM and using proper mount options and layout for security, redundancy, and data integrity.

Hope this clears up any misconceptions. Obviously, I'm clearly biased about the work here, but assembling and then passing a SPECmail test of this size is a gigantic effort. If anyone thinks

I know a guy who works for BlueArc. They do most of the network and filesystem stuff in hardware, hardly anything is firmware. (the rare filesystem permission checks or something)

20Gbps is only 16% short of the 3GBps metioned in this article, and you get up 256 TB. Really expensive shit, but it's pretty cool.