Slashdot Mirror

In the broadcast engineering space, we see a lot of this kind of thing...

Avid Unity ISIS
Omneon MediaGrid
DataDirect S2A

The disks would go in the chassis (see my itemized list). You may not know it but Sun is not the 1st company to use a chassis with vertical bays. Here is one example among many. The price would be more likely around 2 or 3 grands by the way, instead of 1 grand. But anyway this doesn't change the fact that this Sun box is way overpriced, even with a good 40% discount.

Regarding the mobo, just pick one with two AMD 8131 or 8132 PCI-X bridges. This will give you 4 independent PCI-X busses. The two PCI-X bridges would have to be on 2 different HT links in order to not dangerously approach the theoretical one-way data throughput limit of 3.2 GB/s of one 1600 MT/s 16 bits HT link. The two PCI-X bridges could be either connected to different CPUs or to the same CPU because the Opteron XBAR _can_ easily handle the ~3 GB/s you speak about, it has been designed to support 19.2 GB/s of HT traffic and even more with the recent upgrade to 2000 MT/s ccHT links. Now with the 4 independent PCI-X busses, you could put 4 SATA HBAs on the 1st and 2nd busses, and 2 HBAs on the 3rd and 4th busses. This way the first 2 busses will run at 100 MHz and the 2 others will run at 133 MHz, giving a practical throughput of 3.4 GB/s (2 * (100 MHz * 64 bits / 8) + 2 * (133 MHz * 64 bits / 8), and assuming a 90% efficiency as found on most PCI/PCI-X busses), this is enough to handle the 3 GB/s you are speaking about. There are plenty of single AMD 8131 mobo on the market right now starting at $250. I am sure you can find one with two AMD 813x for $500 max.

Now when I think about it you could even use SATA port multipliers in order to reduce the number of HBAs, allowing all busses to run at 133 MHz. I am aware of 12-port and 24-port SATA HBAs (Areca comes to mind) but those are outrageously expensive and are not necessary to handle all that throughput. My experience and those of my friends playing with high-end enterprise gear prove that _very_ simple and inexpensive PCI-X SATA chips such as the SII3124 or Marvell 88SXxxxx are way sufficient to handle the max combined read throughput of any number of disks attached to their SATA ports. The reason being that the designers of such chips have come up with a simple and performant hardware interface optimized to reduce the CPU load. I know for a fact that the SII3124 design is somewhat close to the AHCI spec which is the best example of a performant SATA hw interface.

So I _do_ believe that it is possible to build a $13-14k server with 48 SATA disks in 4U offering ~3 GB/s of raw read throughput. I don't understand why so many people refuse to believe that, especially since other posters in this thread have pointed out that some vendors are already selling similarly priced servers !

www.datadirectnet.com Has been shipping 48 bay enclosures for a while now. SAF2248Put 5, 10, 15, or 20 behind a big RAID head (S2A9500 with 20 back end (disk) links, and 8 FC4 (Fibre Channel 4 - rated at 400 MB/sec) links on the front and it's a fairly big fast system. Lot of national labs going that route. You can snag manuals here if you want more details manuals

--
Good Karma and all I got was this silly sig line

www.datadirectnet.com Has been shipping 48 bay enclosures for a while now. SAF2248Put 5, 10, 15, or 20 behind a big RAID head (S2A9500 with 20 back end (disk) links, and 8 FC4 (Fibre Channel 4 - rated at 400 MB/sec) links on the front and it's a fairly big fast system. Lot of national labs going that route. You can snag manuals here if you want more details manuals

--
Good Karma and all I got was this silly sig line

www.datadirectnet.com Has been shipping 48 bay enclosures for a while now. SAF2248Put 5, 10, 15, or 20 behind a big RAID head (S2A9500 with 20 back end (disk) links, and 8 FC4 (Fibre Channel 4 - rated at 400 MB/sec) links on the front and it's a fairly big fast system. Lot of national labs going that route. You can snag manuals here if you want more details manuals

--
Good Karma and all I got was this silly sig line

If you are just having 'fun' with this - great. Otherwise, you can either get by with what can be done easy with SATA/SCSI and a cheep box, or you are into the low end raid section with things like Apple's box. Need over 180TB going over 1.5/3GB per second on a single system? Try something from here http://www.datadirectnet.com/.

What is it using for disk space? MCR @ LLNL is using the Lustre file system with DataDirect Networks storage.

A few examples:

1. No worry of obnoxious code, as it's FS/OSS.
2. No worry of BSA-auditing and multimillion dollar extortion schemes.
3. No licensing headaches.
4. Infinite scaleability per each individually bought copy (as in, you can install an infinite number of copies with one purchased [or downloaded] CD).
5. Due to #4, ever-increasing savings as the number of computers onto which you install the software grows.
6. Assurance that the product will not die off simply because a company goes out of business, as it is FS/OSS. Any worthy project will be taken up by others if it's original developers move on.
7. Related to #6, ability to develop/implement your own features for your specific needs.

On another note regarding Oracle, it is basically slow crap. The executable alone is 18MB, so it naturally has poor performance; specialized database-systems will outperform it. Btw, data assurance from Oracle doesn't come for free. It costs quite a bit. And for that extra money you spend on it, it'd be better just spending that money doing an audit of FS/OSS code to insure that it won't lose data, and creating backup systems. Using journaling file systems like ReiserFS and XFS is also useful.

Enterprise != Personal systems.

Completely correct. The benefits of using FS/OSS at the enterprise level are even greater. Refer to the many research papers and discussions of companies saving millions by using GNU/Linux over Windows-2000/XP/2003. The MITRE study comes to mind: http://www.egovos.org/pdf/dodfoss.pdf This is a study funded by the government to get an objective evaluation; not some crackpot study funded by MS to make them look better.

Your $300 sale from Gateway doesn't mean shit. A $3M sale, does. They don't give a shit about you. Deal with it. Firstly, this is irrelevant to the rest of the discussion. This was simply a personal digression of mine. The point was that you can get excellent technical support for free within a community of intelligent members. If my $300 doesn't mean shit to Gateway, then they and every other OEM should stop their false advertising of "tech-support" -- because all they're doing is reading from a cookbook which we could have found online. Btw, I don't how many customers Gateway has. Let's say they have 1-million home-user customers, and each customer pays $100 for tech-support (these are obviously conservative numbers). That amounts to $100 million in tech support paid to Gateway by home-users. They damn well better care about the quality of tech support they're giving to home-users.

Lets see some open source clusters

Where have you been the last five years? Some of the world's most powerful supercomputers are Beowulf clusters, using GNU/Linux. See an O'Reilly article for an overview. In particular, GNU/Linux Beowulf clusters are being used for:

• weather forecasting
• high-energy physics problems (e.g., singularities)
• creating lifelike animations & computer-generated graphics (e.g., Matrix, Titanic, Toy Story)
• data mining
• simulation of semiconductors
• CAD systems for developing
• sequencing of the human genome

Yep, this FS/OSS stuff is really useless. It's only made the movie industry more money then from any other movie (see Titanic), assisted in the sequencing of the human genome, and assisted in the prediction of weather patterns, potentially saving lives.

What about SAN support?

Granted, I can not find any FS/OSS implementations at the moment, but there is commercial support available for GNU/Linux:

• href=http://www.globe-it.com/Products/linuxsansolu tion.htm
• ht

About 80 megabytes per second per port. If I'm using all 8 ports, about 80, if I'm using fewer ports ~90 megabytes per second. datadirectnet.com Note - It's not as good if you are doing I/O under 64KB.

DataDirect Networks for the S2A 6200 or 3000, as much storage as you want with 8 (4 on a 3000) Fibre Channel ports going to hosts or switches.
Then SANergy for file system sharing software.

dataDirect has a cool SAN appliance, and the company supports Linux.

Check Data Direct Networks out. The SDD is a I/O monster - 20 pipes of Fibre Channel going to the disks, 8 channels of Fibre Channel going to hosts (or switches, etc). It can handle GIGs of cache too - very cool stuff. Lot of internet media serving, TV/film media production, etc, uses the gear. The SANds side provides file system sharing, etc that works great with the hardware and SAN features of the SDD.