Slashdot Mirror
IP Over SCSI?
morzel asks: "One of the advantages of SCSI based systems is that a plethora of devices can exist on the same high-bandwidth bus, including multiple host adapters - at least: that's the theory. While it seems pretty obvious to me to use this as a low latency/high-bandwidth interconnect between a small number of hosts, I've never seen an actual implementation of such a system. Do these, preferrably IP-based systems, actually exist? I'm not in need of a Beowulf style cluster just yet (I don't have an application for them) but I am interested in the possible usage of SCSI as a _fast_ interconnection for small numbers of load-balancing machines in cluster. A combination with the Linux Virtual Server Project could create a killer solution... Right? Thanks for all input/comments on this!" (Read on...)
"I would think these kinds of interconnects would be ideal for small clusters, or larger clusters where groups of eight nodes could be interconnected with each other, with one node acting as the master node. This would probably provide more bandwidth and less latency than ethernet-based solutions, and on the other hand could be a lot cheaper than special hardware."
-----
[Science] is one of the very few things that raises human life a little above farce and gives it the grace of tragedy.
--Bob
1^2=1; (-1)^2=1; 1^2=(-1)^2; 1=-1; 1=0.
I remember reading the Mach source code a few years ago, and they had a layer called "SCSINet" or something like it. It was a way to do IP over SCSI (which, because of its greater bandwidth, lower latency, and priority levels, etc., could lead to neat hacks like distributed shared memory).
So, yes, it's been done. It's even been done in the open.
It's hacks like this, and the ability to have multiple ethernet interfaces (think: switched private Gb ethernet) that make me wonder just why in the hell people buy proprietary cluster solutions (DEC's memory channel - 40MB/sec.) when open standards are quite possibly better, and certainly less expensive.
Makes me wanna puke.
--Corey
Not only will they not deserve liberty or safety, Mr. Franklin, they will be DENIED both!
Additionally, you can get patches for Linux 2.2.9 and 2.2.14 here:
Hmm. it is worth mentioning that they are currently running positive discrimination - if you are using most common Webbrowsers under Windows (with the noticable exception of Lynx) you will be refused access to the site.
--
-=DaveHowe=-
A quick look through the Beowulf project's network drivers page yields this link...a lphanet/linux/drivers/scsi/IP-over-SCSI/
http://www-internal.alphanet.ch/archives/local/
It would seem to me that someone already has this working in an experimental stage.
-CC
With the Dayna I believe you can use one of their devices on a shared chain, assign two IP addesses to the device, and have Ethernet over SCSI between two Macs (The device just 'relabels'). Saw it done by a junior grade hack at Chrysler, who simply didn't know it wasn't an intended use.
So yes, and we were offtopic.
.sig: Now legally binding!
Are there any gigabit NICs available yet?
:(
Yeah, but they're expensive as hell - Multiwave has them for $300. And a gigabit switch is going to be a pretty hefty price too.
100 mbit switches aren't bad. Netgear makes a nice 8 port 10/100 switch (called the FS108, IIRC), for about $90. I'm planning on getting one sometime the summer, actually: I don't see much need for anything too much faster (though IP over SCSI would be a pretty cool hack).
The coolness factor would be high, as would keeping from having a secondary Eth-switch to host the message-passing traffic to keep it off of the primary gen-purpose network.
TCP/IP as well as Ethernet are general purpose networking solutions, real workhorses. However, high-performance cluster-based parallel programming is not a general purpose use -- it benefits the most from a communications path that is optimized for a constant stream of high-volume, relatively small messages from any one node to any other. Sortof a networking nightmare, eh? Sortof like how a usenet feed is a general filesystem's worst nightmare -- uses the underlying mechanism (transport mechanism in PVMs case: TCP/IP/Ethernet; filestore mechanism in usenet's case: ufs, ext2) in a manner that goes against the grain of the optimization assumptions made by the underlying layers.
I maintain our department's 8-node Sparc PVM cluster. We use hand-me-down machines that get displaced from other upgrades. We use it for teaching parallel programming, so performance isn't a great concern for the future of humanity, but when the students write code that doesn't use the message passing medium effectively (currently a dedicated 100mbit switch), then they get a bit discouraged when their code runs better on a single machine as opposed to the cluster. Oh well -- part of the learning process!
Additionally, you can get patches for Linux 2.2.9 and 2.2.14 here: http://www-internal.alphanet.ch/archives/local/alp hanet/linux/drivers/scsi/IP-over-S CSI/