Slashdot Mirror
Ethernet The Occasional Outsider
coondoggie writes to mention an article at NetworkWorld about the outsider status of Ethernet in some high-speed data centers. From the article: "The latency of store-and-forward Ethernet technology is imperceptible for most LAN users -- in the low 100-millisec range. But in data centers, where CPUs may be sharing data in memory across different connected machines, the smallest hiccups can fail a process or botch data results. 'When you get into application-layer clustering, milliseconds of latency can have an impact on performance,' Garrison says. This forced many data center network designers to look beyond Ethernet for connectivity options."
Long Live the Token Ring!
One Ring to rule them all
In our Data Center, we have a great big vat of steaming salt water and we drop one end of the cat5 cables from each server into the vat....those packets that can't figure out where they're going just drop to the bottom and die ...we have to drain this packet-goo out once a month. (but we do recycle it...we press it into CDs and sell them on Ebay)
(Seriously, haven't people heard cut-through switches which just look at the first part of the header and switch based on that... store-and-forward switches are so "1990s")
TDz.
I don't think I need to read anymore, well, I did verify that the number really appears in the article.
This author does not understand the subject material.
(I suppose you could deliberatly overload a switch enough to get this number, maybe, but that would be silly, and your switch would need 1.25Mbytes of packet cache.)
"(By comparison, latency in standard Ethernet gear is measured in milliseconds, or one-millionth of a second, rather than nanoseconds, which are one-billionth of a second)"
That would be one-thousandth, not millionth (aka micro second). Not a good start...
I just blame it on the ether-bunny.
Actually, even with Gigabit ethernet availability HPTC and other network intensive data center operations have moved to Fibre Channel and things like:
1 21.html
Infiniband http://en.wikipedia.org/wiki/Infiniband
and Myrinet http://en.wikipedia.org/wiki/Myrinet
http://h20311.www2.hp.com/HPC/cache/276360-0-0-0-
HP HPTC site
-What's the speed of dark?
Er, yeah. No kidding.
When I was writing applications at the San Diego Supercomputer Center, latency between nodes was the single greatest obstacle to getting your CPUs to running at their full capacity. A CPU waiting to get its data is a useless CPU.
Generally speaking, clusters who want high performance used something like Myrnet instead of ethernet. It's like the difference between consumer, prosumer, and professional products you see in, oh, every industry across the board.
As a side note, how many parallel apps solve the latency issue is by overlapping their communication and computation phases, instead of having them in discrete phases, this can greatly reduce the time a CPU is idle.
The KeLP kernel does overlapping automatically for you if you want: http://www-cse.ucsd.edu/groups/hpcl/scg/kelp.html
I'm talking performance. Store & Forward hammers your performance. In my experience, you get better performance when you run the server at 100Mb full duplex (along with all the workstations) and use Cut Through than if you have the server on a Gb port, but run Store & Forward to your 100Mb workstations.
So you have an environment with requirements totally unlike the ones described in the article and needing none of the solutions illustrated in the article. Hey...thanks for letting us know. Maybe the other million Slashdot users with environments irrelevant to the post can let us know what they have as well.
I had recently considered using this Tolkien ring until I found out that deinstallation is very difficult. Something about having to take it to a smelter.