Slashdot Mirror
An Application For 10-Gigabit Networking
Chip Smith sent us a short excerpt from a news article on Supercomputing Online:
"Just yesterday Lawrence Berkeley National Laboratory and several key partners put together a demonstration system running a real-world
scientific application to produce data on one cluster, and then send
the resulting data across a 10 Gigabit Ethernet connection to another
cluster, where it is then rendered for visualization." Here's the link to follow if you'd like to read more on this experiment.
Their first project is to create a virtual dictionary and spell checker so that the Slashdot editors make sure that their posts to the front page are spelled properly. As an added bonus, it'll even check grammar! Unfortunately, the scientists aren't sure if there's enough bandwidth available yet to correct all the mistakes.
You're only as smart as your brain.
This experiment shortly followed by a another showing how fast they could move MP3 collections
Jesus saves souls and redeems them for valuable cash prizes
Great, then someone driving by could steal all of your MP3s in under a minute.
It is logically the next step in the evolution of networking, but it currently outstrips even the memory bandwidth of most systems. Even gigabit today is rarely necessary. In both cases, even if you don't use anywhere near your full bandwidth, you still win if you are using more than the 10 times slower generation provides. There are still, however, some compelling applications that immediately present themselves.
As an uplink between network switches, more bandwidth is always needed. This is what was done in the experiment. If you can't imagine a network that busy, consider switches with 256 100MB ethernet ports, or only a few gigabit ports. It is always better, if it all possible, to be able to guarantee that the switch uplink will never become a bottleneck. If they need to do this with many gigabit ports today, be it ethernet or fibre channel, it is done by using multiple gigabit uplinks.
In any cluster where there is substantial communcation between the nodes, more bandwidth and less latency is always useful. Sufficiently low latency and high bandwidth makes many types of computation feasible that would not otherwise be feasible. Shared memory within a cluster can always benefit from decreased latency. A cluster is always better (meaning VASTLY cheaper and VASTLY more reliable) than a many-many CPU system. Anything that makes the cluster nodes seem closer together will make it easier to move existing software onto cluster-based systems.
It is 10 gigabit, not 10.6 gigabit. 10.6Gb/s was the speed they got over a PAIR of 10 gigabit ethernet connections.
What I really want to know is how many LOC's (Library of Congress) this is per second....
This reminds of reading about Neural Nets in the various texts on Artificial Intelligence. They always quote Shepherd and Koch 's "The Synaptic Organisation of the Brain": "The brain incorporates 10 billion neurons and 60 trillion connections."
When I think about these new network technologies I can't help to think it's our connections that we lack these days. Hopefully with more and more advanced technology we can utilize these connections to create things more intelligent. This appears to be on the right track.
Maybe "Jane" will finally come out of the closet. Well, actually we got to have Instantaneous (sp) communication before that happens... Doh, well... we're making slight progress.
No. It's a 10.6 gigabit connection over 2 pairs of 10 gigabit Ethernet interfaces. It doesn't matter how it's made up, the connection speed is the overall speed of the connection.
And, if that's boring, think about the military applications. In order to try and cut costs and save on code duplication, the labs are building systems in which part of the application (the secret part) runs on secure systems, whereas non-secret parts run on machines using commercial code. Having a single physical pipe between the areas rather than, as at present, multiple pipes could make the security setup a lot easier, and make the design of the machine considerably cheaper. We will all sleep a lot more securely knowing that LL is able to design lots of new exciting kinds of nuclear weapon at minimal cost.
By the way, if they use any GPLed software, does that mean they have to release the entire source code for the application? Just a question.
Panurge has posted for the last time. Thanks for the positive moderations.
Must be the middle of the night, because I'm struck by the beautiful improbability of the phrase "real world scientific [computing] application". Looking at the article, I note that the phrase is stolen from it, and that the article also mentions that the app is "Cactus". Looking at the pages for that project, I note that Cactus is a computational framework, not an actual application. Further, there seems to be no indication which app was actually run.
However, most of the Cactus apps seem to be in astrophysics. Sigh. Maybe it's "real-world" astrophysics. Or maybe it's just bedtime.
Let's do the math:
1280x1024 uses 1.25MPixels (that's real Mega as in 2^20) at 32 bpp gives 48MBit/frame. Multiply that with 60 frames/second yields 2880 MBit/second, or just a tad short of 3 GBit/second. That sounds well within the limits, but consider this: Assuming that in a real network this 10 Gbps network will be relatively as efficient as 10 or 100 Mbps networks (some assumption, I know) are right now, you'll get between 30% and 50% of theoretical throughput as the sustained data rate, leaving you with a very narrow margin of bandwidth. And this assumes that you're the ONLY user on the network, which pretty much zeroes out the use for 1280x1024x32bpp, since this is bound to be needed for networked games, which tend to be less than funny when you're the only player.
But ignoring my assumptions and assuming 100% throughput sustained, you'll get 3 players max - still far less than you're likely to really want.
Black holes are where God divided by zero
The throughput from a single hard disk is not that important: in these environments, RAID arrays (typically fiber channel) optimized for read performance allow overall disk performance at e.g. 1.6Gbps. If you have multiple such servers on your network, with many workstations trying to get at the data, 1Gbps Ethernet starts looking a little slow, especially for the backbones.
The main one is for large networks. If you have a large network, gig or dual gig may just not be enough for connections to your major layer-3 switches. Well till now, gig was all you could do on the eithernet spec, you had to go with something like packet-over-sonet to get more bandwidth. 10GigE is nice because you can keep the whole network eithernet, but get more bandiwdth.
At the university where I work I'm sure we'll start using this sonner rather than later. Right now all our distribution routers have dual gig connections to the two backbone routers. Fine, escept that each is feeding 20 to 50 buildings at 100mb each and the redundant set we are going to add will probably be gig. Those gig links to the backbone will fill up fast if each building has a gig to play with. Hence, 10gig eithernet is great since it works with our existing switches and setup, only faster.
Really, desktop or even server use is not the main target of this at least not for a few more years. The main target is removing bottlenecks from the network that supplies those servers.
You're thinking point-to-point, but that's not what networks are for. Imagine the backbone at a hospital with CAT scanners, MRIs, xrays all generating digital images, and doctors around the hospital accessing a database of those images. 10Gbps isn't enough for applications like this. My local dentist's office uses digital xrays, and they complain about the 1Gbps on their little LAN - and they probably don't have more than about 15 workstations.
And as someone else mentioned, rendering and editing of digital video uses up even more bandwidth. You don't have to be Pixar to need to do stuff like this - many companies in the media business can use this.
okay.. I havent' read the 10Gbit standard, so I'm sort of talking out my ass.. but...
if it follows the way 10 & 100 worked..
This is another one of those "Why you can't transmit at 10Gbs from a single computer over 10Gbps ethernet" posts.
First, realize that the 'speed' of ethernet reflects the maximum use of the broadcast medium at maximum capacity. A medium designed for multiple conversations going on at once.
Example: There is a mandatory 9.6 microsecond gap between attemps to transmit frames on the ethernet (this is a minimum limit). In 10Mbps, this is 96 bits worth of time. In 100, it's 960.. if they do the same in 1000, tht's 9600 bits.
So think of that as a per-frame penalty.
If the frame size stays the same.. the mandatory inter-frame gap is about 6 times the size of the data payload now. (okay, so maybe it's different now..). If this IS the case... what's the point of of 10Gbps? More hosts, no switches. That's right.
Bandwidth is the cure for switching.