10-Gigabit Ethernet Standard Approved

A little birdie brings news that that 802.3ae standard for 10 Gigabit/second Ethernet has been approved. Everyone out there with Gigabit Ethernet - you are now officially obsolete. The new standard is fiber only, no more of that nasty copper stuff.

Comments

What an informative link.

[taliver]

Here's one that might be a little more informative. I leave the google link to someone else.

Re:not obsolete

[larien]

One word: striping. If you put enough disks in, you can get more than 1gbps out of a disk array. Realistically, though, you're limited to using this in two places:

1. Large server with many, many disk controllers and even more disks
2. Network backbones

It'll creep in to the second quickly enough (once Cisco et al support it in hardware), I'd imagine (we already have a 4gbps backbone using 4 gigabit lines in our site) and the former will start happening at the top-end installations of E15K's and the like.

Fine, I'll do it myself.

[taliver]

This is the google link.

In meaningful terms

[Throatwarbler Mangro]

10Gigabit/sec = 1.25Gigabytes/sec

1 LoC (Library of Congress) = 10 Terabytes = 10,000 Gigabytes

That's 0.000125LoC/sec, or roughly 2.22 hours to transfer the entire contents across 10GigE.

Wow.

Re:In meaningful terms

[Dark Nexus]

As a slight correction, when it comes to baud ratings, 10 Gigabit/sec = 1 Gigabyte/sec

It's 8:1 for storage, but generally 10:1 for network ratings (an example - more for serial ports, but it still applies), thanks to a header and a footer bit sent with every byte. Sometimes (rarely), throw in a parity bit for good measure.

Mind you, that's still only 2.78 hours.

Re:In meaningful terms

[psychos]

This is incorrect. Low speed serial interfaces do tend to use a start bit and a stop bit, but higher speed interfaces generally do not.

I'm not very familiar with 10gige technology yet, but my brief research shows that it uses 64B/66B coding (e.g., 2 overhead bits out of every 66). Running at a clock rate of 10.3125GHz, that gives you a full 10Gbps of throughput, or 1.25 GB/sec.

100baseT uses 5B/4B coding, which does result in 2 overhead bits out of every 10 just like your serial line example. However, 100baseT actually runs at 125MHz so you do get a real 12.5 MB/sec out of it.

Of course, if you really want to be picky about "LoC/sec" or whatever pointless measure the popular media has latched onto this week, you need to consider the overhead of TCP headers, whether or not you want to allow jumbo frames in your calculations, and so on.

Re:Can I actually use that?

[larien]

As others have pointed out, this is irrelevant for "average" PCs. Where it comes in useful is for high-end servers, network backbones and (possibly) clusters which throw a lot of data around.

Re:Beowulf Cluster

[DHam]

Actually, for many parallel applications the killer is latency rather than bandwidth. That's why we end up shelling out so many Euros for proprietary networks like Myrinet. I don't know what the latency on 10Gbit is but Gbit ethernet is not really much better than 100Mbit.

Re:Hit me with the clue-stick please!

[GigsVT]

revcom

IEEE

Consider yourself hit with clue-stick.

Re:not obsolete

[Ogun]

Oh, you mean like this:
Cisco 12000 10Gb line card
or like this:
Catalyst 6500 10Gb line card

Cisco did announce these a while ago.

Re:not obsolete

[questionlp]

The highest speed PCI-X (64-bit @ 133Mhz) is capable of reaching ~1GByte/sec which is just about the speed of 10 Gig Ethernet. There was/is the promise of Araphoe (sp?) that resembles AMD's HyperTransport but would be used for expansion cards rather than a chip-to-chip pathway.

The other bottleneck with even high-end Intel-based servers could easily choke when dealing with not only 10 Gig Ethernet but also add Fiber Channel, multiple channels of Ultra 160 or Ultra 320 SCSI RAID, etc., since the memory bandwidth (and processor bus speed?) would then become the possibly the next bottleneck. RISC servers don't have that much of a problem just yet, but sooner or later it will be.

Re:wither Cat6 ?

[Barche]

This thing runs at 10 Gbps. Ethernet uses Manchester encoding (+-=1, -+=0), which means you have to double the bps to obtain the bandwidth. So you need (about) 20GHz of bandwidth on the cable. At that frequency, losses in a copper cable are just too high. You'd need to use either wave conduits (big metal pipes, not an option) or optical fiber.

Many are missing the point here

[buss_error]

Sure, you can't use all 10G on ONE machine. Even a server can't use all that speed. Even using many NICs. (Buss congestion, ya know.) That isn't the point here. The point is that instead of having to segment a lot of traffic off to a vlan or other workaround, that traffic can be supported on one lan. This reduces equipment, interconnections, configuration, and alot of other headaches. In short, you can reduce the total points of failure.

And remember, Intel isn't the only hardware platform out there. While I don't know of a hardware platform that can make fully support the speeds needed, there are some that can support better than 4000 Kbps now.

Re:wither Cat6 ?

[ivan256]

Gigabit ethernet took all it's electronic specifications from fibre channel. When Gigabit came out, there was already copper available for fibre channel, and there was nothing stopping you from using those GBICs. The recent development was that they figured out how to get the signal over regular CAT5.

I'm sure that there will be a copper spec for 10 gigabit too, it's probably just not ready yet. Consider that people will be wanting to use this on the backplane of embedded network hardware, and blade servers.

Hairy smoking golfballs

[alienmole]

(come to think of it, this implies that physical size is a fundamental limiting factor on the speed of computers - it does no good to have an infinitely fast CPU if its parts can't communicate rapidly due to speed-of-light delays...)

Researchers have realized this for decades. Before enormous silicon chip densities became ordinary, engineers at IBM (IIRC) used to say that the future of computers was "hairy smoking golfballs". This captured a number of important characteristics of very fast computers:

• Hairy because of all the wiring that would be involved (like the rubber strings in the core of old golfballs)
• Smoking because of all the heat that would be generated
• Golfball-sized because of latency issues, as you said
• Golfball-shaped because a sphere is the most efficient use of space for components that need to be interconnected.

Since those days, Intel and its competitors have fulfilled all of these predictions except for the spherical shape, which is much more difficult and not as important as the other characteristics.

A Pentium 4 is hairy - those 55 million transistors have a lot of connections; and smoking, as anyone whose CPU fan has broken can attest. It's smaller than a golfball in cross-sectional area. That size isn't just to make them more convenient! If a physically bigger CPU would be faster, you can bet someone would be building them.

Re:not obsolete

[questionlp]

Oops... forgot to mention that the currently available chipsets that support one or more PCI-X busses include the Intel E7500 and the ServerWorks Grand Champion (GC) series (either the HE or the LE, depending on the number of processors required).

The "northbridge" of the Intel E7500 supports two PCI-X busses (more information about the chipset can be found here.

The ServerWorks GC series support for PCI-X start from 2 independent busses (the GC-SL) up to six PCI-X busses (the GC-HE). Specs on the ServerWorks stuff is located here.

I'm not completely sure if the AMD Hammer chipsets will include PCI-X support initially, but if one were to give up AGP 8x (which isn't really needed on a server) then you can turn that into a PCI-X bus to support a single 10 Gig Ethernet controller.

Of course, there is still the bottleneck of the memory subsystem which can make or break a high-end system.