10-Gigabit Ethernet Standard Approved

A little birdie brings news that that 802.3ae standard for 10 Gigabit/second Ethernet has been approved. Everyone out there with Gigabit Ethernet - you are now officially obsolete. The new standard is fiber only, no more of that nasty copper stuff.

Comments

Approved != Affordable

[CodeMonky]

Approved or not it will still be some time before costs come down enough so that comapnies can justify replacing their gig backbone with 10gig.

Re:Approved != Affordable

[ajvtoo]

Be good for education and research though. JANET is planning to be 10Gbps on the core by late-Summer so they'll be pleased the standard has been approved (see http://www.ja.net/superjanet/index.html)

Re:Approved != Affordable

[Cmdr Taco (luser)]

Gigabit?

Hell, where I work, we're still patting ourselves of the back for getting rid of that rotten old coax. We'll probably be languishing in 100Base-T land for a while yet.

The early adopters of 10Gb ethernet are certain to be:
Universities
e-Commerce/ISP outfits
Large corporation's data centers

There is still plenty of life left in gigabit ethernet. In fact, it is still just gaining momentum.

Re:Approved != Affordable

[Anonymous Coward]

Ten bucks (gbpounds if you must)
says that that upgrade will be
a 10 GB/s OC-192 SONET link.

not obsolete

[Mortin]

considering hdds can hardly transfer at 1gbps, gigabit is hardly obsolete... yet :)

Re:not obsolete

[larien]

One word: striping. If you put enough disks in, you can get more than 1gbps out of a disk array. Realistically, though, you're limited to using this in two places:

1. Large server with many, many disk controllers and even more disks
2. Network backbones

It'll creep in to the second quickly enough (once Cisco et al support it in hardware), I'd imagine (we already have a 4gbps backbone using 4 gigabit lines in our site) and the former will start happening at the top-end installations of E15K's and the like.

Re:not obsolete

[GeckoUK]

You are correct, but only in the case were you have a network of two computers.

In the real world a company deploying this is likely to have hundreds if not thousands of machines all connected at once.

Re:not obsolete

[Anonymous Coward]

not too many people would hook up a single box to a 10GB pipe (although many of us would like to). who knows - it might happen eventually!

Re:not obsolete

[Anonymous Coward]

"looks at the 10 gigabit line he just ran from uunet to his house and scratches head." Damnit! I knew they were ripping me off.

Re:not obsolete

[GigsVT]

We're going to need something to replace PCI before we can use 10Gbit ethernet fully though. Even 64bit 66Mhz PCI has a max (in theory) of 528 Megabytes/sec.

On a side note, I have sucessfully pulled 130Mbytes/sec out of 5400 RPM IDE Disks on 3ware controllers, with a cost less than $9000. 3 controllers, 24 disks, 64 bit 33Mhz PCI. RAID 0 over 5. So the potential is there to exceed current GigE, without too many disks or controllers, or getting too expensive.

It would also help a lot if we could get regular gigabit ethernet working well first. I think there was a story here on Slashdot not long ago that showed that most GigE cards had trouble pushing over 400Mbits even with large frames. Only the expensive $500 one came close to it's full potential (900Mbits). My experience is that without jumbo frames, there is hardly any advantage with lower end GigE cards.

Re:not obsolete

[Ogun]

Oh, you mean like this:
Cisco 12000 10Gb line card
or like this:
Catalyst 6500 10Gb line card

Cisco did announce these a while ago.

Re:not obsolete

[Astrorunner]

Thats silly.

Consider first that you can only utilize about 38% of the bandwidth over Ethernet before collisions start dragging things down. Then consider that it is not just one person who is on that ethernet, but a multitude of people. While one person may not be able to fill up the entire pipe. And do we really want to do that?

Re:not obsolete

[questionlp]

The highest speed PCI-X (64-bit @ 133Mhz) is capable of reaching ~1GByte/sec which is just about the speed of 10 Gig Ethernet. There was/is the promise of Araphoe (sp?) that resembles AMD's HyperTransport but would be used for expansion cards rather than a chip-to-chip pathway.

The other bottleneck with even high-end Intel-based servers could easily choke when dealing with not only 10 Gig Ethernet but also add Fiber Channel, multiple channels of Ultra 160 or Ultra 320 SCSI RAID, etc., since the memory bandwidth (and processor bus speed?) would then become the possibly the next bottleneck. RISC servers don't have that much of a problem just yet, but sooner or later it will be.

Re:not obsolete

[13Echo]

I can pull around 90MB/s off of my dual 7200 RPM drive RAID array. I could imagine that 4 striped standard ATA drives could do well over 130-150 MB/s making full use of gigabit ethernet on a fast bust and fast ethernet card.

Re:not obsolete

[monkeydo]

Please put down the CCNA study guide and back away slowly. Ever since the invention of switches most people haven't had to worry about collisions. We're talking about full duplex, 2 hosts on a collision domain. In other words no collisions. If you are getting collisions on GigE or switched FastE you are doing something very wrong.

Re:not obsolete

[_|()|\|]

We're going to need something to replace PCI before we can use 10Gbit ethernet fully though.

If a network has more than two nodes, as most networks do, then each node isn't expected to saturate the network. Think of adding more lanes to highway, rather than increasing the speed limit.

Re:not obsolete

[ivan256]

I've seen system controllers that had the memory controller, CPU bus, and 10 gigabit ethernet in the same chip. If you can get 10 gigabit to memory, you can use all the bandwidth. At least in bursts...

Re:not obsolete

[cheezedawg]

We're going to need something to replace PCI before we can use 10Gbit ethernet fully though.

Good point. We need something like 3GIO. Plus something has to be done about the bandwidth between the northbridge and the southbride. Right now it is at 266 MB/sec with plans to increase to 533 MB/sec.

Re:not obsolete

[Telastyn]

Intel gig cards can do ~ 900mbits with drivers tuned for running that high. The "normal" drivers expect to be usually running around 200mb or so, and 1gb as a peak.

Re:not obsolete

[questionlp]

Oops... forgot to mention that the currently available chipsets that support one or more PCI-X busses include the Intel E7500 and the ServerWorks Grand Champion (GC) series (either the HE or the LE, depending on the number of processors required).

The "northbridge" of the Intel E7500 supports two PCI-X busses (more information about the chipset can be found here.

The ServerWorks GC series support for PCI-X start from 2 independent busses (the GC-SL) up to six PCI-X busses (the GC-HE). Specs on the ServerWorks stuff is located here.

I'm not completely sure if the AMD Hammer chipsets will include PCI-X support initially, but if one were to give up AGP 8x (which isn't really needed on a server) then you can turn that into a PCI-X bus to support a single 10 Gig Ethernet controller.

Of course, there is still the bottleneck of the memory subsystem which can make or break a high-end system.

Re:not obsolete

[Amazing Quantum Man]

Mmmmm... my own private 10Gb/s pipe.

Let's see, that's about 1GB/s, so I can download the three ISOs for MDK8.2 in oh... 3 seconds or less!

Re:not obsolete

[Anonymous Coward]

Doubtful, since while your computer is probably able to handle sturating GigE and can probably come close to handling 130 MB/s from the drives, it can't do both at the same time.

Re:not obsolete

[alienmole]

I could imagine that 4 striped standard ATA drives could do well over 130-150 MB/s

A standard 33MHz PCI bus would limit the top end to about 132MB/s, which is the point the other poster was making - that current buses will make it difficult for a single machine to saturate 10Gbit Ethernet. Not that that's a serious problem, exactly...

Re:not obsolete

[Detritus]

Ethernet is not Aloha. See Measured Capacity of an Ethernet: Myths and Reality, Boggs et al.

Re:not obsolete

[questionlp]

The connection between the north and southbridges with desktop chipsets do start at PCI speeds and go all the way up to 266MB/s for the current Intel and Via (the latter using V-link) chipsets and much higher the chipset maker uses AMD's HyperTransport (right now, it looks like nVidia is the main desktop chipset maker that is using it, but SiS is also looking into using it in the Hammer chipsets).

Server chipsets usually are more complex and can provide multiple high-speed interconnects between the "northbridge" and the different "southbridge" chips (like the Intel E7500 chipset supports a total of three 1Gbyte/sec connections between the northbridge, the MCH, and I/O controllers, two of which support PCI-X hubs that provide up to 1Gbyte/sec to the expansion boards).

Re:not obsolete

[Pii]

I think you, and a number of others that are pursuing this "disk access" angle, are missing the purpose of 10GigE.

While I have no doubt that you'll see a 10GigE interface card soon, and that large hosts (Big Iron, Big Unix) may be able to take advantage, it's really not about Host-to-Switch connectivity.

It's about Switch-to-Switch connectivty.

I deploy a lot of networks where you have a tremendous number of ports at the access closet, and you typically end up with 1 to 2 Gbps to the closet (2 Gbps when channelling).

As an example, a Cisco Cat 4006 chassis, fully loaded, can support up to 240 users. That many users can certainly chew up a 1 Gbps uplink to the core.

Additionally, what about when you need to cascade multiple switches in a closet, or expand your core? Sure, maybe I get 64 Gbps across the backplane of a given switch, but what about between the two switches? I'm limited to 4 Gbps between them in a channel configuration, and in so doing, I have chewed up 8 (!) Gigabit ports.

Now, for those same 8 ports, I get 40 Gbps between the switches. Odds are, I can get away with only 1 or 2 connections, using just 4 ports.

Re:not obsolete

[Eraser_]

I find your post funny :)

Interesting fact: i have seen "ethernet switches" sold which have collision lights. It makes me wonder how dumb the network cards are by that comapany that they can't get their own carrier detection right. :) Either that or they are selling to the masses something which doesn't do actual switching, or has such a pitiful buffer that it still needs a collision light.

Disclaimer: I am a CCNA, however i saw my classmates for who they were. A trained monkey could pass those classes.

Re:not obsolete

[monkeydo]

Interesting fact: i have seen "ethernet switches" sold which have collision lights. It makes me wonder how dumb the network cards are by that comapany that they can't get their own carrier detection right. :) Either that or they are selling to the masses something which doesn't do actual switching, or has such a pitiful buffer that it still needs a collision light.

You missed the third option, which is that they know more about Ethernet than you do.

There is nothing wrong with putting a collision light on a switch. Ethernet and Fast Ethernet switch ports can certainly have collisions.

In older networks switches are frequently used to agregate hubs. Even though the switchport represents a single collision domain, if you plug a hub in there may be many hosts. It is very likely you will see collisons on the switch in this case. It is also likely that you will see collisions on Half-Duplex segments with a single host if it is very busy. In this case however you should see very few.

In Full-Duplex mode collisions cannot occur, and in fact collision detection is turned off. Do not make the mistake of assuming that because a single device is plugged into a switchport that its operating in Full-Duplex or that you won't see collisions.

The most common way to misconfigure a switch is to have all the ports set to auto, and the hosts hardcoded full duplex.

Disclaimer: I am a CCNA, however i saw my classmates for who they were. A trained monkey could pass those classes.

No comment.

Re:not obsolete

[afidel]

Cisco shipped their first 10Gb ethernet module back in November of 2001. They are for the Catalyst 6500 of which I have about 10 onsite, though none run 10Gb yet, just a 1Gb blade per and the datacenter one has a second one for some of the high volume servers.

Re:not obsolete

[KjetilK]

Yeah, besides, my serves says:
root@pooh:~> hdparm -t /dev/hda

/dev/hda:
Timing buffered disk reads: 64 MB in 6.75 seconds = 9.48 MB/sec

Though I admittedly don't really know what this means, something tells me that the bandwidth between my HD and the rest of the computer needs to improve a lot before I need to worry about the bandwidth outside the box... :-)

Re:not obsolete

[Anonymous Coward]

You forget one thing,this is 10 gig ETHERNET.
A typical IDE drive will realistically get 40 mbps.
Sure using top of the line SCSI & RAID solutions in servers, and having arrays with tens or hundreds of serrvers could easily use 1 gbps, but if you have such a setup you'll already have a much faster solution, something like OC-768.(4 terabit per sec.)

Few people that could use 1 gbps use typical ethernet.
They got high end solutions.
(Last i checked one light frequency(channel) is capable of 40 gbps, and one single mode fiber capabel of 40 channels)

Re:not obsolete

[Sycraft-fu]

We'd actually talked about these at work since our backbone is built all on 6500s. Not that we need the bandwidth right now but it never hurts to stay ahead of the curve espically since we're starting to run gigabit out to buildings. Right now the big problem is port density. The gigabit modules we use now have 16 ports per blade, however the 10 gig modules are only a single port per blade. The chassies would get real full real fast if we tried to do that. However I'm sure it won't be long before higher density modules are available.

Re:not obsolete

[Sycraft-fu]

"The most common way to misconfigure a switch is to have all the ports set to auto, and the hosts hardcoded full duplex."

Or try to use a 3com card with auto anywhere. I don't know why but 3com 905s can't seem to autonegoiate to save their lives. When I started at my current job the room we're in is run from a 3548. the ports were all set too auto which worked just fine with all the Intel NICs in everyone's system. It all negoated to 100 full and it was good. However my system had a 905 in it and for whatever reason it kept getting a duplex mismatch with the switch. Both sides had to be set to 100 full.

Now the really weird thing is I have a 905 at home and a 3com 3300 switch (24 port 10/100). You'd think their network card and their switch could auto negoiate. Nope.

I don't know what their major malfunction is, Intel cards don't seem to have any trouble at all.

Re:not obsolete

[mjoconnor81]

Even 64bit 66Mhz PCI has a max (in theory) of 528 Megabytes/sec.

Remember that Gbit is not Gbyte. 8 bit's to a byte. 528*8=4224.

Also keep in mind that this is currently designed for desktops, but servers and interswitch communication.

Re:not obsolete

[GigsVT]

Even 64bit 66Mhz PCI has a max (in theory) of 528 Megabytes/sec.
Remember that Gbit is not Gbyte. 8 bit's to a byte. 528*8=4224.

Put down the crackpipe. :)

64bits=8 bytes

8*66=528 Megabytes/sec

Re:not obsolete

[Anonymous Coward]

3com ethernet equipment is standards-violating crap. It's insane than anyone buys them.

Re:not obsolete

[pandaman9000]

I have had experience with several single and dual processor servers and workstations, of various speeds and scsi/ide configurations. Two things were held in commom for all of them: 1) They all had some iteration of the 3C905B ethernet card in them. 2) They were all connected to Cisco 29i00 series switches that had to be set up at full duplex/100Mbit for both card AND switch to not fall back to half duplex. Personally, I get better performance on a 10Mbit full duplex link than on a 100 Mbit half duplex.....

Re:not obsolete

[flonker]

This would be the case if you were using a hub, but not if you're using a switch. With a switch, each node has access to the full bandwidth of the network.

(There are some other issues at work here, but in general, this holds true.)

Re:not obsolete

[rew]

On a side note, I have sucessfully pulled 130Mbytes/sec out of 5400 RPM IDE Disks on 3ware controllers, with a cost less than $9000. 3 controllers, 24 disks, 64 bit 33Mhz PCI. RAID 0 over 5. So the potential is there to exceed current GigE, without too many disks or controllers, or getting too expensive.

What ARE you talking about?

Get an Asus A7M266-D based computer, two Athlons, and 4 maxtor 160G 5400 RPM drives on the on-board controller and you get 130Mb per second. $2500?

Roger.

Re:not obsolete

[GigsVT]

I was just posting my experience, I didn't say it couldn't be done with less.

Re:not obsolete

[zeno_2]

Hmm.. im pretty sure what the above poster was stating is that this 10gigabit network is not for a single machine, you could hook up 10 1gigabit machines to this and use it as a single pipe.

Anyone using gigabit networks is going to be using a switch, there's really not much of a point in using hubs anymore. The idea that a 10gigabit network is stupid because pci can't keep up, that really doens't matter though, as the above poster stated, its not making one single machine communicate faster, its making a network of machines being able to communicate with each other without running into situations where all the bandwidth is saturated.

Re:not obsolete

[rew]

Same here :-)

Approved by Revcom, who the hell is that?

[Anonymous Coward]

Uh, doesn't the IEEE approve these things?

Am I missing something here?

Re:Approved by Revcom, who the hell is that?

[cryptor3]

http://standards.ieee.org/faqs/stdsbdFAQ.html#c3

sweet

[trueimage]

too bad i still can't afford gig-e
too bad my pc couldn't take advantage of 10gig-e either

heh

still nice to dream...

Re:sweet

[Detritus]

The NICs for 1000BASE-T are affordable. The problem is the lack of cheap switches.

Re:sweet

[pboulang]

hmmm, is $39 not cheap enough?

Re:sweet

[Anonymous Coward]

Too bad "your PC" is probably hooked to a cable modem or DSL anyway.

Re:sweet

[Detritus]

10/100BASE-T switches are cheap. 1000BASE-T switches are still expensive.

Re:sweet

[ncc74656]

hmmm, is $39 not cheap enough?

You might want to read the original message more carefully...the problem is that Gigabit Ethernet switches are still expen$ive. The $39 switch to which you linked is a Fast Ethernet switch, which is an order of magnitude slower.

I've seen 5-port Fast Ethernet switches for $25. The cheapest Gigabit Ethernet switch I saw on Pricewatch just now was the D-Link DGS-3204, a 4-port switch for $300.

Re:sweet

[pboulang]

I stand corrected.. serves me right for missing a 0 :)

Re:sweet

[Ex-MislTech]

Well I only had two systems with IDE raid so
I just used two Gig-ethernet cards and a
cross over cable between them for passing
cd images or other large 500meg+ files .

I just set them up on a 2nd class C network
in addition to the 100 mega-bit one I have been
using for years .

Can really cram alot across it, in a short
amount of time .

By the way, 100 Base-t does not hit line speed
most of time either, test it with data chunks,
its horrible un winblow$ , *nix is alot better.

Ex-misltech

What an informative link.

[taliver]

Here's one that might be a little more informative. I leave the google link to someone else.

Fine, I'll do it myself.

[taliver]

This is the google link.

Re:Fine, I'll do it myself.

[Anonymous Coward]

Looks like you posted this just reply to your own post as fast as you possibly could. 0800 and 0802. Hmmmmmm.

Re:Fine, I'll do it myself.

[satanami69]

Best Karma Whoring ever!

Re:Fine, I'll do it myself.

[Anonymous Coward]

Actually, I got the link, went to it, it looked OK, I posted it, went back to the link, saw it was a registration required site, so decided to post the google search as well.

Posting anonymously since it's well off topic.

In meaningful terms

[Throatwarbler Mangro]

10Gigabit/sec = 1.25Gigabytes/sec

1 LoC (Library of Congress) = 10 Terabytes = 10,000 Gigabytes

That's 0.000125LoC/sec, or roughly 2.22 hours to transfer the entire contents across 10GigE.

Wow.

Re:In meaningful terms

[Iamthefallen]

yes yes, but how much pr0n can I download?

Re:In meaningful terms

[stevelinton]

About 1000 DVD quality video channels, or a feature length movie in a few seconds, or a bit longer for some future HDTV standard.

Streaming digital holographic porn, anyone?

Re:In meaningful terms

[glwtta]

I thought the time honoured (and extremely relevant) measure of LoC/s has been officially replaced with HG/s (Human Genomes per second)? Mostly because it allowed for a lot more flexibility in making up figures that don't really tell you much, if I remember correctly.

Re:In meaningful terms

[Anonymous Coward]

How many RI's would the paper for that cover?

(RI= 1 Standard Rhode Island Unit)

Re:In meaningful terms

[Dark Nexus]

As a slight correction, when it comes to baud ratings, 10 Gigabit/sec = 1 Gigabyte/sec

It's 8:1 for storage, but generally 10:1 for network ratings (an example - more for serial ports, but it still applies), thanks to a header and a footer bit sent with every byte. Sometimes (rarely), throw in a parity bit for good measure.

Mind you, that's still only 2.78 hours.

Re:In meaningful terms

[Anonymous Coward]

Many. Why is Rhode Island even a state? It's just a suburb of Boston.

Re:In meaningful terms

[psychos]

This is incorrect. Low speed serial interfaces do tend to use a start bit and a stop bit, but higher speed interfaces generally do not.

I'm not very familiar with 10gige technology yet, but my brief research shows that it uses 64B/66B coding (e.g., 2 overhead bits out of every 66). Running at a clock rate of 10.3125GHz, that gives you a full 10Gbps of throughput, or 1.25 GB/sec.

100baseT uses 5B/4B coding, which does result in 2 overhead bits out of every 10 just like your serial line example. However, 100baseT actually runs at 125MHz so you do get a real 12.5 MB/sec out of it.

Of course, if you really want to be picky about "LoC/sec" or whatever pointless measure the popular media has latched onto this week, you need to consider the overhead of TCP headers, whether or not you want to allow jumbo frames in your calculations, and so on.

Re:In meaningful terms

[monkeydo]

Since the amount of overhead really varies with the transmition medium it makes a lot more sense to use Byte=8 Bits and then add a fudge factor for overhead. It's easy to say it takes 10 bits to send a byte on a serial link, but in networks you have to take into account things like MTU, different protocol headers, CRCs and congestion limitations.

Re:In meaningful terms

[Reality Master 101]

Let's see if that number is right... I'm curious. According to this, the LOC is "the largest library in the world, with more than 120 million items on approximately 530 miles of bookshelves. The collections include more than 18 million books, 2.5 million recordings, 12 million photographs, 4.5 million maps, and 54 million manuscripts."

18,000,000 books * 200 pages, scanned, * 100K/page = 343 Terabytes.

2.5 million recordings * 3 minutes per * 1 meg/minute MP3 = 7.5 Terabytes.

12 million photographs * 250K per = 2.9 Terabytes.

4.5 million maps * 20 meg scan = 0.9 terabytes

54 million manuscripts * 50 pages, scanned * 100K/page = 257 terabytes.

My conclusion: The LOC is more like 611 Terabytes, if you're talking a complete collection with all detail. And that still leaves 29 million items that they didn't identify (out of the stated 120 million items). But those may not be media materials.

I'm not sure about this 10 Terabyte number. If you just talking books, that's 610K / book. That seems pretty high for just text, but low even for a low-res scan. I think someone just pulled it out of their butt.

Re:In meaningful terms

[Anonymous Coward]

I think we have to start using Kazaa netwrok contents. Last night I saw 2000 Terabyte.... ;)

10GE. Great. I can use up my monthly allowance for my old ISP in 4 seconds flat.

Re:In meaningful terms

[Anonymous Coward]

Don't forget to that the monikers for 10/100/1000/10000 are the THEORETICAL capacities. In reality it is very rare that you will actually acheive that high of a transfer rate. Given all of your transport protocol's overhead, need for retransmits with errors, etc your not likely to ever actually hit real 10 Gig.

Why Bother ?

[tealover]

Cable broadband won't allow you to use unlimited bandwidth. Soon, the Internet will be just like professional sports. You'll need corporate sponsorship to really enjoy the experience.

Digital Divide? Here we come again.

Re:Why Bother ?

[CausticPuppy]

Actually, physics won't allow you to use "unlimited bandwidth."

Re:Why Bother ?

[HowlinMad]

thats assuming that our current phyics theories are actually correct.

Its not fair!

[GnomeKing]

*looks at his 14.4k modem*

*looks at the article*

*looks at his modem*

*cries*

Re:Its not fair!

[Jugalator]

Upgrade to 56k, you cheap, whining bastard! ;-)

Re:Its not fair!

[Anonymous Coward]

yeah you stupid ass fucknut

Re:Its not fair!

[Abreu]

Some ISP's still can only connect at thirty-something kbps... And some "free" ISP's limit you to 28.8

Re:Its not fair!

[Reziac]

I did. Verizon's shit phone lines downgraded me to 28.8 (26k to be exact) :(

BTW I have some 2BaseT NICs, available at a good price :)

Re:Its not fair!

[Amazing Quantum Man]

2BaseT? That's 2Mb/s on Twisted Pair!
If you're referring to ThinNet, that's 10Base2.
Thicknet is 10Base5.

Re:Its not fair!

[Reziac]

The really old slow stuff, yeah, that's what I was referring to. I may have misremembered what to properly call it, and managed to make something up :)

BTW did anyone else see the gigabit ethernet review in Windows&Net magazine? seems so far it's not really living up to its name.

Re:Its not fair!

[UranusReallyHertz]

BAH! As I type this, I am connected at 24.0 Kbps. Rural life has many advantages, but bandwidth is not one of them.

With this annoucement

[SkyLeach]

It should be obvious that to burry copper is completely obsolete. Per yard, fiber should be cheaper to manufacture and bury.

10Gb speeds should be enough for anybody, so start building the infrastructure now and leave the telcos in the dust.

Will they do it? No. Why not? Because they think that they should bury the copper/fiber hybrid cable that they have been burying and come back and do it again later.

Burying cable is the most expensive part of telecomm.... retards.

Re:With this annoucement

[pgilman]

"10Gb speeds should be enough for anybody..."

uuh, why does this sound familiar? seems to recall something about 640k... ;-)

Re:With this annoucement

[Hieronymus Howard]

10Gb speeds should be enough for anybody

Just like 640K is enough for anybody.

HH
--

Re:With this annoucement

[Anonymous Coward]

For retarded people currently availible telco equipment transmits @ 1,6 Tbits/sec and more over single fiber (a normal cable has what about 100 fibers), there is enormous overcapacity with people having up to 80% of barried fibers unlit.
The availibility of this standart has no impact
on longhaul telco networks.

Re:With this annoucement

[stevelinton]

Actually, burying pipe is the most expensive part..... I worked briefly for a university computing service a few years ago and they spent an absolute fortune to buy a network of yellow plastic pipe connecting all their buildings. A relatively trivial incidental expenditure was to pull some cable through it. When that sort of cable is obsolete, a further trivial expenditure will replace it, etc....

Re:With this annoucement

[Anonymous Coward]

"barried"?

Do you mean "buried"?

It is ironic that you called someone a retard in that same post.

Re:With this annoucement

[trueimage]

the cost is in the implementation, not the cable itself. you'd have to replace all the cabling, and re-do the whole network setup... the man hours and downtime and glitches would cost, even for a school, let alone a business

Re:With this annoucement

[haa...jesus christ]

10Gb speeds should be enough for anybody, so start building the infrastructure now and leave the telcos in the dust.

I remember hearing this about memory once upon a time...I forget who said it though... :)

Re:With this annoucement

[runcible]

10Gb speeds should be enough for anybody No.

I will have enough bandwidth when I get transfer rates comprable to the those on my motherboard.

Once it no longer matters where the physical components of my computer are. Once my processor is rented cycles in Korea, my RAM is in orbit, and my storage media is distributed in 10K chunks all over the former Soviet Union, and there is no noticable performance hit. Then I'll have enough bandwidth.

Re:With this annoucement

[Anonymous Coward]

gee. i wish my motherboard was that fast.

Re:With this annoucement

[PhxBlue]

10Gb speeds should be enough for anybody, so start building the infrastructure now and leave the telcos in the dust.

It doesn't matter how much bandwidth you give me, I will always want more. And so will the people who design software to run on higher-bandwidth networks.

Re:With this annoucement

[captaineo]

I will have enough bandwidth when I get transfer rates comprable to the those on my motherboard.

Yes, that's a good observation... You have to watch out for the latency though! Today most software hardly "maxes out" DDR/RDRAM memory because of fetch latencies. Then consider that the pure speed-of-light delay to Korea is on the order of 0.1 seconds...There's no way you're going to be able to run general-purpose software with that kind of latency; heck you would even be able to see the delay on each mouse click!

(come to think of it, this implies that physical size is a fundamental limiting factor on the speed of computers - it does no good to have an infinitely fast CPU if its parts can't communicate rapidly due to speed-of-light delays...)

Re:With this annoucement

[Anonymous Coward]

Gigabit ethernet should be enough for you then. Unless you have one of those fancy PCI64/66MHz Boards, your PCI bus has a theoretical upper limit of 133 MBytes/s and much lower practically achievable rates. And if you got a server board: 10 GBit Ethernet is 10 times faster than Gigabit Ethernet while PCI64/66MHz is only 4 times faster than PCI32/33MHz... In other words, the PC-architecture computer which can saturate 10 Gigabit Ethernet hasn't been built yet.

Re:With this annoucement

[Reziac]

The price of cable (copper or fibre) by the foot is not the real factor. Most of the real cost lies in labour, equipment, permits, etc.

Here's a real example from SoCal: by the spool, telephone line itself costs about $5/foot. But the total cost to lay underground lines is about $40/foot. (Compare to stringing now-mostly-prohibited overhead lines at $16/foot.)

Re:With this annoucement

[roju]

I'm suprised they didn't already have a network of tunnels connecting everything. I was under the impression that many schools do, especially older ones - ones that generate their own heat at a central location.

Re:With this annoucement

[BreakWindows]

It should be obvious that to burry copper is completely obsolete.

Wrong! Copper is already strung around every city and home in America (probably a hefty portion of the world). And, there's a standard for gigabit over copper:

Deployment Guide
[PDF]

It's limited to 100 meters, but for communities, home networks and any switched network, I don't see a point in passing up what is already laid in the building. For future digs, they could go either way, and I'll agree fiber is the way to go. But let's not ditch copper just yet...it seems to have some usefulness left in it.

Re:With this annoucement

[DickPhallus]

The university of waterloo does this, this page explains it all:

U of Waterloo tunnles

Re:With this annoucement

[Anonymous Coward]

10Gb speeds should be enough for anybody,

Damn, I gotta save this comment! Look back 10 years. Now imagine ten years from now. Or, well, 10 years from when this is implemented. What's uber-fast today will be a joke tomorrow. You can already buy 200 GB IDE hard drives! My first computer had 20 MB and 1 MB ram and that was ~1991.

Re:With this annoucement

[BESTouff]

10Gb speeds should be enough for anybody, so start building the infrastructure now and leave the telcos in the dust.

I already heard that before ...

Re:With this annoucement

[Anonymous Coward]

It should be obvious that to burry copper is completely obsolete.

Oh yeah? I'd like to see you efficently transfer electricity with fiber.

Another (0, Troll) from Out-of-context Troll.

Re:With this annoucement

[b1t r0t]

It's limited to 100 meters, but for communities, home networks and any switched network

Dammit, and I'm about 160 meters from my DSL head end. I guess I won't be getting 1000BT delivered to my house any time soon. Gives me more time to afford that gig module for the Cisco switch in my house, though.

Re:With this annoucement

[Yunzil]

Speed of light, runcible. Runcible, speed of light.

I'll leave you two to get acquainted. :)

Re:With this annoucement

[Anonymous Coward]

10Gb speeds should be enough for anybody

Just like 640K is enough for anybody.

While I'd never said "640GB (GB not KB) of RAM should be enough for anybody (example a Toy Story quality graphics game), agree that 10Gb should be enough for any end-user. Becouse if it's not enough, you can always just use your computer as a full-multimedia-ass-kicking terminal to a supercomputer somewhere connected to other supercomputer with even 10Tb/s pipes. (And today, using X-Window compression protocol you can browse the net with acceptable speed using netscape running on remote mashine and X-server on local one connected with a 33.6kbps modem.)

Re:With this annoucement

[roju]

Oddly enough, I was basing my assumption on UW :)

Although this is the first actual evidence I've seen of their existance. Thanks for posting that link.

Re:With this annoucement

[Macka]

The base note said:

no more nasty copper

And you said:

It should be obvious that to burry copper is completely obsolete.

Are you guys nuts? Working with copper is so much easier than with fibre. Fibre is fragile (fixed bend radius to avoid cracking) and comes in fixed sizes so it's more awkward to do a neat and tidy job because you have to loose the excess somewhere. Then there's the extra cost of the GBIC's and higher cost of the hardware it connects into. Copper cables on the other hand you can make yourself, customising the size to your exact requirements. It's so much more durable, and you can mix and match the same patch panels with 100Mbit and 10Mbit networks.

If you think that copper is undesirable, you've obviously not had any experience with sizable projects in this area.

I'd choose copper Gigabit over Fibre any time.

Re:With this annoucement

[stevelinton]

It was Cambridge (UK). The university is scattered through and around the town, and certainly doesn't have anything as centralized as a steam heating system (or, at that time, any central management worth a damn either). The network installation was further complicated by the fact that, in large parts of the town practically everything is a listed historical site of some kind, so huge and strange efforts had to be made not to damage anything. Rumours also mention problems around the Old Cavendish lab (in the town centre). Rutherford split the atom there in 1908 and parts of it are apparently still radio-active.Finally Cambeidge is basically built on a swamp and about 1m above sea-level, so if you dig a hole it tends to fill with water pretty quickly.

Caveman

[led]

Obsolote ?!? I must be a caveman then, with my el cheapo 10Mps "backbone" at home...

Re:Caveman

[Anonymous Coward]

Make that two cavemen.

10Gb is nice for quick infrastructure, but
the real trend for home/small networks is
WiFi, obviously.

Re:Caveman

[Anonymous Coward]

Gosh, I still have Coax Ethernet.

Re:Caveman

[Anonymous Coward]

Erk, 100Mbps is more along the lines of something acceptable if you use a server for network storage in your house. It just takes ages with 10BaseT. And you can't get 100Mbps with WiFi (yet), so you're stuck with pulling cat-5 through the walls.

Re:Caveman

[Amazing Quantum Man]

Your Root-ness, I'm just a simple caveman. Your high speed data access frightens me.

-- Frozen Caveman Sysadmin

Re:Caveman

[bluGill]

my Sun3-60s only have access to 10Base, so that is what I use. I'm planing on a wireless network, but again as you say, not fast. My 386-25 still does useful work, the Suns are more for bragging rights.

I've thought about 100Base, but I only have one machine (the server) that could use the bandwidth, so it hardly seems worth the upgrade.

Re:Caveman

[Anonymous Coward]

I'll upgrade when I can get 10Gb/s to my house. Then we can download full 5GB DVD is a few minutes.

Beowulf Cluster

[gartogg]

With 10gbps ethernet, imagine the parallel processing! ummmmmm.... large primes.....

Re:Beowulf Cluster

[DHam]

Actually, for many parallel applications the killer is latency rather than bandwidth. That's why we end up shelling out so many Euros for proprietary networks like Myrinet. I don't know what the latency on 10Gbit is but Gbit ethernet is not really much better than 100Mbit.

Re:Beowulf Cluster

[joib]

Yes, and myrinet (wulfkit/quadrics/via/whatever) makes Gb ethernet seem dirt cheap by comparison...:(

Re:Beowulf Cluster

[Anonymous Coward]

An increass in network bandwidth will have very little effect on the search for large primes. Other parallel tasks may be expediated, however.

Quite L0000ng

[Anonymous Coward]

And I'm just waiting for the new 10/100/1000/10000 NIC's to appear

...

iSpeed ... do you?
http://www.ispeed.com

I prefer copper

[Anonymous Coward]

Hi,
anyway, why use fiber, when you can have copper and squeeze it between doors, windows and everything that closes away the server's hum from a peaceful, quiet home? As far as I know, using fiber would be a *snap* between a door.
OK, having said this, 802.11 should rule. But too expensive. snif.

ineiti

Re:I prefer copper

[Anonymous Coward]

Squeeze your twisted pair copper between doows, windows and everything and perhaps under your desk and rolling chair will cause damage to the cable, affect the twist of the cables and cause problems with crosstalk, interferance, etc. The commer cable is twisted for a reason.

... and the fiber cables I've seen seem to be pretty robust and just may survive one squash to run them initially although I would not want to be constantly 're-squashing' any cables, optical or copper in doors and windows.

Copper vs. Fiber

[jandrese]

IIRC the original Gig-E hardware (if not the original spec) was Fiber only as well. Eventually people started coming out with copper hardware to save on costs. In most cases, the only real advantages to fiber are the long cable runs and the immunity to interference in noisy EM environments (like your typical computer room). The downside is the cost.

Re:Copper vs. Fiber

[joib]

IIRC, some researchers had succeeded in making cables suitable for data transmission from some plastic. dirt cheap, and quite fast too (of course not as good as single mode fiber, but better than copper).

Re:Copper vs. Fiber

[Anonymous Coward]

If you go and read the mailing list archives from the working group, you will find that HP wanted a 10Gbit copper standard. They proposed one with a maximum cable length of 10m. Cisco didn't like the idea.

A cluster I'm building could actually benefit from it for connecting the gigabit switches in each sub-cluster together. The Cisco pre-standard 10Gbit links are something like $50-100k each so anything that cuts into the cost is a good thing from my perspective.

Oh the possibilities!

[Yoda2]

Lynx will rock!!!

Re:Oh the possibilities!

[Anonymous Coward]

Actually 500000 frames/s in Lynx is not any less stupid than 500 fps in Quake... It's faster than the human eye anyway.

Re: Everyone out there with Gigabit Ethernet - you

[lhaeh]

I feel sorry for all the people that spent $1500 each on Gigabit cards. Then spent a ton of money to get all their cat5 cable pulled and replaced by cat5e.

Can I actually use that?

[SuperCal]

Will an adverage PC be able utilize that speed? Can I send data to my harddrive at 10 G-Bit? Sorry for the ignorance, and thanks for the inlightenment which is sure to come.

Re:Can I actually use that?

[larien]

As others have pointed out, this is irrelevant for "average" PCs. Where it comes in useful is for high-end servers, network backbones and (possibly) clusters which throw a lot of data around.

Re:Can I actually use that?

[amorsen]

Will an adverage PC be able utilize that speed?

No. The answer is easy because the only reasonable way to connect high speed to a standard PC is via the standard 33MHz 32-bit PCI bus. That is 1056Mbps theoretically, but you'll be lucky to see 900Mbps in practice. Therefore an average PC can almost utilize the speed of gigabit ether. 10 gigabit is out of the question.

The solution is to go to PCI-X which is a 64-bit 133MHz bus in version 1. Even then, the theoretical bandwidth of ~8Gbps cannot saturate 10Gbps ether. PCI-X version 2.0 will provide 266MHz and 533MHz speeds, but whether that ever shows up in a standard PC is doubtful.

Mmmmm fiber

[Anonymous Coward]

Your network, brought to you by
Kellog special K. Make sure you get your fiber
every day.

Will Cable companies be using this?

[Viewsonic]

Someone ought to tell them their local tokenring network needs this stuff .. Then they can stop complaining about bandwidth hogs ... And charge us less per month...

Re:Will Cable companies be using this?

[GigsVT]

There's plenty of local bandwidth, it's their upstream you are hogging.

Re:Will Cable companies be using this?

[NoMoreNicksLeft]

Yes, my good old DOCSIS to 4/16 cable modem kinda sucks.

LOL.

Could someone please explain to the idiot above that unusual/old networking technologies do not necessarily fall into the "tokenring" category? That used to be my duty here on slashdot, but I'm getting really burnt out with it.

Cable companies almost universally use what is known as DOCSIS. Not token ring. Not even close. Token ring is not FDDI, nor ARCnet. They are all different things, using different cable types in different enviroments. They are not ethernet or 10base2/5.

wither Cat6 ?

[green pizza]

My building recently had new copper installed. Previously had Cat5 (great for 100BaseT) but was upgraded to cable meeting the specs for the latest Cat6 draft spec (rather than just Cat5e).

Is 1000BaseTX the end of the line for copper? Or will there eventually by a 10000BaseT that will run on Cat6?

Re:wither Cat6 ?

[Klox]

Using Cat5e for 1Gb was a big hack. Our hardware guys wince at the thought of such a crappy medium for high-speed applications. Once you start running longer than a couple of meters you really need to use optical. 10Gb will not run on copper.

Re:wither Cat6 ?

[Barche]

This thing runs at 10 Gbps. Ethernet uses Manchester encoding (+-=1, -+=0), which means you have to double the bps to obtain the bandwidth. So you need (about) 20GHz of bandwidth on the cable. At that frequency, losses in a copper cable are just too high. You'd need to use either wave conduits (big metal pipes, not an option) or optical fiber.

Re:wither Cat6 ?

[ivan256]

Gigabit ethernet took all it's electronic specifications from fibre channel. When Gigabit came out, there was already copper available for fibre channel, and there was nothing stopping you from using those GBICs. The recent development was that they figured out how to get the signal over regular CAT5.

I'm sure that there will be a copper spec for 10 gigabit too, it's probably just not ready yet. Consider that people will be wanting to use this on the backplane of embedded network hardware, and blade servers.

Re:wither Cat6 ?

[Anonymous Coward]

You're somewhat out of date, that's 10Mb which uses Manchester encoding, but:

100Mb uses 4B/5B encoding, i.e., each nibble is transformed into a 5 bit symbol, properly scrambled etc.., so it uses 125Mb/s.

Gb Ether on copper uses all 4 pairs on an 8 pin connectors and a 5 level encoding, making 625 possiblities. It turns out that is is actually a byte parallel bus at 125MB/s more than a serial link.

Look up the specs onthe web for the more nasty details.

Re:wither Cat6 ?

[Barche]

ah, ok... in our course they just said "ethernet uses manchester". I didn't bother to check if that was only specific to 10Mb :)

Re:wither Cat6 ?

[jefftp]

Only 10 meg EtherNet uses Manchester Encoding.

100 meg uses 4B5B/NRZI (non-return to zero, inverted) encoding.

Gig changes the rules yet again.

Just like a 9600 bps modem is still running at 1200 baud (baud is a unit of sound), but 4 bits get stuffed into one baud, so to did the encoding methods change to accomodate the speed changes in ethernet.

Re:wither Cat6 ?

[cgori]

Gigabit Ethernet (and 10GigE) is called "Ethernet" basically because it keeps the same frame size restrictions (64-1518 bytes), CRC algorithm, packet structure (DA, SA, etc), and extensions (802.3x flow control, 802.1p prioritization, 802.1q VLAN, etc).

Ethernet is really a layer 2 protocol, with many Layer 1 implementations (i.e. 10Base5, 10BaseT, 100BaseT, 100VG, 100BaseFL, 1000BaseT, etc) -- the physical signalling for almost all of those is different based on the characteristics of the signalling medium. Unfortunately the term "Ethernet" is often confused with the original Layer 1 implementation of 10 meg Ethernet.

What you can do with 10Gbps

[Jugalator]

Time to...

Download a typical 100K pr0n JPG: 0.00001 s
Umm...
Download a 650Mb ISO: 0.52 s
Hm...
Download 2 650Mb ISO's: 1.04 s
Eeh?!
Download 100 650Mb ISO's: 52 s
Wow!
Download 1000 650Mb ISO's: 8.7 min
Jeez!
Download an image of CowboyNeal: 12.31 hours
Bah... Tech still need to catch up.

Re:What you can do with 10Gbps

[Anonymous Coward]

time until laughter ensues...
still waiting

Re:What you can do with 10Gbps

[Anonymous Coward]

Your math is off - 10Gbps is not 10GBps. 1GB is 8Gb. But that stuff is still pretty damned fast.

Re:What you can do with 10Gbps

[NoMoreNicksLeft]

Download an image of CowboyNeal: 12.31 hours

Huh? Are you using some kind of submolecular scanning technology? Besides... who would want an exact duplicate of him anyway?

Re:What you can do with 10Gbps

[Anonymous Coward]

maths are right. capitalization wrong. 10 gigabits ~= 2 isos.

Re:What you can do with 10Gbps

[Anonymous Coward]

what he wanted to say was probably something like:

. O

Re:What you can do with 10Gbps

[Mr. Sketch]

Download a typical 100K pr0n JPG: 0.00001 s
Download a 650Mb ISO: 0.52 s
Download 2 650Mb ISO's: 1.04 s
Download 100 650Mb ISO's: 52 s
Download 1000 650Mb ISO's: 8.7 min

You forgot one:

Download Windows 2003: 1h 35m 12s

Re:What you can do with 10Gbps

[Anonymous Coward]

You forgot one:

Download all of Mr. Sketch's lameass Windows jokes... oh nevermind.

Oh Man! My Network was finally upgraded 100mbs.

[jellomizer]

This really stinks. It took me 3 years to avocate 100mbs. And now its 2 revs behind.

Re:Oh Man! My Network was finally upgraded 100mbs.

[frovingslosh]

For all the world is a stage and the people are mearly green.

And it is Soilent.

No more copper-bandits

[.sig]

Maybe now those damn geeks will stop tearing the copper pipes out of old buildings to reuse as network cabling. Now its time to toughen security on our fiber!

Re:No more copper-bandits

[erpbridge]

Yup.... make sure to watch out for bandits trying to make off with Metamucil fiber supplements!

Hit me with the clue-stick please!

[Andy_R]

Can someone bring me up to speed?

1) The link shows it has been approved by "Revcom" - who are Revcom, and why should I be interested in their approval?

2) Seeing as ethernet seems to speed up by an order of magnitude each time, why does the standard not allow for many more x10 jumps?

3) How far is 10Gb Ethernet from getting to the consumer/business market?

Re:Hit me with the clue-stick please!

[GigsVT]

revcom

IEEE

Consider yourself hit with clue-stick.

Re:Hit me with the clue-stick please!

[ivan256]

3) How far is 10Gb Ethernet from getting to the consumer/business market?

I know of companies that have had 10 gigabit ethernet chips working internally for over 3 years now. They were just waiting for the standard to come out. Now they'll tweak their chips to meet the standard and release them. You should be seeing these in stores Real Soon Now(TM). Expect them to cost between $1k and $3k per HBA at first though. They probably won't reach an affordable level for 5 years or so. We still haven't completed the transition to gigabit.

Air...

[EyeOfTheBeholder]

What about 100Mbps Airnet though

As if 1000BaseT didn't suck enough CPU cycles

[green pizza]

I'm guessing 10GbitE will be used for inter-switch and inter-router connections long before it gets to the desktop. Ever looked at performance comparisions between 100BT and 1000BT between just two PCs? A couple years ago the difference wasn't much... NICs weren't efficent enough and the host PC's didn't have enough CPU power to handle that many tiny packets per second. Jumboframes and faster CPUs have helped a lot since then, but we're still a long ways away from even 90% utilization between two PCs with 1000BT. And here we are with 10GigE, with 10x as many packets per second.

I'm I the only one that thinks the only efficent 10GigE NICs are going to be PCI-X cards with an onboard 2.6 GHz P4 co-processor and 512 MB of buffer?

Re:As if 1000BaseT didn't suck enough CPU cycles

[Enry]

There was an article in the Linux Journal a few months ago (February issue I think) that talked about intelligent network cards. They had an onboard XScale CPU and its own OS and TCP/IP stack.

What would happen is the OS (Linux) would get intercepted at the socket layer and pass the data to the network card. The card would then handle the process of building the packet and all the remaining layers of communication.

This allowed for a high amount of main CPU time left over for actually doing processing while the network card CPU was focused on handling the TCP/IP packet work. IIRC, you could saturate a 1Gb line with data at only 5% main CPU usage.

Re:As if 1000BaseT didn't suck enough CPU cycles

[bluGill]

My company once did this for a 25Mhz dec machine. We discovered that you now need a new protocol to the adaptor card, and the overhead of that protocol is equal to a well tuned tcp/ip stack. So if they can actually make this work, what it really means is that linux hackers should spend some time to tune the stack.

Note though that tuning the stack may come at the expense of maintainability, or flexability.

Re:As if 1000BaseT didn't suck enough CPU cycles

[EvlG]

That sounds really neat.

How long until these things are actually in the hands of consumers?

Re:As if 1000BaseT didn't suck enough CPU cycles

[JordanH]

• I'm I the only one that thinks the only efficent 10GigE NICs are going to be PCI-X cards with an onboard 2.6 GHz P4 co-processor and 512 MB of buffer?

Sure, today. I'm still glad to see that networking standards are being pushed far forward in advance of computing equipment. 10MB/sec Ethernet was hard for computers to keep up with when it first came out, but I'm glad they didn't wait for the computers to catch up before establishing the standard.

New low-end desktops today can comfortably handle 100mb/sec no problem. High end handles 1Gig/sec. In about 5 years, by Moore's law, the new high end machines point to point will be able to use up all of that 10Gig/sec Ethernet. Ethernet is supposed to support all the machines on a LAN.

The fact that we can bearly support 10Gig/sec Ethernet now seems pretty irrelevant to me.

Re:As if 1000BaseT didn't suck enough CPU cycles

[billh]

Ethernet is supposed to support all the machines on a LAN.

That is what switches are for. You don't need to share bandwidth with other computers on the LAN. If you have too much traffic between two computers, and are saturating the link to either of them, then you have a network design issue.

Re:As if 1000BaseT didn't suck enough CPU cycles

[ahfoo]

Trivial point but. . .
TSMC is currently in production at 130nm, starting 90nm process by the end of this summer and jumping to 65nm process in 2005. IBM says 40nm is the final limit for CMOS transistor gates. If it is indeed the middle of 2002, then I don't think Moore's Law is going to be holding in five years.
Of course there's always multiple processor configurations, advances in circuit designs and better nanotech (since processor designs already are properly classified as nanotech at the 0.1 micron level) and all sorts of things to push the limits one more time, but Moore's Law and CMOS are more or less at the end of the road once you're dealing with resolutions of a few dozen atoms which is what you've got at 40nm. And that information is according to the people who have the most to gain by denying it --IBM, Intel, TSMC, UMC etc.

Re:As if 1000BaseT didn't suck enough CPU cycles

[Anonymous Coward]

Damnit, that's is supposed to read less than 0.1 microns

Re:As if 1000BaseT didn't suck enough CPU cycles

[janolder]

The death of Moore's Law has been greatly exaggerated many times before. My Dad (chip designer at IBM) used to tell me "Oh, the naysayers are at it again. Moore's law won't be holding in five years, according to them."

This was around 1979.

Re:As if 1000BaseT didn't suck enough CPU cycles

[Anonymous Coward]

At 10G speed, you actually would need OC192 class Traffic Processor. CPU like Xscale is only good for the control plane not the data path.

Trust me, been there, almost done it before the big crash.

Re:As if 1000BaseT didn't suck enough CPU cycles

[p3d0]

The easiest way to guarantee you'll be wrong is to predict that computer advances will cease at some definite time.

Re:As if 1000BaseT didn't suck enough CPU cycles

[JordanH]

• That is what switches are for. You don't need to share bandwidth with other computers on the LAN.

To prevent your having to share bandwidth with no other computers on a LAN, you'd have to have switches at every point in the LAN. Ultimately, in such a configuration, you still have the potential for contention. Sure, switch all the 10Gig/sec lines so that they essentially talk to each other with no interference locally. What if those same two machines decide to talk to a third machine somewhere else on your network. You're bandwidth limited.

• If you have too much traffic between two computers, and are saturating the link to either of them, then you have a network design issue.

Sure, but I was pointing out that it won't be too long before two computers talking to each other could potentially swamp a 10Gig/sec Ethernet.

Now, I agree that it's highly unlikely that two computers will be swamping a 10Gig/sec Ethernet in a well designed network anytime soon, but the possibility of it leads one to believe that 10Gig/sec just isn't the infinite room for growth that we assume it is today.

Re:As if 1000BaseT didn't suck enough CPU cycles

[Anonymous Coward]

So what you are saying is that you are planing to buy a PC for your PC?

How big and busy is your network though?

[Chas]

Seriously. Unless you're pushing quad-digit node-counts or are sharing streaming video all over the place (or just have lots of 0-day servers), 10Gb isn't going to really provide you with any appreciable performance gain over 100Mb.

In most cases, small files are sucked down well before your bandwidth usage ramps up that far. And even larger files would probably only be sucked down a few seconds faster (mainly because of the speed of the storage medium on your system).

Re:How big and busy is your network though?

[maharg]

but on the other hand, if you are pushing quad-digit node-counts and you are sharing streaming video all over the place, then this is a Good Thing.

My employer has just implemented a gigabit backbone for a project which really could do with 10 gigabit capacity to allow room for growth. In fact, we asked the supplier to draw up a migration plan from gigabit to 10 gigabit, so we are serious about moving that way. The only thing that prevented us from implementing 10 gigabit in the first place was the lack of standards.

Of course, as many posters have pointed out, you need seriously fast storage to really use this sort of bandwidth.

Yes, we are streaming video (over 300Mbits/sec on the backbone at the moment) and yes we are pushing a quad-digit node count :o)))

Someone had to mention it...

[keep_it_simple_stupi]

Imagine a Beowolf cluster of these... Ha!

Seriously though, this might be a strong catalyst for more clustering of many different types of application servers etc. A faster link is always better... :)

Re:Someone had to mention it...

[Anonymous Coward]

thank you, captain obvious!

Makes no difference, in practice

[zilde]

Most Broadband VLSI companies have their own internal standards that closely track the 802.3ae, and so does the chip spec. For this reason, many XG chips sampling or in the fab now differ very slightly from this golden standard (may be 2-3 months at the most, and minute in terms of difference of functionality ). So, practically speaking, this is not significant. However, waiting for this Godot is finally over and that is great news. :-)

obsolete...

[jaseman21]

I would hardly call gigabit ethernet obsolete. Most home networks and many small business LAN's still use old reliable...

12 mb/s twisted pair.

It still meets all gaming and small office needs.

great for backbones and so forth but...

[st0rmshad0w]

but why would you need networking to the desktop that's so much faster than the data transfer rates if the internal components?

Re:great for backbones and so forth but...

[Anonymous Coward]

Good question. All the switches here are Gbit capable, but all of the nics are 100Mbit or less. Many use 10mbit and don't notice.

Re:great for backbones and so forth but...

[Scott BaioWulf]

You're right you don't. You'll use this to connect between switches or between your cisco 7500s and the PIXs.

Re:great for backbones and so forth but...

[SquadBoy]

And then of course there will be plenty of bandwidth for the bandguys becuase your firewall fails *open*! Real men use Checkpoint.

Now all I need

[faring]

Now all I need is an expansion slot that can handle 10Gb/s to plug that new NIC into.

Re:Now all I need

[Turbyne]

oh great, back to PCI video...

Broad-Band Hiaku

[DoubleWhiskey]

there is loose soil dig and bury the goodness next to my f$@%'n house!!!!! (well its not much of a hiaku)

Slashdot Effect

[WolfDeusEx]

10Gigabit still woun't standand up to the slashdot effect

Re:Slashdot Effect

[jdavidb]

Don't be too proud of this technical monstrosity you've constructed. The ability to transfer the Library of Congress in less than three hours is insignificant compared to the power of the slashdot effect.

Rats. It would have been funny if you hadn't gotten to it first. :)

Re:Slashdot Effect

[fruey]

Except that the SlashDot effect is not so much a bandwidth issue, as an inability to serve multiple connections at the same time all at varying speeds. That's what really screws servers, not the available bandwidth.

Re:Slashdot Effect

[Anonymous Coward]

oh for fuck's sake. It was a joke, you pedantic asshole. You know, "it's funny. laugh"

Re:Slashdot Effect

[fruey]

In being attacked frequently by pedants, I have become one myself. Shit.

However, I would have to say that jokes about the Slashdot effect, StarWars characters giving their end of the story, or Beowulf clusters are worn and boring, no? That's what brought out the pedant in me.

Re:Slashdot Effect

[Anonymous Coward]

Yeap, you need at least a Beowulf cluster of those to resist a Slashdot effect.

Unless...

[Conspiracy_Of_Doves]

...you used this stuff

Oops, never mind

[Conspiracy_Of_Doves]

That'll teach me to read the article :)

Where's the bottleneck now?

[StupidKatz]

Soon, all humans must upgrade their "fiber channel" from eyeball to brain to take advantage of this new technology.

Things to upgrade:
Vid card
CPU
Monitor
Eyeballs
Brain

Re:Where's the bottleneck now?

[rjch]

You forgot the required fibre connector that taps directly into the brain stem...

Gates claims 640k statement is myth...

[xneilj]

See this.

ah, the pleasures of a government job

[caveat]

that's the great thing about working at a national lab - in my office i have a gigabit network connection straight to the backbone (the advantages to being tech-savvy in a generally retarded department..."oh come on, the 100/1000 card is like $25 more than the 10/100...and it's not your money anyway"). wonder how long before they upgrade the network, those iso's take *forever* at 700KB/s...
yes, i know i'm not pushing my connection at all @ 700K, and i know 10-gig ethernet wouldn't make a rat's ass of a difference, but i like to gloat (/. on mozilla 1.0 takes, oh, 0.981 seconds to load and render)

Wow, the implications

[Daath]

Now it seems feasible to actually share RAM over the ethernet - That would be nice :D

Re:Wow, the implications

[Hydrogenoid]

Bandwidth isn't the main problem for sharing ram, in fact... Latency is more troublesome...
And designing an efficient cache coherency method for concurrent access, etc, etc...

Re: Enough for anybody

[cnladd]

10Gb speeds should be enough for anybody

Just like 640KB of RAM should be enough for anybody? :)

Switch prices

[stevelinton]

Switches for these speeds are still kind of large, awkward and pricy. We had a visiting lecturer from one of the major players in this level of kit talking here about 6 months ago, and their top-end product (he showed a photo) was a 48-way full bandwidth 10Gb switch, It filled two full height 19" racks, consumed 20kW and cost upwards of $2M.

Of course they've probably come down a but in the last few months...

Re:Switch prices

[Anonymous Coward]

Oh, THOSE things? I saw a coupon for one in last week's newspaper...

Re:Switch prices

[afidel]

Cisco 6500 series has had 10Gb modules for a while now. It is about 20U and doesn't cost anywhere near that kind of cash. How many 10G ports the backplane supports I am not sure, but there are 10Gbit solutions that are within meer mortals budgets.

Re:Switch prices

[Gigabit Switchman]

12-port gig fiber w/10 gig uplink
1U
http://www.enterasys.com/products/item s/1G694-13/
Should have 802.3ae compliance in a few months, from what I hear.

Pushing the limits of hardware

[Klox]

10Gb Eithernet is pushing the limits of hardware. Everyone working on this are only running the optical link at 10Gb, but then splitting the signal into 4 lanes (called XAUI) so the signal can be processed at sane speeds. Both the 10Gb Ethernet spec. and the 10Gb Fibre Channel spec. take this into account, so all the data is 128-bit alligned.
Companies won't have hardware in their labs until early next year, so don't expect that you will see and 10Gb NICs at Best Buy any time soon.

Speed v. Latency

[Hershmire]

Most clustering applications depend on the latency of the network, not the overall speed. A majority of the applications written do not require massive amounts of data to be transferred in order to distribute jobs to the nodes.

What is optimal is a low-latency network protocol that can quickly pass jobs to other nodes. If you're into clustering applications that require the movement of large amounts of data AND want low latency, then products like Myrinet are a good choice. Be prepared to shell out some cash for it, though.

Backbone?

[Anonymous Coward]

I've got a line of cat5 running across my carpet. It's taped down, even, so no one trips on it and causes me to kill them. :)

The sad thing is 100 mbps seems slow at times. :/ But I have no problems playing mp3s off my server while transfering the latest errata updates over, so hey :p

about copper lines

[Anonymous Coward]

it was my understanding that there had been some tricks discovered allowing copper to transmit over 100Gbs. I am probably way off, but my real point is not the number but the fact that many believe that there is more to be squeezed out of copper lines.

I don't care one way or the other, mind you but I wonder about the logistics and derived costs associated with initial setup, maintenance and rollover to these systems from any current non-fiber system in place.

Is it ok for researchers into copper line data transmission to stop and go on to something else? Hey! thats like world wide downsizing. Anyway, I am now curious how long it will be until Linksys produces a Home Switched 100Gbs Network Kit.

Cisco has 10GigE modules already

[Anonymous Coward]

They follow the IEEE 802.3ae spec.

http://www.cisco.com/warp/public/cc/pd/ifaa/6500gg ml/prodlit/10c65_ds.htm

"There is no limitation in the number of modules supported per chassis, resulting in up to 12 ports of 10 Gigabit Ethernet in a 13-slot chassis."

10Gb over copper? Won't happen!

[xt]

The reason is material properties.

Six months ago, I had the chance to talk with the 3Com technical manager who was on the board drafting the spec.

What he said was very simple; all tests indicated that the only way to have 10Gb over copper is to limit the connection distance to centimeters!

1Gb already pushed the envelope for copper, using all pairs, multiplexing, and error correction; 10Gb is just not possible.

Re:10Gb over copper? Won't happen!

[ivan256]

the only way to have 10Gb over copper is to limit the connection distance to centimeters!

Why is that bad? Centimeters is plenty for backplane traffic.

Also, who said it had to be over CAT5?

Re:10Gb over copper? Won't happen!

[xt]

It's not bad, but the point of this spec is to retain compatibility with and seem natural to those familiar with Ethernet, for server to desktop data distribution, not to help design a backplane.

The decision to go fiber only, was based on technical and manufacturing factors and was not taken lightly. Pretty much everyone having something to gain from 10Gb Ethernet, big customers and manufacturers alike, was involved.

One big reason for this decision is that 10Gb Ethernet can't keep compatibility with existing copper-based wiring solutions. If one is to change a paid for CAT5e copper-based wiring solution, already capable of 1Gb, due to bandwidth needs, the path is fiber.

Re:10Gb over copper? Won't happen!

[Wesley Felter]

4x InfiniBand (8Gbps) can run over copper, although the cabling is expensive. My guess is that it's only a matter of time before someone adopts this approach for 10GbE (for better or worse).

Re:10Gb over copper? Won't happen!

[ivan256]

the point of this spec is to retain compatibility with and seem natural to those familiar with Ethernet

Why? This technology won't be for links to the desktop for a decade. This is pretty much backbone only stuff right now. Ethernet has had it's share of phyiscal links. There is no reason for them not to brake compatability with existing cabling.

Re:10Gb over copper? Won't happen!

[xt]

I don't know the exact reasoning of the decision, but compatibility was a major factor. I believe they need to retain compatibility, in order to speed up its approval and the rate at which it will be adopted.

You are right about the backbone usage. At this speed, it's not strictly LAN stuff. It can also be used for MANs and WANs. Backbone providers were also part of the draft board and one of its intended usages is this.

I can't really tell you why these decisions were made, but basically, every decision was a compromise between the needs of LAN and WAN guys.

Re:10Gb over copper? Won't happen!

[sumdumgai]

If it was not compatible, would you call it Ethernet? No, you would call it OCX or Invisiband or something else.
Why should companies totally revamp thier development for an architecture that is not known or proven in the industry, (at least to the PHBs). This way they only need to develop some new encapsulation and timing routines, and yur half way home.

Re:10Gb over copper? Won't happen!

[ivan256]

They call it ethernet because of the frame type, not because of the physical layer. You don't need to do any encapsulation.

Orbital RAM

[Rupert]

You thought the latency was bad with RDRAM!

The speed of light will be a hindrance to your plan.

bus is the limiter now

[photon317]

Do the math - even on a "high end" server:

Sun SBus - 25mhz x 64bit = ~800mbps
PCI 33mhz x 32bit = ~1000mbps
PCI 66mhz x 64bit = ~4000mbps

And that of course is the raw speed for the whole bus. It's shared between multiple device - and even then you usually can't get the real theoretical maximum throughput.

Until busses at least 3x faster than 64/66 pci become common on server hardware, this will only be realistically deployable as network infrastructure (eg Inter-Switch Links between high end Cisco Catalysts). Even at 3x 64/66 pci, one 802.3ae card will saturate the bus.

Of course 10Gbit Fibre Channel is also coming down the pipe soon - hopefully between the two there will be a real drive for newer bus architectures to actually go mainstream in the server market.

Re:bus is the limiter now

[photon317]

oops, my quick mental math led me astray - SBus would be ~1600mbps, not ~800mbps. In any case, doesn't change the point :)

Fiber only - for the moment

[Ashurbanipal]

Every time they come out with a new standard for ethernet it's the same old schpiel - "you need this special expensive coax/shielded-pair/fiber-optic etherhose to make it work; you canna change the laws o' physics Cap'n!"

Then eight months later somebody figures out how to run it on old lamp cords and string.

Don't rush out to buy fiber unless you need the noise isolation (glass is great for that!) and don't care about the cost.

Re:Fiber only - for the moment

[stienman]

Don't rush out to buy fiber unless you need the noise isolation (glass is great for that!)

When I saw this bit of news I ran to my local fiber cable company, bought a 5km spool, and placed it in front of my monitor.

Now I don't see any of the noise from trolls on slashdot. Glass is great for noise isolation!

-Adam

Re:Fiber only - for the moment

[b1t r0t]

Don't rush out to buy fiber unless you need the noise isolation

...or the distance. In any case, don't go thinking you're "future-proofing" yourself by stringing fiber through the walls of your house. Because while there's only one kind of Cat 5e copper, there's at least two kinds of fiber (single mode and multi-mode) to deal with. And some corollary of Murphy's Law will predict that whichever kind of fiber that you've spent money to install without an immediate need will be the wrong kind when the need arrives.

The only kind of "pipe" which is truly future-proof is an empty one, with air inside. One inch is a nice size. And maybe a length of nylon lanyard to help pull something through.

Re:Fiber only - for the moment

[Ashurbanipal]

The only kind of "pipe" which is truly future-proof is an empty one, with air inside. One inch is a nice size. And maybe a length of nylon lanyard to help pull something through.

Got that right! 1.25" is nice too... make sure it's metal, continuous ground. And airtight is beautiful if you can get it, pneumatics are fun for all ages.

Gee, thems is short bits

[TootsMutant]

Doesn't that work out to about 1-1.5 inches per bit?

Re:Gee, thems is short bits

[Amazing Quantum Man]

Yep.

damn tags

[Anonymous Coward]

. <- earth CowboyNeal -> O

Many are missing the point here

[buss_error]

Sure, you can't use all 10G on ONE machine. Even a server can't use all that speed. Even using many NICs. (Buss congestion, ya know.) That isn't the point here. The point is that instead of having to segment a lot of traffic off to a vlan or other workaround, that traffic can be supported on one lan. This reduces equipment, interconnections, configuration, and alot of other headaches. In short, you can reduce the total points of failure.

And remember, Intel isn't the only hardware platform out there. While I don't know of a hardware platform that can make fully support the speeds needed, there are some that can support better than 4000 Kbps now.

Re:Many are missing the point here

[cybercuzco]

there are some that can support better than 4000 Kbps now.

I think everyone is supporting better than 4000 kbps now: 4000 Kbps=4Mbps

Arg.

[be-fan]

Now my PC133 RAM is *really* obsolete. It can't even handle an ethernet connection!

Re:Arg.

[Anonymous Coward]

Moderate the parrent funny... no... moderate it sad...

Re:!not obsolete

[frovingslosh]

Doesn't matter that you're using many computers, as long as you're using switches rather than hubs. (Not trying to run that 10 Gig down a coax daisy chain are you? ;-) ). Since it would make no sense to adopt 10 gig bleeding edge technology and then use hubs rather than switches, the fact remains that 1 gig ethernet is all that virtualy everyone currently can make use of, the bottle neck isn't in the ethernet. Still, nice to see the technology evolving. Maybe it will even be afordable by the time I can use it.

What use is this? I can think of a few...

[sacremon]

I work in a data center for a major ISP/backbone provider. While we've got OC48's coming in from the backbone, it's 1Gb ethernet to the LAN distribution routers. With 10Gb ethernet, we can finally fully utilize that incoming bandwidth without having to use a lot of ports.

Another good use is the emerging use of iSCSI, or SCSI over ethernet. 1Gbps ~ 100MBps, but more likely around 60-80Mbps. With 10Gbps, a SAN based on iSCSI will actually be able to use the throughput of those SCSI drive arrays.

Eventually this will trickle down to the desktop, but not right now. So it doesn't really matter what PCI can handle - this isn't presently meant for it. BTW, 133MHz PCI-X will give 10Gbps, so if you have a dedicated PCI-X bus to that adapter, you can handle it will today's technology.

Obsolete?

[mindstrm]

I know you say this jokingly.. or do you?

This is not THE new standard, it is A new standard.

It is THE standard for 10Gbps ethernet. Nothing more.

Gigabit is hardly obsolete when a) very few corporate networks are using Gigabit outside the server room, and...

Your average workstation can probably not even push 10Gbps, or anywhere near it in the first place. (Of course, that's not as big a deal, because it's ethernet, right? A single host can't max it out anyway.. the higher capacity means more hosts with lower latency.)

Re:Obsolete?

[Anonymous Coward]

Gigabit is hardly obsolete when a) very few corporate networks are using Gigabit outside the server room

MS DOS 3.1 is hardly obsolete, when very few corporate networks are using MS DOS 3.1 outside the server room.

Re:Obsolete?

[Anonymous Coward]

> Your average workstation can probably not even push 10Gbps, or anywhere near it in the first place.

"Your average workstation" is obsolete too. So?

latency?

[_|()|\|]

Where it comes in useful is for high-end servers, network backbones and (possibly) clusters

HP's TruCluster is designed to use a proprietary cluster interconnect called Memory Channel, with a bandwidth of about 100 MB/s. Gigabit Ethernet is cheaper, but can't compete when it comes to latency. Any idea how we can expect this new standard to compare?

Re:latency?

[larien]

I'd assume it would be slightly faster due to the increased bandwidth (i.e. it takes less time to transmit x bytes). However, it would depend on your requirements as to whether latency or bandwidth are your priority when you choose a transport.

The main clusters to benefit would be ones like Beowulf which tend to use "commodity" hardware (not that 10Gbps is commodity... yet!).

Re:latency?

[_|()|\|]

The main clusters to benefit would be ones like Beowulf which tend to use "commodity" hardware

TruClusters can use Ethernet as the cluster interconnect, but Memory Channel is recommended, because of its lower latency.

I was also thinking of Oracle 9i RAC. The Oracle / Dell / Red Hat bundle uses Gigabit Ethernet. I'm not familiar with the Oracle cluster file system, but I suspect that it would benefit from a low-latency interconnect.

Dr. Science

[Anonymous Coward]

Dr. Science:
How can data arrive before I send it?

Dear AC:
It's all in how you look at it. When data arrives so quickly that you don't have to wait for it, then you lose all sense of time; your senses are tricked into thinking that you received the data even before it was sent.

I'm smarter than you are because I have a master's degree -- in science !!

Can you say Distributed Computing?

[cybercuzco]

If you can connect two computers together across the office and run them as if you had the two processors in one box, this is a big leap for distributed computing. The main problem today with distributed computing is that the network is the bottleneck, so you can only run tasks that can be easily broken into small chunks. You therefore cant use software designed for the big 64 way IBM Big Iron, because all their processors are on the same bus so you dont have to split stuff up, the processors talk to each other realtime. 10G ethernet allows you to string 64 cheap boxes together and run them as if all the processors were on the same bus, so you can run all that nuclear explosion simulation or weather simulation software that youve always wanted to but couldnt find the spare $10M to buy a supercomputer. Id wager that apples Xserve will be one of the first widely available computers to run 10G (you could get 1G from them like 2 years ago) Imagine a beowulf cluster of those.... ;-)

Re:Can you say Distributed Computing?

[photon317]

It will make it better, but you 10G ethernet doesn't match the speed of an SMP interconnect. If a processor has a 133 Mhz DDR bus (like an AthlonMP), that's ~ 17Gbps. You might assume that by the time 10G ethernet is widely deployed, processors might be utilizing 200 Mhz DDR busses for ~25Gbps. It's also considerably lower latency across those little copper traces on the board compared to going through 10G ethernet.

Technology will improve for all sorts of networks and busses, but it will almost always be universally true that a tight interconnect inside a single machine will perform better than an externally cabled network between machines.

Re:!not obsolete

[mjpaci]

I don't believe that there are even 1000Base-T hubs. It's all switches.

tacions for a signal?

[frovingslosh]

Unless you're using tacions for signaling I think you'll find out pretty fast (or pretty slowly depending on how you look at it) that your processor in Korea and your RAM in orbit has a bit of a signal delay problem. Put your ram in geosync orbit (so you aways know where to find it and it doesn't wander to the other side of the earth) and you'll have a read/write cycle of several seconds. Just doesn't seem like the best use of the top end bandwidth you seek.

Re:tacions for a signal?

[Amazing Quantum Man]

Geosync has a MINIMUM .25s latency. That's just ONE round trip (45000 miles). Things get slower from there.

Unless, of course, you've invented subspace radio.

ONLY $9000?

[Brightest Light]

Less than $9000, you say? Wow. Let me grab my checkbook and head over to staples...

Re:ONLY $9000?

[GigsVT]

That's pretty cheap for 2 terabytes.

Bandwidth bottleneck on motherboard - AGP?

[Anonymous Coward]

Okay, now PCI is a bottleneck. Even 64bit PCI, quad-pumped, would still only support around 8Gb/sec... So I suggest that we repurpose the AGP port. We can go back to boring old PCI for the graphics card, so lets implement AGP network cards! Of course, it won't be the "Accelerated Graphics Port" anymore, it will be the... "Always Generous Pornography" "Accelerated Game Piracy" "Automatic Grits-to-Pants"

Re:Bandwidth bottleneck on motherboard - AGP?

[adambwells]

Regarding AGP network cards ... it's been done, by Apple's XServe. In its standard configuration, one of the gigabit interfaces is an AGP card in the AGP slot.

Re:Bandwidth bottleneck on motherboard - AGP?

[Anonymous Coward]

no, that's a pci/agp slot. the ethernet card uses the pci function of the slot. the slot supports both standards

Your Sig

[Anonymous Coward]

Just a suggestion: It might be funnier if you instead of "* not guaranteed" put "* not an actual guarantee".

Anyone knows the MTU?

[Oestergaard]

Does anyone know how big packets one can send thru such a pipe ?

100MBit maintained the same MTU as 10MBit, 1GBit maintained the same MTU too - leading to severe problems with performance. It's bad enough on 100Mbit, it's horrible on 1Gbit, to think that they maintained the 1500 byte limit on 10Gbit gives me the shakes...

Yes, I know about "jumbo frames", and I challenge you to find an affordable 1Gbit switch that actually supports it.

Anything below 64KByte packets would be insane as I see it.

Anyone knows ?

Re:Anyone knows the MTU?

[Anonymous Coward]

Gigabit ethernet can use 9500 (??) bytes packets...

Re:Anyone knows the MTU?

[trybywrench]

well on congested links a smaller packet size actually works better, thats why ATM cells are only 53 bytes in length. Also, when 2 or more interfaces are on the same collision domain the 1500 byte MTU is required for collision detection assuming you want the max. link length to remain the same.

I would guess that if they maintained the same MTU it was to "cover all the bases". You can do some tuning for a specific LAN like messing with window sizes and MTU's to get optimal performance for your particular case.

Re:Anyone knows the MTU?

[scosol]

"collision domain"?

As if *anyone* using 10Gbps NICs would be using them in a shared-medium "hub" environment?

Please... :)

Re:Anyone knows the MTU?

[timster]

You may know this, but even in a fully switched environment, collisions and collision domains are still relevant, as it's always possible for two interfaces to attempt to transmit *to the same interface* at the same time.

Re:Anyone knows the MTU?

[Ziviyr]

You may know this, but even in a fully switched environment, collisions and collision domains are still relevant, as it's always possible for two interfaces to attempt to transmit *to the same interface* at the same time.

So its silly to assume a single switch would know better than to try to send two packets at the same time to the same wire?

I guess I've underestimated teh intelligence of these things a good bit...

Hairy smoking golfballs

[alienmole]

(come to think of it, this implies that physical size is a fundamental limiting factor on the speed of computers - it does no good to have an infinitely fast CPU if its parts can't communicate rapidly due to speed-of-light delays...)

Researchers have realized this for decades. Before enormous silicon chip densities became ordinary, engineers at IBM (IIRC) used to say that the future of computers was "hairy smoking golfballs". This captured a number of important characteristics of very fast computers:

• Hairy because of all the wiring that would be involved (like the rubber strings in the core of old golfballs)
• Smoking because of all the heat that would be generated
• Golfball-sized because of latency issues, as you said
• Golfball-shaped because a sphere is the most efficient use of space for components that need to be interconnected.

Since those days, Intel and its competitors have fulfilled all of these predictions except for the spherical shape, which is much more difficult and not as important as the other characteristics.

A Pentium 4 is hairy - those 55 million transistors have a lot of connections; and smoking, as anyone whose CPU fan has broken can attest. It's smaller than a golfball in cross-sectional area. That size isn't just to make them more convenient! If a physically bigger CPU would be faster, you can bet someone would be building them.

He would, wouldn't he? [OT]

[alienmole]

It's harder for him to retract his claim about prime numbers:

"The obvious mathematical breakthrough would be development of an easy way to factor large prime numbers."
-- Bill Gates, "The Road Ahead", Viking Penguin (1995)

at least not 300 baud ...

[Lewis Mettler, Esq.]

Just remember those days with the 300 baud acoustic units.

What's so bad about copper?

[Rorschach1]

I can strip copper wire with my teeth, and terminate it with a Leatherman tool. Until I can do that with fiber, my network's sticking to good old-fashioned electrons.

Re:What's so bad about copper?

[Ziviyr]

I can strip copper wire with my teeth, and terminate it with a Leatherman tool. Until I can do that with fiber, my network's sticking to good old-fashioned electrons.

How dare there be tempest unfriendly networks?!?!

Learn the SI

[Anonymous Coward]

http://www.bipm.fr/enus/3_SI/si-prefixes.html

m -mili = 10 ^(-3) = 0.001
M - mega = 10 ^(6) = 1 000 000

that is 1 000 000 000 times difference.

Sun SBus - 25mhz x 64bit = ~800mbps
0.025 * 64 = 1.6 bps

PCI 33mhz x 32bit = ~1000mbps
0.033 * 32 = 1.056 bps

PCI 66mhz x 64bit = ~4000mbps
0.066 * 64 = 4.224 bps

Re:Learn the SI

[photon317]

Learn to be practical.

When was the last time you hear anyone use the term "milihertz"? 1 millihertz would be roughly around 1 cycle every 17 minutes. We're discussing computer networks and system busses. Get with the program, and stop being an anal nitpicker.

...having spent time in these tunnels

[aaarrrgggh]

I'd have to say that burying a new pipe is often more attractive for a university. The reason that most campuses have the tunnels is for steam heat.

At my old school, the tunnels were typically 3 feet wide by five feet tall on the branches, with 12" dia steam supply and return pipes. Also sharing the tunnel is 12kV power, sanitary lines (in some locations), domestic water, and copper telephone trunks.

The problem is that the steam pipes have a life of 30-50 years, and if you get a leak you will be hard-pressed to get someone out of the tunnel! Ignoring that, the tunnels are damn hot, so working in them is a challenge. Today, with OSHA, that same tunnel would need to be 6' wide and 7' tall, with exits every 200' or so... not as attractive as in the old days.

They do run fiber conveyance in them, but that was more a function needs changing faster than you can dig up the roads.

bandwidth is sometimes overrated

[mrm677]

It is well known that ethernet has lousy latency. My favorite analogy is if you calculate the bandwidth of filling a freight train full of 80gb hard drives and steam it cross country. Great bandwidth, lousy latency.

This is why building a cluster using ethernet is not a great idea if the communication to computation ratio is very high. Some clusters I've seen use Myrinet, which offers high bandwidth and low latency at the cost of wire distance (and price).

1Gb obsolete?

[Gyorg_Lavode]

Man, now I'm really going to have to find what's pulling my network down to 10Mbps or I'll ever be able to face my friends.

nice

[rocket97]

can you install linux on it?

Re: End of Moore's Law

[DustMagnet]

For over a decade I've been hearing that Moore's Law will only last for five to seven more years. 40 nm may well be the end for CMOS, but I'm sure someone will think of something. If I knew which technology will be used, I could be filthy rich in five years.

The same is true with hard drives and ethernet over copper. Both have exceeded "theoretical maximums". I had to laugh when the story said the standard was for fiber only. I've heard that one before too.

No Copper for 10G Ether

[jefftp]

The only way 10G Ethernet could support copper would be to use two strands of coax, or twinax. This was batted around by the IEEE committee for a great deal of time, but even with twinax the distances for 10G on copper were short (75 meters) and the cost of twinax is higher than fiber.

So they dropped copper support completely.

Don't expect to see 10G to the desktop anytime soon. 1G to the desktop is still a waste of money for 99% of the desktops out there.

10G Ethernet will see the most benefit for switch/router to switch/router connections in WANs and MANs.

Fiber

[phriedom]

Naw, the sheathed single-mode fiber looks like heavy fishing line, so you have any overhead lines at all you could just spiral the fibre down those lines and no one would see them. They might notice you in the cherry picker putting the line up though. ...and then there is the matter of a 12km link at both ends and deserializing the bits and...

Re:Fiber

[morph3ous]

Yeah, I wish I could just run my own fiber, but it's not that easy. :(

Ethernet as a WAN protocol

[trybywrench]

10 gig ethernet really only makes sense in long haul, heavily utilized backbones. Which is funny because Ethernet isn't all that great of a WAN protocol ( in case Mr. Metcalf is reading, yes i know it was originally intended only for LANs ). Its great success is mainly due to a sharply falling price curve. ATM as a protocol is head and shoulders above TCP/UDP+IPV4+Ethernet with respect to the WAN but it costs more. Until IPv6 with its enforced QOS and flow labels is widely adopted I don't see Ethernet doing alot in the WAN.

Re:Ethernet as a WAN protocol

[Anonymous Coward]

Yes, you are right. But I would go ahead and
say that Ethernet will never be a great WAN
protocol even with the introduction of IPv6.

IPv6 can be placed on fiber easier than IPv4, thus
IP over Light will kill ethernet as a possible packet switched wan technology.

High bandwidth, high latency

[shrikel]

Just pump those photons through this thing and you'll end up with the interesting combination of extremely high bandwidth with extremely low speed.

fiber not for homes or small/mid corps yet

[sahmed]

Until there are cheap fiber cables, and equipment to make arbitrary length ones. It's just much easier to network with copper and 99.99% of users simply don't need the speed.

Besides these days most home users use their lan just to get to the router and on to the outside world. That's true of a good number of corp users too. Since a connection is only as fast as the slowest link a 10Gps line is pretty much useless to most people. Large coporations may have some use for it as their internal backbone, but 10Gb/s to the desktop is quite far away from being implemented en masse.

Which 3Ware controller?

[StRex]

Were you using the 3Ware 7810 or the 7850 controller? I'm just curious because the 7850 supposedly has higher RAID5 throughput though it seems according to their specs the higher throughput primarily occurs with small random reads/writes.

I'm assuming the "RAID 0 over 5" description is because you're running each controller as a RAID 5 array, then using software RAID to stripe these as a RAID 0 array? I'm currently spec'ing out a system that would use this arrangement, so it's nice to hear someone else [seems to be] doing this.

Re:Which 3Ware controller?

[GigsVT]

The RAID is arranged as you assumed. We are using 7810 cards. The 7850 is primarily faster in RAID5 write performance, which is pretty low on the 7810. The numbers I quoted were RAID5 read performance.

We are using Tyan TigerMP motherboards. Motherboard selection is critical, there are a lot of little quirks between chipset, driver, linux kernel, with certain combinations. Make sure to read up on all bug reports and on google that mention your proposed hardware conbination.

Keep cable lengths as short as possible, 18-24 inches with 80 wire cables are best. If you have the budget, get 3wares official hot-swap trays, they are nice, but expensive.

Why not use the sewer system?

[hokanomono]

Most cities should have a big sewer system. Here we have it and maybe you know it from "The third man". Recently they use it for the fibre that they need for quantum teleportation. So why have i never heard about the idea of a city wide fibre network inside a sewer network?

Copper rocks for very short distances

[billstewart]

Those annoying copper-wire people keep taking our "we'll finally make you buy fiber now, this time for sure!" standards and implementing them on copper for much less money :-) I can get a GigE-on-copper NIC card at Fry's for $59 bucks, though the cheapest switch I saw that supported it was about $300. It's almost tempting to do gigabit copper at home just because I can :-)

Obviously if you're actually going anywhere, you want fiber, and the ability to go 10-20km over fiber (for GigE and also for storage-network technologies like FibreChannel) is starting to change the economics and scaling decisions about where you put servers and storage, how much clustering you do, how you handle disaster recovery, etc., but for inside wiring, copper is still convenient.

Large what

[Anonymous Coward]

Did anyone else read "large penises," or do I just have cock on the brain?

Re:bus is the limiter now -use AGP

[Anonymous Coward]

Hey, why not use the AGP 8X bus for this?

..and a PCI Gfx card...

Nonsense! I'm sitting here looking at one.

[Anonymous Coward]

10GBE copper works right now for short distances.

There's already a spec and there was working hardware shown at the last NewWorld/Interop.

I've got a test chip on hooked up to my Infinium 86100A right now.

EH?

[Anonymous Coward]

> We discovered that you now need a new protocol to the adaptor card, and the overhead of that protocol is equal to a well tuned tcp/ip stack...

You must have had really, really, bad design types to end up with a CPU-to-Adaptor interface that was anywhere near, even remotely, as heavy as TCP.

I can't wait

[attackiko]

I can't wait to plug my DSL into this thing.. wohoo!

Avoiding network upgrades by overkill

[billstewart]

A decade or so ago we did a consulting gig out at a different national lab, which then had a classified-projects design network and a regular network, and they'd decided they wanted to do the fastest hairiest network they could for the classified net, because the security decisions involved in changing or upgrading things were so painful and difficult. They decided that a gigabite/second per user was about enough - because that's enough for two 32-bit-color 30fps 1024x1024 displays, so the bottleneck is no longer the user's computer, it's the user's eyeballs. In reality, they could use a bit more today, because we're close to 2Kx2K displays, but back then that was about as greedy as they could get, and there wasn't really anything that quite did that:-)

Ironic

[tlambert]

I think it's ironic that this is being herladed as "the death of copper".

It's my understanding that what held it up was the optical parts vendors couldn't make transceivers that went fast enough.

You've actually been able to buy 10Gbit *copper* hardware from HP for almost a year now, and the copper part of the draft standard has remained unchanged since then.

I rather expect that the next speed increment will have similar problems with the relative ease of making copper vs. optical transceivers, so the next bump will end up being "copper first" again.

Re: 2400bps over copper? Won't happen!

[mks113]

I recall in university that we were told that a phone line was limited by physics to 1200 baud. They had a couple horrendously expensive Hayes 1200 baud modems in the lab to play with.

My 1.5 Mb/s DSL line runs over those identical copper lines.

Yes, physics limits what we can do over copper, but perhaps our understanding of those limits isn't quite as finely tuned as it will be in a few years.

That said, I have no problem with fibre for new installations, but even that will be upgraded over the years. This is cutting edge. It will become commonplace before too long.

just think........

[Anonymous Coward]

think of how fast you download a copy of Episode 2 in with that connection

bending copper

[spacey]

Aren't you aware that copper also has limits on the radius you can bend it, based on the thickness of the bundle? You can damage copper by bending it, too.

-Peter

Re:bending copper

[Macka]

But for Gigabit copper you use Cat5 cables, and you'd have to be deliberatly aggressive over a period of time to damage one through bending.

Re:bending copper

[SkyLeach]

HAHAHAHAHAhahahaheheheheeeee... *whew*

Don't see many telcos burying cat 5 no matter what they are laying... know why? Because they need at least 100x that much bandwidth on the pipe itself. You are thinking of hte cat five to *each pc*. Sure, if that could carry all the internet bandwidth then it would be fine, but it's not.

It's way to expensive to put a switch and booster in every 100m, so they have to bury fiber.

'nuf said.

Re:bending copper

[Anonymous Coward]

Hi, like you I protect and whore my Karma at every moment.

So like I was saying, when I saw this,

"I'm sick of motherfucking moderation crap going to shit on slashdot. This was supposed to be funny, and it was most DEFINITELY ON TOPIC.

The topic: the game produced by the EFF.

What am I talking about? The character in the game? If that's not on topic, nothing is.

You don't like it: fine, don't read it. But don't fucking bury it with "Offtopic" when it's not.

Slashdot is turning into a whorehouse for pussy brats with nothing better to do than screw up a good news site.

Fuck it, I got my 50 karma a long time ago. I don't need this shit so I'm just going to read from now on and let the comments go to hell where they belong.

My $0.02 will always be worth more than your 0.02, so :P " I knew I found a brother in arms.

I want you to know you are brave to voice yourself like this and jeopardize your Karma. I will always look up to you and try to mod you up! Maybe I can email you and we can go hang out and get a burger or something. I will post this anonymously to protect my Karma.






BTW, suck me off, please. -RWS

Re: End of Moore's Law

[ahfoo]

Okay you skeptics. I hope you're right. I live in Taiwan and the economy will be totally trashed if I'm right and everything will be okay if you guys are right. So, let's hope I'm just being misled by IBM, EETimes and all these gossip rags sporting this "we're already doing nanotech and it's getting old fast" nonsense. However, let us not forget the boy who cried wolf. Just because people have called it wrong in the past doesn't mean it disappears. I didn't see anybody disputing that CPUs are already at nanoscale.

Now vs. Then

[morph3ous]

For now, I doubt that this is going to affect us too much as there is no way that most ordinary hardware can keep up with such tremendous speeds. I do though think that this is a huge development for the future. It seems that bandwidth just keeps getting cheaper. This I beleive and correct me if I'm wrong is due to fiber technology. Tons of information can be crammed on one fiber and then this can be run huge distances. Fiber to the home and in the home seems like it will be a ways off. Unless of course you run an ISP or hosting company at your home.