Ethernet at 10 Gbps

An anonymous reader writes "This article talks about 10 Gigabit Ethernet and asks, 'But just how much data can a person consume?' Currently at work, we're working on a major project to re-architect our core application platform so that the different systems can be de-coupled and hosted separately. The legacy design implicitly relies on systems being in the same LAN due to bandwidth-expensive operations (e.g., database replication). Having this much bandwidth would change the way we design. What would you do with this much bandwidth?"

Comments

Hmm...

[ravenspear]

What would you do with this much bandwidth?"

Check out more unusual positions.

What would I do?

[Anonvmous Coward]

The company I used to work for was sending very high resolution images from multiple cameras uncompressed from one unit to another to perform analytical operations on them. I think they manged to work at a gigabit, but 10 would be much nicer for them.

What would Joe Sixpack do with it? I'm not sure at the moment. Thing is, since we're working within our limitations today it's hard to concieve of whta use it'd be. However, what happens when it becomes commonplace? It does open doors. Imagine if cable companies traded in coax for ethernet. They could easily send uncompressed HDTV. That'd be pretty slick.

Re:What would I do?

[nine-times]

Thing is, since we're working within our limitations today it's hard to concieve of whta use it'd be.

Isn't that always the way? I remember having a 20Mhz IBM PS/2 and wondering "How am I going to use all this power?" And the 30 MB hard drive- how would I ever use all that space?

It seems like when we have the capabilities, we find something to do with the extra. HDTV sounds probable, and more bandwidth can only help working over networks on a mass scale (remote home folders and roaming profiles, VNC/Citrix), but you never know. When processors were getting to the 1Ghz point, a bunch of industry analysts were predicting "Now that we have enough power to make working speech-recognition software, we can finally ditch those keyboards!" Yeah, right.

The big concern is, with the extra bandwidth, will Microsoft see this as an opportunity to release new, extra-inefficient network protocols?

Re:What would I do?

[egomaniac]

In your world, 'hdtv' might be 1280x1024, but in the world at least a few of us live in, HDTV (notice the caps) isn't, it's 960x540(assuming square pixels) and I imagine less than 24 bit color (All this analog stuff!). I don't know, but I would also guess that it's the same frame rate as regular old TV (~30 fps).

There are two HDTV resolutions in current use, known as 720p and 1080i. 720p is 1280x720 60fps, and 1080i is 1920x1080 30fps (60 interlaced fields). Both of them are 24-bit truecolor.

I have no idea where you got 960x540 from, as it does not correspond to any HDTV resolution. I'm also not sure what the reference to "all this analog crap" is supposed to mean, as HDTV broadcasts are entirely digital.

Remote Virtual Immersion

[weston]

But just how much data can a person consume?

If I was going under the knife remotely, I'd want the surgeon to have as much bandwidth as possible (and very, very, very low latency).

Re:Remote Virtual Immersion

[MP3Chuck]

Imagine a malpractice lawsuit!

Exhibit A: Surgery Log
[DR]Surgeon opened Xx[Patient]xX's abdomen with a scalpel.
[DR]Surgeon punctures Xx[Patient]xX's stomach with forceps.
Xx[Patient]xX: OMGWTF??!!
[DR]Assistant: ROFL PWNED!!1
[DR]Surgeon: STFU N00B i ping 350
Xx[Patient]xX: w/e

What would I do?-Horror-vision.

[Anonymous Coward]

"What would Joe Sixpack do with it? I'm not sure at the moment. Thing is, since we're working within our limitations today it's hard to concieve of whta use it'd be."

The Goatse.cx experience in holographic, 5.1 surround-sound, smello-tactile-vision.

The Network Is The Computer (tm)

[gusnz]

OK, so for stuff like streaming MP3s and so forth, this is a little overkill for the current style of usage. However, where I think this will come in useful is for stuff like remote disk and memory access over IP.

With a 10G LAN, you'd be able to come up with a great distributed computer system (e.g. for compiling software). IIRC protocols are in the works now for native-ish memory access over networks, turning a network into one huge computer, and you can already access remote disks with the right software. Imagine the simultaneous distributed encoding of several HDTV streams to redundant archives on several different computers, and you'll probably find that more bandwidth = better.

So yeah, there'll definitely be possibilities for this sort of stuff, even if it is only as a base requirement for the post-Longhorn Windows version :).

NC-PC-NC

[basking2]

So, we used to have little dumb terminals that talked to the big smart backend. Then computer became cheaper and we had Personal Computers, but we have to manage and distribute all these updates and it's a real pain and it sometimes destroys your computer during the upgrade/install process. Now we can swing the pendulum back towards the Network Computer a little more.

This isn't a new idea. Software companies like MS would love to sell you a subscription to MS Office which you renew and they in turn patch and maintain the software on your company's server or on the MS servers. It's a neat idea for sure. Companies like Novel have made some interesting claims about Network Computers.

There is also the whole Plan9 type of mentality too.

Re:10 gigabit is kinda much

[cynical kane]

PCI is 33Mhz, not 33 MB/sec. 33 X 32-bit-bus = 133 MB/sec. PCI-X goes up to 133 Mhz and 64-bit, so that's 800 MB/sec.

Presidential Bioinformatics

[buckhead_buddy]

But just how much data can a person consume?'

100 Megabytes per chromosome
x 23 chromosomes per gamete
x 20 million gametes per ejaculation

Therefore Ms. Lewinsky can consume roughly 46,000,000,000 megabytes
(assuming that there is no overflow to a dress)

How much can you consume?

Re:Presidential Bioinformatics

[Anonymous Coward]

Close, but not entirely accurate.

A single gamete has 1.5 billion individual base pairs. Of course, that's base-4, since DNA doesn't work off of binary. ACGT is what you're made of. :) In other words, you have 3 billion bits per DNA strand. The average male ejaculation contains around 150 million sperm. This means that there is a total of 450,000,000,000,000,000 bits of information, which turns into 56.25 petabytes of information. That number is close to yours, but your information is wrong.

The fact that I just corrected you is pretty sad as well.

Way overkill

[JRHelgeson]

As a CCIE, I have been designing networks for years. I have analyzed traffic to/from desktops and watched traffic to the average desktop never even get above 27mbps. This is due to the average file size of the transfer which is rarely above 10 megabytes. At 10 megs, it only takes a few seconds to get it transfered and it only has a few seconds to get up to speed, by the time it gets all revved up, the file transfer is complete.

High-end workstations such as CAD with gigabit connections, working with 500 mb files, or multi-gigabit video files will occasionally reach 500 to 600 mbps, and even then only for a couple of seconds. At these speeds, power users can use that network connection as if it were a local drive, because at those speeds you are matching the speed at which you're reading/writing data to your local hard drive.

The only time I've ever seen near gigabit traffic at a steady pace was at network servers, where traffic can reach a steady 600mbps on a single gig link - which is maxing out the speed at which the server drive can read/write data to its hard drive. Think of it this way, a 1 gigaBIT link can transfer a 1 gigaBYTE file in about 10 seconds, that's FAST! Conversely, it takes nearly 20-30 seconds just to write that large a file to the hard drive.

Now, on a Cisco 6500 core switch, or a Cisco GSR 12000 where traffic is aggregated, these are the only places where I've actually seen multi-gigabit traffic rates, and that was across the switch fabric - not all directed to a single interface.

The 12000 GSR already has a 10gb interface, it is a single line card that takes up a full slot. It sells for about $60,000 and is used to move data from the switch fabric of one GSR to another GSR, which means you need to put in 2 of them at a mere $120,000 to get the two connected.

Moving to optical links, you can get up to 36Gbps using Dense Wavelength Division Multiplexing on multimode fiber. This uses several colors of laser light to transmit multiple 'channels' across a single fiber link.

Even at these tremendous speeds, they are only used at traffic aggregation points, again because any network device, even a turbocharged SAN couldn't handle reading/writing at those speeds for anything longer than a quick burst.

I say this: If you think that 10gig/sec is your answer, you're looking at the wrong problem. You can get the performance you need at gigabit rates.

I'm not saying that we'll never need 10gigabit to the desktop, just not until we solve the hard drive bottleneck. Solid state storage could solve the problem, but we'd need to have solid state drives that store 100gb of data in order to match the throughput of the network.

Re:Way overkill

[dbarclay10]

Most of your argument rests on people not being able to read/write data from hard drives fast enough to use the network bandwidth. Some examples:

The only time I've ever seen near gigabit traffic at a steady pace was at network servers, where traffic can reach a steady 600mbps on a single gig link - which is maxing out the speed at which the server drive can read/write data to its hard drive. Think of it this way, a 1 gigaBIT link can transfer a 1 gigaBYTE file in about 10 seconds, that's FAST! Conversely, it takes nearly 20-30 seconds just to write that large a file to the hard drive.

More:

Even at these tremendous speeds, they are only used at traffic aggregation points, again because any network device, even a turbocharged SAN couldn't handle reading/writing at those speeds for anything longer than a quick burst.

And lastly, your conclusion:

I say this: If you think that 10gig/sec is your answer, you're looking at the wrong problem. You can get the performance you need at gigabit rates.

Given your premise, you argue for your conclusion quite well. I don't, however, think your premise is accurate. Or perhaps better, I don't think it's relevant. First and foremost, there's all sorts of storage mechanisms which can transfer data as fast or faster than 10Gbps. Think solid-state drives and some decent-sized drive arrays (they don't need to be *that* large, we're talking roughly 1 gigabyte per second; that can be done with 5-10 consumer-grade drives, let alone the arrays of hundreds of high-end 15kRPM SCSI drives and the like). So on the basis of storage speed alone, your argument fails.

Second, what does storage speed have anything to do with it? You mention servers not needing this - a *huge* number of servers never touch their drives to read the data they're serving. Drive access == death in most Internet services, and people invest thousands of dollars in huge RAM pools to cache all the data (they used to invest tens of thousands, but now RAM is cheap :). So for a huge number of servers, drive speed is simply irrelevant; it's all served from RAM and generated by the CPU, so unless you're trying to say that CPUs can't deal with 10Gbps (which you aren't, and quite rightly), the conclusion falls down again.

Do desktops need this? No, of course not. If that's what you're really trying to say, then all fine and dandy, just say it. Acceptable reasons would be "people don't need to be able to transfer their 640MB files in less than 10 seconds" and "their Internet connections aren't even at 10Mbps yet, they certainly don't need 10Gbps!" However, you'll find that this technology quickly percolates downwards, so at some point in the future people will be able to transfer their 4GB (not 640MB at this point) files in a few seconds, and their "little" 640MB files will transfer near-instantaneously.

oooh

[iamdrscience]

IDE over IP. Yes, it does exist.

Re:What would I do with this much bandwidth?-Music

[Lord Prox]

Ya know, so far everyone seems to think of this as a long distance pipe. It's not, it ethernet. RTFA useful distance is in meters *NOT* kilometers. This is an intraoffice connection not a WAN pipe.

Re:Argh! No more!

[optikSmoke]

I hereby banish this to the Beowulf cluster of memes, along with Soviet Russia/Hot Grits/Profit!

Umm, ya. Well done. The, um, banishing of things into..... popularity. That'll be effective. We all know how unused each of those oft-repeating jokes are. Oh, wait......

damn.

All your base are belong to porn?

Re:What would I do with this much bandwidth?

[leapis]

This much bandwidth isn't going to help you do any of these things. I upgraded my network to gigabit ethernet about a year ago (from 100 mbit), and much to my surprise, the speed increase was only about 3 times when copying files from one machine to another. I did a little math, and found the answer. Your average ATA hard drive, even at max bus speed, only delivers 0.8 Gbps. And in the real world, you are lucky to get half that from a single drive. In my own test transfers from RAID1 and RAID5 arrays, my transfer rates never once exceeded 0.70 Gbps. Until there is a fundamental increase in the amount of data you can get off a spinning disc, its not likely that a home user is going to saturate a 1 Gbps line, much less a 10 Gbps line.

Dumb terminals? Cluster computing?

[Entropius]

Let's see. There are about a million pixels on my screen (1280 x 800). Assume 24 bit color, so that's 24 megabits per frame.

This at 60 fps will be 1.44 Gbps.

So 10-Gbps ethernet is enough to stream the output of a monitor, *uncompressed*, at full framerate, to either a dumb terminal or another computer. Even the most elementary compression (only reporting changed pixels, or PNG/jpeg techniques) could cut this to a fraction of 1.44Gbps.

More generally, it could allow more of the things that are currently on the PCI/USB bus to become external, and could become a more flexible replacement for USB. Scanners, cd writers, audio devices, you name it ... lots of things could be externalized and generalized. This would also allow more devices to be shared across networks more easily, since they're *on* the network in any case. With the Internet, nobody cares about the physical location of the machines they access; likewise, with this system peripherals aren't associated as strongly with one specific computer.

This sort of thing might also have applications for cluster computing, allowing more sorts of things to be done with clusters since you have higher inter-node bandwidth.

Re:It's not the bandwidth

[kasperd]

I have to count picoseconds for the kind of stuff I do

Unless you are working with individual gates inside a chip, I doubt picoseconds really matters. On ethernet we are certainly not talking picoseconds. We are still limited by the speed of light, so it would take the signal 100 picoseconds just to get through the RJ45 connector. With a 1.5m ethernet cable there will be at least 10 nanoseconds of roundtrip time, because that is the time it takes light to travel 3m.