The Road To Terabit Ethernet

stinkymountain writes "Pre-standard 40 Gigabit and 100 Gigabit Ethernet products — server network interface cards, switch uplinks and switches — are expected to hit the market later this year. Standards-compliant products are expected to ship in the second half of next year, not long after the expected June 2010 ratification of the 802.3ba standard. Despite the global economic slowdown, global revenue for 10G fixed Ethernet switches doubled in 2008, according to Infonetics. There is pent-up demand for 40 Gigabit and 100 Gigabit Ethernet, says John D'Ambrosia, chair of the 802.3ba task force in the IEEE and a senior research scientist at Force10 Networks. 'There are a number of people already who are using link aggregation to try and create pipes of that capacity,' he says. 'It's not the cleanest way to do things...(but) people already need that capacity.' D'Ambrosia says even though 40/100G Ethernet products haven't arrived yet, he's already thinking ahead to terabit Ethernet standards and products by 2015. 'We are going to see a call for a higher speed much sooner than we saw the call for this generation' of 10/40/100G Ethernet, he says."

Comments

Re:More data forces the need for more bandwidth

[conureman]

I reckon I'd best roll out the gigabit switches around here. I hate when stuff gets obsolescent before I deploy it.

10 mbit/s ought to be enough for anybody

[Shakrai]

;)

Re:10 mbit/s ought to be enough for anybody

[El_Muerte_TDS]

Don't you mean 5 Mebibit per second?

That gives you 640KB per second.

Re:10 mbit/s ought to be enough for anybody

[Garrett Fox]

A joke, yes, but what applications will this next generation of bandwidth be for? Is there (or will there be) really a big demand for high-definition, full-screen video streamed through the Net? Or is this bandwidth advance more about increasing the number of devices attached to the Net, so that we can have pervasive surveillance, er, sensor networks?

Re:10 mbit/s ought to be enough for anybody

[stevelinton]

Initially this is to connect disks to database engines and to push entire virtual machines onto servers to handle demand spikes and things like that. Later to handle the upstream end of pushing multiple HD video streams out from servers towards large numbers of clients.

Re:10 mbit/s ought to be enough for anybody

[takev]

3D high definition voxel video.
That will probably eat a bit of bandwidth.

By that time the minimum resolution will be 2048 x 2048 x 2048 voxels 8 bytes per voxel for RGBA at 128 frames per second. Which adds up to 8,796,093,022,208 or 8TB/sec. maybe we can get 1:1000 compression ratio, so that will end up with 8GB/sec.

Re:10 mbit/s ought to be enough for anybody

[Urza9814]

you get 10mbit/s??? Lucky!

Re:10 mbit/s ought to be enough for anybody

[Locklin]

Sweet, more pixels than retinal ganglion cells. Where can I get an eye upgrade?

Re:10 mbit/s ought to be enough for anybody

[BrokenHalo]

If you were going really retro you would have said 3 mbit/s :)

No, 300 baud should be enough.

Re:10 mbit/s ought to be enough for anybody

[Chabo]

Maybe for you, but my vision is significantly better than average. I won't be happy until we have twice that in each dimension.

Physics?

[happy_place]

Does anyone know what are the physical limitations of highspeed ethernet? I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

Re:Physics?

[Shakrai]

I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

Nah, at that point you just place the whole ethernet infrastructure within a subspace field, modulate the deflector dish a little bit and you'll be off and running.

Re:Physics?

[Penguinoflight]

That's more a problem with copper wiring. Cat5e seems to have problems reaching above 700-800 Mbits/s, I assume cat6 does better but wouldn't expect to see 10Gbit or even close.

At this point we haven't really started to see limitations on how fast a fiber optic connection can be switched, although I wouldn't doubt there being a theoretical limit.

Re:Physics?

[MichaelSmith]

Does anyone know what are the physical limitations of highspeed ethernet? I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

Very roughly, at one terahertz you can transmit one terabit. Now whats the frequency of an XRay laser expressed in hertz?

Re:Physics?

[ckhorne]

It depends on the total bandwidth (measured in Hz) and thus the medium itself. The frequency allowed over the line, coupled with the signal encoding and speed (IIRC, roughly speed of light, although slower over copper) should give you the theoretical maximum. I don't have the formulas handy to figure this out.

However, there is a limit to how *fast* electrons can be moved- the speed of light. How many electrons (and their encoding) determine total bandwidth.

Re:Physics?

[Junior J. Junior III]

Does anyone know what are the physical limitations of highspeed ethernet? I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

The speed of light limitation will limit ping times over a set distance. Upgrading to terabit speed doesn't make the end nodes further apart, it widens the pipe between them. So, no, I don't see a theoretical limit to how wide the pipe can be. At some point, you'd need a really thick cable, I suppose, which could become impractical.

There's other bottlenecks, too, such as the speed of the systems' internal busses, or storage devices, though.

Re:Physics?

[Shakrai]

Umm, I've reached 950 Mbit/s on Cat5e before. Fairly long (70-80 yards) runs of it too. Are you sure you aren't running into a limitation of your hardware? I've seen a lot of PCs with crummy NICs or slow PCI buses that can't reach full gigabit speeds no matter how good the cabling is.

Re:Physics?

[empiricistrob]

That's a bit hard to say. But here's a way of thinking about it:

The Shannon-Hartley theorem states that the channel capacity (e.g. the data bandwidth, measured in bits per second) is related to the channel bandwidth (measured in hertz). If we assume a very pessimistic signal to noise ratio of 1:1, the SH theorem says that the cable's bandwidth in hertz will be the same as the cable's bandwidth in bps.

So if we want a cable capable of transmitting information at 1tbps, the cable will need a bandwidth of roughly 1000 GHz. That means that it would be impossible to carry that amount of information using even microwaves. We're talking about at minimum infrared light. Or in other words -- we're talking about fiber optics, not cat5.

Re:Physics?

[setagllib]

We can always keep adding more bandwidth - in the extreme case (as in TFS) by trunking together more of the same links. But latency is not really improving. Ethernet itself is very high-latency compared to e.g. Infiniband. But fundamental limits of latency are impossible to overcome, and the best you can do is get closer and closer, perhaps asymptotically so. Between our planet and another, any latency in hardware is going to be a rounding error compared to the latency in the electromagnetic waves themselves, which propagate at "only" the speed of light.

Re:Physics?

[Penguinoflight]

Most of the time even a regular PCI bus will be able to saturate gigabit ethernet. At the moment I just have two systems that are gigabit capable, both on PCIe and both have onboard marvell yukon chips. Which chips are you using?

Re:Physics?

[Bandman]

I think the grand parent was talking about using copper wire as a conduit.

I agree, optical will go much, much, much faster. And later, when we have vacuum-optic cables...well, all that much faster, I suppose.

It's those pesky optical logic gates that are holding us back.

Re:Physics?

[jgardia]

This is really pessimist. I remember in University time, even phone cables reached over 80 db of signal/noise ratio. In a telephone cable, the available bandwidth is about 2-3 kHz, and you can move over 30 kb/s. I would expect next technologies to start using phone-like modulation, instead of +5/-5 V for a 1, and -5/+5 V for a 0.

Re:Physics?

[Penguinoflight]

That all depends. While networking has traditionally been a serial connection, there's nothing stopping multi-mode connections and in fact multi-mode already has some implementations.

Spread your 1Tbit connection across 10 lines and you only need 100Gbit's per line.

Re:Physics?

[Vellmont]

We're talking about at minimum infrared light. Or in other words -- we're talking about fiber optics, not cat5.

Except for the fact that:

1. Twisted pair ethernet uses electrical signal modulation, not photon modulation.
2. There are 4 pairs of wire in twisted pair ethernet cable.
3. The S/N ratio is a LOT higher than 1/1

Re:Physics?

[rossdee]

You left out the 'tachyon pulse' part. Thats the way to get your data moving faster than light.

Re:Physics?

[Anonymous Coward]

You have a fundamentally flawed understanding of how waveforms propagate over a medium.

Electrons in a physical signal such as one that is transmitted via ethernet do NOT move at the speed of light or anywhere near it. If that happened, there would be near infinite current; which would fry anything and everything it touched.

The signal propagates as a wave. The electrons only shift slightly because we are talking about very small current here. I won't go into the details of electron behavior with respect to its random behavior and current.

The waveform moves much faster than the electrons themselves and its speed is based upon the conductivity of its medium (when talking about signals over a copper wire anyway - in optical, electrons do not move at all)

Re:Physics?

[BlueParrot]

Does anyone know what are the physical limitations of highspeed ethernet? I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

Depends what you mean with speed. The lowest possibel latency is limited by teh speed of light.

Bandwidth is limited by the number of "tubes" you can run and how much data you can push down each tube. In principle there's nothing stopping you from doing something crazy like encoding your data on DNA strands that you dissolve in a soup and push down an actual tube with cross section 10m, moving thousands of liters of DNA soup per second. The theoretical maximum bandwidth of such a setup would be staggering, but the latency would be horrid.

So basically latency is limited by the speed of light while the theoretical limit to bandwidth is so massive that other practical and economic concerns will limit it rather than fundamental physics.

Re:Physics?

[TheRaven64]

Electrons move very slowly (at least, compared to the rate the data travels) along the wire. The effect is roughly analogous to a Newton's Cradle, where the balls move slowly but the shock wave when they are together travels at the speed of sound so the data (one bit - the ball has impacted on one end) travels at the speed of sound. Using light or electrons, the speed of the signal is the speed of light. This give 67ms as the theoretical minimum signal time (134ms ping time, since pings are round trips) anywhere in the world, given by dividing the circumference of the earth by the speed of light.

In practical applications, the latency is greater for two reasons. The most obvious is that we are not laying cables in a straight line. If I ping the machine I have on the other side of the park from here, the data goes via London, a few hundred kilometres out of the way. If you use satellite relays, then the signal is bouncing up and down between the surface and the satellite's orbit at least once, adding to the distance.

The second reason is switching time. The signal travelling along the wire is very quick, but even on a single-segment network that data has to be processed by two network cards, encoded going out and decoded coming in, transferred to and from userspace process's address spaces and so on. Things like infiniband lower this latency by allowing userspace code to write directly to the card, which removes some but not all of the overhead. If you are using fibre then the transformation between an electrical signal and a sequence of photons, and then back again, adds still more latency. In a switched or routed network (like, for example, the Internet), this has to be done several times because (outside of labs) we can't route packets without turning them back into electronic signals. Most routers will queue a few packets while making decisions and at the very least they typically read the entire packet off the line before routing it, which, again, adds a bit of latency.

In terms of throughput, there is no theoretical limit. If you can send one bit per photon, you can double the throughput by doubling the number of photons (i.e. just use two fibres). The limit is set by cost, rather than by physics. There are a few physical limits which affect this. Shannon's limit gives an upper bound on the number of symbols per second you can send across any given link, given an amount of signal bandwidth and a signal-to-noise ratio. This is quite misleading, however, because the number of symbols does not directly correlate to a number of bits. Early modems used two tones and got speed increases by switching between the two faster. Later ones used a number of different tones and so transmitted the same number of symbols per second but more bits. The same is done with fibre, for example using polarised photons or photons of different wavelengths to provide different virtual channels within a single fibre. These can be detected separately and distinguished from each other. If, for example, you send photons of four different wavelengths, you can send two bits per photon instead of one. If you use 16 different wavelengths, you can send four bits per photon.

When it comes to radio transmission, there are some even more interesting effects. If you've tried receiving analogue TV between hills, you will have seen a ghosting effect because your signal comes via two different paths. It turns out that, with two different transmitters, you can distinguish between them even if they are transmitting on the same frequency, by measuring the different paths each takes. This is particularly interesting for things like WiFi, because in urban environments (where you have the most people trying to use the same radio bandwidth at once), you get more possible return paths (due to more objects that bounce the signals), and so (given enough processing power), you can discern more individual transmitters, giving more usable bandwidth. There are lots of tricks like this - probably a great many that no one has thought of yet - that can provide greater throughput in exchange for more signal processing power.

Re:Physics?

[Shakrai]

Most of our network cards use intel chipsets. What OS are you using? I've never been able to fully saturate gigabit or even 100mbit ethernet under Windows. Even when using simple protocols like FTP it usually tops out at 80-90% of network capacity. Using more complicated ones (SMB/windows file sharing comes to mind) will reduce it even further. Transfers between two Linux machines are another matter. I've been able to saturate 100mbit networks easily using a variety of protocols and achieve the aforementioned 950 Mbit/s transfer.

I just fished through all of my cacti graphs and found a sustained (25 minutes) period of 980 Mbit/s data transfer between two of my switches. The link between them is regular Cat5e with a run of about 60 yards. So I'm not sure that you can attribute your issues to the physical layer, although anything is possible.

Re:Physics?

[Xiph1980]

Electrons don't move through a conductor with the speed of light. They collide too much, reducing their speed to mere millimeters per second. The signal however travels with approximately 0.75c. This can be explained by an analog. If you have a row of marbles in a gutter of the same width of the marbles' diameter, you can push against the first marble, and simultaneously the last marble would move. The "signal" of you pushing the marble gets propagated through the umpty marbles inbetween and reaches the last marble far before your hand has reached its location.

In theory it is possible to create a system that transmits informations faster than the speed of light. Taking a perfect weightless incompressible solid marbles, and place them in a 1 lightyear long gutter made of a similar perfect inelastic material. place a compressive spring at the receiving end, and push the marbles from the transmitting end in a pattern. The very moment you push the marble in a bit, and let it relax again, the far end, 1 lightyear away, will see that exact same movement, thus transferring information faster than the speed of light.
It is however obviously impossible to make those perfect materials, thus we're bound to sub-c communications.

Re:Physics?

[cats-paw]

Realistically the SNR of cable is very high. The bigger problem is reflections and impedance mismatches due to the high frequency of the data. As others have pointed out you can (as Gb does) use multiple pairs of wires.

Even at 10 bits/s/Hz on 4 pairs of wire, you are talking 25GHz of bandwidth. At those frequencies RF effects, i.e. mismatches are very important and will require very high quality connections and very high quality cable.

Coax should be used at 25GHz, and small diameter coax, not twisted pair.

There is also the problem of the electronics required to convert a 1 Tb/s data stream into some sort of parallel data stream for the compute blocks. Non-trivial.

I say toss the wire and work on more economical ways to connect fiber, so that I can run fiber at home.

Re:Physics?

[JesseMcDonald]

In theory it is possible to create a system that transmits informations faster than the speed of light.... It is however obviously impossible to make those perfect materials, thus we're bound to sub-c communications.

I don't think it's accurate to says that FTL communication is possible, even in theory, if the materials the theory would require are themselves impossible.

Re:Physics?

[gblackwo]

The electrons actually move quite slowly regardless to how fast you send stuff down a cord.

Re:Physics?

[evanbd]

They've been doing that for some time. GigE uses a 5-level trellis code, and even 100Mb used 3 signaling levels.

Re:Physics?

[Xiph1980]

In most situations when you're starting to be educated about mechanics, you start with perfect springs and perfect elastic bands. Similarly in physics, you start with perfect materials. This makes the calculations a lot easier, and in that way gives the student a better understanding of the subject. Ofcourse I could explain that even for a material with a density of 1g/m (atmospheric air has a density of 1.2kg/m) the weight of a string of 1 lightyear long and 1 millimeter diameter would add up to be about 8 million kg, and thus the inerta to push that rod forward and back is nigh impossible to overcome. Also, I could explain to him that the soundwave-propagation through a marble of dense material results in the reaction with the next marble, and that even though the speed of the soundwave through a solid matter is rather large compared to the speed of sound through air under atmospheric pressure, it's still a far cry from lightspeed, thus that it takes a long long time for the marble at the receiving end to react on the marble on the transmitting end. This all however doesn't make it easy for anyone not in this field of expertise to understand the analog.

In theory you can have the perfect liquid. In theory you can have the perfect solid. In theory you can have the perfect vacuum. In practice you can't any of those. But that doesn't mean you can -- in theory -- continue working with theoretical perfect materials. In theory doesn't mean in fact.

Re:Physics?

[Xiph1980]

Missing a few superscript 3's there for volumetric values. Apparently the actual superscript 3 symbol doesn't get recognised.

Re:Physics?

[JambisJubilee]

If, for example, you send photons of four different wavelengths, you can send two bits per photon instead of one. If you use 16 different wavelengths, you can send four bits per photon.

How about this: What if you used a monochromatic light source in a single mode, polarization-maintaining fiber and had your "bits" be the polarization phase? If you divided the phase (0-2*Pi) into 256 chunks, you could send a byte per photon! With more sensitive equipment, you could get even more. Anyone heard of this?

Re:Physics?

[Cassini2]

The Shannon-Hartley theorem is not the relevant limit. The hard limit for copper is the cutt-off frequency, and for optical systems other technical challenges come into play.

Any given copper wire has an associated cutoff frequency. Passed this frequency, it is almost impossible to get significant amounts of energy to pass through the cable. The cutoff is very steep.

For most types of coaxial cable, the cutoff frequency is on the order of 1 GHz to 8 GHz. Since the bandwidth required for a working communications link is generally higher than the bandwidth of the cable, copper wiring will top out at something on the order of a few GHz for most practical applications. UTP cable, as used in existing Ethernet, will perform worse than coaxial cable. For practical purposes, we have probably used all of its available bandwidth for 1 Gb Ethernet. UTP has a cutoff frequency on the order of 300 to 500 MHz, if memory serves. As such, the 1 Gb Ethernet specification resorts to uses all four pairs to achieve the 1Gb rated speed.

To increase bandwidth further, either microwave or optical waveguides can be used. Microwave waveguides are not practical for personal computer use. This leaves optical fiber, which is an optical waveguide.

Optical fiber has an essentially unlimited bandwidth, on the order of 500 Tb/s. Its performance is primarily limited by cost reasons and technical reasons relating to receivers and transmitters. It is difficult to generate the variable frequency light sources required to make use of the vast amounts of light spectrum. Separation of the light sources at the receiver is also a major issue. There are optical dispersion problems relating to the cable, but these are easier to deal with than the problems of creating a precision wide-band variable frequency laser.

In general, the technologies at optical speeds are not as well developed as the electrical technologies for microwave, broadcast, and copper communications transmission. It is much more difficult to use all available bandwidth at optical speeds, than at copper speeds. However, since the theoretical bandwidth at optical speeds is huge, much higher communication speeds are possible with optical.

Re:Physics?

[Nukenbar]

I also remember when everyone thought that the highest download speed you could get over a telephone wire was the 56kbps modems of yesteryear because that was the limit of copper phone wire.

I was very confused when DSL modems came out.

Re:Physics?

[hankwang]

So if we want a cable capable of transmitting information at 1tbps, the cable will need a bandwidth of roughly 1000 GHz. That means that it would be impossible to carry that amount of information using even microwaves. We're talking about at minimum infrared light.

As others already pointed out, with a better S/N ratio you can lower the frequency bandwidth, although impedance mismatches become more of a problem. If you try to send a 250 GHz signal through the cable, the wavelength is about 1 mm, which means that an impedance fluctuation over roughly 0.5 mm is enough to create a significant reflection. That could happen if the distance between the two wires in the twisted pair varies a bit, if the cable is bent, etcetera.

Another problem is that a lot of materials tend to absorb radiation at that kind of frequencies, which means that the insulation around the wire will be heated by the signal that travels through the copper wire. It might be hard to get useful signals after more than a few meters of cable.

Re:Physics?

[Tubal-Cain]

Taking a perfect weightless incompressible solid marbles, and place them in a 1 lightyear long gutter made of a similar perfect inelastic material.

Why go through so much trouble? Make a very long rod. Push and pull it from one end, and you will transmit information.

Re:Physics?

[FooBarWidget]

How about using 2 wires to send 2 bits simultaneously? Doesn't that solve the problem?

Re:Physics?

[Xiph1980]

Was still in the electron particle analog mode, as in many individual sphere-thingies pushing against eachother :)

Re:Physics?

[Trepidity]

I believe transmitting information faster than light is explicitly ruled out by the special theory of relativity, and leads to all sorts of paradoxes if it's posited to be possible. That's not just a practical limitation on materials science, but a fundamental physical law, so I suspect there must be some other explanation for why this marble thought experiment doesn't work.

My guess is that the intuition of the incompressible-marbles-transmit-signals-instantly example discounts relativistic effects, so only really works at much-lower-than-c speeds. I can't offhand explain what exactly the relativistic effect that makes it fail would be, but one guess is the one explained here, that you're setting up a scenario with infinite phase velocity, but where information can still only be transmitted at a lower group velocity.

Re:Physics?

[dirvine]

except for entangled neutrino's !

Re:Physics?

[Vadim Makarov]

Propagation of a mechanical wave (i.e., sound) in materials is ultimately field interaction between atoms. Field changes do not propagate faster than c. Actually, the speed of sound is far, far slower than c, because atoms electrically screen each other and the next atom only sees the electric field change after the previous one has moved. The inertia of the atom moving in the field slows down the process quite a bit.

This is my naive (and quite possibly incorrect) understanding of the mechanical wave propagation process. Hope somebody can explain it better ;).

Re:Physics?

[digitalsushi]

when i worked for the 10 gigabit ethernet consortium at the UNH IOL (www.iol.unh.edu), i had to do a 10 gig demo once. not naming vendors, i wasn't able to get it above 1.1 gigabit/second with commodity pc hardware. we had pattern generators, but those don't count in the real world.

for 1 gigabit, the best line utilization i ever got was about 97%, using two linux boxes, netcat, and piping /dev/random into /dev/null across it. i'm not a math guy so i can't say what the theoretical max is.

Re:Physics?

[perryizgr8]

yeah. electrons just drift and that speed is usually in the order of a few cm/s.

Re:Physics?

[TheRaven64]

Yes, there are techniques which do this. Sending polarized light down a fibre is actually very difficult. Remember that the light bounces around over the length and may bounce a different number of times for two subsequent photons, and each bounce slightly affects the polarity. Frequency is easier to work with for two reasons. The first is that it is not affected by travelling through the fibre. The second is that it's easy to separate out the individual channels. You can send a photon beam into a prism and you will get out a spread of beams at different frequencies which can all be detected by individual receptors.

Re:Physics?

[ubercam]

Have you tried disabling the QoS bandwidth reserve thing? IIRC, Windows reserves a portion of the network throughput for QoS stuff (something like 10-20%). I forget how to do it exactly... I think you have to go into the Group Policy Editor or something. Although I do know for a fact that if you use nLite to make an XP install you can remove it entirely.

Re:Physics?

[Trepidity]

I don't think that works either: the consensus of physicists at least presently is that entanglement alone cannot transmit classical information, stated as the "no-communication theorem". Quantum entanglement can be used together with a classical communication channel to transit information via quantum teleportation, in which information that doesn't flow over the classical communication channel nonetheless gets spookily transmitted. But the requirement of a classical communication channel does still mean no information can be transmitted faster than the speed of light.

Re:Physics?

[JesseMcDonald]

Yes, but when using perfect materials to teach basic concepts it's important to limit their use to situations where the differences between theory and practice are insignificant. Your attempt to describe a means of FTL communication applies perfect materials to a situation where the differences fundamentally influence the result.

Re:Physics?

[Alef]

I'm not a physicist, but if you're talking about single photons I'm fairly certain that is impossible. To determine the polarization of the photon, you'll essentially have to "test" one direction, and by doing that you destroy its state.

Re:Physics?

[fwr]

Electricity does not flow through cables at the speed of light. It is more like 2/3rd the speed of light...

Re:Physics?

[Penguinoflight]

The issue with 56k connections was never about copper limitations; in fact some of the older houses actually were using copper-clad steel wiring. The FCC introduced a law to limit the connection speed shortly after the introduction of 56k modems, this resulted in most being limited to 51-53k instead.

The real issue that kept higher speed modems from becoming commonplace was a lack of incentive to ISPs. Providing line quality better than needed for voice conversations didn't have any pay-back for telephone companies.

Re:Physics?

[Shakrai]

I also remember when everyone thought that the highest download speed you could get over a telephone wire was the 56kbps modems of yesteryear because that was the limit of copper phone wire.

Nobody ever thought that was the limit of copper phone wire. It was known as early as the 60s that the physical layer was capable of transmitting higher frequencies than those used for plain old telephone service. The limitation came because of the fact that modems operated over POTS, which was limited to 64k of bandwidth per voice channel.

Re:Physics?

[Chabo]

So maybe if we were able to cool everything down to absolute zero, the atoms would be close enough together that pushing on one atom would exact instant force upon the next one!

So maybe if we manage to reach absolute zero, we'll also be able to transmit information at faster-than-light speeds!

Re:Physics?

[Junior J. Junior III]

Electricity does not flow through cables at the speed of light. It is more like 2/3rd the speed of light...

Right, but when you're talking about data, you could be talking about fiber optics, which do use light. The speed of light in a fiber optic is less than the speed of light in a vacuum, which is basically the theoretical speed limit for non-exotic physical processes. Unless we start sending data packets comprised of tachyons or something.

Re:Physics?

[MichaelSmith]

It's those pesky optical logic gates that are holding us back.

Oddly enough I know a physicist who is working on exactly that. He uses diamond as a substrate, doped with atoms which function as logic gates.

Re:Physics?

[petermgreen]

note: in this post I will use the term bandwidth in the classical communication theory sense and use the term bitrate for what computer geeks often reffer to as bandwidth.

If we assume a very pessimistic signal to noise ratio of 1:1, the SH theorem says that the cable's bandwidth in hertz will be the same as the cable's bandwidth in bps.
If we assume a more realistic SNR but also assume real modern modulation schemes (which can't reach that theoritical limit) then the channels maximum bitrate is likely to be arround 5-10* times the bandwidth since you can have more than one bit per symbol.

Also a cable does not have to be just one channel, your typical cat5 cable for example has 4 pairs each of which can be used as a seperate channel.

It was through the techniques of multiple bits per sybol and aggregating multiple pairs that we got first gigabit over cat5 (though 5e is reccomended for long reliable links) and then 10 gigabit through cat 6 (though IIRC it's been too much of a power hog for wide deployment, one way to increase SNR is to throw power at the problem).

However I think it is fair to say that we have mostly used up the potential for expansion in the direction of more bits per symbol (there is a big diminishing returns issue, each extra bit you add to a symbol DOUBLES the transmit power you need to maintain adequate signal to noise ratio,). More pairs is a possibility but it's going to get unweildly pretty quick (can you imagine hooking up your 1 terabit link with a 400 pair cat6a cable? ).

Fiber has it's own issues ofc, it has insane bandwidth but actually using that bandwith is limited by the need to interface it to electronics. Most of the current 100 gigabit proposals seem to be based on bonding up multiples of 10 gigabit stuff (though be doing it at a lower level I suspect they can get a better efficiancy than ordinary link aggregation) and then either running them through multiple fibers or using WDM to put them onto a single fiber.

*low number is a figure common with real multi-level encoding schemes, high figure is a guestimate (and probablly a pretty optimistic one)

Re:Physics?

[Linuxmonger]

I agree, optical will go much, much, much faster. And later....

Why? C for an electron in a copper cable is about 98% of C for a photon in a glass cable. Both show losses in similar manners, both degrade with distance. I've run GBEth on Cat5e copper in a production environment to a bit over 450 feet when a forklift caught the fiber and destroyed it. I was getting 875 MB/Sec on copper and 890 on glass when we got it replaced the next week. What is the basis for this statement? Citation needed!

Re:Physics?

[MichaelSmith]

Going back 15 or so years that wasn't the case. Copper was really slow then. Somebody up the page is talking about evacuated light guides instead of fibre. That sounds like a good way to cut into latency, and we are doing more with shorter wavelengths these days so maybe bandwidth over fibre improve.

Re:Physics?

[PAStheLoD]

/dev/random can be very slow. there are/were even DoS attacks based on this.

Just sayin' :)

Re:More data forces the need for more bandwidth

[petermgreen]

I don't see gigabit being superseeded for connections to end systems anytime soon. 10GBASE-CX requires expensive cable and has annoying run-length limitations. IIRC 10GBASE-T is a power hog. Fiber is both expensive and a PITA for such applications (I very much doubt fiber patch cords would last very long in a typical desktop environment)

It might be an idea to select gigabit switches with the capability to handle 10 gigabit uplinks though.

Gee, that's great.

[Doodoo Browntrout]

I'd settle for gigabit speeds from the gigabit hardware I have now.

Re:Gee, that's great.

[ap7]

Sure. But then, I'd also love to see USB 2 do its maximum 480 mbps, something not easily achievable even as USB 3 prepares to make inroads into the device market.

So why not simply treat the 1 Gigabit connection like a 500 Mbps one. 10 Gigabit is coming along, probably provides real world speeds higher by an order of magnitude as well. I'd welcome the speed increase anyway. Heck, Broadband providers do the same thing, and so do mobile phone operators ;-).

Re:Gee, that's great.

[Eil]

Then you're not involved in HPC applications that demand an extremely fast physical layer (e.g., clustering).

Re:Gee, that's great.

[LoRdTAW]

The problem today is we have low end motherboards with on board gigabit Ethernet. Gigabit chip sets are sometimes the limitation (eg garbage Realtek stuff) as well as the OS in use. Running a Linux server with a 3 disk SATA software raid5 (mdadm) I can get speeds of upward of 50MBps when copying from the server to my Windows Vista system. At that point the single vita disk, a 250GB 7200RPM SATA disk becomes the bottleneck. The Linux Server has a two port PCIX Intel gigabit adapter with only one port hooked to a Netgear GS108T switch. The Vista box has an on board realtek gigabit PHY and the AMD/ATI 790G chip set gigabit controller. With a fourth SATA disk in the server I can top out at 40-45 MBPS when copying from the raid 5 to the single 400 gig disk (both formatted XFS). One of these days I would like to see if I can setup another Raid array or solid state disk and really benchmark the maximum network speed.

If you think about it gigabit is more than enough for most of today's desktop needs. I would rather see faster Ethernet used for internet/datacenter backbones. In a real large network environment I can definitely see 10Gb and 100Gb useful for server links and backbone stuff. 10 gig ethernet used for simple workstations is a bit extreme at this point. Many applications used today can comfortably run on 100BT networks. The only application where 10GBT might be helpful is high end workstations doing heavy AV/Rendering/Pro Photo work.

Re:Gee, that's great.

[F34nor]

What kind of cabling is it going to need?

Re:Gee, that's great.

[LoRdTAW]

There are already10Gbt copper cards from Mellanox. They can tolerate cable runs of 55 meters using cat5e and cat 6 or the full 100 meters using cat6A. That might be a problem for current cabling infrastructure but backbones most likely use fiber it will be no problem. Work stations and desktops wont need 10gbt any time soon so the existing 1gbt over cat 5 is fine.

And as for 40/100 and beyond Ethernet who cares what cable media it will use. Most likely will be fiber and most likely wont really be necessary for the desktop/workstation any time soon. Gigabit should last us another 10 years for desktop and laptop use. 10 gig would be good for use on workstation and server boards.

Re:Gee, that's great.

[petermgreen]

There was the same issue in the early days of 100 megabit. While normal PCs with cheap controllers (e.g. the (in)famous RTL8139) could link at 100 megabit they couldn't really push data that fast.

With an adequate OS, NICs, internal interconnects (that means PCIe), protocol implemenations, storage, cable, switches etc it is possible to get very close to the theoretical maximum with gigabit ethernet. Cheap shit OTOH.........

Re:Ethernet or Token Ring

[drinkypoo]

When Token Ring died it was because 100Mbps ethernet was cheaper than 16Mbps token ring. I was there. Token couldn't keep up; case closed.

Re:Bring it on!

[value_added]

Personally my home LAN is outgrowing GigE ...

As would the home LAN of any self-respecting geek. ;-)

On the other hand, let's not lose sight of the fact that outside our basements, things will stay pretty much the same.

ifconfig vr0 | grep media
                media: Ethernet autoselect (10baseT/UTP)

As a side note, here in California we can only dream of having basements.

"We think we'll need even more speed... later on"

[jimbudncl]

News at 11.

Re:Ethernet or Token Ring

[Tragedy4u]

The Ethernet slow downs you describe are only in proportion to users when it's a half duplex hub and users are sharing the same broadcast domain and collisions abound. In modern networks ethernet switching reduces the amount of needless crosstalk with full duplex on each port, the only time you see a broadcast is if it's intentional or a device needs to discover where a MAC address is. So I'm really not sure I understand your arguement, it seems irrelevant to me.

Re:More data forces the need for more bandwidth

[Bandman]

10 years, maybe 15, and we'll all be on fiber.

Except for those places that are still using BNC connectors on their coax, of course. They'll never change.

Re:Ethernet or Token Ring

[Anonymous Coward]

Many companies should have sticked to Token Ring, so there wouldn't be this slowdown during backups, updates etc. In the end Ethernet is just slow because of the amount of users on the network, they yell for a bigger integer before "bit" instead of changing technology.

Who modded this informative?

You're a moron or in love with token ring. Token ring doesn't magically create bandwidth out of thin air. Even with token ring, the network has a finite, fixed maximum speed and a finite, fixed maximum bandwith.

If you're moving terabytes on the network for big backups, there is less idle capacity on that segment of the network for other traffic, regardless of the network technology (fiber, ethernet, token ring, ATM, MPLS etc.).

Token ring does prevent COLLISIONS, but so do full-duplex ethernet switches. It may be easier to implement QOS and traffic shaping on token ring, but that is a completely different story.

Re:Ethernet or Token Ring

[petermgreen]

While we have retained the ethernet name and frame format CSMA/CD has pretty much gone the way of the dodo (it's supported but virtually never used at gigabit and not supported at all at 10 gigabit)

Token ring gives each device on the ring roughly equal time, I'd imagine switched ethernet with a decent switch would have similar behaviour. I beleive some of them can also prioritise data.

they yell for a bigger integer before "bit" instead of changing technology.
because throwing bandwidth at the problem typically solves it without needing to redesign the network or make value judgements about whose data is more important.

Re:More data forces the need for more bandwidth

[bcmm]

Being able to push more content, move more data, combined with data files being that much larger, is the real driving force behind this push.

+1 Insightful; until now I never truly understood why people wanted more bandwidth.

Re:Bring it on!

[Bandman]

OK, I'll bite. What are you doing on your home network that you're being hindered by GbE?

Re:Ethernet or Token Ring

[jgardia]

I assume you never cabled computers using token ring, nor added a new desk because the boss wanted someone else in the office. I think token ring concept is excellent, but implementing it in the real work is a PITA.

Re:More data forces the need for more bandwidth

[scotch]

Being able to push more content, move more data, combined with data files being that much larger, is the real driving force behind this push.

Interesting, please subscribe me to your newsletter.

Re:More data forces the need for more bandwidth

[petermgreen]

10 years, maybe 15, and we'll all be on fiber.
On what do you base this claim?

How do you propose to get arround the problem that fiber patch cords are easilly damaged?

What applications do you think will require this kind of bandwidth? HD video with moderate compression should easilly fit into a gigabit.

Bah

[BigBlueOx]

All *we* had was an acoustic coupler. And an Ohio Scientfic. S-100 bus. 8k of memory IF you were looky. AND we read the bits as they came over the phone AND typed them in ourselves.

And you tell that to the kids today and they won't believe you. Bah. Spit.

Re:Bah

[jbeaupre]

Ah, our trusty J-Cat. It may have survived a lightning strike, but you could still type faster than it could transmit.

Re:Bah

[Linuxmonger]

My first computer was a Commodore 'Teachers PET' with a home made serial interface hooked to a DEC-Writer green-bar printer/keyboard. Later we hooked it up to a 300 pound cast iron teletype with roll paper and a separate paper tape punch on the kitchen counter (Mom Loved it!). I still have a few print outs of the Mona Lisa and Last Supper done as ascii art - the latter is 24" high by 72" wide and took about two hours to print. My modem was 300 baud and my e-mail address was all numbers at compuserve when they still only allowed users to connect after 6:00pm.

Why?

[quenda]

Why?

Re:Why?

[Jeppe Salvesen]

Cloud computing.

Re:Why?

[julesh]

Cloud computing

Doesn't really answer the question. To elaborate: most current computer systems are incapable of maxing out Gigabit ethernet. For any nontrivial application you're going to be loading data from disk, and unless you have very fast disks you're not going to hit 1Gbps.

Now I can see 10 or even 100Gbps being sensible for high performance computing applications. But terabit? Isn't that a little OTT?

Re:Why?

[ionix5891]

Porn

Re:Why?

[MukiMuki]

Terabit, at the moment, is entirely for enterprise, where the amount of hard drives connected to the bigger machines lies in the triple to quadruple digits. However, 10G is at least useful for higher-end consumers.

8 hard drives in a RAID6 array managing full speed (approx. 20mbytes/sec. per drive?) hits 120 megabytes/sec., already reaching gigabit's limits. Add two more arrays and 10G becomes useful. While I personally can't see much of a use for that beyond, say, murdering load times in modern games (and maybe the scratch disk needed while editing a poster-sized image at 600dpi with several layers), I'm sure plenty of people can find plenty of good uses for such a setup, which is becoming cheaper daily.

It isn't about using up a terabit, it's about getting bottlenecked at the level below it and needing the breathing room.

Re:Why?

[petermgreen]

Doesn't really answer the question. To elaborate: most current computer systems are incapable of maxing out Gigabit ethernet. For any nontrivial application you're going to be loading data from disk, and unless you have very fast disks you're not going to hit 1Gbps.
The reason is backbone links. Consider a tree of 1000 very active machines and suddenly exceeding 100 gigabit seems quite feasible at the pinch points of that network

You may ask why use ethernet for such things, well the answer is if your whole network is ethernet it is easier to just transparently replace a link with a faster one than to start trying to mix technologies

Terabit is probablly OTT right now but given the speed at which standards bodies move it is nessacery to start planning it sooner rather than later ;)

The other thing that TFA doesn't mention is that current 100 gigabit proposals rely on multiple "channels" (either completely seperate links or WDM wavelengths). I imagine that in paralell with the development of terabit ethernet there will be a push for a single channel 100 gigabit standard.

Infiniband: a better technology in the datacenter

[Anonymous Coward]

Infiniband is faster than Ethernet today and can scale better with lower latency. It is also significantly cheaper and can be used for SANs and LANs to eliminate bulky cables.

The desktop will only need Gigabit, arguably 100Mb/s for next 5 or 10 years. It is the servers and network trunks that need the speed not the desktops.

Re:Bring it on!

[Abcd1234]

Personally my home LAN is outgrowing GigE

Uhh... seriously, *WTF*. What do you have, a dozen teenage boys streaming HD porn 24 hours a day from a central server or something?

Re:More data forces the need for more bandwidth

[nbvb]

Corning's bendable fiber. There ya go.

http://www.xchangemag.com/hotnews/77h23134942.html

Please make IEEE-1588 a standard part of 1TbE

[wowbagger]

An open letter to any hardware vendor considering making chips for these higher speed protocols:

Please add the timestamp counters needed to support IEEE-1588 Precise Timing Protocol. These counters don't add much in the way of complexity when added to the NIC, but they are VERY complex to add after the fact.

Being able to synchronize the clocks of 2 hosts to 5nS or less may seem esoteric right now, but for these sorts of transfer speeds, you are going to have a significant number of users (Test and Measurement folks like me, scientists at places like CERN and FermiLab, grid computing) who will need that kind of time sync.

Re:Please make IEEE-1588 a standard part of 1TbE

[TechyImmigrant]

An open letter to any hardware vendor considering making chips for these higher speed protocols:

Please add the timestamp counters needed to support IEEE-1588 Precise Timing Protocol. These counters don't add much in the way of complexity when added to the NIC, but they are VERY complex to add after the fact.

Being able to synchronize the clocks of 2 hosts to 5nS or less may seem esoteric right now, but for these sorts of transfer speeds, you are going to have a significant number of users (Test and Measurement folks like me, scientists at places like CERN and FermiLab, grid computing) who will need that kind of time sync.

http://www.ieee802.org/1/pages/802.1as.html

There you go.

Re:Please make IEEE-1588 a standard part of 1TbE

[Anonymous Coward]

He didn't say, "Please make a specification for this", he said, "Please take the time to add support for this to your hardware." It doesn't matter if the specification exists or not if nobody takes the time to add it to their implementation.

Re:More data forces the need for more bandwidth

[silentsteel]

If that "typical desktop environment" needs the transfer speeds that fiber optic cable allows, most companies see replacing patch cords as a cost of doing business. As more companies discover the need for that bandwidth, and start considering fiber to the desktop more seriously, the cost of fiber will continue to fall. This is especially true as the tolerances for abuse decrease significantly with copper patch cords above a CAT 6 rating.

Disclosure: I did start out working in IT/Telecom as a cabling/phone technician so I have seen firsthand how easy it is to ruin copper connections, inadvertently.

Re:More data forces the need for more bandwidth

[fuzzyfuzzyfungus]

Nice. I wonder how it deals with being stepped on, having rolly chairs rolled over it, being run under furniture, and waxed to the floor by cleaning crews that can't be bothered to move it?(true story)

Re:More data forces the need for more bandwidth

[mc1138]

Ok, but I warn you, the size of the news letter increases exponentially every day.

And it will be integrated directly into the CPU

[egghat]

cause the PCIe bus is way too slow for transporting terabits.

Or am I wrong?

bye egghat

Re:And it will be integrated directly into the CPU

[Penguinoflight]

You're wrong. Even the original PCIe specification supports around 2000MBytes/sec (or around 20Gbits/sec) on an 8x link. You get double that with PCIe2, and there's always the option to go with x16. All together the maximum theoretical throughput currently available on PCIe is around 80Gbits/sec per card.

PCIe3 will be introduced years before terabit ethernet, doubling theoretical throughput again.

Re:And it will be integrated directly into the CPU

[egghat]

But that's not terabit/s. Even with PCIe3.

Or am I missing sth?

Bye egghat

Re:And it will be integrated directly into the CPU

[Cassini2]

The PCIe bus is not the only problem. Even at 10Gb Ethernet, the load on the CPU of processing each interrupt corresponding to each packet arriving on the network becomes significant. At 1 Tb/s, your network speed is substantially higher than both the peak hard drive bandwidth (3 Gb/s) in your average desktop computer. At 1 Tb/s, you could fill a 1 TB hard drive in 8 seconds flat!

For the moment, these high speed technologies will be primarily used in the network and cabling closets, where the aggregate bandwidth of many computers (or at least many hard drives) can be gathered together.

Re:And it will be integrated directly into the CPU

[perryizgr8]

The PCIe bus is not the only problem. Even at 10Gb Ethernet, the load on the CPU of processing each interrupt corresponding to each packet arriving on the network becomes significant. At 1 Tb/s, your network speed is substantially higher than both the peak hard drive bandwidth (3 Gb/s) in your average desktop computer. At 1 Tb/s, you could fill a 1 TB hard drive in 8 seconds flat!

For the moment, these high speed technologies will be primarily used in the network and cabling closets, where the aggregate bandwidth of many computers (or at least many hard drives) can be gathered together.

i've never seen any hard drive do 3 gb per second. maybe that's just the interface maximum and actual data rate off the platter is much lower.

Re:And it will be integrated directly into the CPU

[Penguinoflight]

If you read the summary, you'll notice that the intermediate goal is 40-100g-base, which is mostly currently achievable. As faster ethernet standards are developed, faster bus speeds will be.

Re:And it will be integrated directly into the CPU

[egghat]

English is not my native language. There is a rather high probability that my German is better than yours ;-)

bye Anonymous Coward

Re:And it will be integrated directly into the CPU

[egghat]

I'm sure that buses will get faster. I just wanted to add a perspective *how* fast Terabit/s is.

bye. egghat.

(better, acs?)

Re:And it will be integrated directly into the CPU

[EvolutionsPeak]

Uh no,
10^12/8 = 125000000000 = 1.25*10^10 bytes / s
1.25*10^10 / (1024*1024) = 119209.289 Megabytes/s

Re:And it will be integrated directly into the CPU

[petermgreen]

According to my calculations PCIe 3 x16 would give arround 128 gigabit per second. That is less than a terabit but MORE than 100 gigabit.

But really the thing you are missing is that ethernet is not just about connections to end systems. You also have to think about the connections from switch to switch. In any decent sized network your requirements for those is likely to be considerablly higher than your requirements for links to end systems.

Ethernet has traditionally moved in fairly large jumps. This is IMO a good thing as it helps keep the number of different standards in use to a managable level.

Re:Ethernet or Token Ring

[disi]

Sorry if I upset someone here :) Anyway about 9 years back I worked in a company using Token Ring and Ethernet (this was actual when they were about to switch to Ethernet). For my experience the connecting part was pretty much the same cabeling using normal star topology.
Of course the technology went further and todays ethernet switches work around the problem that Ethernet brings with it's design. Even though you have in theory a chance of getting 100Gigabit you will never get it and it breaks down to the amount of users sharing the network.
Lets assume Token Ring would have got the same attention as Ethernet?

Stragglers in the March of Progress

[Cyanara]

The other day I had a small business for a client and was amazed to discover that they were running a significant network through a 10Mbps hub. Being able to upgrade that to a (rather affordable) Gigabit switch was quite satisfying.

By 2015...

[castironpigeon]

...we'll be able to use our monthly bandwidth allowance in under one second. Hooray?

Re:Ethernet or Token Ring

[dave420]

Jesus Christ now I've seen it all. Token ring. Hahahaha! You're funny.

Re:Article summary

[Jae686]

Linux just isn't ready for the desktop yet. It may be ready for the web servers that you nerds use to distribute your TRON fanzines and personal Dungeons and Dragons web-sights across the world wide web, but the average computer user isn't going to spend months learning how to use a CLI and then hours compiling packages so that they can get a workable graphic interface to check their mail with, especially not when they already have a Windows machine that does its job perfectly well and is backed by a major corporation, as opposed to Linux which is only supported by a few unemployed nerds living in their mother's basement somewhere. The last thing I want is a level 5 dwarf (haha) providing me my OS.

Wait, what? There ARE TRON fanzines? Hummm I must get my Terabyte gear NOW!

Re:More data forces the need for more bandwidth

[Opportunist]

10 years, maybe 15, and we'll all be on fiber.

1990 called, they want their news back.

I remember the time when fiber was the next big thing (in said 90s). Everyone was anxious, everyone thought it was really great, yeah, sure, it was expensive but give it time and mass production, and after all it's SO DAMN FAST (Gigabit! Absolutely impossible on copper wires!).

It's a bit like that IPv6. Yes, it would be nice, yes, it's a great technology. But companies loathe investments that aren't really, really, REALLY necessary.

And consider this: IPv6 is a change in software. Fiber instead of copper means digging, running cables and buying a LOT of new hardware. I'd guess we'll see IPv6 in wide use before we see fiber.

40 Gigabit Ethernet explained

[IGnatius T Foobar]

In case anyone was wondering "40? Why 40 Gigabit?" here's the answer: 40 Gigabit Ethernet reuses existing OC-768 technology. So it's actually not exactly 40 Gbps, it's actually 39.813120 Gbps. The idea is that Ethernet encapsulation and framing are being applied to existing components that are electrically (and optically) OC-768. (For the nitpickers out there, yes, I know there's more to it than that, but let's not get bogged down in details.)

So that's why we're making a stop at 40 Gbps instead of going straight to 100 Gbps. Existing technology is being reused to get a useful product to market faster.

Incidentally, 10 Gigabit Ethernet is similarly based on OC-192 technology, so it's actually 9.953280 Gbps.

Re:40 Gigabit Ethernet explained

[dotwaffle]

Well, the 10G WAN PHY is 9.953Gb/s but the 10G LAN PHY is certainly 10Gb/s.

Re:40 Gigabit Ethernet explained

[bored_lurker]

Well, not exactly. Yes, it is true that OC768 is in the 40G range but that is not the full story. 40G signals are carried over DWDM systems using digital wrappers (all of this is based on the ITU-T G.709 standard). So an OC768 is carried in an OTU3 (OCH430) wrapper. But if you want to carried a 40GE signal is can be carried in an OTU3e1 (OCH445). Yes, I know this is an extension of the current OTU standard but my point is that the speed of Ethernet is not really bound by SONET as most of the transport is over DWDM systems, not MSPP SONET/SDH systems. It is carried digitally wrapped over DWDM ROADM systems.

Oh, and yes, I am currently working w/ 40G systems and soon 100G.

Re:40 Gigabit Ethernet explained

[bored_lurker]

Again by the ITU-T G.709 standard an OC192 WAN PHY and LAN Phy can be transported in an OTU2 (OCH 107). This has a payload (+OPU) of 9.9953G. To get true wire speed 10G you have to use an OTU2e (OCH 111, also called transparent mode) which has a payload +OPU of 10.3125. FWIW, there is also an FC standard OTU2f (OCH 113) which has a rate of 10.51875G. I keep these numbers on a chart I made on my wall - too much to memorize :-)

Fiber in the future...

[Firethorn]

Personally, much like how BNC is still hanging around in a few spots, I think 15 years for 'more than half' would be optimistic. On the other hand, I have actually dealt with installed fiber to the desktop systems, so I have a bit of experience.

Fiber patch cords aren't as easily damaged anymore, especially for the plastic multimode stuff. There's also nothing preventing them coming out with patch cords that are armored to the diameter of today's cat6 cables. That's a LOT of armor. ;)

Another option would be to steal a bit of PoE technology - make the computer's ethernet port support PoE, which feeds a media converter in the wall. Other options include fiber with a couple of small gauge wires with it to provide power to the MC in the jack, wiring AC to the jack, etc...

Why I see fiber eventually winning, even to the desktop.
1. Cost - Copper keeps going up in price, while fiber remains stable or even drops, relatively. Even today bulk gigabit+ capable fiber can be obtained cheaper than bulk cat6 cable. What currently kills fiber to the desktop is generally connector cost, combined with higher adapter cost because they're 'special purpose'. Still, laser tech keeps getting cheaper. Many motherboards today have optical connectors on them for the audio. Network adapter is a different matter, but the potential is there. Cat6 connectors are a bit harder to terminate and are also a bit more expensive. Thus far, the higher speed copper ones I've read about have been even harder. So that advantage copper has is going away.
2. Speed - Gigabit cat5e/6 costs more than old style cat5, which is more than phone quality cat3. They're looking at having to add wires to break gigabit speeds, and change the connecter so it's no longer RJ45 compatible. This, to me, breaks the backwards compatibility that has allowed twisted pair to win for so long.
3. Range - With a large building, the difference between fiber and copper can be the difference between having 1 network room and 8 or more network closets with powered equipment in them. If fiber was a bit cheaper, I'd run large multifiber wires to the closets, and merely have a patch panel inside to distribute the lines out to the various jacks.
4. Weight & Bulk - Cat6+ is getting heavier and heavier - computer density is still increasing today. With the increase in weight and bulk, existing building cable trays and runs are becoming overloaded. Adding more is an expensive proposition, and I estimate that I can fit two times as many fiber cords into a given cable tray, at half the weight over copper runs. Even more if you put in patch closets so that you run many pair.
5. Emissions - fiber doesn't emit or be affected by EMF radiation.
6. Future proofing - copper is pushing it's limits, fiber installed today would likely only need minimal modifications to support terabit speeds in the future.

What applications do you think will require this kind of bandwidth? HD video with moderate compression should easilly fit into a gigabit.

Well, how about HD 3D video? 120-150HZ refresh rates combined with blink glasses to display those 3D videos that movie theaters are showing?

Still, for most business uses, I tend to say that even 10meg connections are more than enough for most users. Seriously, we still occasionally find a 10 meg hub with some users on it. Thus why speed is only one of the advantages fiber has. Cost, Range, and bulk are bigger ones. Range and bulk because, well, they increase costs.

What I think fiber to the desktop needs is the equivalent to 10baseT - an open, low cost standard that is cheap and easy to use. Right now you have a dozen of propriatary connectors. Some are tougher, some are cheaper, etc... We need the equivalent of the RJ-45.

For fiber I'd consier a standard specifying optional small gauge metallic wires for power transmission to compete with PoE, one of the things keeping copper alive. Being pure power, it could be injected cheaply and effectively just about anywhere. Just keep the voltage low enough to not hurt anyone - depowering such as system could be a nightmare.

Re:Fiber in the future...

[petermgreen]

Gigabit cat5e/6 costs more than old style cat5
Can you even buy old style cat5 anymore? almost all the cable I see for sale nowdays is cat5e or cat6 (cat6 being a bit more expensive than 5e but not hugely so)

Still, for most business uses, I tend to say that even 10meg connections are more than enough for most users. Seriously, we still occasionally find a 10 meg hub with some users on it.
I know here at the university of machester most ports are still 100 megabit and a few are still 10 megabit. I'm sure there is the odd gigabit port arround but as you say for most people 10 megabit is sufficiant (though sometimes annoying) and 100 megabit is more than enough.

What I think fiber to the desktop needs is the equivalent to 10baseT - an open, low cost standard that is cheap and easy to use.
And to make it cheap you need a lot of users, and that means you need a killer application that means a lot of users need more than gigabit to the desktop.

Re:Fiber in the future...

[Chabo]

What I think fiber to the desktop needs is the equivalent to 10baseT - an open, low cost standard that is cheap and easy to use. Right now you have a dozen of propriatary connectors. Some are tougher, some are cheaper, etc... We need the equivalent of the RJ-45.

I worked through college at a networking lab dealing with Gigabit Ethernet, but 10Gig, but while there are tons of connectors out there, but only two are really ever used on the NIC: SC and LC

A link from Belkin on the types of fiber:
http://www.belkin.com/networking/fiber/

SC has historically been used more, but LC has taken over as the dominant interface, because you can have a much higher port density; with LC, you can have a 48-port 1U switch, and you can have a 4-port NIC in a standard PCI/PCIe slot.

Personally, I think if you wanted to use fiber in a home or small office setting, 1000BASE-SX will be the best choice for a while. SX (850nm light) is cheaper than LX (1310nm), and it'll work well over any distance you'd see inside a building. Also, it's "fast enough" for most applications, especially since you're unlikely to have anywhere near 1Gbps to the external network in that kind of setting.

Re:Fiber in the future...

[Firethorn]

I worked through college at a networking lab dealing with Gigabit Ethernet, but 10Gig, but while there are tons of connectors out there, but only two are really ever used on the NIC: SC and LC

I've had to deal with three different proprietary connectors for desktop use... Not fun.

Oh, and I was referring not to the connection to the switch, but for the connection to the desktop, density isn't too terribly important there, but expense is. LC is nice enough, and you can get adapting patch cables in bulk if you need to.

Re:Fiber in the future...

[Chabo]

I've really only seen SC and LC on switches, too. I've seen a few really old switches that used MTRJ, but nothing made since about 2001.

I mean, switches often have GBIC or SFP slots instead of hard-wired interfaces, but I'd consider that a plus, since fiber ports can wear out; it's easier to switch out a dead SFP on a 48-port switch than to try to diagnose an issue with the hard-wired port.

Nowadays at the switch, SFPs are pretty standard, and LC fiber is pretty easy to work with. Even at the NIC, where I'd rather have a hard-wired port because it's cheaper than an SFP, LC is pretty standard, and SC is the only other interface I've seen.

Re:More data forces the need for more bandwidth

[IGnatius T Foobar]

+1 Insightful; until now I never truly understood why people wanted more bandwidth.

Warez, pr0n, and MP3's. That's all. Take those away and we might as well be using 300 bps modems.

Re:Bring it on!

[TheRaven64]

The only thing I can think of that a home user might be doing is using a networked PVR which directly captures something like a HD DVB-T stream and then sends it over the network. This can be close to 30MB/s, which is well into the kind of level where very cheap GigE hardware will start to struggle. Maybe if he has a NAS which stores HD recordings and has the PVR dumping footage on it and other machines reading from it (or more than one channel being recorded at once, using something like GNU radio) then it will be possible to saturate even good GigE hardware. At home I use 802.11g, and rarely find that it is too slow.

Re:Ethernet or Token Ring

[rwyoder]

When Token Ring died it was because 100Mbps ethernet was cheaper than 16Mbps token ring. I was there. Token couldn't keep up; case closed.

It's not dead; It's just resting.

Re:More data forces the need for more bandwidth

[Panzor]

My first comp-sci teacher had an upright trashcan waxed to the floor. It was like a little clear igloo with no door or blocky-texture.

For a moment there...

[ryzvonusef]

I thought The heading was "The Road to Terabithia" :)

Re:More data forces the need for more bandwidth

[afidel]

It's not like high speed ethernet cables handle any of that well at all! As soon as you deform a Cat6A cable it will no longer be able to handle 10Gbit.

Re:More data forces the need for more bandwidth

[windsurfer619]

You first ;)

If they push any more data though ethernet cables.

[jameskojiro]

They are going to break some law of causality eventually.

Might as well work on this

[ThatsNotPudding]

seeing as how 802.11n finally been approved and all.

/sarcasm

Re:More data forces the need for more bandwidth

[Gilmoure]

Most of our buildings have been wired with fiber to the desktop for the last 4-5 years. Biggest breakdown are the stupid transceivers. Their power supplies go wonky and we can't get just the power supply. Have to swap out entire unit. New machines are coming in with fiber cards but still have older machines with ethernet only.

Re:More data forces the need for more bandwidth

[Jeremy Erwin]

What applications do you think will require this kind of bandwidth? HD video with moderate compression should easilly fit into a gigabit.

Why settle for "moderate compression?" Dalsa Raw requires 3.2 Gb/s.

Before a movie can be put on bittorrent, it must be edited.

Re:More data forces the need for more bandwidth

[vux984]

And consider this: IPv6 is a change in software.

IPv6 is only relevant on the internet as a whole, and has a chicken/egg problem. There's no real benefit to itself for a single business switching to ipv6, and the ISPs don't need it if their customers and peers aren't switching.

Businesses do however derive benefit from upgrading the speed of their LANs and therefore are motivated to do so, individually. Hell, I'd buy a 10GB lan for my home office if the price dropped. I upgraded to gigabit a few years ago already. Just for moving VM images around on my LAN, doing backups, etc, it was worth it.

Re:Ethernet or Token Ring

[dirvine]

You need to remember ethernet is a shared bus technology, using CSMA/CD for collisions. This is how it works at layer 2 with a layer 2/3 switch you can limit this to an extent.

Re:More data forces the need for more bandwidth

[severoon]

"Obsolescent?" Is that like "scrumtrilescent"? (Invented by Will Farrell's James Lipton character on the spot to describe an actor's work.)

Re:More data forces the need for more bandwidth

[anegg]

From what I have seen, fiber manages quite well in an environment that is fairly abusive, being underfoot and kicked around. I'm not sure I know of many examples of it being rolled over in a chair, but I've seen a lot of it stuffed under desks and workstations, kicked around on a daily basis, with amazingly few failures.

Re:Ethernet or Token Ring

[NeoSkandranon]

Pining, perhaps?

Re:Ethernet or Token Ring

[anegg]

Its pining for the Fishkill fjords

Re:Bring it on!

[Vanders]

Consumer 10GbE is a long, long way off. There's no way in hell you're going to wire your home for 10GBASE-CX cabling, and 10GBASE-T is only just begining to appear. The cost of the switch ASICs is so high that creating an 8 or 16 port switch isn't much cheaper than making a 48 port, and adaptors are still complex, expensive devices: a lot of them have SFP connectors so you can plug in a suitable transponder, which makes things expensive. Other are basically HPC interconnect cards running in Ethernet mode I.e. cards produced by Myricom.

Most small & medium sized server rooms won't start to use 10GbE for at least the nex five years, yet alone consumers.

Re:Ethernet or Token Ring

[anegg]

Token Ring was a dead end in the marketplace, despite a heavy-hitter pushing it hard (IBM). It was passed over in favor of Ethernet because Ethernet was in practice superior to Token Ring. At the time, Token Ring got every bit as much attention as Ethernet, if not more. By 9 years ago (2000CE) Token Ring had already died and its rotting carcass was beginning to stink. If you like, you can consider FDDI as the last gasp of Token Ring, and even that is (largely) dead and buried.

There were many attacks on Ethernet technology by Token Ring advocates, most of them were easily disproven by those who had implemented Ethernet. Please, let Token Ring go...

Re:Ethernet or Token Ring

[joib]

You're a moron or in love with token ring.

I don't think those two are exclusive of each other.

Re:Bring it on!

[BrokenHalo]

...and, in any case, if you can afford a GigE pipe to the outside world, you are wealthy beyond my wildest dreams. Unless you are running data-intensive operations across a cluster, most people only notice any real latency when copying big files over WiFi connections.

Re:Ethernet or Token Ring

[Firethorn]

Token ring gives each device on the ring roughly equal time, I'd imagine switched ethernet with a decent switch would have similar behaviour. I beleive some of them can also prioritise data.

Ethernet switches do indeed often support QoS, which is the prioritization of traffic.

Even my home router box does, though as a router it's smarter than at least some switches. I'm not aware of any modern gigabit switches that aren't at least that smart though.

Re:Ethernet or Token Ring

[Chabo]

Nope. Token Ring died because all of the tokens fell out, and couldn't be found.

Re:Ethernet or Token Ring

[Chabo]

GigE has half-duplex in the standard, but practically nobody supports it. Autonegotiation is required at Gig speeds, and I haven't come across a single device that supports half-duplex, but not full, so there should never be an instance in which two devices negotiate a half-duplex Gig link.

10Gig finally dropped half-duplex for good.

Even if you're using half-duplex, as long as you're using switches and not repeaters, you'll never have more than two devices on a collision domain, so it's with modern equipment, Ethernet is a point-to-point technology, even though it was originally a shared bus technology.

Re:Ethernet or Token Ring

[Chabo]

I've met plenty of morons that don't love token ring.

You may be on to something for the relationship the other way, but being a moron does not imply love for Token Ring.

Re:Bring it on!

[Firethorn]

You have to ask?

Porn. It's always Porn. ;)

After all, the internet is for porn.

Re:Bring it on!

[Chabo]

My biggest problem with Wifi is that I live in a densely populated area, so there's a ton of interference. Sitting on my couch, my wireless NIC's software can see 10 networks that don't hide their SSID.

In reality, 10BASE-T might even be an improvement over my current connection, from my couch to my wireless router, 15 feet away. 100BASE-TX certainly is.

Re:More data forces the need for more bandwidth

[PONA-Boy]

Regardless of whether this was meant seriously or not...

In 10 or 15 years, the need to physically PLUG your computing device into something will, likely, be obsolete. Wireless will be matured to the point wherein our ever-more-mobile society will be completely unplugged. Maybe large carriers will still continue with physical media connections between devices but end-users will be free from such restraint.

Re:Ethernet or Token Ring

[afidel]

Token ring didn't die, it just evolved and shrank. SONET and FDDI are both descendants of token ring.

Re:Infiniband: a better technology in the datacent

[afidel]

Infiniband cheaper than Ethernet, you MUST be off your rocker. HP dual port 10Gbit Ethernet card costs $700, DDR Infiniband card is $1200. Sure it's cheaper to buy one Infiniband card than two Ethernet cards but few applications need more than two 10Gbit ports today. Also 10Gbit Ethernet is rapidly becoming comoditized in switches, Infiniband due to small market penetration basically never becomes commoditized.

Re:Ethernet or Token Ring

[petermgreen]

and you will notice that having killed off token ring at the low end ethernet is now taking over at the higher end too ;)

Re:Article summary

[Artuir]

Modded troll, but instead of modding him troll why don't you point out counter arguments? This is a large part of the anti-linux sentiment and dispelling these "myths" if they are indeed such would go a long way in public perception.

As far as I see it, he's right aside from the ad hominem attacks.

Re:Ethernet or Token Ring

[petermgreen]

note: what are commonly reffered to today as hubs and switches are considered to be repeaters and bridges respectively by the standard.

You need to remember ethernet is a shared bus technology, using CSMA/CD for collisions.
Historically yes, theese days no.

Old fassioned 10BASE5/10BASE2 (which are pretty much the same other than the cable) was a shared bus technology. Busses could be joined with either repeaters (which made the busses into a single collision domain) or bridges (which made each of thier ports a seperate collison domain)

10BASE-T and the various 10 megabit fiber standards supported both CSMA/CD and full duplex links. However there was no autonegotiation at this time and hubs were far more common than switches so 10BASE-T was usually run half duplex.

Fast ethernet supports both CSMA/CD and full duplex. Any reasonablly recent 100BASE-T NICs and switches will support full duplex and will autonegotiate full duplex mode by default. CSMA/CD is used primerally with hubs (which have mostly dissapeared off the market as switch prices have come down) and possiblly with other old or weird equipmen.

1000BASE-T supports both CSMA/CD and full duplex but afaict is very rarely used in CSMA/CD mode.

10GBASE-T and above support full duplex only.

Re:Ethernet or Token Ring

[petermgreen]

Lets assume Token Ring would have got the same attention as Ethernet?
Then I would guess like ethernet it would have abandoned it's roots and moved to a network of full duplex point to point links. It's really the only sensible topology at modern speeds.

Modern ethernet is only ethernet in name and frame format. Shared bus networks and CSMA/CD were relegated to network backwaters noone cares about

Re:Ethernet or Token Ring

[dirvine]

Are you sure theres a full duplex / half duplex distinction ? with layer 2 devices everything is on the one 'bus' so to speak. In this case the rules of csma/cd apply whether full or half duplex regardless.

If the bus is transmitting - your screwed till it quiet by at least 2X transmission time (min packet size X speed X distance) speed in this case is 10Mb/s ?

This is the carrier freq which data can be transmitted on !

this is exactly why theres minimum ethernet packet sizes ! so transiting stations can detect they have not sent the whole packet before a collision

There is talk of near light speed electron movement in this thread which is wrong as even in a vacuum (which seems not to exist now - like infinity) was only every touted at 3X10^7 m/s. Max packet sizes were more arbitrary from what I can remember!

I could be very wrong in my thinking here - some of which is way back ! but interseting thread never the less.

Re:Ethernet or Token Ring

[petermgreen]

Are you sure theres a full duplex / half duplex distinction ?
Positive.

with layer 2 devices everything is on the one 'bus' so to speak. In this case the rules of csma/cd apply whether full or half duplex regardless.
I think you have your layer numbers a bit screwy. Current convention seems to be that layer 1 is the very low level hardware interfaces. 3 is the IP layer and 2 is everything in between. Personally I think layer numbers confuse more than they help with describing such things...

Coaxial bus based ethernet (10BASE-2 and 10BASE-5) is naturally half duplex. As you say only one station can transmit at a time on such a network.

Most twisted pair and fiber ethernet (there are a couple of exceptions, e.g. 10BASE-FP and 100BASE-T4 but none are in common use) is naturally full duplex. However for compatibility and to support use of hubs they also support a mode where they act like a half duplex link.

Just to make things absoloutely clear, lets consider 4 simple cases (if you understand theese you should be able to understand more complex ones too)

* an old fasioned 10BASE2 or 10BASE5 bus. All nodes share a shared bus and use CSMA/CD to arbitrade.

* a 10BASE-T or 100BASE-T network with a hub. The 10BASE-T links are in half duplex mode and the whole network is still one collision domain. As before the nodes use CSMA/CD to arbitrate.

* a 10BASE-T (or possiblly very old 100BASE-T ) network with a switch. Since there is no autonegotiation half duplex links and CSMA/CD are still used but rather than being the whole network the collision domain is only one link.

* A modern network with a switch. The links autonegotiate into full-duplex mode and there is no collision domain at all.

this is exactly why theres minimum ethernet packet sizes ! so transiting stations can detect they have not sent the whole packet before a collision
And half duplex gigabit had to increase the minimum packet size to keep a reasonable network size possible but since hardly anyone uses half duplex gigabit that's kind of academic.

There is talk of near light speed electron movement in this thread which is wrong as even in a vacuum (which seems not to exist now - like infinity) was only every touted at 3X10^7 m/s.
If there is (I haven't spotted it myself) it probablly results from confusion between the speed of electrons (which is pretty slow) and the speed of the electrical wave (which iirc is about a third to a half the speed of light depending on the material of the cable).

Re:Ethernet or Token Ring

[petermgreen]

* a 10BASE-T or 100BASE-T network with a hub. The 10BASE-T links are in half duplex mode and the whole network is still one collision domain. As before the nodes use CSMA/CD to arbitrate.
Just to clarify that should say "The 10BASE-T or 100BASE-T links are in half duplex mode"

'Road'? Shouldn't that be 'Bridge'?

[Namarrgon]

Didn't anyone else think 'Bridge to Terabit Ether'?

What a missed headline opportunity.

Re:'Road'? Shouldn't that be 'Bridge'?

[rant64]

"There is nothing more helpless and irresponsible than a man in the depths of a Terabit ethernet binge".

No, seriously. What a foresight these guys at IEEE have. They've run out of single-letter standards already.

Re:Bring it on!

[BrokenHalo]

If your problem is interference from non-hidden networks, chances are they are also set to run on the default channel (here it tends to be channel 6) so you could try setting your router and NIC to use (say) 1 or 11. Some of the others might be worth a try, but there is more overlap. You could also improve your reception by using an antenna like this, bearing in mind that since this "shouts louder" than your neighbours, a bit more care is required to secure your connection.

Re:More data forces the need for more bandwidth

[bcmm]

As wireless improves, so will wired. I don't think wireless is going to overtake wired in terms of transfer speeds any time soon, and there will always be people who want/need better transfer speeds (someone above mentioned moving VM images around, for example).

Re:Bring it on!

[quenda]

HD DVB-T stream and then sends it over the network. This can be close to 30MB/s,

Bits maybe, not bytes. Around here, A DVB channel is 22Mbps max. The wifi certainly struggles, but not the 100Mbps LAN.

Be visionary!

[Jeppe Salvesen]

Well - you are here assuming that the majority of the network traffic will be fetching and pushing blocks to the storage system. That may be partially true. However, with terabyte interfaces we should also be able to move entire virtual machines between servers in less time than we currently need. Applications will also be able to run inside a large cloud where calculations are distributed across machines and collected afterwards. The better the network throughput and network latency, the smaller tasks can be distributed and still run faster than in serial on one processor.

I mean - cloud computing can be kinda like supercomputing: Lots of processors with high-speed interconnects.