Slashdot Mirror
10GbE: What the Heck Took So Long?
storagedude writes "10 Gigabit Ethernet may finally be catching on, some six years later than many predicted. So why did it take so long? Henry Newman offers a few reasons: 10GbE and PCIe 2 were a very promising combination when they appeared in 2007, but the Great Recession hit soon after and IT departments were dumping hardware rather than buying more. The final missing piece is finally arriving: 10GbE support on motherboards. 'What 10 GbE needs to become a commodity is exactly what 1 GbE got and what Fibre Channel failed to get: support on every motherboard,' writes Newman. 'The current landscape looks promising. 10 GbE is starting to appear on motherboards from every major server vendor, and I suspect that in just a few years, we'll start to see it on home PC boards, with the price dropping from the double digits to single digits, and then even down to cents.'"
10GE Motherboards are still pointless when 10G routers & switches are still way too expensive.
Biggest reason I can remember from when we were looking at upgrading SAN and LAN equipment in our data center was the price/performance point. We didn't need 10 GbE performance yet and the price was pretty far above what we were using. That was 3 years ago though, so I'd have to poke around some of the newer equipment to see if we have any boxes with it. I just took a gander through the HP and Dell offerings and it's not even an option on anything but the top tier equipment. I think that pretty much explains the situation itself.
Of course its growth was going to be lower.
The primary use of 10GbE is virtualization. The use of network cards are a function of the number of chassis, not the number of hosts. Numerically, 10GbE is not 10 1GbE cards. You can split the 10GbE between a lot of hosts. You can easily double, triple, or even quadruple that to making that 10 GbE card the equivalent of 1 GbE cards on 40 servers, depending of their load and use. Instead of buying 40 servers and associated cards, you're buying one larger chassis with larger pipes. In a large farm environment, and it makes sense.
Throw in the fact that network is only as fast as its narrowest choke point, there is no reason to put in a 10 GbE card behind a 7MB DSL connection.
What 10GbE needs to become a commodity is a) end of any data caps, b) data to put down that pipe, and c) a pipe that can handle it.
Show me fiber to my door and then, it will be a commodity.
Ten gigabits per second is 1,250 megabytes per second. High-end consumer SSDs are advertising ~500 MB/sec. A single PCIe 2.0 lane is 500 MB/sec. Then there's your upstream internet connection, which won't be more than 12.5 MB/sec (100 megabits/sec), much less a hundred times that. I guess you could feed 10GbE from DDR3 RAM through a multi-lane PCIe connection, assuming your DMA and bus bridging are fast enough...
I'm sure a data center could make use of 10GbE, but I don't think consumer hardware will benefit even a few years from now. Seems like an obvious place to save some money in a motherboard design.
Visit the
What you describe exists. It's not uncommonly used for IP cameras outside the 100m limit of TP Ethernet (on perimeter fences, etc.). The problem with fibre is that it's a bitch to terminate compared to copper, and therefore quite a bit more expensive to install on a large scale. Fibre still only makes sense when you need the long cable runs.
its also not needed for most work environment.
It is extremely convenient when doing large building and/or campus networking, though...
Sure, it makes very little sense to do 10Gb to the drop(barring fairly unusual workstation use cases); but if all those 1GbE clients actually start leaning on the network(and with everybody's documents on the fileserver, OS and application deployment over the network, etc, etc. you don't even need a terribly impressive internet connection for this to happen), having a 1Gb link to a 48-port(sometimes more, if stacked) switch becomes a bit of an issue.
Same principle applies, over shorter distances, with datacenter cabling.
Oh for crying out loud. Where do you people get off with this kind of thinking? How are you even allowed in technology fields with a mind like that?
It's not needed...technology is about advancing because it's WANTED. It's not run by committee, and it's not run by determination of some group need, because if it were, we'd still be living in caves and worshiping rocks, because fire isn't needed by anyone.
And the reason, reading between the lines, for it taking so long to be adopted, is because everyone has become cheapskates when it comes to technology. The idea of a separate NIC to handle network traffic is a lost cause, as is a dedicated sound card, and now video card. Why? Because you're trying to justify to a group of people who refuse to educate themselves why it would be in their own best interest to pay a little more.
I applaud the people behind 10GB E, and hope they have enough resources / energy to bang out 100GB E. This is progress we can measure, easily, and it should be rewarded.
I am John Hurt.
it's trivial to enable LACP to bond several 1 gbps links. no new equipment, no new cabling. that would have slowed down my 10 gbps deployment.
10x1gb != 1x10gb. Your LACP bond still limits a single stream to a single link. Even with multiple streams, you would have to have a lot in order of them to hash out to all the links.
10GBe won't be as fast as a nice cheap SSD
It doesn't matter, remote storage is faster than gigabit. I don't need to hit 10 to get a benefit.
but not even an SSD can keep up with an avalanche of data requests from multiple systems unless that remote server is pretty damn beefy by home standards.
What? That's the whole point of fileservers. They need to meet the usage, of course, but that's an always increasing spec.
Simpler to keep your OS local, and trivial as far as cost.
Consolidating is always cheaper (per unit of storage) and it's easier to back up and manage, keep on UPS power, etc.
You'll need a heck of a RAID array for that, but it's buildable. Or, you could just stick with GigE, since that still tops out at 125MB/s and that pushes local (non-SSD) storage.
eh, my current central storage is 5 hard drives in a ZFS raidz2 with one SSD split up for L2ARC (cache) and ZIL (write cache). The entirely of the setup difficulty is:
cd /etc/yum.repos.d
wget url-to-repo
yum install zfs
(reboot or modprobe)
zpool create home raidz2 sda sdb sdc sdd sde cache sdf6 log sdf7
Oh, I had to plug in 6 SATA cables. Typical throughput is about 340MB/s. The only reason they're not all SSD's is because SSD's are expensive and unreliable. If it wasn't a home machine, the ZIL would be on a mirror of SSD's.
Correct me if I'm wrong, but my understanding is that this requires a lot more than just a high-speed connection. High-end connection + craptastic router = terrible latency when dealing with high load.
Switch, not router. There are problems with current buffer management techniques that effectively means that higher ceiling room means latency improvements. Google 'bufferbloat'. Things like CoDel will make this better when the pipes are more full, but they're not widely deployed yet.
"I have my home wired up like a datacenter. Everyone else should want a huge amount of network capacity and capability so that it makes my already extravagant costs slightly cheaper."
JHFCOAS - this is Slashdot. What we're doing now is what will be sold in a box for $200 at WalMart in five years. I'm amazed to find tech geeks who don't even know that normal people have been buying inexpensive Buffalo and WD SAN solutions at the office supply store since 2008. And with all this shit going on about the NSA, you can bet people are going to be pulling some of their stuff back out of the cloud.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Thats why you have a few 10GbE uplinks on the access switch, that way everyone generally gets 1gbit at all times.