Slashdot Mirror
Mixing Gigabit, Copper, and Linux
iampgray writes: "With copper-based gigabit cards selling for less than $36 these days, what kind of performance can you expect -- especially in the often-overlooked Linux market? We sought out to test exactly what you could expect from copper-based gigabit solutoins for the desktop interface through the cluster-targeted products. Name brands and off-brands were put through the wringer. How'd they fare? Interesting results to say the least."
Are we even to the point when a normal PC could handle Gigabyte? And if so, why not use optical? I mean, saying I've got a fiber optic home network is a lot cooler then saying I've got a gigabyte eth home network. I mean to a geek, (to anyone else, that would just be lame... er...)
How much more expensive is the optical stuff for GigE? I'm mostly using optical audio connections from my home sterio, and that's not to much money
autopr0n is like, down and stuff.
Are there any NICs using the AGP? Not many boards have 64bit PCI yet, let alone PCI-X, but every board has an AGP slot. This would be great for cheap 1U cluster nodes, with an appropriate riser card of course.
Did you know you can fertilize your lawn with used motor oil?
There are basically two types of latency in PCI. The first latency is the amount of time it takes for a target to return a requested word. This is 16 33MHz clocks, according to the PCI 2.2 spec.
The second type of latency is the amount of time it takes a target to return a second word in a Burst transaction. This is 8 33MHz clocks, according to the PCI 2.2 spec.
The setting you are playing with in BIOS is probably the first latency... which is basically a setting in the PCI master, deciding how long to wait for data from a target before deciding to change the transaction to a delayed read. A delayed read basically frees the bus, and the master will check back with the peripheral at a later time to see if it has the data ready yet or not.
delayed reads slow down access to that peripheral, because no other transaction is allowed to take place with that peripheral until that delayed read is finished.
Older PCI cards didn't have the 16 clock limit on returning the first word of data, and they usually took longer. On new systems that try to be pci 2.2 compliant, to prevent a bunch of delayed reads from taking place, you have the option of increasing the latency timer in bios, so that it won't time out exactly on the 16 clock boundry, thereby speeding up access to that peripheral, at the cost of hogging the bus.
So anyway, adjusting the latency timer isn't likely to have an effect on newer peripherals... unless you make it too short, causing a bunch of delayed reads, and then your system will slow down.
--Scott
But I thought there was a register (oddly named latency) that governed how long a busmaster could burst when someone else wanted the bus.
First, you can't just stick a gigabit card in a machine and expect it to work at full capacity. The basic design of ethernet was not really designed for gigabit speeds, but we've managed to squeeze it out - barely.
With 10 mbit cards, having the card generate an interrupt with ever incoming frame wasn't too bad. And on 100-mbit, it's still managable - but at a full gigabit, it really, really starts to bog down the machine. Some cards get around that by using interrupt coalescing, where they buffer up a few frames before they trigger an interrupt. That has a drawback, though: It increases latency. The trade-off has to be at some point, and not choosing the RIGHT point can affect either throughput or latency.
Furthermore, to get the full benefit out of your card, you generally need to enable jumbo-frames on both the card and the switch - and of course, your switch has to support that feature.
To make matters even worse, you can't always pump out (or receive) a full gigabit in any other than testing situations. Say you're receiving a large incoming file via FTP, NFS, or the protocol of your choice. Can your machine *really* write data to the disk at over 100 megabytes per second? And if it can, can it really handle both receiving a gigabit from the card, processing it, and writing the gigabit out to the disk? Unless you've got a very large amount of money in the machine, it probably won't.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
Actually, the EIDE ports on modern chipsets are not connected to the PCI bus anymore. They use proprietary high-speed buses (V-Link, HyperTransport, etc.), which is a good thing, because one Ultra-ATA133 Port has already the same bandwidth as the whole PCI(-32/33) bus.
Did you know you can fertilize your lawn with used motor oil?
For those of you who don't know how this works, here's a bit of a primer: basically, you set the port on your big data center grade switch to "trunk" and then you enable 802.1q on your Linux box. Then you don't just have one Ethernet interface with one address --- you have up to 4096 virtual ones, each on its own VLAN and each with an IP address that's valid on that VLAN. So you'd have eth0.1, eth0.2, eth0.3, etc... each talking to the machines on that VLAN.
Once you've got that running, you can do all sorts of neat stuff, including:
As you can see, it's limited only by your imagination. And with that much stuff potentially running through the box, you're going to need that 1 Gbps of speed. Happy hacking!
Tired of FB/Google censorship? Visit UNCENSORED!
"Only in their dreams can men truly be free 'twas always thus, and always thus will be."
--Tom Schulman
We've just setup up a gigabit cluster with new
u pe rServer6012P-6.htm
:-} I havn't been able to test how much more CPU headroom they give me.
1u servers from supermicro:
http://www.supermicro.com/PRODUCT/SUPERServer/S
I've tested it using a CISCO catalyst 3500XL switch. Our new 4000 switch goes online this week.
I can get a fully saturated link using a program I wrote:
http://mxtraf.sf.net
I'm using a 2.4.18 kernel with the latest drivers from Intel.
BTW, anybody know how to enable jumbo frames in IOS?
-- Buck