Slashdot Mirror
Gigabit Networking for the Home?
The Clockwork Troll asks: "I've had a whole-house audio/video distribution project on the back-burner for a while now. As gigabit networking hardware prices come down to earth, I'm tempted to jump on the 1000BaseTX bandwagon. As far as I can tell though, the current crop of consumer-priced hardware/software doesn't address a couple key issues, namely: fragmenting jumbo frames for the benefit of legacy clients - this is critical as some of the devices on my network will not tolerate the 9000+ byte Ethernet frames which are needed to get the most out of gigabit; and OS support - do Linux and Windows require much tweaking to take advantage of gigabit? Will most drivers automatically optimize themselves? A Google search didn't reveal too much consensus, especially on hardware choices. What switches and software configurations have Slashdot readers been using for home gigabit networks, in particular mixed ones (100/1000BaseTX?"
Hard as I try, I can't imagine ever having enough stuff in my house to warrant gigabit. Damn.
at Gigabit speeds successfully on your home LAN, your slow ass drives ain't gonna deal with the flow of bytes.
Dude that is like trying to use jet fuel in a 1984 Capri.
I've had a good experience with a Dell PowerConnect hub (the 8- or 16-port model, I forget which). It was quite inexpensive and claims to support Jumbo Frames (however I haven't actually gotten this to work; when I enlarge the frame size on Linux it loses the connection). Oh, and I had to disable one default feature on the hub (tree-spanning something or other) to get it to work.
For clients I use Intel gigabit cards (the 64-bit PCI "server" model). I wouldn't skimp here since indications are that cheap gigabit cards don't have any hope of getting wire speed. NFS file copies max out at 20-30MB/sec, but I know that is limited by my server's disk array. I did a test for raw network bandwidth (just sending zero bytes as fast as possible) and got around 60-80MB/sec.
Everything is connected to my existing Cat-5 cable with no problems. This includes several Linux systems, one Mac and one Windows PC.
I will caution you not to expect anything like gigabit wire speeds with typical clients. My Mac G4 in particular seems to have trouble getting good bandwidth (I think the problem is either the network stack or NFS client).
If anyone has a success story with jumbo frames, I'd love to hear about it. The only references I could find are for mega-dollar Cisco/Foundry type equipment.
I have this friend who goes to South Dakota School of Mines and Technology. He got a bunch of free Cat-6 from one of our mutual friends, whose brother owns a audio/video installation company, so he wired his entire dorm with Gigabit. He brags about it all the time, too. He's done some other weird stuff in his day, though at least he didn't cover his entire dorm room walls with AOL CD's.
Oh, wait....
There's no point in upgrading to gigabit unless your boxes are equipped with 64-bit PCI slots. 33mhz/32bit PCI's theoretical limit is 133Mbit/sec. This is how most consumer-grade Wintel motherboards are equipped. Thus, the 32-bit "gigabit" cards sold sold by the consumer networking outfits are a rip-off, they're barely faster tha 100Mbit.
If you're lucky enough to have a board with 66mhz/32bit slots, the data rate doubles, but all devices on the bus must run at 64mhz, since the bus will run at the speed of the slowest device.
To get a tangible benefit from gigabit ethernet, you need at least 66mhz/64bit PCI, which runs at 512Mbit/sec. This will get you half way there. To saturate gigabit, you need 133mhz PCI-X.
Other devices, like disk controllers and the disks themselves, can be bottlenecks. Better plan on upgrading to that dual-channel U320 SCSI RAID setup!
Bottom line? A true implementation of gigabit ethernet requires commercial-grade hardware, which costs plenty. The consumer stuff is little more than fluff. Unless you plan to go "all the way", save your hard-earned dough for something better, like a contribution to the FSF or an OSS project of your choice.
This isn't the sig you're looking for... Move along.
Does it support jumbo frames. How well does the Asante detect jumbo frames if it's an unmanaged switch? I know their next one up(GX5-1600) did. I was going to go gigabit also, I'm running a cluster, but the only cheap solutions ($200 or less) for an 8 port switch, that I could find where from SMC and Buffalo technologies which both claim to support jumbo frames. The SMC and Buffola are both unmanaged.
You COUULD do that on 100Mbit, easily, depending on what exactly you're expecting out of your cameras (two camera streams could fit into 100Mb, plus a dozen VOIP lines, and god knows how many mp3 streams)
Then again, you could also go and get a 4 port 100Mbit card, and use a bonding driver (you are planning on using Linux, afterall, right? And segment your camera system/voip/servers--400Mb/s could be tons of room for dozens of cameras, hundreds if they're only snapshotting, and hundreds of simultanious voice lines. At some point it's (the bandwidth) going to be too much for your consumer motherboard to take, (better have 64Mbit/66Mhz PCI ports if you really want to push it).
You'll have wait for PCI-X or go SGI.
In networking: giga = 1000
When refering to any type of computer storage: giga = 2^10.
This is mostly because computer storage is addressed by a processor in some way and processor registers happen to be binary storage devices.
You can't build a 1000 byte RAM chip and expect to address it without doing a calculation to distinguish a valid address from an invalid one.
A 1024 byte RAM chip makes it simple. Just connect 10 address pins to it and any combination is valid.
Networks don't use the 2^10 convention because their rates are not required to be based on powers of 2.
I have a NetGear 4 port gigabit switch. I have found I can transfer files about 2.5x as fast as with 100mbit (without jumbo frames). In my book, that's worth the few extra bucks a gigabit switch will cost you.
A warning though, I've heard most of the cheap gigabit switches have fans in them. Fans reduce the reliability of a switch many fold and make them LOUD. I like my 4 port Netgear and they now make an 8 port version which is also fanless and very reasonably priced.
Does anyone have a Linksys or D-Link gigabit switch who can confirm or deny the presence of a fan?
One note I'd like to throw in: Gigabit ethernet requires Cat-5 cable. Not Cat-5e, Not Cat-6, Cat-5. Better cables may be less prone to issues but they aren't part of the gigabit ethernet standard so don't go out and re-cable your house just for a little Gig-E.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
As for fragmenting down, it might be easier to do that with a router that you actually have software control over (i.e. an old, low power linux box). I don't really have any experience with this on a home network, so...
Sujal
politics, food, music, life: FatMixx
I went through this... I bought netgear gs105 and netgear nics, all really cheap at amazon.
Like me you'll probably find you don't get a 10x increase in speed, but maybe 25-50%, like from 8 MB/s to 13 MB/s when you transfer stuff between two computers.
This is because your hard drive is fragmented, and this will completely, and drastically affect performance when you copy stuff. You don't realize it, but you will take a massive hit when you try to copy your isos, movies, etc across the LAN.
I went from 13 MB/s to like 30 MB/s after i defragmented my source and destination drives.
The main thing is that with Gigabit Ethernet, you have to think of the entire network as a system that works completely together. There has to be a complete unity between all components on your network because you will see the bottlenecks a lot easier.
Also, none of the netgear cheap stuff support jumbo frames. The more expensive NICs do, but the gs10X ports do *not* support jumboframes.
As well, they get really, really, really hot. Unnecessarily hot if you ask me, like burning to the touch, and could really heat up the inside of your CPU. In fact, even the gs105 switch is hot to the touch, too.
I instead bought 2 Intel Pro 1000 MTs. They are much more reliable, they do support jumbo frames (but I can't use it until I actually get a jumob frame compatible switch) and they don't get hot at all.
I know Gigabit Ethernet would be great for thin clients. Their really making a come back, too. It's incredible to think that you can run faster over wires than you can with the local system bus. Having Gigabit ethernet at home would be pretty wild. For windows, Windows 2000 should be fine for Gigabit ethernet, as long as you have the hardware to support it. Use Service Pack 3. SP4 has been reported to be buggy and problematic. Windows 2000 is awesome when it comes to networkability. You could have a NIC card on a Windows 98 machine with no newer drivers available. You could then use the same NIC card with Windows 2000 and there'd be a good chance that it would have updated drivers. And I don't think it's because Windows 2000 is newer than 98. It's just that Windows 2000 was made for networking. NT's pretty old and I'm not sure it wouldn't be worth while to try to get drivers from a modern system and set it up. I'm not a big fan of XP, but it shouldn't have a problem supporting Gigabit Ethernet. For best performance, check your motherboard manufacturer for the latest chipset drivers. Updatd chipset drivers work wonders on new boards.
Agent Smythe, meet RFC1323 - Long Fat Pipes. If you can use more throughput between the same machines with multiple streams, then window scaling is for you.
When information is power, privacy is freedom.
Jumbo frames are a non-standard(!) solution to the old problem of Gbit hardware not being able to handle enough frames at wire speed (or related issues, like interrupts per received frame). Modern hardware shouldn't suffer from those problems, so all the advantage to be had is a minor reduction in overhead. And the headache of trying to run a non-standard feature...
There is no way to negotiate jumbo frames between hosts, so on a single L2 broadcast domain, either all hosts should be capable of the same size jumbo frames, or none should use them at all. A router should be able to fragment them, provided the hosts are on different subnets, but most L3 routers (which you'd essentially for the required performance) won't do fragmenting in hardware.
You're probably going to get firewire cards for cheaper than gigabit ones, and I have seen demo setups with firewire wall plates so you can network your home (though I don't know if they're commercially available yet). But this would seem to be an alternative worth looking into.
First, it seems many people around here are not THAT up to date on what you can actually buy right now. It is correct that Gigabit is not really THAT useful when you're using a PCI card stuck to the 133MB/s PCI bus (although I would not consider around 60-70MB/s THAT bad compared to a standard 100MBit network card, it's still 8-9 times faster...). But you CAN buy motherboard integrated GBit cards that ARE on their separate bus right now, at consumer prices. Just look for an Intel 875P board with Intel CSA GBit, e.g. an ASUS P4C800E Deluxe. German c't magazine tested various home GBit solutions and they got around 110 MB/s over consumer priced hardware, if you just choose the right components.
:)
Second, the speed depends of course mainly on what the two sides of the connection are capable of in read speed (from disk) and write speed (to disk). If you copy files from A to B and one side is only using a cheap-ass 10 MB/s hard disk, you won't get anywhere near the theoretical maximum network speed.
I have a LAN here with my main machine being a machine with Intel CSA, and then there are three other machines - two with a PCI GBit card and one with a motherboard-integrated PCI 3com NIC. Depending on which machine copies to which machine, I get transfer speeds of 30 MB/s (copying to my old Celeron PC) to about 70 MB/s (the last only when I copy files from a machine with a fast hard drive to my main machine, which is using the CSA GBit and the SATA stripe set, which is also using a separate bus away from PCI - in this case the network speed seems to be limited by the read speed of the other machine).
So I would say that right now the home GBit is limited mainly a.) by the combined speed of hard disk and PCI GBit card being smaller than 133MB/s in the case of a machine with a PCI network card and b.) the hard disk read/write speed being slower than the max GBit speed in the case of a machine with CSA GBit. I would guess that if I had a second machine like my fastest one (both hard disk and GBit away from PCI and the hard disk stripe set being able of read/write speed greater than 100MB/s) I would finally be in GBit heaven
As far as components go - look, as was said, for the motherboard integrated, non-PCI solutions if you buy a new PC. If you're upgrading an old PC, PCI cards are OK - they are a DEFINITE improvement over 100MBit cards, even if you just read 30MB/s. As for the switch - don't buy the cheapest one, the Realtek chips (they're the ones most likely using in there) seem to have some real issues. Also, if you are noise sensitive, look for one without a fan, those little buggers can get pretty annoying real soon. I bought a 3com 5 port 10/100/1000 switch for (half a year ago) 150 Euros, and I'll probably stick another one on top of it pretty soon. That thing (3C1670500) is small, has no fan and simply does what you want it to do. And it's pretty cheap for a brand name product. And all the components which don't use GBit (like the print server, the DSL router and the Access Point) I simply left on the old 100MBit switch, so the five ports limitation wasn't really one.
It may be expensive, but I have to say I love the 6500 series, especially our new 6513s. They really are beautiful boxes. Dual SUPs, Firewall Services Module, Content Switching Module, so amazing. Check out this if you're wondering why the parent and I are giddy.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
Yeh, right. You're fancy expensive switches *might* impress the novice, but what happens when you need to support your legacy systems?
No rackmount arcnet hub, for the TRS-80 Model II that runs the thermostat software?
No Synoptics 3030 with 3 Lattistalk blades (switched localtalk) for those old Mac SE's running the custom, undocumented Filemaker db's?
What about econet, 4mps token ring or FDDI? Do you have any ATM25 or ATM155 for those Alcatel DSL modems that will do atm rather than ethernet?
My god man, where is the HIPPI? Or are you just going to leave that Cray J932 standalone? And if you do, how will you ever crank your numbers up at seti@home?
Sure, gigabit is fine for all these newer systems, but unless you want to spend millions rewriting the software that does water management for toilet flushing, then where are you at? Don't believe the X10 lies, folks... your old-fashioned wired smarthome is a work of art, and nothing is ever going to approach its reliability.
Well, "host" isn't necessarly an accurate term. We are Voyeurweb, among other things. :)
:)
:)
:) Hopefully no one will even notice, other than getting slightly better speeds.
:) You don't know what kind of nightmare that is to have running. I had to write our own sync program to keep all the servers updated. Normally we'd use rsync, but it's too big for that. Every time we'd try to run it, it would suck up several hundred Mb of ram, and then pretty much kill the machine, and still never have copied the first file. And that was 2 years ago. So there's another 2 years worth of content in there now.
:)
:) It'll probably go live sometime this coming week. We're still working out bugs. Well, not programming bugs exactly, mostly we're going back and forth changing the look and function.
:)
Through some connections several years ago, Igor hooked up with us. We used to just run small porn sites, but then Igor hooked up with us to do Watchcams.com (rest in peace).
I'm trying to make the clear distinction that we didn't "buy" Voyeurweb. There was a rumor about that for a while. It's still Igor's. We (well, my bosses) made a partnership with him, but he never lost control over what goes on the site.
The majority of our network and bandwidth are for Voyeurweb. Imagine that, a big free site, and a bunch of horny fuckers on the Internet. It's a great combinations for using up bandwidth.
Right now (well, at least on Mondays), we peak somewhere just over 1Gb/s across 3 cities (Tampa, Florida; New York, NY; Los Angeles, California). We found that we *HAD* to spread out across major cities, providers start bogging down. Tampa is particularly bad. i don't quite understand all the dirt with it, but it has something to do with Sprint handling all the last mile links in the city, so even though we are in a colo facility with a direct connection to our provider, Sprint gets overwhelmed, and our bandwidth would go flat at about 400Mb/s. Now if we detect any city doing that, we shunt some traffic over to other cities.
We're working on something else to do this more efficiently for us. Think of it like a local director on crack.
Sometimes I impress myself with Funbags.
Just made a new revision to it's sync program, so now it works even better. We were having problems with one of the guys who reprocesses the contrs to put on Funbags putting trailing spaces on some (but not all) filenames. Either I was going to fix thousands of filenames, and update the associated HTML (yada, yada, yada), or fix the sync, so I did.
Today, I'm working on a little something new for Voyeurweb. Igor dropped me one of his "I have this great idea" Emails, which means a bunch of programming for me.
"I want this.."
"Ok how about this.."
"Can we add this?"
"Sure"
"Ok, how about this?"
"That'll mean rewriting everything we just did. Ok, it'll be ready tommorrow."
At least writing Igor's stuff is somewhat fun, and I know people like it when it's done well. It's not like I'm writing forms and CGI's just to collect data "Please enter your demographics here, and you could win $20"..
Serious? Seriousness is well above my pay grade.