Slashdot Mirror
Gigabit Networking for the Home?
The Clockwork Troll asks: "I've had a whole-house audio/video distribution project on the back-burner for a while now. As gigabit networking hardware prices come down to earth, I'm tempted to jump on the 1000BaseTX bandwagon. As far as I can tell though, the current crop of consumer-priced hardware/software doesn't address a couple key issues, namely: fragmenting jumbo frames for the benefit of legacy clients - this is critical as some of the devices on my network will not tolerate the 9000+ byte Ethernet frames which are needed to get the most out of gigabit; and OS support - do Linux and Windows require much tweaking to take advantage of gigabit? Will most drivers automatically optimize themselves? A Google search didn't reveal too much consensus, especially on hardware choices. What switches and software configurations have Slashdot readers been using for home gigabit networks, in particular mixed ones (100/1000BaseTX?"
Go for the gusto: 1000baseFX!
It's a shame that even a low-latency gigabit connection couldn't keep you from FAILING IT.
Check out the 8 port Asante GX5-800P. You can find them for ~ $160.
That's gigabyte. Bit prefixes go by 1000 (10^2), byte prefixes go by 2^10.
Giga really does mean 1000. I think 1024 would be gibibit. The hard drive manufacturers do have a point about bytes and bits, you know.
Gotta shuttle all that porn around the home network huh? ;-)
It's either on the beat or off the beat, it's that easy.
I moderate therefore I rule!
--
Of course, if your needs are more extensive you may need something more...
Al Qaeda has ninjas!
Hard as I try, I can't imagine ever having enough stuff in my house to warrant gigabit. Damn.
If you have 100BaseTX with 1000BaseTX you will take a big performace hit. I worked in a data center that had to be converted to 100BaseTX because not all devices are offered in 1000BaseTX and the conversion between 100 and 1000 is a big performace problem.
Nick Powers
Encryption: I may not agree with what you say, but I will defend your right to encrypt it...
at Gigabit speeds successfully on your home LAN, your slow ass drives ain't gonna deal with the flow of bytes.
Dude that is like trying to use jet fuel in a 1984 Capri.
In Mac OS X, there's a setting right in the Network Preference Pane that is under "Ethernet" and it allows you to scale up the packet size depending on the immediately aparent network appliances. I haven't been able to use this feature because:
A: Some clients have nice network hardware, but legacy copper
B: Some clients have gig copper, but not enough hardware
I can't wait to see the transfer rates on Gig with Jumbo packets though. *Drool*
I got nothin'.
Just curious how many people have some type of storage that can pump out enough bits to even need gigabit? What kind of storage (on a budget) can be used for a project like this?
I've got an Abit motherboard with Intel gigabit built in and WindowsXP loaded on it. My GF has a Powerbook with gigabit built in. We bought the cheapest gigabit switch we could find. We got Cat 6 cable.
:-).
Everything was autodetected and the speed improvement over 100mbit was dramatic. Highest performance increase I've ever gotten for doing basically zero work (I did plug in the cables all by myself
Now, this obviously doesn't answer all your questions, but for anyone out there who doesn't have legacy issues all I can say is go for it, it's a no-brainer.
BTW, I use a Linksys WAP-Router for internet. It didn't so much as burp when we plugged it into the gigabit switch.
TW
Bit prefixes go by 1000 (10^2)
Your elementary school math teacher must be very upset right now!
I think the biggest thing about gigabit is that PCI isn't really fast enough to support it. You can shovel 133MB/second over a PCI bus, or 1064Mb.... very slightly more than a gigabit, but that's with NOTHING else happening on the bus. Generally, since the hard drive controller is also on the Southbridge, I think about the best you're going to get off most PCs, even very, very fast ones, is about 300 megabits sustained.
To really take advantage, you're going to need machines that run the network card off the Northbridge. Presumably, PCI-Express network cards will also keep up pretty easily. From what I can see, you're probably best to wait another year to eighteen months before upgrading; by then, PCI-X should be pretty common, and gigabit networking shouldn't be very expensive.
Note that I don't have any direct experience with gigabit: these are just back-of-the-envelope calculations. I could be completely off, so pay attention to replies.
No tweaking required.
Doh! That should read: Bit prefixes go by 1000, or 10^3.
I've had a good experience with a Dell PowerConnect hub (the 8- or 16-port model, I forget which). It was quite inexpensive and claims to support Jumbo Frames (however I haven't actually gotten this to work; when I enlarge the frame size on Linux it loses the connection). Oh, and I had to disable one default feature on the hub (tree-spanning something or other) to get it to work.
For clients I use Intel gigabit cards (the 64-bit PCI "server" model). I wouldn't skimp here since indications are that cheap gigabit cards don't have any hope of getting wire speed. NFS file copies max out at 20-30MB/sec, but I know that is limited by my server's disk array. I did a test for raw network bandwidth (just sending zero bytes as fast as possible) and got around 60-80MB/sec.
Everything is connected to my existing Cat-5 cable with no problems. This includes several Linux systems, one Mac and one Windows PC.
I will caution you not to expect anything like gigabit wire speeds with typical clients. My Mac G4 in particular seems to have trouble getting good bandwidth (I think the problem is either the network stack or NFS client).
If anyone has a success story with jumbo frames, I'd love to hear about it. The only references I could find are for mega-dollar Cisco/Foundry type equipment.
Now I know this is /. but before everyone says "you don't need gigabit!" and "bah, who needs that kind of speed" gigabit ethernet is genuinely useful. Even copying 500mb files can take intolerably long when you want it done 4 minutes ago. If the poster wanted a bunch of nonsense about why he shouldn't do it and why its a dumb idea, he could have gone to Circuit city (they don't sell gigabit so they would try to sell him 10/100). Instead he asked us for an informed option and information on the matter.
Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
And you have enough cash to be utilizing a JBOD to back up your pr0n, there is a problem.
What the hell are you babeling about? Gigabit is 1024 Megabits per second. What the hell does 100/1024 mean? By the way for the other people that responded to this if you want to convert to Gigabytes divide by 8. Technically a byte isn't defined to be 8 bits, but that is pretty much the defacto standard.
I have a mixed network and have not had any problems with speed or the switches flaking out.
I have 3com gigabit cards in three computers and a 3com 100Mb card in one.
One gigabit machine is a redhat 8 machine that is used as the network attached storage (NAS) box feeding media throughout the house and acting as the DNS for the house (This is so much faster than relying on your ISP!) and to filter packets for the kids computer (Damn Pr0N!)
One gigabit machine is my personal desktop.
One gigabit machine is in the family room sucking media from the NAS.
The 100MB machine is upstairs and the kids use that one.
The gigabit machines are plugged into a LanReady gigabit switch that I bought for 60 bucks Ebay.
The 100MB machine is plugged into a 3com superstack.
Both switches are then plugged into the cable router.
Speeds between the gigabit machines average 50 Meg a second depending how large the files are and if it's streaming or copying, The 100Mb box pulls 7-8 MB a sec from the others.
I'm happy with the speed.
Karma means nothing to me, so suck it...
I bet it would have made it more relevant. Too bad the cities in utah didn't have the budget to fund it. But then again, I hope private sector has improvements in the offing. I am curious, what would be the biggest improvement on the next generation broadband connection other than connection speed?
Activists United
No, really. I'm serious. Not at home, anyway.
Unless you get a very hot, brand new PC with motherboard integrated gigE, your PCI bus can't push the bandwidth. The same goes for switches. You'll be doing good to get 400 mbps out of a cheap gig switch.
Even if you have a $5000 gigE switch and a PC that can handle it, what are you going to talk to, your cable modem? The only place gigabit ethernet makes sense is when you are aggregating traffic from multiple computers to a centralized server or set of servers, and are using applications that actually require that kind of bandwidth. Even if you want to move that much data around, and have a way to do it (hint - neither scp nor samba can talk that fast), the best benefit you'll see is about double the performance you get with 100.
Here in the networking world (where I live and play), recent advances in traffic management systems have begun to punch holes in the time-worn theory that throwing bandwidth at a network problem = fixed. If you really want network performance, go check out the Linux advanced router/ traffic control site. (lartc.org) There, you'll learn to get lightning response from ssh and your first person shooters, all while running a 2gig/month web server through your home dsl's 256K uplink. And it won't cost you a dime.
who are those slashdot people? they swept over like Mongol-Tartars.
I don't think his rapid hand to penis stroking would be able to keep up with the amount of pr0n being transferred as well.
I have this friend who goes to South Dakota School of Mines and Technology. He got a bunch of free Cat-6 from one of our mutual friends, whose brother owns a audio/video installation company, so he wired his entire dorm with Gigabit. He brags about it all the time, too. He's done some other weird stuff in his day, though at least he didn't cover his entire dorm room walls with AOL CD's.
Oh, wait....
And of course, the chipset will have a backdoor username and password with no disgruntled nVidia employee will ever be allowed to see under *any* circumstances! :-)
Gigabit pipes are needed for stuff that can actually utilize it, like when you have 100+ servers needing to be backed up throughout the day to your SAN, or when you are serving out 600-800GB from your SAN to your servers. This is why you find gigabit pipes at the core and throughout the datacenter, but not from your workstations to your switches. Not yet, anyways.
Tests using PCI Gigabit chips (e.g., broadcom, 3com, intel) get around 500Mbps or so.
/. story! Of course, that controller is attached to the processor by a 6.4GB/s link!
Intel CSA attached gigabit chips (on Intel chipset motherboards only) perform better. CSA is a dedicated link from the northbridge to a gigE controller.
Of course, nForce3 250Gb integrates gigE inside, and gets over 800Mbps performance. See the preceeding
Also, PCI-X != PCIe. PCIe (PCI Express) is the upcoming high speed serial version of PCI that operates on a point-to-point basis. PCI-X is the extended faster variant of 64-bit 66MHz PCI running at up to 133MHz (1GB/s PCI essentially) in a bus configuration.
This is the major point that is overlooked when people talk Gb networks. Only with PCI-X slots do you see a major improvement in performance, and I would doubt that a home network contains even one PCI-X slot.
once you get around the IDE or SATA, the audio, the USB2 or Firewire (if we're talking video editing) etc etc etc, you would be better adding another standard network card and teaming them for your major data stores in the network and leave everything else as it is.
Also on a side note a 1 X PCI Express slot is ~250MB in each direction (about ~500MB total) so yes a 1 X PCI-E slot will do Gb ethernet fine
Normal people worry me!
Dude
Jet fuel has far less octane then your average gasoline
Try saying "it is like nitro in 1984 Capri..."
When a TCP connection is established both sides report their MSS.
If one end is "normal" ethernet it will only report 1500 btyes (while the Jumbo frame end will say 9000).
The jumbo frame comp will send 1500 byte packets because that is what the other end asked for, and the "normal" comp will send 1500 byte packets because thats it's interface's MTU.
In other words TCP will handle it for you, stop worrying about it.
Jumbo frames didn't work for me, using this hub, either.
There's no point in upgrading to gigabit unless your boxes are equipped with 64-bit PCI slots. 33mhz/32bit PCI's theoretical limit is 133Mbit/sec. This is how most consumer-grade Wintel motherboards are equipped. Thus, the 32-bit "gigabit" cards sold sold by the consumer networking outfits are a rip-off, they're barely faster tha 100Mbit.
If you're lucky enough to have a board with 66mhz/32bit slots, the data rate doubles, but all devices on the bus must run at 64mhz, since the bus will run at the speed of the slowest device.
To get a tangible benefit from gigabit ethernet, you need at least 66mhz/64bit PCI, which runs at 512Mbit/sec. This will get you half way there. To saturate gigabit, you need 133mhz PCI-X.
Other devices, like disk controllers and the disks themselves, can be bottlenecks. Better plan on upgrading to that dual-channel U320 SCSI RAID setup!
Bottom line? A true implementation of gigabit ethernet requires commercial-grade hardware, which costs plenty. The consumer stuff is little more than fluff. Unless you plan to go "all the way", save your hard-earned dough for something better, like a contribution to the FSF or an OSS project of your choice.
This isn't the sig you're looking for... Move along.
We use GigE fiber for our server networks, and pass up between 400Mb/s and 600Mb/s on high traffic days from each one.
The one thing I can say is that you'll probably never use it. There's really no need at this time. most protocols aren't any good at sucking up that much bandwidth on a single stream.
I've had many people prove this to me. They'll transfer files as single transfers. They can use up to about 10Mb/s. But if they transfer lots of files, they can use lots more. Try it through a switch that you can monitor bandwidth on. Through FTP, SMB, SCP, or whatever, you won't use up 100Mb/s. But, running multiple concurrent sessions, you can try to come close.
Heroinewarrior has a library called "firehose", which uses up all the available bandwidth, and will stripe across multiple connections to use up more. So, if you have 3 100Mb/s cards in a machine, you can come close to transfering at 300Mb/s.
You should also consider the other factors. Can your machine really send that fast? Is your hard drive fast enough to send over 100Mb/s ?? A nice fast SCSI drive, or a SATA drive can do it, but most IDE drives will fall short (specs be damned, try it in real life).
I transfer stuff around on the GigE lan all the time. We do exceed 100Mb/s, but it's usually with multiple machines.
The highest bandwidth usage machines we have are voyeurweb.com . They send out 150Mb/s through TEQL (Linux kernel option) combined 100baseTX cards, with several copies of thttpd running.
thttpd is a web server that is very small, and works very efficently. Apache has one process per connection, but thttpd has one process for everyone. Well, at least theoretically. It was around 80Mb/s of regular web site files, that it started flaking out. So, we run 4 copies of it on seperate IP's and let it scream.
As for our network, I'll outline our largest network.
We have a 1Gb/s uplink to Level3. This goes to a Cisco Catalyst 3508 (8 GBIC ports).
The remaining 7 GBIC ports go to 7 switches, mostly Cisco Catalyst 3550-48 (48 100Mb/s ethernet, 2 GBIC), and the servers are attached to the 100Mb/s ports. We have one Dell switch, which does 1000baseTX on all the ports, and a few machines with 1000baseTX cards. They can't pull anything resembling 1000Mb/s between each other. it simply doesn't happen. Honestly, doing transfers through http, ftp, or scp doesn't ever use over 100Mb/s on individual transfers. Sure, we can do it with multiple concurrent transfers, but at home, how many hundred or thousand users are you really trying to supply?
For home, you'll never use it. 100Mb/s is usually overkill. I set up my house with 802.11b, and at 11Mb/s peak, I see no difference than my old house, where we had copper run to every room and a Catalyst 2924 managing it. 11Mb/s is more than sufficent for a home network.
Spend your money on a *GOOD* 100Mb/s switch. I highly recommend Cisco, like a 2924, which you should be able to get relatively cheap used. Even if you put GigE cards in the machines, you can at least monitor your bandwidth now, and see what you really use. If you start flat-lining at 100Mb/s (bandwidth graphs make things really obvious), then you could consider upgrading.
Serious? Seriousness is well above my pay grade.
I was fortunate enough to buy 2 TI-Chipped Firewire cards ($20 total) and use them to network my main WS to my Server w/ 5-foot cable. You can save a lot of money going this route if you can. MM
omg my '84 Capri would be r0x0rz with nitro jet fuel! How did you do it?????
Will it work in a '79 gremlin too????
I'm not quite a Macophite yet, but for throughput Apple has been thinking ahead. Since the Beige G3 Apple has had 64-bit PCI slots and the latest release of the G5 has PCI-X. Also for about 3 years almost every machine has come standard with Gigabit Ethernet. While the bottle neck is really not the networking, it is something I can't even find on any home machine from any of the computer companys.
I would agree with many previous posts about Gigabit having no real purpose at home yet. But I when they do have a purpose and a need I'm certian I'll still have a PS2 port and 32-bit PCI.
Who needs to preview
Jet fuel is kerosene. A 1984 Capri wouldn't run on it at all.
--I am Sun Tzu of the Borg. Resistance is feudal.
On a 100BaseT NIC the theoretical max transfer rate is 12.5 MBps with a realistic speed of 8 MBps. Multiply that by 10 to get a rough estimate of Gigabit speed. Most ATA HDD's can transfer around 40-60 MBps. You can easily saturate a 100BaseT network with bargain basement machines.
Gigabit Ethernet is faster than what your typical ATA drive will absorb, but it is still going to be quite a bit faster than 100BaseT.
Spend the Money on a nicer HDD or a decent RAID setup and you will be able to make full use of a Gigabit pipe.
Why is this rated insightful?
Sure, the guy can do basic math, but the
entire premise is wrong. Just because you
can't get the full gigabit on your current
computer DOESN'T MEAN that you can't get
enough more bandwidth to make the installation
worthwhile.
If something you spend a lot of time doing
can be sped up by a factor of 3, are you going
to complain you didn't get a factor of 10?
A friend of mine just went nuts when he found out about a new switch from SMC, the SMC8508T. While it's unmanaged, it offers non-blocking architecture across the entire line as well as support of jumbo frames up to 9K, which is extremely unusual for SOHO stuff. Not even a lot of expensive Cisco stuff does jumbo frames. And he paid $150 for it.
Why should you care about jumbo frames? I found this nice guide about that here.
-R
One of the boards I read a review for a while back (was it the new nForce3 stuff also on slashdot today?) had a separate dedicated bus for the Gb ethernet so it didn't ever go to PCI. There might be more of that in the near future, while we're waiting for PCI-X
But you don't have to be able to pump the full 1000 Mbps to take advantage of gigabit ethernet. As long as you can pump more than 100 Mbps, then gigabit will give you a speed improvement over 100 Mbps ethernet.
(Or, a good location for the ceiling is "anywhere above your head").
There's a company called ServeSoft (they have a weird url that I can't remember at the moment) that does exactly what you're looking for. They have ServeSoft branded Gigabit switches that are a great value (12 ports for $110 I think). I bought some things from them, but not anything fancy (I just needed to get video from my home server to a single home theater PC, not multiple machines which I gather is what you need). Beware though, they don't have a return policy for working items. I'm glad I liked my stuff and didn't get stuck with it.
It really does matter how the PCI buses are set up in the machine, and how many times the data has to cross the same PCI bus. A bigger question, though, is where the data is going to and coming from: if it's a fairly normal disk subsystem, then that will probably be the choke point. If the network subsystem is doing a data copy on each transmit or receive, that will hurt too.
Another consideration is that using standard ethernet frames (not jumbo frames) can dramatically increase the cpu overhead. Throwing a gige nic into an old machine will probably not saturate the PCI bus, for this reason.
In networking: giga = 1000
When refering to any type of computer storage: giga = 2^10.
This is mostly because computer storage is addressed by a processor in some way and processor registers happen to be binary storage devices.
You can't build a 1000 byte RAM chip and expect to address it without doing a calculation to distinguish a valid address from an invalid one.
A 1024 byte RAM chip makes it simple. Just connect 10 address pins to it and any combination is valid.
Networks don't use the 2^10 convention because their rates are not required to be based on powers of 2.
If you're using a 32-bit, 33MHz PCI bus, which nearly everything does, the top speed is going to be 400Mbit. OTOH, that's going to be the top speed of your drive controller too. I got a dual-P3 Tyan board from Ebay and intend to run a 66MHz RAID controller in one 64-bit slot and a 64-bit SysKonnect Gigabit Copper Ether card in another. My hub is an Asante GX5-800 (I beleive the internal switching is 7GBit, the "800P", recommended earlier, doesn't have a published internal speed). I was hoping that "Extreme PCI" and the BTX form factor was going to make it easier and the parts more mainstream, but that's been slow in coming and I got impatient. I computed that 640x480 pixel, 24-bit images in a 60-frame-per-second slideshow, uncompressed, required 600MBit's of throughput and need about 400GB of storage to hold two hours. I do intend to use compression, and I've found that my animation tools fall apart at 60frames per second, but that was my goal for the network and server. It wasn't possible when I wrote the spec, but now it's nearly cheap! Happy Networking, -Mark THIS TAGLINE INTENTIONALLY LEFT BLANK
The least expensive switch I have found that support jumbo frames are from SMC, the SMC8505T http://tinyurl.com/3by3v and the SMC8508T http://tinyurl.com/nhaz. The links are to the smc site. The 5 port version is approximately $100-120 and the 8 port is $140-$150. SMC also has 16 and 24 port versions. As far as support for Jumbo frame support Windows 2000/XP and Linux both have them as long the NIC has drivers that support them. I know the major NIC manufacturers like Intel, Broadcom, and 3Com have driver support for them. One tip: if you are using Dells with Intel 1GigE embedded on the motherboard make sure to use the latest drivers from the Intel support site since the default Windows drivers from Dell do not show the Jumbo frame option. As far as the optimal Jumbo Frame size, that would depend on the type of traffic you are carrying. Simply putting in 9K frames on everything might not be optimal. It will take some experimentation to find the right sizing.
I have a NetGear 4 port gigabit switch. I have found I can transfer files about 2.5x as fast as with 100mbit (without jumbo frames). In my book, that's worth the few extra bucks a gigabit switch will cost you.
A warning though, I've heard most of the cheap gigabit switches have fans in them. Fans reduce the reliability of a switch many fold and make them LOUD. I like my 4 port Netgear and they now make an 8 port version which is also fanless and very reasonably priced.
Does anyone have a Linksys or D-Link gigabit switch who can confirm or deny the presence of a fan?
One note I'd like to throw in: Gigabit ethernet requires Cat-5 cable. Not Cat-5e, Not Cat-6, Cat-5. Better cables may be less prone to issues but they aren't part of the gigabit ethernet standard so don't go out and re-cable your house just for a little Gig-E.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
Jet fuel has an "octane rating" of 100-130, far MORE than automobile gasoline.
That is why Intel i865/i875 has the option of direct connect e1000 gigabit (CSA) to the northbridge. Most motherboards with gigabit built on that uses either of those chipsets use the e1000 CSA gigabit chips.
The only reason you are seeing 500Mbps or so is because the PCI bus segment the NIC on is too slow. If you are running at 64bit/66Mhz PCI then you should see up to 940Mbps (theoretical maximum for TCP throughput over GigE because of overhead) with a PIII 1Ghz or faster processor going single direction.
Also, PCI-X 2.0 has DDR and QDR versions that double or quadruple the data rate just like modern processor FSB. Plus, PCI-X is much much more efficient than PCI. You get about 60% bus utilization out of PCI, and about 80% for PCIX.
Because of that, if you aren't using anything else on the PCI 32/33 segment, you will see about 633Mbps of throughput, which is why you get 500Mbps or so.
For GigE to run at full line-rate, we are actually talking about needing 2Gbps because it is full-duplex and it is 1Gbps each direction. Therefore 64bit/66Mhz PCI just about covers that.
Of course PCI-E (depending on how many lanes) will be much faster. Typically high throughput devices such as video cards will have at least x8 lanes.
AHAHAAA HAHA HAAAAAA!!!!
Umm "Dude" no it doesnt .ca division
AVGas is typically has a 100 Octane rating (r+m)/2
Jet A, Jet a-1 and Jet b run from 100 - 130 Octane ratings
Fightercraft effectively run kerosine aka jet a-1 with additives at 130 Octane+ ratings. so I don't know where you buy your fuel but if you are snagging 100+ octane rated fuels let me know 'cause I will increase the compression on my STS or add a paxton centrifugal supercharger and come visit. For more information on fuel grades check out our friends at the Royal Dutch Shell Oil company's
Bad Panda! No Bamboo for you! In matters of importance ACs will not be responded to. Want to say something critical,OK
The Screen Savers did an on-air demo of 2 pairs of identical machines (hard drives, processors, RAM), one pair with 100 NICs, and the other pair with Gigabit....
They were transfering files between the pairs, and on a 500 MB (roughly) movie file, the Gigabit was done transfering in about 1/5 the time.
It was enough to convince me that I wanted Gigabit in my home. So far I have the Cat6 wired to 4 rooms of my house, all ending in my coat closet. Next I need to purchase a router. (My Dells came with on-board Gigabit)
Most ATA HDD's can transfer around 40-60 MBps. You can easily saturate a 100BaseT network with bargain basement machines.
I beg to differ. The numbers you quote there are empty benchmarks of an ATA drive alone within an OS and a benchmark tool, or some OS-less independent method devised by manufacturers across and IDE bus. I said the drives couldn't handle it.. any sustained transfer at that rate, even if the drive would support a streamed write for a sustained period, being fed at "good" gigabit speeds of ~800Mb/s, would surely melt the drives. But in practical terms -- (I should not have said just the drives, earlier) -- the pure drive metrics are useless for this discussion. You must take into consideration that all of the following will destroy every good number you might have had: the data (or file) transfer method, the capabilities of the OS itself, and how it's tuned, the application in use, and how it handles checking, transmits, and writes, any number of ethernet based faults, retransmits, etc, IP fragmentation, packet reconstruction, TCP window size and frag size tuning (or lack thereof), the position of the moon at the start of transfer..
IMO gigabit in house is a waste. Take the money you would have spent on a switch and NICs and buy some good champagne and cigars.
I dont get it. People here are bitching that the best throughput they see on gigabit ethernet is 400Mbps. Thats 4x the speed of regular 100Mbps ethernet. 4x still seems like a hell of an improvement, especially when you consider gigabit switches can be had for $100-150. I'd take a 4x faster HDD, processor, memory, etc anyday! Why snub your noses at at 4x network speed increase?
As for fragmenting down, it might be easier to do that with a router that you actually have software control over (i.e. an old, low power linux box). I don't really have any experience with this on a home network, so...
Sujal
politics, food, music, life: FatMixx
We use an HP Procurve switch, and it has blades you can put in to add different functionality. Now, we have a 100Base network, though our Netapp needs 1000Base. Since we were ordering it, we picked up 3 1000Base cards for the Procurve, and no throughput depreciation. If you're willing to pay the cash for a Procurve in a datacenter, along with the cards, that's the way to go.
-- This space for lease, low setup fee, inquire within!
I can transfer large files from one machine to another at arround 25mb/s with my gigabit switch. Try doing that with 10/100.
Just because you can't saturate the gigabit doesn't mean you won't get a benefit from it.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
I'm p[retty sure modern Southbridges have a dedicated link to the Northbridge (think VIA's V-Link, SIS' MuTIOL, or nVidia's nForce2 Hypertranport from southbridge to northbridge), so if IDE port are supplied by the Southbridge they get dedicated bandwidth separate from the rest of the PCI traffic. Also the same goes for LAN adapters and sound processing integrated into teh Southbridge (again, nForce2 APU).
I went through this... I bought netgear gs105 and netgear nics, all really cheap at amazon.
Like me you'll probably find you don't get a 10x increase in speed, but maybe 25-50%, like from 8 MB/s to 13 MB/s when you transfer stuff between two computers.
This is because your hard drive is fragmented, and this will completely, and drastically affect performance when you copy stuff. You don't realize it, but you will take a massive hit when you try to copy your isos, movies, etc across the LAN.
I went from 13 MB/s to like 30 MB/s after i defragmented my source and destination drives.
The main thing is that with Gigabit Ethernet, you have to think of the entire network as a system that works completely together. There has to be a complete unity between all components on your network because you will see the bottlenecks a lot easier.
Also, none of the netgear cheap stuff support jumbo frames. The more expensive NICs do, but the gs10X ports do *not* support jumboframes.
As well, they get really, really, really hot. Unnecessarily hot if you ask me, like burning to the touch, and could really heat up the inside of your CPU. In fact, even the gs105 switch is hot to the touch, too.
I instead bought 2 Intel Pro 1000 MTs. They are much more reliable, they do support jumbo frames (but I can't use it until I actually get a jumob frame compatible switch) and they don't get hot at all.
Speaking from personal experience?
I know all that! I was replying to the original post which was talking about standard PCI and low throughput, and I was giving some real world figures to his speculated figures, and explaining that gigE is available already on motherboards in non-PCI limited versions (Intel CSA, nVidia integrated, etc). I also corrected the common statement that PCI-X is PCIe.
PCIe graphics cards will be 16x from the start. It looks like 1x, 4x and 16x will be the common configurations (and not 2x, 8x and 12x which are the other options). 1x gets 250MBps in each direction (more than enough for a discrete gigE controller), and 4x will get 1GBps in each direction.
To your point about disk though, my two main servers are 64-bit/66MHz PCI boxes with striped arrays that seem have enough sustained throughput to saturate a gigabit link with just a few clients' worth of video demand. And at this point my equipment is virtually antiquated - I suspect the striped WD Raptor SATA arrays that are typical of many power users' servers could do the job just as easily.
There are no karma whores, only moderation johns
That's what the external WLAN HDD is for. Duh.
"Um, it's not mine... I, er, was just browsing"
if you are going 100 mbit, get a good switch. I found how how large a hole my linksys was when our cable operator upgraded from 3 megabits to 12 megabits. when I called to bitch about the speed, I couldn't get more than 300 k/s on a download, the elightened tech said your router sux. So I set up a linux router and I'm getting great downloads and the lan speed went from about 3mb/s average to about 7mb/s when copying dvd rips around. If I hadn't learned this I probably would have gone to GigE by now. That said when I move a lot of DVD rips around from one machine to another even 7mb/s isn't fast enough for some of us twitch freeks and gigE would be handy.
I have a gig network at home... The key here (as others have mentioned) is gig is useful if you transfer large files and have a relatively new PC. AMD chipsets with HyperTransport and Intel 865/875 chipsets take the onboard gig and take it OFF the PCI bus. This is a good thing.
Make sure you have Cat5e (or 6), a gig switch, and new PCs, then your gig network will function with good speeds.
Bear in mind that having a slow hard drive will slow down your file transfers too...
Why should I bother? Unless I've got a nice big fat SCSI RAID on each machine in my house (consisting of six or so Linux and Windows machines) and constantly swap data between them, whats the point for a home user like me to invest in new hardware? I've run stats on transfers, and the most I've gotten out of my two strongest computers on Crossover running Gentoo Linux is 60-75% utilization. If I were to get SATA drives and a Dual-Channel RAM mobo, then sure, Gigabit MIGHT be worth it. But until then, 100Mbps is just fine for me.
When refering to any type of computer storage: giga = 2^10.
Not disks.
I've got news for you, your HD probably does 40Mbit if you're lucky, so HTF are you gonna fill a 1000Mbit pipe? 100Mbit switched is more than enough for even the largest files on a household network. Where are you gonna keep 1000Mbit of data every second? RAM? I hope you've got a terabyte of it and one serious kernel patch to support it on i386.
Unless your transfering data from RAM on PC1 to RAM on PC2, you'll never see full utilization; your disk IO will never keep up.
I am using a Netgear GS108 gigabit switch, coupled with a Netgear FS108 with no real problems. Servers and workstations are on the GS108, and print servers and the gateway router are on the FS108.
One of the big things you are going to find about GigE is when you're doing something like copying a file, you don't get 10 times the speed. GigE is fast enough that your hard drive's ability to read and write data becomes the limiting factor. When copying files between two machines, both having the identical 80 GB Seagate 7200.7 drive, I clock in about 3 times the speed with the GigE than I do with 100-Base-T. Despite this, however, there has always been enough bandwidth to stream video between to machines in real time.
HD streams are ~ 20Mbps MPEG2. I save HDTV programs to a file server over a 100Mbps network. Two is no problem.. three or four streams could run into problems if other traffic hits the network.
Nothing that I've run into in multimedia streaming needs real high bandwidth. It's only the bulk transfers that will fill the pipe, like backing up a system over the network.
But, it's always good to have headroom. Gig-E is cheap enough now that it makes sense for the high-tech home.
I agree with what you say about 100Mb/s being sufficient, but 802.11b is not good enough. I need 802.11g to stream video.
A waste of what? Hardly money anymore, if you have a small network. And what are you suggesting - that hard drives can literally not break 8MB/second in practice? That is absolutely ludicrous. Transer a Gig or two over a 100-base network between decent machines, caclulate the time it takes, and *then* tell me that the network isn't the bottleneck. The guy was right, you're not gonna saturate the 1000-base network but you are gonna return the bottleneck where it belongs - to the hard drives.
I've got a gigabit network set up based on an 8-port Ovislink Gigabit Switch and Ovislink Gigabit NIC's. After one bad toss with a faulty NIC, everything works fine now. We have a RAID0 server and several machines, and seeing as we tend to download a lot as well as watch a lot of movies, the GE pays off. The speeds are high enough that I wouldn't even bother worrying about tweaking unless you really need it; you're only going to notice the difference if you need to shovel over vast amounts of data from one comp to the other on a daily basis.
:)
Also, unless you have SCSI's, I think GBE is still marginally faster than most EIDE setups (they pull what, 150mb/s? As opposed to full duplex 2Gb/s, divided by ten for easy math, makes around 200mb/s?) So it may also be pointless to tweak.
On the other hand, a GBE NIC costs 20 bucks or so, so upgrading the whole house in one go isn't the colossal investment it was a year ago, either. Sticking to all the same manufacturer and setting jumbo frames on will probably already get the most out of your setup already. Just juggle your variables and see which come out on top
- Jynx
A positive attitude may not solve all your problems, but it will annoy enough people to make it well worth the effort.
My home network consists of the following:
A 6Mb DSL circuit
Cisco Catalyst 3550PWR-24 Switch
Cisco 1751-V Router with 2 FXO's running CallManager Express (3.3 beta)
Cisco Aironet 1100 802.11g IOS AP
6 Cisco 7960 Phones and 1 7920 WiFi Phone
It ain't much. I have separated voice, data and guest VLANs each with their own dedicated wireless SSID (and WEP keys) and quality of service on the LAN and WLAN. I need some 1000baseTX GBICs so I can plug in the gigabit NICs on my Athlon 64 and Compaq Evo laptop. I'd also like to get IOS firewall feature set fired up so I can authenticate my guest users in a walled garden and control their outbound Internet access. Keeps the punk in-laws from running Kazaa!
I won't have much time to play or study the next couple of weeks, because I'll be learning how to wire and install phone lines! w00t!
This thread from anandtech deals with the same issue. There are some benchmarks in there if you scroll down and onto the next page. Check it out, it may give you some ideas of what you guys want to do.
Well, if GigE is effectively twice the bandwidth of 100Mbs, then why not just slap two ethernet controllers together for channel bonding?
If you have intel nics, you can use their bonding linux drivers(which may give you double the bandwidth).
SMC switch and SMC cards are up to the mark. One thing I could not do is get windows machine go any faster than 180MBits on File copy from a Samba machine.
Did anyone have a Samba benchmark of copying a 2GB file where you are copying from samba server to the windows machine?
- People who believe other people have no right to live, got no right to live ...
I know Gigabit Ethernet would be great for thin clients. Their really making a come back, too. It's incredible to think that you can run faster over wires than you can with the local system bus. Having Gigabit ethernet at home would be pretty wild. For windows, Windows 2000 should be fine for Gigabit ethernet, as long as you have the hardware to support it. Use Service Pack 3. SP4 has been reported to be buggy and problematic. Windows 2000 is awesome when it comes to networkability. You could have a NIC card on a Windows 98 machine with no newer drivers available. You could then use the same NIC card with Windows 2000 and there'd be a good chance that it would have updated drivers. And I don't think it's because Windows 2000 is newer than 98. It's just that Windows 2000 was made for networking. NT's pretty old and I'm not sure it wouldn't be worth while to try to get drivers from a modern system and set it up. I'm not a big fan of XP, but it shouldn't have a problem supporting Gigabit Ethernet. For best performance, check your motherboard manufacturer for the latest chipset drivers. Updatd chipset drivers work wonders on new boards.
Another thing to worry about with GigE is that each frame recieved needs to send an interrupt to the cpus. If you increase your interrupt throttle on any card with decent drivers, you take more cpu power but also get increased performance.
With 100MBit networks, the performance hit of these interrupts are negligible but that's not the case with faster networks.
Jumbo frames should help with this but even on a network with all the same high end Intel cards and all the same SMC switches, we still saw drastically reduced network performance when they were enabled. I don't think they work at all the way they're supposed to between different vendors.
The Intel e1000 drivers that we use in linux started auto adjusting their interrupts with the 2.6 kernel and we found that it resulted in shitty performance. By manually tweaking the InterruptThrottleRate option on the module, we got the best bandwidth to performance ratio. It seems like Intel probably tunes their drivers to work best under sporadic activity though, while we needed performance for long periods of high load.
Of course, I only have experience with the e1000 cards so YMMV.
"Now I know this is /. but before everyone says "you don't need gigabit!" and "bah, who needs that kind of speed" gigabit ethernet is genuinely useful. Even copying 500mb files can take intolerably long when you want it done 4 minutes ago."
So really by that logic, then even Gigabit will not be good enough. What if he wants it one minute ago? Or three seconds ago? Or one tenth of a millisecond? Remember "people will never be satisfied with what they have". No HD will be big enough. No GPU or CPU will be fast enough. No case will be nice enough. No woman will be pretty enough.
HTF are you gonna fill a 1000Mbit pipe
Errm, probably from the IDE or SCSI raid array that you're likely to have if you're considering Gb networking. The limiting factor isn't the disk speed, but the PCI bus speed. On the presumption that you're spending a wodge of money on GigE, you'd probably have spent money wisely on a server and got PCI64, or at least 66MHz PCI32
A good RAID can beat 100 megabytes a second. This will saturate a 1000Mbit connection. xbitlabs.com has some benchmarks on various RAIDS and RAID controllers that I found interesting.
I bought a $100 5-port Netgear non-copper Git-E switch and a $39 non-copper gig-e card. My new computer was an i875P which had a gig-e on board. I'm burning a lot of TV shows to DVD. Now I'm actually bottlenecked by the HDD. I can copy over the network as fast as I can disk to disk. Now need faster disks!
'I don't think anyone needs more than what I've used'. Please, put a bullet in your head.
Spend the Money on a nicer HDD or a decent RAID setup and you will be able to make full use of a Gigabit pipe.
No you won't, because you spent the Money on drives, so you can't spend the Money on that Gigabit pipe. Duh.
I'm still using 1.5 Mb ArcNet, you insensitive clod!
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Actually if you put everything that's been happening for three years, then you can see were the home is heading. Everything is becoming integrated, and low cost. Software is maturing and becoming low cost. Computers are showing up everywere from our clothes to our cars, to our kitchen, making it all "smart". From wired to wireless. Imagine having an enterprise quality computer(s) (maybe a blade design) with a couple GigaE lines going out to the rest of the house some ending in desktop clients, others ending in WAP, with some specialized "application appliances" scattered around. Might even be a TV, or Radio on the end. Everything able to talk to everything else. From Laptops and Cellphones, to PDAs, to our clothes. In house HDTV VOD. Quadrophonic sound. Cooperative models isn't just for OSS. MS showed some of what can be done in a house of the future. But here we are, and the future is nearer than we think.
I regularly get about 10.5 MB/s over 100BaseT using scp, despite the encryption (this is between fairly modern FreeBSD and Linux machines with decent CPUs).
I have several machines that have gigabit-capable NICs, but I don't think I need that kind of transfer rates often enough to make it worth looking at...yet.
"Three words...
Multiple HD Streams"
Ok, so that's Left eyeball, Right eyeball, and Third eye.
Jumbo frames are a non-standard(!) solution to the old problem of Gbit hardware not being able to handle enough frames at wire speed (or related issues, like interrupts per received frame). Modern hardware shouldn't suffer from those problems, so all the advantage to be had is a minor reduction in overhead. And the headache of trying to run a non-standard feature...
There is no way to negotiate jumbo frames between hosts, so on a single L2 broadcast domain, either all hosts should be capable of the same size jumbo frames, or none should use them at all. A router should be able to fragment them, provided the hosts are on different subnets, but most L3 routers (which you'd essentially for the required performance) won't do fragmenting in hardware.
I don't think they would be open to false advertising claims, no. Not if they just said "Gigabit Ethernet" - which is when it comes down to it merely a name for a standard. If they said "Capable of full gigabit-per-second speeds" then I expect the false advertising claim /might/ have a chance ... but I doubt it.
After all, if a 400Mb/s "gigabit" switch is false advertising, then "108Mb/s" WiFi would be <i>doomed</i>. What a pleasant thought. "108mb/s, when running in full duplex with full utilisation of both directions, in lab conditions, including all link-layer overheads, without encryption, maybe." By those idiot's logic, 1000baseTX would be "Two Gigabit Ethernet!". Show me two-gigabit throughput in normal usage situations, and I'll care.
that's assuming there isn't any other traffic running on your network. HDTV will bring it up to ~19Mbit, which might leave little room on the pipe for other people.
I make backups over the network and everything moves faster. Gkrellm shows 30mb/sec over the interface (and that's about right, the max speed of the servers harddrive.) While I can't fully take advantage of gigabit (as my harddrives aren't that fast) it's still a worthwhile upgrade in my opinion. I got 3 nics and an 8 port gigabit switch under $200.
If you don't want someone to copy something, don't give it to anyone.
There's a lot more to do than watch video. Burning files from 100Mbps to DVD i get a maximum safe speed of about 4-5x. The burner is 8x, and it would certainly speed things up to be on Gigabit. Now if two of us were burning DVDs grabbing files from a server, i'd hate to think of the buffer underruns a 100Mbps connection would serve.
[note: Mb = Megabit, MB = megabyte]
... while you don't need gigabit ethernet, others may - and it can work very well indeed.
While I agree that for many usage patterns, 100Mb/s is more than enough, this is not true for everybody.
As for 11Mb/s (802.11b) being sufficient - that's probably true if you're using your LAN for basic 'Net access, synching documents between machines, sharing small files, etc - but rapidly falls down under any greater loads. For one, you're on a shared medium - and _4_ 802.11b clients starts to suck in a hurry. Many things do see large improvements when run over faster networks than basic WiFi, too. I find this particularly noticeable with remote X applications (which I use a lot) and huge print jobs.
I also do a lot of work with disk images. When you're lobbing 20GB images around a lot, you start to enjoy gigabit ethernet. While it's true that disks are almost always incapable of meeting the full demands of a gigabit link, they ARE capable of going faster than 100Mb/s. As such, I see a huge improvement with gigabit.
Additionally, disk issues depend on the workload. A basic IDE or SATA disk will miserably fail to cope with 10 concurrent random read/write threads - I've seen them fall below 10 MB/s. On the other hand, a decent SCSI disk or a good SATA disk with TCQ will often cope very well. A good OS helps a lot, too.
On the flip side, my bog-standard Western Digital and Maxtor 7200RPM 120GB disks can easily chuck 40MB/s over the wire in single-client streaming reads - the Maxtor sometimes clocks 50MB/s. When imaging a machine, this is very nice indeed.
So
Fragmentation can be a reason for poor performance, but it depends on what you're doing, and what filesystem you're using. If you use FAT32 or HFS, then of course framentation will be a major problem, as the FS makes no effort whatsoever to reduce it.
If, on the other hand, you use NTFS, ext2, ext3, reiserfs, UFS, or any other modern filesystem, you will probably find framentation to be much less significiant.
Random-file I/O will always be much slower than sequential transfers of large files, of course - hence the issue of different workloads.
If your disks are getting that hot, too, you NEED TO COOL THEM BETTER OR THEY WILL FAIL. Set up at least dedicated fan cooling for the disks. My case has a cage for the disks in the lower front, with two large fans blowing over them - my disks only get a little hot when under very heavy loads.
Before messing with jumbo frames, make sure your TCP window sizes are sensible. Many OSes set them up stupidly, and an appropriate window size can make a massive amount of difference to throughput.
Likewise the 2.4 GHz devices in your home (microwave, 802.11 networking, cordless phones, etc) all use radio waves in the 2.4 * 1000 * 1000 * 1000 cycle/s range.
Computer storage is the oddball here. 1 Megabyte is 1024 * 1024 bytes not because mega == 1024^2 but because it's easier to design computers with powers of two in mind.
Communication equipment on the other hand uses the standard meaning of the terms.
My LAN at work works wonderfully in a mixed 1000baseTX/100baseTX environment. The core switch has two gigabit uplinks (to the core servers), and all other clients are on 100baseTX ports.
I haven't observed any performance degradation for the 100Mb/s clients, and when communicating with the servers over gigabit things work extremely smoothly, with six clients often getting full speed transfers at once (not just file access; we use remote X and a lot of other things on our servers).
>I can transfer large files from one machine to another at arround 25mb/s with my gigabit switch. Try doing that with 10/100.
:-)
My 10/100 card can manage 25 millibits per second just fine
--
Jon
You're probably going to get firewire cards for cheaper than gigabit ones, and I have seen demo setups with firewire wall plates so you can network your home (though I don't know if they're commercially available yet). But this would seem to be an alternative worth looking into.
I also opted to mirror the media instead of going the backup route, Since I set the backups as a cron that runs when I sleep, I'm not aware of the time it takes copy the updated files. According to little meter on gkrellm, I'm getting about 10-12 MB/s thoughput (oddly I sometimes get a reading that says it's copying at 14.1MB/s. A friend claims explains that the reason why this is greather than 100Mb/s is because of compression at the NIC level) and this is roughly confirmed with howlong it takes to copy a 1.1 GB feature:
patrick@pappy > time cp pirates_of_the_caribbean_xvid.avi ~/ real 1m54.645s user 0m0.230s sys 0m17.740s [~/media/movies/] patrick@pappy >
I can transfer large files from one machine to another at arround 25mb/s with my gigabit switch.
;-)
25 millibit per second?
Try doing that with 10/100.
Any time.
You can shovel 133MB/second over a PCI bus, or 1064Mb.... very slightly more than a gigabit, but that's with NOTHING else happening on the bus.
Don't forget that 1000baseT transfers 1Gbps in both directions, so potentially you need 2Gbps of bandwidth over the bus. And as you said the HD controller is also on the south bridge so if you copy stuff from disk to network you'll need more than that to fully utilize the hardware.
Hopefully PCI Express with give us more bandwidth. If you must buy now make sure you get one on CSA and not just a on-board chip on PCI, it doesn't cost much more and makes quite a difference.
Too bad CSA is Intel-only for the moment.
You must not have bought disks for *years*!!
The low-end server that arrived last month is reading over 500Mbit/s from its disks.
A faster system, maybe with RAID, would do well over that.
Well, as they state themselves :
http://gentoo.org/~spider/netgear.txt
I didn't do this, now did I?
Maybe somebody is working on such a standard, which is said to be gigabit over 2 pair of copper wires. However, almost all the current stuff on the market are 1000baseT, which is Gigabit over 4 pairs of wires in a CAT5e cable.
Do a dd if=/dev/hda of=/dev/null, let it run for a while, then Ctrl-C. The data rate it shows will quickly disabuse you of the notion that an ATA HDD will operate *sustained* at 40-60MBps. They can certainly burst a high rate (thanks to the cache), but sustaining that rate is a whole different story.
Most ATA HDDs won't sustain a data rate capable of saturating a 100Mbit/s network, let alone a gigabit network.
Oolite: Elite-like game. For Mac, Linux and Windows
Boy this turned into a bit of a tome.
/proc/sys/net/ipv4/tcp_rmem /etc/samba/smb.conf
For a switch I went with an 8 port SMC EZSwitch 8508T. I chose it since:
1. It supports jumbo frames. According to my testing it will pass ethernet packets up to 9212 bytes which should correspond to a 9198 byte MTU.
2. It doesn't have a cooling fan. A definate plus since in my experience the little fans in switches such as this can become quite annoying as they age.
3. It comes with rack mount ears.
4. It's affordable. I purchased it from Securemart.com for $139.31 shipped. Ordered it Thursday or Friday, it arrived Monday or Tuesday.
As to NICs, one of my PCs already had an Intel gigabit port on the motherboard. In addition I purchased 4 more Intel Pro 1000/MT Desktop Adapters. Since:
1. They have good driver support on both Linux and Windows.
2. They support jumbo frames. Supposedly up to around 16000 bytes.
3. They're supposed to be pretty fast/efficient. It's kind of dated but you can find a comparison of some 32-bit gigabit NICs here.
4. They'll do 66Mhz if your motherboard supports it and of my systems does.
5. They have DOS NDIS2 drivers so I can use Ghost to make/restore images over the network.
One I purchased through Intel's evaluation program for $35.31 shipped. As I recall it took over a week to show up. The other three I ordered from OnlineMicro for $28 each plus $11.32 shipping. Be sure to change the shipping option from ground to 2 day air if you order more than 1, it's cheaper. They shipped them out the day of my order and they arrived on time.
One of the Intel NICs died about 4 hours after I installed it. I swapped it with another and the replacement has been working fine for a few weeks now. I ran the diagnostics on it and other all but the link test passed. When the OS is booted up the switch shows no link lights but sometimes when the PC is off the link lights do come on. I've also tried it in another PC where it exhibits similar symptoms. I haven't yet contacted Intel about getting it replaced.
I spent a lot of time tweaking various things. Some findings:
1. With default SO_RCVBUF sizes a MTU in the neighborhood of 4000 or so bytes seems to get about the best network/application wide throughput. Specifically the otherwise fast NF7-S system below would lose almost 50% throughput with 9000 byte MTUs with the default SO_RCVBUF size. Linux to Linux lost around 30% as I recall.
In theory you can change the default SO_RCVBUF size on linux by echoing appropriate values to:
Other than that you appear to have to change this setting in each individual application. One application of note that allows you to easily make this change is samba. See your:
2. If you crank the SO_RCVBUF size up to 200ish k or more then a 9000ish byte MTU can eek out another 5ish percent more bandwidth. Thus for the moment I've decided to just stick with 4076.
3. MTUs that are not of a size of the form 8x+4 cause Linux to behave oddly when it performs path MTU discovery. Namely for jumbo sizes that don't fit that form the discovery decides that the PMTU is 1492. You can read more detail about it in a Usenet post I made here. I still don't have a good picture of what'
First, it seems many people around here are not THAT up to date on what you can actually buy right now. It is correct that Gigabit is not really THAT useful when you're using a PCI card stuck to the 133MB/s PCI bus (although I would not consider around 60-70MB/s THAT bad compared to a standard 100MBit network card, it's still 8-9 times faster...). But you CAN buy motherboard integrated GBit cards that ARE on their separate bus right now, at consumer prices. Just look for an Intel 875P board with Intel CSA GBit, e.g. an ASUS P4C800E Deluxe. German c't magazine tested various home GBit solutions and they got around 110 MB/s over consumer priced hardware, if you just choose the right components.
:)
Second, the speed depends of course mainly on what the two sides of the connection are capable of in read speed (from disk) and write speed (to disk). If you copy files from A to B and one side is only using a cheap-ass 10 MB/s hard disk, you won't get anywhere near the theoretical maximum network speed.
I have a LAN here with my main machine being a machine with Intel CSA, and then there are three other machines - two with a PCI GBit card and one with a motherboard-integrated PCI 3com NIC. Depending on which machine copies to which machine, I get transfer speeds of 30 MB/s (copying to my old Celeron PC) to about 70 MB/s (the last only when I copy files from a machine with a fast hard drive to my main machine, which is using the CSA GBit and the SATA stripe set, which is also using a separate bus away from PCI - in this case the network speed seems to be limited by the read speed of the other machine).
So I would say that right now the home GBit is limited mainly a.) by the combined speed of hard disk and PCI GBit card being smaller than 133MB/s in the case of a machine with a PCI network card and b.) the hard disk read/write speed being slower than the max GBit speed in the case of a machine with CSA GBit. I would guess that if I had a second machine like my fastest one (both hard disk and GBit away from PCI and the hard disk stripe set being able of read/write speed greater than 100MB/s) I would finally be in GBit heaven
As far as components go - look, as was said, for the motherboard integrated, non-PCI solutions if you buy a new PC. If you're upgrading an old PC, PCI cards are OK - they are a DEFINITE improvement over 100MBit cards, even if you just read 30MB/s. As for the switch - don't buy the cheapest one, the Realtek chips (they're the ones most likely using in there) seem to have some real issues. Also, if you are noise sensitive, look for one without a fan, those little buggers can get pretty annoying real soon. I bought a 3com 5 port 10/100/1000 switch for (half a year ago) 150 Euros, and I'll probably stick another one on top of it pretty soon. That thing (3C1670500) is small, has no fan and simply does what you want it to do. And it's pretty cheap for a brand name product. And all the components which don't use GBit (like the print server, the DSL router and the Access Point) I simply left on the old 100MBit switch, so the five ports limitation wasn't really one.
1 Gb/s is just 125 MB/s, rather lower than your U320 SCSI bandwidth, so a small RAID array suffices. Remember old 32-bit PCI buses won't go beyond 133 MB/s. You should use 64-bit PCI-X, solic state drives and 10Gb Ethernet, but that's a little more expensive.
Forget how many DVD's, MP3's and telephone calls you can transport...how many Libraries of Congress can it transport per minute/hour/day/week/month/year?
Trying is the First Step to Failing --Homer Simpson
I've got a lot of 2 gig ghost image files lying around.
/home4 $ time copy /home3/dna3.gho .
:)
[d:\]timer on & copy \\INDEPENDENT\home3\dna3.gho . & timer off
Timer 1 on: 5:46:48
\\INDEPENDENT\home3\dna3.gho => D:\dna3.gho
1 file copied
Timer 1 off: 5:48:02 Elapsed: 0:01:13.75
[d:\]dir \\INDEPENDENT\home3\dna3.gho
4/07/2004 20:35 2,147,482,153 ___A________ dna3.gho
Hmmm let's see:
2147482153/73.75*8/1E6=232.9 megabits/second
And for fun a disk to disk copy on the linux box going between two channels on the onboard sil3114 sata controller:
independent
real 0m59.826s
2147482153/59.8*8/1E6=287.2886 megabits/second
So not bad for the boring disk subsystem I have.
Don't forget the impact of the Ethernet interframe delay on GigE performance. It's a basically irreducible effect which varies with the traffic mix and cable-run lengths, and will cut your max observed performance often to the 2-300 Mbps range. Jumbo frames alleviate the effect however :-).
However, it is true that this is just a theoretical maximum; there's other overhead in the PCI protocol and some implementations are worse than others. A typical PCI bus will drive a single device at about 50 MB/s (give or take). This is faster than a 100 Mb/s card can go, but not as fast as a gigabit card, so if you want to squeeze out all the performance, you'll need a wider/faster version of PCI. So your conclusion is correct, even if your math isn't.
Am I part of the core demographic for Swedish Fish?
If your filesystem fragmentation is limiting the speed of your multimedia (or other) transfers I would recommend a couple of solutions:
1) Use XFS instead. SGI designed it for streaming video/multimedia applications. It uses special algorithms to minimize fragmentation for large stream images.
2) Use Reiser4, and make sure you also install the new background defragmenter. Also make sure it's niced enough to not get in the way of the file transfers.
Good luck
just think a little bit about:
100Mbit means (theoretical) ~12MegaByte per second. Let's talk about 8Megabyte per second in this example.
Most movies have a averange duration of 90Min. so you could transfer with 8 Megabyte per second 43.2 Gigabytes in this time.
another oint you should take accocunt: your pc is only so fast, like the slowest part in it (bottlenecks everywhere!).
this means, that when you use 1gbit networking and you use a (e.g!) 5200RPM harddrive you should shoot yourself.
there are many components you have to so think about, if you NEED it realy and what you will earn if you do so :)
The latest NForce3 has gigabit built in to the chipset - so it's not /all/ Intel.
Instead of posting repeats, maybe the editors would have a look at pending submissions like CAD machine recommendations or "new" concepts like mine-finding flail tanks. Just a thought.
PCI-X is "here" and has been for ages. It's just only found in server and high-end workstation boards. PCI-X is essentially 133MHz 64bit PCI, and while fast is otherwise uninteresting. You can get it now if you buy a high end Xeon, Athlon MP, or Opteron board. It's also available in many non-x86 systems, such as the Apple G5.
I suspect you mean PCI-Express, otherwise known as PCIe or PCI-E, which is the emerging replacement to the PCI standard.
Craig Ringer
Personally, I'm still astonished that nobody has mentioned 10 Giabit Ethernet, and said something like "Pfft! Who needs that pokey gigabit stuff anyway?"
Slashdot and people saying "you don't need something that fast" is strange and frightening. I can at least understand people pointing out that you are unlikely to get full gigabit speeds, but then those people go on to say "so you don't need gigabit" - completely missing the point.
two and a half years ago when I was learning about Gig-E in grad school, it was cat 5E (note the upper case). Now I'm building a house and I want the walls wired; naturally I ask about cat 5E. And the builder tells me "cat 5e is standard"- thats because there is ONLY cat 5e these days.
In the future, I would want to not be isolated from my friends in the Space Station.
Perhaps you have been forgetting to set the blocksize to the right values? Do a man dd.
Nearly all my 7200rpm HDDs can sustain >=40MB/sec.
In fact when I compress and image them for backup purposes I get about 28MB/sec which is much higher than 100Mbps.
e.g. time dd bs=131072 if=/dev/hda | lzop -c > host-20040409.lzo
Ok, here's the deal with jumbo frames.
Don't worry about them. Only very, very expensive systems will be able to take advantage of them.
If you have 32/33 pci, you arent going to get max throughput from GbE anyway. I've managed to get around 90mbyte/sec using ttcp, which is about 750mbit/s.
Because the hardware does all the work for you (hardware checksum, interrupt mitigation, etc). the cpu usage is very low even at that rate. And thanks to polling, the interrupt rate isnt an issue either.
Your bottleneck will be your PCI bus, plain and simple. You arent going to get the full 132mbyte/s from 32/33 pci, period.
Unfortunately 64bit/66mhz PCI motherboards are somewhat expensive and 64/66 cards are 3-4x the cost of 32/33 ones.
Framesize is a function of hardware capability.
If you have legacy 10/100 devices that are plugged into that segment, jumbo gigE frames will NEVER work with the legacy devices. gigE frames appear to be L2 MAC errors as the preamble, source, destination, length addressing may line up in the front of the frame, but the crc at the rear will never line up. (Ethernet II frame illustrated below)
Preamble|Source MAC|Destination MAC|length|data|CRC
This is exactly like MTU's not lining up.
But anyways, I think there are demonstrations with some workloads saturating a gigE w/o using jumbo frames.
[snip] from http://sd.wareonearth.com/~phil/net/overhead/
Gigabit Ethernet with Jumbo Frames
Gigabit ethernet is exactly 10 times faster than 100 Mbps ethernet, so for standard 1500 byte frames, the numbers above all apply, multiplied by 10. Many GigE devices however allow "jumbo frames" larger than 1500 bytes. The most common figure being 9000 bytes. For 9000 byte jumbo frames, potential GigE throughput becomes (from Bill Fink, the author of nuttcp):
Actually, the biggest problem with GigE ime is that you can't handle the interrupts fast enough to support it. There will be network cards on the market soon (level 5 networks for one) that address this issue, and it needs to be addressed before there is a chance of getting 10Gbps in a regular machine.
This should help with a purchasing decision :)
/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$/i
I have a linksys 8 port gbe switch. I do largish (100-4200mb) file transfers between 2 machines with Intel desktop gbe cards and 3Ware ATA RAID cards w/ 5400rpm drives.
No tweaking, went from ~10-12mb/s to 20-24mb/s switching from 100 to 1000. Lots of variations between Linux and Win2KPro, and I had to move my Linux system to +1Ghz to get decent speed from samba. But worth it to me. Speed was measured with Win2k perfmeter or timed so YMMV.
I'd suggest not going with the Linksys because it has a cooling fan.
I've been using GbE for home LAN for about a year now. Here's the hardware I use:
h ttp://yoda.uvi.edu/InfoTech/rj45.htm
Switch:
Linksys Instant Gigabit 10/100/1000 8-port switch
I think I paid ~$200 for this.
Cards:
Intel PRO/1000 MT Desktop Adapter (~$50 ea)
Use the e1000 driver in 2.4.x or 2.6.x.
Netgear GA302T Copper Gigabit Adapter (~50 ea)
Use the tg3 driver in 2.4.x or 2.6.x
The tg3 chipset runs rather hot, the e1000 is tiny and runs cool. I havent noticed a performance difference between either, and both chipsets run fine regardless of whatever PC I put them in.
Motherboards with embedded GbE typically use e1000 (if theyre good), or realtek (if theyre cheap).
Jumbo frames:
See my post on that here.
Cabling:
Hand crimped cat5e. Works fine. One interesting note about GbE, you no longer have to worry about crossover cables -- the GbE spec requires that devices autodetect crossover. You can make all your GbE cables "straight through" cables.
Do pay careful attention to following strict T568 wiring code though. You can no longer get away with incorrectly wired cables which just happened to work for 100bt. Since all pairs are now used in GbE, your wiring order must be 100% spec.
Here's some wiring guides:
http://www.lanshack.com/make-cat5E.asp
First off all, the wareonearth article is a call to arms to provide one fix to a problem that is much more fundamental to the scale and deployment of the current internet as we know it today. In other words changing the frame size to 9000 in his example will accomplish nothing unless every router in the world also changed it frame sizes. In the real world however there are many cases (I did not say all) where I don't want 9000 byte packets being dropped due to loss/congestion. It will kill throughput.
.0000001 loss into the example given at the website and see what your throughput is.
Throughput is a function of MSS in COMBINATION with MANY other factors. In fact, plug 5ms (your home network latency) and
The bigger problem with most home Gige networks is not the switch, its the NIC. When you are purchasing cheap nics you are relying on the CPU to do time critical operations (interacting with the TCP stack) which is both inefficient and slow in comparison to dedicated hardware. Move up to hardware - offloading NIC cards and your throuhgput will go up dramatically even with low end off the shelf switches. You do not need jumbo frames to do this and even ifyou did, TCP will take care of the frame fragmenting for you in a mixed environment..
And whats with all the confusion about Mb vs MB, I have never seen so many people confuse the potential bandwidth of their CPU, PCI slot, IDE, networks, etc... please people, get it together..
on my oldest machine, p3 450, 12gig udma2 drive, i'm getting ~16Mo/sec read and ~14.7Mo/sec write, through the filesystem (Ext3), based on copying 1Gio at a time to /dev/null or from /dev/zero with dd bs=1024k count=1024 (including sync after the write)...
Because you'll find that you can't write to a filesystem on a single disk much faster than 100mbit anyway. Gigabit is significantly faster than the I/O that a single drive can provide.
Government of the people, by corporate executives, for corporate profits.
No - I've experimented with numerous block sizes. The best I've seen on the systems I've played with is around 11Mbyte/sec - around 8 seems more normal. Perhaps it's just the crappy systems we have ;-)
Most cheap home systems are more likely to be crappy though.
Oolite: Elite-like game. For Mac, Linux and Windows
On a 100BaseT NIC the theoretical max transfer rate is 12.5 MBps with a realistic speed of 8 MBps.
Not true... I consistantly get a rate of 10 MBps. All the time, every day.
I'm looking to move to gigabit though. I could have one disk array for the entire house.
They can read at that speed. They can't write at that speed. You will need large memory buffers (similar to the size of the files) on either end of the network to handle the slowdown when waiting for the disk, or a stripe across several disk spindles.
Then of course for smaller files there's the seek times, you don't get anything like the maximum theoretical throughput from the drive. As to waiting for 1/4 of the time, it depends whether it's 0.01s or 60s.
Government of the people, by corporate executives, for corporate profits.
FYI, any new mac you buy will have gigabit on the mobo, including the laptops.
One of these days I may upgrade to a gigabit switch in the basement so the house network runs 1000bt, but I'm in no great hurry. HD i/o speeds here are already being pushed with the 100bt so I don't know how much gigabit will help anything.
I work for the Department of Redundancy Department.
When it costs $10 for a switch and $5 for a NIC.
Till then, the only time my 100mbit LAN gets remotely taxed is when I run Bacula backups of all of my machines.
Government of the people, by corporate executives, for corporate profits.
You say that like it's a bad thing!
"The invisible and the non-existent look very much alike." -- Delos B. McKown
...I get 1Mbit = 1024kbit line here (or the authoroties will kick their ass for false advertising) and hard disks measured by base 10 gigabytes.
The only way to end this is to use Gib / GiB consistantly for base 2 sizes. And forget the pathetic gibibit/byte names, if you have to distinguish between them, use "binary gigabit/byte" as opposed to "decimal gigabit/byte".
Kjella
Live today, because you never know what tomorrow brings
Betcha. Unless of course you've got 4Gb of filesystem buffer in each of your machines that is...
Government of the people, by corporate executives, for corporate profits.
98 is commonly available here in the UK, which can easily get to over 100 with octane boosting additives. There are also a few places that still sell what used to be called "five star" fuel (100RON), but they're getting quite rare now.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
Doing some of the same research for myself I noticed that many unmanaged switches, including the linksys workgroup switches linked off this story, do not support jumbo frames.
If you plan to use jumbo frames make sure the swtich you buy supports MTU sizes up to 9000. Just about all managed switches I've seen do this but they cost considerably more than unmanaged switches.
Once exception I found are the SMC Gigabit EZSwitch unmanaged switches available in 8 - 24 port versions.
-- The BlueCamel
Several recent studies have re-ignited the Jumbo frame religous war. ( i.e. Do jumbo frames actually improve performance? The answer in these papers seem to indicate no.)
For the applications you want to do, jumbo frames are a bad idea. Why? The larger the packet the more interpacket jitter that occurs. For things like VOIP you want small packets.
For data transfers, you might consider jumbo frames. But any retransmit will have a larger impact on your performance.
For the above reasons, I would not care about support to jumbo frames.
Thingsmith.
Currently at the college I attend they have Gigabit for Business and tech park. You need to make sure that the switch is compatiable with your Ethernet controller. Also some of the Gigabit switches require that you go Cat 6 others like 3com are suppose to work fine with Cat 5.
For linux/windows you should not have to do any real tweaking. I would say stay away from the 3c2000 for linux unless you want to build your own module or go to 2.6 kernel, and stay away from e1000 if your using some brands of cisco switches.
It takes routers to fragment packets. Which means one of a few things: a) no jumbo frames b) buy a box to act as a router.
Just to restate though, jumbo frames will only do good in large data transfer environments.
However, about two years ago when I was doing a high-speed IDS deployment, the scanner was an 1.3 P3 Xeon and it could generate 800mbps of traffic using gig fiber and 1500 byte frames. Try adjusting your TCP window size and application MTU size before resorting to jumbo frames.
PCI will do it just fine, you just need 66MHz and/or 64 bit PCI slots. Sadly, most motherboards, even for Athlon 64, and a lot for Opteron don't have anything better than 33MHz/32 bit slots. Xeon systems have been shipping with 66/64 PCI standard for four years now, so I've been playing with some cast-off workstations.
I got a decent price for a 4 port gigabit hub. It was around $250, which was much cheaper than any other alternatives at the time. About a month after I spent the money, I found out that in the real world, gigabit ethernet is only about twice as fast as 100 megabit. I think this is the original article I read is here. Make sure to look all the way to the end at the "Using the speed" section. Basically, it says that if all you are doing is copying from hard drive to hard drive, gigabit is only about twice as fast.
Yea, that's 98 RON, which is not what we grade in America. We grade R+M/2. A nice way to boost octane, though is to use Toluene (R+M/2 = 114) or Xylene (R+M/2 = 117). They can typically be added as up to 30% of your gasoline mixture.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
HD is a wasteland right now. Some of the networks are in HD some of the time, if network sitcoms and a few sporting events is your idea of watching TV. There's HBO and Showtime, if you get either one, and then there's a PBS and a Discovery HD which are almost just a loop. Beyond that and the re-hashed crap on HDNet there really isn't anything terribly compelling in HD.
I was thinking of doing a similar sort of thing, but most of the machines in my house are laptops. I looked but couldn't find any cardbus GigE adapters. Only PCI.
Anyone else find one?
I had no idea Gb Ethernet switches had dropped so much in price. If I was buying a new switch today I'd definitely be buying one of those $100 Linksys switches.
No you wouldn't be buying a Linksys, because they and the others in that class do not support Jumbo Frames, thereby diminishing one of the best features of GigE, and increasing the interrupt requirements on every one of your GigE NICs by a factor of... well, more than enough to make a sizeable performance hit -- someone else can do the numbers.
I was hovering over the "Add to Cart" button on the Linksys two days ago -- I noticed how CHEAP they are now and wanted to get rid of the crossover between my primary box and my 1/2TB RAID5 box used for audio/video). Thank goodness I did a little more research. I would have been really pissed to buy a 'Gigabit Ethernet Switch' that didn't support Jumbo Frames...
FYI, the SMC 85xx series switches DO support Jumbo Frames, and at almost the same price point. I don't know why Linksys, D-Link and Netgear cheaped out on Jumbo Frames support in their firmware/hardware. Pretty lame if you ask me.
But not as lame as Amazon.com taking off the SMC unmanaged gigabit switches once I (and probably others) pointed out that Buy.com was selling them for $4 cheaper ... heck, $142.99 for an 8-port unmanaged GigE switch? I may click the Buy Now button NOW!
7MB/s is pretty darn close to the limit of 100Mb/s. notice the difference between B and b... unless the math people have changed something, like adding few new numbers between 7 and 8.
I have one of those noise cancellation devices that reverses the phase of sound to cancel it out. I pointed it at the Linksys, and it made the switch a whole lot quieter.
Only problem is, the noise canceller has a noisy fan in it.
--Rob
Towards the Singularity.
...what kind of bandwidth streaming DVD content actually needs. It's 6-9Mbps which means you could probably serve 6-8 "terminals" simultaneously with a freaking 100Mbit hub. With a 100Mbit switch, it's a no-brainer.
He's also clearly clueless about consumer hard drive transfer speeds. Unless the OP has a nice RAID setup on his server (which is doubtful if he's un-savvy enough to come to Slashdot for networking advice) there is no way that this theoretical 120GB workstation backup will be appreciably faster. I say this as someone who actually has Gig the Fast Ethernet running on a home network with a server that uses ATA drives in both RAID and non-RAID configurations, not as someone who plugs numbers into a calculator and makes ridiculous assumptions.
cybermace5, stop trying to spend someone else's money based on your incomplete understanding of both the requirements and the technology.
I'm surprised no one has mentioned clustering of computers yet. Sure, no current hard drive --> hard drive transaction will ever flood the full gigabit of bandwidth, but what about CPU --> CPU? In a year or two when Apple's XGrid becomes a standard part of the OS, I'd be happy to have my little network of Macs become a clustered beast.
Shades of Grayden
Hint, that's the maximum theoretical read performance. Hard disks read significantly faster than they write.
/dev/zero.
Test it on your system:
Reading:
dd of=/dev/null if=/tmp/file bs=64k count=131072
Writing:
dd if=/dev/zero of=/tmp/file bs=64k count=131072
You should try it with different count values to see how your filesystem buffer affects the speed. Every file you read has to be written somewhere (unless streaming video for instance) and when you have very large files (e.g. 4Gb) your filesystem buffer will be flushed through unless you have configured a 4Gb buffer of course. To take any sort of advantage of gigabit, you need large enough buffers to make sure you aren't being limited by the write speed of the receiving drive.
I predict that you won't get anything like the 32Mb/s quoted, never mind 58Mb/s once you're running at the disk speed rather than the buffer speed. Even with the ideal condition of dd'ing from
Government of the people, by corporate executives, for corporate profits.
No, Gigabit is 1 (US) billion bits per second
;-p
No, Gigabit is 1 billion bits.
Gbps = 1 billion bits per second
As for the whole 1 (US) thing, I wasn't aware that we had a different value for 1 than the rest of the world. Guess I'm just another ignorant american.
Sorry, I'm a jackass.
Casual Games/Downloads
this Discussion seems so circa 90s when 10 vs 100 debate was in effect Bottom line..... 100s had its day in the sun, and its had one HELL of a good run, with the the current prices of GIGE 8 port switches is what i paid for my 10/100 arkteck in 96 so yea id say its worth the upgrade name one other comp component you get that kind of life expectancy out of
If you are only getting read speeds of 8 megabytes per second sustained, yes it may be time to upgrade. Perhaps check to make sure the drive and controller are set to DMA instead of PIO.
Even crap consumer grade current generation drives it boring single drive configurations are capable of read speeds of 30M/s on the low end and 60M/s on the high end. Shell out $500 for a RAID-0 SATA array (two drives) and you will get sustained read throughput of roughly 100MB/s with a low of about 50MB/s.
http://www.tomshardware.com
Glonoinha the MebiByte Slayer
Scyld computing (of Don Becker fame) did a lot of work on gigabit drivers for linux, and open source as well. If you can dig up an free copy of their beowulf operation system, you can probably get the drivers from there. Their website isn't too helpful after the Penguin computing acquisition, though.
Try this page.
if(!toilet_paper) roll.replace(new roll);
Network throughput is measured in megaBits per second.
Hard drive throughput is measured in megaBytes per second.
Yes 40 is less than 100, until we factor in the 8:1 advantage that bytes have over bits. Then all of a sudden we are looking at 320Mb/s vs 100Mb/s and by upgrading to a GigE pipe you move the bottleneck to the hard drive and your entire system moves data over the network three times as fast.
Glonoinha the MebiByte Slayer
Unfortunately, with one linux box, the NIC doesn't work at all :-( and I haven't figured out why. When it works it's great, and the prices are cheap enough to make it viable. It just isn't quite as stable as 100 mbits is.
-dB
"It if was easy to do, we'd find someone cheaper than you to do it."
Presumably, PCI-Express network cards will also keep up pretty easily. From what I can see, you're probably best to wait another year to eighteen months before upgrading; by then, PCI-X should be pretty common, and gigabit networking shouldn't be very expensive.
PCI-X and PCI-Express are not the same thin!. PCI-X is almost the same as PCI as far as hardware is concerned but the protocol is way different. You find it mostly in servers. PCI-Express on the other hand is way different in every way. It is a lot like Ethernet in fact! The software interface is about the same, so it will just look a lot like PCI to higher level software.
I want max headroom
Well, *I* never found him that attractive... but if stuttering does it for you...
Even if a man chops off your hand with a sword, you still have two nice, sharp bones to stick in his eyes.
Jumbo frames are not configured by default, and you don't need them to take advantage of gigabit. Jumbo frames are great for a dedicated network where all the devices (interfaces) are configured to use jumbo frames. Applications would be
r ne t-with-jumboframes->[databaseServer]/[NAS-Filer ]
[Internet]->-FastEthernet->[WebServer]-GbitEthe
Well, Anonymous Coward, stop trying to KEEP people from spending money on tech gadgets when they can afford it. My career is somewhat linked to the strength of the technology industry, what about yours? Besides, having gigabit ethernet CAN'T HURT.
...
In fact they've been doing so for years. 100Mbit networks are often a bottleneck. Gigabit tends not to be, which is nice.
I think the biggest thing about gigabit is that PCI isn't really fast enough to support it. <snip/> I think about the best you're going to get off most PCs, even very, very fast ones, is about 300 megabits sustained.
But 300 Mb/s would be a huge improvement over 100baseT.
Lots of people here are calculating what it would take to use all of Gigabit's bandwidth, which IMHO misses the point.
You write your nine symphonies, then you die.
I've forwarded this topic to our CFO and have asked him to chime in....he's wired his home for gigabit, along with other things, so he may have some useful info.
If you're not outraged, you're not paying attention.
I've considered gigabit Ethernet for HD streaming too -- I mostly get smooth playback over my 100Mbps network, but occasionally there's a little glitch when the player app moves to the next file, which doesn't happen when playing from the local disk. Hasn't been important enough to make me shell out the money, though.
My pall and I both have gigabit nics in our mobos, but neither of us have any other gigabit hardware... yet.
So when we got togeather we pluged in a normal crossover cable. the two systems connected at 400Mbit. nice, but not gigabit. But I had this bright idea about a gigabit cross, that, believe it or not, worked!
on a normal cross cable, the green pair is swiched with the orange pair, while the brown and blue pairs are left alone. this works just fine for 100baseT because it only uses the green and orange pairs. gigabit uses all four pairs.
So what i did was cut off one end and switch the brown with the blue, while keeping the green and orange in their (already switched) places. i recrimped it and pulged it in. just like that, Viola! a gigabit connection with no gigabit hardware inbetween!
we saw a noticible increase in speed, although i never actualy measured it.
Nathan Friedly
Why go to Gigabit? A 100BaseT LAN on a switch can easily handle several video loads. Several of my PCs can pump out 6Mbit/s, (DVD quality). But there is still 94Mbit/s left! Surely you're not going to want to watch 16 videos at once, (100 6 /)?
Also, I don't know of many PCs that will be able to make use of the GigaBit speeds to it's full extent.
-- main(s){printf(s="main(s){printf(s=%c%s%c,34,s,34
** draws sword **
WHACK WHACK!
Your response was quite humorous, you almost seemed paniced at the thought of forgetting the HTML.
You are full of it.
All the computer storage makers use the stuipd 1K = 1000 because it makes it look like they can store more. Any decent file system actually reports the size with 1K = 1024,
The reason this all started has to do with addressing memory. You have to have an address composed of bits. This means every maximum addressable size is a power of 2. So, the habbit started of refering to computer memory in powers of two, and that has translated into using it for most things computer related.
Is that not everyone is playing with crappy 320x240, compressed to hell MPEG-4 data. Like let's say you want to play with video in the DV format (it's an actual format, just just an acronym for digital video). That is 25Mbits/second just for the video component. What that works out to is one of those little mini-DV tapes holds 10GB of data for an hour of video (more with audio). You get a couple hours of video, and copying it over a 100mbit network is just painful. It still takes a long time with GigE, since the disks are now the limiting factor, but much less than it does with FastE.
going on about why you don't need Gigabit? Aren't we all geeks here? Isn't that a good enough reason to get it?
Hardisk rates are spec'd in megabytes, networks in megabits. So a 100mbit network is a 12.5mbyte network. However, you don't actually get that kind of throughput. I find a good rule is to divide by 10 when going from bits to bytes in a network, that will give you a realisitc maximum rate you'll actually see on your system, and even that is a makimum, not an average.
Try it out some time, get a system with two good harddrives, that aren't fragmented, and another system on a 100mbit lan. Try a direct disk-disk copy, try a copy over the lan. Notice how much slower the lan transfer is. Good 7200rpm drives can write considerably faster than 100mbit/sec provided the data is contigous.
For my uber-HTPC with HDTV-recording, etc I bought the SIIG JU-2NG011 gigabit eth/firewire/usb 2.0 card for a mere $80. The Netgear gigabit switch was $60 and cat5e cable was $15. It's clear when using it, that hard-disk I/O is rate-limiting, but that's ok as I usually stream HD streams for recording to my much faster RAID server. Most new server/workstation machines already have gigabit ethernet on the motherboard, so that's like a freebie. I also use it for DVD ripping over the network, and numerous other things I play with.
tcboo
Cat5 cable has 4 pairs. While 10 and 100 didn't use them, they were all in the spec and cables conforming to the Cat5 spec must have them.
IEEE802.3ab or 1000Base-T is for gigabit networking over Cat5 UTP cable. From what I understand, Cat5e is not not in the spec and is not necessary.
That said, if I were running or buying cables, I'd go with Cat5e. The price premium is minimal and totally worth it. Just don't think you have to rip out Cat5 and replace it with Cat5e to get your gigabit ethernet working.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
That probably has something to do with the fact that 66MHz/64-bit is completely useless for consumer workstations. There's a reason why it's on the xeon.
While I'm sure it's all optimal and everything to put everything over one cable using one megafast network card, I think people are ignoring the potential of aggregating multiple NICs over multiple channels to get the same total bandwidth. Hey, if 1 Gbps is good, 4x1 Gbps is even better, right?
Geez what do you want. There is more programming available in HD now, than there was OTA programming 20 years ago.
I'm using Intel Pro/1000MT desktop NICs and the Netgear GS105 5-port copper switch. My desktop machine is an Athlon XP 2500+ running XP and my file server is a VIA C3 at 800MHz running Linux. Iperf gives me TCP throughput of about 650-700Mbps. I'm probably not getting quite that much on actual file transfers since there's a disk involved, but it is quite a bit faster than running at 100Mbps.
For those who claim gigabit isn't needed for home use... sorry, it is. I was tired of waiting for things to transfer at 100Mbps. There's my need right there. At $30 for a NIC and $85 for a switch, it's not like I'm throwing tons of money at it, and I'm seeing a significant improvement over 100BaseT.
Netgear (and a lot of the other consumer switch manufacturers) claim wire-speed performance on their gigabit switches, even the little 5-port versions. I don't doubt it since they're using Marvell or Broadcom switching silicon. I haven't had a chance to put the Smartbits on my Netgear to prove/disprove the claim though.
i have been thinking about gigabit for awhile now my I.T. friends tell me that it is pointless unless you have dedicated Gigabit-Lan onboard
they tell me the add-on cards are no good because they are limited by the PCI BUS speed and cant take full advantage of the Card
I agree tho that 10/100 is perfectly fine for playing video over Lan
Eh, that doesn't match my experience.
:). My current project is 2 TB.
I've got an Athlon MP 2600+ workstation with an Intel gigabit card, which connects via a $199 LinkSys Gig switch to an ancient Mac OS X dual 450 box running 10.3 Server. I get ~470 Mbps copying from the server to the workstation, which is a heck of a lot better than I had with 100Base.
I'm probably unusual in that I work with uncompressed HD files from home
My video compression blog
I actually have one of these!
:)
I'm only running 10/100 Base but I have 2 48 port cards in it, and one management card. I have both power supplies and they are running off of 2 APC Smart-UPS 3000s.
I know what your thinking. 'Yeah right he's dreaming.' But I really do have it! Left over from a tech company here in Seattle.
I bet they wish they had a gigabit network.
My advise: don't bother trying to find equipment that can handle Jumbo frames. Most unmanaged GigE switches don't, and equivalent managed switches (many of which do handle Jumbos) are priced at a premium over their unmanaged brethren. You should only buy a managed switch if you actually need a managed switch. It takes a lot of effort to configure a network to use Jumbo frames for certain links and not for other links, and it just isn't worth it 99.9% of the time. If you want performance, focus on smart GigE cards that offload a good chunk of the packet processing (e.g. like checksum calculations and fragment reassembly). The Intel cards are a good start.
As to whether your network needs GigE or not... well, it depends on what you are using your network for. I use NFS all over my network and with GigE I get virtually the same disk performance over the network that I get with local disk. I also run backups over the LAN and the backups that run over GiGE go a lot faster then the ones that run over 100BaseT. Even more to the point, I've started distributing video over the network and 100BaseT just doesn't cut it for more then two or three streams.
If you are going to buy a new switch, you might as well get a GigE switch.
-Matt
Man, you've really been missing out. It's almost as if you're still in the 1990s world of hdds.
Are you sure you have DMA on?
linux: hdparm
freebsd: atacontrol
Make sure your drives, cables and ATA controller supports DMA first.
Use the 80pin ATA cable. The 40pins often don't make the grade for DMA, some are ok enough for Ultra ATA 66. But you want ATA-100 to leave you some headroom - some of my ATA drives hit 50+MB/sec at the outer cylinders and they're not top end stuff.
And Jet Blue (aka AVGas & Jet A) in the UK is 103 - 107 Octane Or it was when I was first learning to fly down at Lashden Airfield in Kent some 14 years ago. Besides which, I would look gormless driving my STS from the wrong side of the car :)
Bad Panda! No Bamboo for you! In matters of importance ACs will not be responded to. Want to say something critical,OK
you forget to mention that the speed of the PCI bus is only 200 or 300 Mbs unless your running a mobo with an integreted gigabit controller your nto going to get the full speed also you would need a good reaid array to write/read at those speeds
Hardware:
Linksys EG008W
Intel PRO/1000MT Network Adapters (except the mac which is built in)
Various IDE HD's, CPU's and motherboards
Bottlenecks:
32bit 33Mhz PCI - ~100MBytes/sec
Hard Disks - 5-30MBytes/sec (varies widely 15-20 is typical)
CPU - Stats are from receiving machine copying TO the local machine
55% - G4 733Mhz ~18.5MB/sec
36% - Athlon 1.46Hgz 17.6MB/sec
Its definitely the disks at this point. Fragmentation and even more so multiple disk accesses at once both make a huge difference in performance. Simultaneously accessing multiple files on the same disk seem to throw most consumer IDE drives for a loop. 10-20 Milliseconds is just too long to wait...
After the disks, the slower machines (ie 1ghz) is going to be the next limiting factor.
(ie the ones 1ghz)...
Please correct me if I am wrong.
I've had a serious lack of luck trying to find anything on the sub $3000 market that supports Jumbo Frames. It appears that only Cisco, Nortel and some high end HP switches supported Jumbo Framing. I have the Netgear 4 port GigE which does not support Jumbo Frames. I looked at Netgear's Managed Layer 3 24port switch, which also doesn't appear to support it. Two network cards doing a crossover cable had no problem with it, but the switch sure didn't like it. Linksys is the same way with their GigE switches. Only the 'big boys' get to have Jumbo Frames
-- If we don't stand up for our rights, now, there will be no right to stand up for them later.
Unfortunately, you won't find any switch devices which will fragment packets based on link MTU.
By intentional design, fragmentation only happens on layer 3 devices like routers (or L3 switches, I guess). True layer 2 switches like a home gig-e switch will not do so.
The "fix", and the best practice, is to always keep the MTU the same inside each subnet. If you want a jumbo frame network, put a router in there between the mtu boundaries.
The reason this is done is that fragmentation is truly a layer 3 issue-- while there is a fragmentation field in the IP header (and many other L3 protocols), there is nothing of the sort in the ethernet frame header. A device would have to be smart enough to actually rewrite the packets at the L3 level and regenerate them in order to keep the layer2 checksums correct....and at that point, you have a router again.
mark
>I'm tempted to jump on the 1000BaseTX bandwagon.
Me too. A lot of new mobo's come with integrated.
>As far as I can tell though, the current crop of consumer-priced hardware/software doesn't address a couple key issues, namely: fragmenting jumbo frames for the benefit of legacy clients - this is critical as some of the devices on my network will not tolerate the 9000+ byte Ethernet frames which are needed to get the most out of gigabit;
You may be able to set this by port on some switches. Never tried it.
> and OS support - do Linux and Windows require much tweaking to take advantage of gigabit?
MTU is pretty simple to set on windows. You can grab a utility at DSL reports for this.
on linux:
man ifconfig
>Will most drivers automatically optimize themselves?
Sadly enough no. Shoot, we haven't even gotten autosense working correctly in a vendor neutral full duplex switched environment. You hard set, or get the wrong (or half) duplex half the time.
> Google search didn't reveal too much consensus, especially on hardware choices. What switches and software configurations have Slashdot readers been using for home gigabit networks, in particular mixed ones (100/1000BaseTX?"
check the MTU capabilities of the switch. BTW, if you set your MTU high for LAN speed, your packets going to the internet will end up getting hella fragmented. This will decrease internet networking performance, depending on your connection speed, this could be quite significant.
never again before coffee, sorry.
I've got a sort-of Gigabit eth setup at home. Only the fileserver and my workstation have gigabit, the rest if a hybrid of 100 and 10 ethernet.
The gigbit NIC's are a no-name 64bit NIC based on the Natsemi 83820 chipset, and the second is a broadcom intergrated into my mobo.
The switch was another dirt cheap no-name thing, which I liked for the fact that it had 2 gigabit ports and 8 100 ports, which could serve the rest of the house.
The file server (Debian) has one of the gigbit cards, and has 3 IDE RAID1 arrays (thanks, 3ware!) totally about 700GB serving music, movies etc to the rest of the house. Giganit ensures that they can all use as much bandwidth over their 100's as they like and never experience stuttering, except when I max out the connection (transferring any new DVD renders I've done) with my workstations gigabit adapter, but this only happens at night via an automated rsync-based backup script. The rest of my boxes also use the same script to backup their important files as well (two mythtv boxes for instance).
In short, gigabitting the house out cost not-very-much (about 120 for the 64bit card and the switch together) and has meant that even when the server is under heavy load there's never a problem with network congestion.
Moderation Total: -1 Troll, +3 Goat
Just a thought: consider multicasting for around-the-house media distribution.
"GNU/Linux is free freedom." -- Me
The (US) thing is because for the longest time the brits insisted that billion == million million, and trillion == million million million.
In the US, billion == thousand million and trillion == thousand thousand million.
They kind of had a point, since the whole bi == 2, tri == 3 thing makes sense, but by now even the BBC isn't trying to convert the world anymore.
It's also smaller and uses less power (I measured both). It actually uses less power than the early-model 100Base-T switch I previously had. 1/3rd as much.
Netgear 8 port switch
No, actually I am correct. I never disputed that storage "makers" use the SI meanings of these terms because they inflate the sizes, nor that filesystems and operating systems report using the units that are most natural and useful for computer users.
In fact, I mentioned that using powers of two is done because of the way computers work--binary computers are easier to design, program and use than base ten computers.
On a personal note, I think that labeling computer disks using base 10 units is deceptive, intended to convince the customer they are getting more than they actually are. It is also deceptive in that these prefixes attached to non-SI units like bit or byte do not have standard meanings, so common usage should guide the understanding of such terms. I really do understand the emotion in this thread. (re: You are full of it.) The discussion was about networking equipment however, not disk drives.
Sports, popular TV shows, and most of the movies on HBO really aren't that compelling, which is why I said that. HD PPV is hardly universal, and the selection isn't that compelling anyway.
Besides, if your cultural diet is satisfied by spots, popular television and popular cinema, you're not that compelling, either.
Here's a link explaining the "What's a billion?" thing better than I did.
yes... and 640k should be enough for anyone...
No todo lo que es oro brilla
better still use iozone or some kind of dedicated benchmarking program. I've managed to get about 40 mb/sec with iozone on my maxtor 7200 rpm ide disks and about 30 on a 5400 rpm disk.
I spent quite a while benchmarking high end RAID systems and even those struggled to get above 80 mb/sec for a single virtual disk in a RAID 5 configuration.
What do you want to watch that is not currently in HD?
The 4500 Series has a backplane capacity of 64 Gbps.
The 6500 Series has a backplane capacity of 256 Gbps.
For those that would die defending it, Freedom
has a sweet taste that the protected will never know.
You don't have to have any experience with voice over IP to figure out the math. If you have a 1.5 mb DSL connection, or even a 3 mb cable connection, you're not going to do anything over the internet that is going to push your local network over 100 mb!
There may be some benefits to a faster local network (although the original posts questions correctly ask if any home network would really take advantage of the supposed speed), but voip isn't a real issue here.
I'm an American. I love this country and the freedoms that we used to have.
"The Office" and other BBC America shows, many of the movies on IFC and Sundance, I'd like for all the major news programs (networks, CNN, Fox) to be in HD, especially their field reports.
Basically I consider about 10-20% of TV content to be at all worth watching, and maybe 20% of that is offered in HD.
I'm glad I get it, HBO Sunday is great, but overall the odds that something you will want to watch *right now* is in HD are pretty low.
The one reason I have actually found to possibly justify upgrading is moving massive amounts of files around. Like moving hard drives worth of mp3s or ISOs and the like ... or just doing large backup images/tar archives. Those things zip across the network. Doesn't happen often, but when it does - the gigabit is appreciated.
not on a 32 bit 33 mhz pci bus. do the math thats about a gigabit right there of total pci bandwidth. and then think that your raid controller is sharing the pci bus with the gigabit card. you wont see anything close to a gigabit until you move to a 64 bit bus or a faster clocked bus. presently my gigabit network (without jumbo frames) maxes out at 300 megabits, and thats going strait from ram on one machine to ram on the other machine. if there was heavy disk io in there i would expect that to drop.
i'm supprised no one has mentioned the pci bus limitations to this whole situation yet.
but yea gigabit is cheap so you might as well.
I just picked up an 8 port 10/100/1000t Micronet SP668B switch a few weeks ago, on sale for cad$180 or so. Its regular price is cad$234 (from ncix.com), or about usd$180.
Anyway, its major plus (to me) is serial port-based management and support for 802.1q vLAN tagging, port bundling (like Cisco EtherChannel), port mirroring (if you wanted to use an Intrusion Detection System), auto MDI/MDI-X, etc. It also comes with ears to mount it on a standard 19" rack.
Sadly, no jumbo packet support is mentioned.I was shocked to find a switch this cheap that supports 802.1q. Makes creating DMZs off your firewall quite easy, as well as storage, management and production traffic vLANs, should you desire making those things. ;) I have such vLANs in my home server room, and this makes it easy to extend those vLANs to my downstairs workroom.
For such an inexpensive switch to offer vLAN tagging, it is no longer a question of why use vLANs in your home network but why not!
Jet Fuel is kerosene... Your capri would stall, as it is not particulary flammable.
No biggie. I do the same thing all the time.... makes me feel like dirt when I look like I don't have a clue, which I have this habit of doing often on Slashdot.
Karnal
I use VoIP, and I have my box turned up to full-duplex maximum quality, it consumes about 90Kbits/sec each way, when someone's talking. That's about 10 Kilobytes a second, you could do full-quality VoIP over two bonded 56k modems!
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
That's nothing.
Back in the early 1980's, when most people had a right to be confused by home PCs, I was in a Radio Shack buying a matching transformer. (can you buy shit like that there anymore?)
Anyways, I overhead a salesman successfully selling a nice middle class couple 2 Tandy color computers, one for each of their children. "For homework" but no fucking printer.
I thought jumbo frames were also required for long-distance networks so that collisions would be seen by both ends of the cable?