Slashdot Mirror


10Gb Ethernet Alliance is Formed

Lucas123 writes "Nine storage and networking vendors have created a consortium to promote the use of 10GbE. The group views it as the future of a combined LAN/SAN infrastructure. They highlight the spec's ability to pool and virtualize server I/O, storage and network resources and to manage them together to reduce complexity. By combining block and file storage on one network, they say, you can cut costs by 50% and simplify IT administration. 'Compared to 4Gbit/sec Fibre Channel, a 10Gbit/sec Ethernet-based storage infrastructure can cut storage network costs by 30% to 75% and increases bandwidth by 2.5 times.'"

173 comments

  1. Math on /. by Idiomatick · · Score: 5, Funny

    i'm worried they had to say 4 * 2.5 = 10 on /.

    1. Re:Math on /. by Anonymous Coward · · Score: 0

      I'm concerned that they did say it.

      I don't know that much about the fibre channel spec. Ethernet was designed so it would work okay over crappy cable and unreliable links, it wasn't designed for throughput. It can only get 80% saturation. I'm going to guess that fibre-channel is significantly better than that. So it's probably more like 2x.

    2. Re:Math on /. by haifastudent · · Score: 1

      i'm worried they had to say 4 * 2.5 = 10 on /. I'm worried that they'll make a super-high-speed network and forget about latency again. Consortium engineers seem to favor headline-making numbers over real-world-benefit numbers.
      --
      Thank for reading to the sig. You may stop reading now. It is safe. There is no more content. Why are you still reading?
    3. Re:Math on /. by Shakrai · · Score: 2, Insightful

      It can only get 80% saturation

      Do you have a citation for that? I've seen Ethernet networks with decent switches approach 95% of the rated capacity.

      --
      I want peace on earth and goodwill toward man.
      We are the United States Government! We don't do that sort of thing.
    4. Re:Math on /. by K.+S.+Kyosuke · · Score: 1

      At the time when Ethernet was being designed for much-less-than-100% saturation, Bob Metcalfe knew much less about network switches than he does now... The equations for CSMA/CD just do not apply today...

      --
      Ezekiel 23:20
    5. Re:Math on /. by rdebath · · Score: 3, Informative

      Modern ethernet 100Base-T switched or 1000Base-T can work to 100%. With the switched medium all the links run full duplex and packets for busy links are stored in memory like a router. With a good switch packets for non-busy links get 'wormholed' to the output before they arrive (arrive completely that is).

      Normally this means that modern lans won't lose any packets; if your lan is losing packets you have a hardware problem. Perhaps you have an unswitched hub somewhere or a seriously overloaded switch that's running out of memory. But even a low spec switch that can't keep up with the net speed shouldn't lose anything, it should just block the senders till it can deal with the data.

      In fact 10Base-2 (cheapernet) was the last ethernet standard that that you couldn't avoid congestion collapse.

    6. Re:Math on /. by petermgreen · · Score: 1

      Modern ethernet networks do not use CSMA/CD they use switches and point to point full duplex links. As such with good quality equipment they can get very close to 100% saturation.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    7. Re:Math on /. by gallwapa · · Score: 2, Informative

      CSMA/CD still applies, except for the fact that on a switched architecture your collision domain is only a single port on the switch. Therefore the problem will lie between the switch and the device itself.

      CSMA/CD is still important in modern ethernet networks, due to the fact that some devices do not properly auto-negotiate. Some devices doesn't obey the RFC's for interpacket spacing in an effort to improve their throughput that can wreak havoc on networks.

      In many cases, if a link fails auto negotiation it will default to a half duplex link, where CSMA/CD is of vital importance.

    8. Re:Math on /. by petermgreen · · Score: 1

      CSMA/CD still applies, except for the fact that on a switched architecture your collision domain is only a single port on the switch.
      CSMA/CD does not apply on full duplex links. On a full duplex link collisions simply cannot happen.

      Yes decent CSMA/CD support may be important for good interoperation with legacy equipment where half duplex is unavoidable (say an old device with broken autonegotiation connected to an unmanaged switch) but it should not be in use on any link where top performance is required.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    9. Re:Math on /. by petermgreen · · Score: 3, Informative

      BTW 10 gigabit ethernet has completely dropped support for half duplex links.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    10. Re:Math on /. by Anonymous Coward · · Score: 0

      you should lower your reserve bandwidth setting in windows to make full use of your connection. Its in the registry, somewhere...

    11. Re:Math on /. by Anonymous Coward · · Score: 0

      if the link is full-duplex CSMA/CD DOES NOT apply - that is the nature of full-duplex!

    12. Re:Math on /. by r33per · · Score: 1

      Network traffic doctrine states that =/C (Ro=lamba/(Mu*C) which equates to Utilisation = input rate / (output rate * constant bandwidth). The ideal is for is to be no greater that 0.8. If it goes above 0.8, then congestion occurs and packets get dropped. However, it should be noted that this applies to routed networks, not switched.

    13. Re:Math on /. by r33per · · Score: 1

      =/C

      I had Greek characters and everything in there!!!!

    14. Re:Math on /. by Alioth · · Score: 1

      Actually, 10Gbit ethernet abandons both CSMA/CD and half duplex operation. CSMA/CD is not supported at all by the standard.

    15. Re:Math on /. by Shakrai · · Score: 1

      The ideal is for is to be no greater that 0.8. If it goes above 0.8, then congestion occurs and packets get dropped.

      That might be the ideal, but the GP seemed to imply that Ethernet breaks down at and/or can't exceed 80% utilization. That's simply not the case for a well designed network with decent (read: not el-cheapo SOHO 4 port switch/router combos) switches, cabling, NICs and full-duplex operation.

      Hell, this is anecdotal from my own experiences, but I've seen hubbed (i.e: half-duplex and non-switched) Ethernets running at 60-75% of capacity without any major issues or hiccups. It's far from ideal but it works well enough for some applications and users.

      --
      I want peace on earth and goodwill toward man.
      We are the United States Government! We don't do that sort of thing.
    16. Re:Math on /. by gallwapa · · Score: 1

      There are some new devices that don't conform to specs that don't properly auto negotiate.. :)

  2. Fibre only? by masonc · · Score: 5, Interesting

    From their white paper,
    "The draft standard for 10 Gigabit Ethernet is significantly different in some respects from earlier Ethernet standards, primarily
    in that it will only function over optical fiber, and only operate in full-duplex mode"

    There are vendors, such as Tyco Electronic's AMP NetConnect line, that have 10G over copper. Has this been discarded in the standard revision?

    --
    CM www.cometenergysystems.com Blog: http://caribbeanrenewable.blogspot.com/
    1. Re:Fibre only? by sjhwilkes · · Score: 3, Informative

      Not to mention 10 gig CX4 - which uses the same copper cables as Infiniband, and works for up to 15M - enough for many situations. I've used it extensively for server to top of rack switch, then fiber from the top of rack switch to a central pair of switches. 15M is enough for interlinking racks too provided the environment's not huge.

    2. Re:Fibre only? by gmack · · Score: 2, Interesting

      If that's true I'm going to be a tad pissed. I payed extra when I wired my apartment so I could be future proof with cat6 instead of the usual cat5e.

    3. Re:Fibre only? by Anonymous Coward · · Score: 0

      If that's true I'm going to be a tad pissed. I payed extra when I wired my apartment so I could be future proof with cat6 instead of the usual cat5e. They'll reimburse you the $10 bucks. Just send them a S.A.S.E.

      People get pissed over absolutely nothing.
    4. Re:Fibre only? by mrmagos · · Score: 3, Informative

      Don't worry, according to the task force for 10GBASE-T (IEEE 802.3an), cat6 can support 10Gbit up to 55m. The proposed cat6a will support it out to the usual 300m.

      --
      Never start vast projects with half-vast ideas.
    5. Re:Fibre only? by Klaus_1250 · · Score: 1

      Future proof would have been cat7.

      --
      It only takes one man to change the Wisdom of the Crowd to Tyranny of the Masses.
    6. Re:Fibre only? by Belial6 · · Score: 4, Insightful

      Unfortunately, you made a fundamental, but common mistake. You cannot future proof your home by running any kind of cable. You should have run conduit. That is the only way to future proof a home for data. When I renovated my last home, I ran conduit to every room. It was pretty cool in that I didn't run any data cables at all until the house was finished. When The house was done, I just pulled the phone, coax and Ethernet lines to the rooms I wanted. If and when fiber, or a higher quality copper is needed, it i will just be a matter of taping the new cable to the end of the old, and pulling it through.

    7. Re:Fibre only? by oolon · · Score: 1

      Yes you have wasted your money unfortunately cat 6 only supports 2.5 Gbs, and unfortunately no one produced equipement to work at that speed because the wiring people came up with the standard before asking if anyone wanted to produce equipement to work over it, cat5e is certified for 1Gbs use. The newer standards like 10Gbs ethernet has been designed with buy in from equipement manufacters, copper 10 or cat 6e was probably what you need. However as 10Gbs (copper) ethernet currently uses 45 watts a port... you probably would not want it for some time yet (next appartment?).

    8. Re:Fibre only? by Anonymous Coward · · Score: 1

      Sorry if this sounds like a dumb question, but how do you pull the first cable since there's nothing to attach it to? And if there is something, how do you pass it into the conduits so that it reaches the exit you need?

    9. Re:Fibre only? by JLester · · Score: 1

      You can: place a string in the conduit as you glue them together, use a fish tape (thin, stiff metal or fiberglass wrapped on a reel) to push through the conduit to the opposite end, use a vacuum with a small spongy ball slightly smaller than the conduit, etc.

      --
      "FORMAT C:" - Kills bugs dead!
    10. Re:Fibre only? by jimicus · · Score: 1

      Conduits join at junctions, and when you put the conduits in you run a piece of string with a weight on it through them. Use the string to draw the cable to the first junction box, tie the cable to another string going in another direction off the box and pull again. Lather rinse and repeat until complete.

    11. Re:Fibre only? by LukeCrawford · · Score: 1
      you use a conduit snake, or a push-pull rod. For short distances, you use a push-pull rod- a flexible fibre-glass rod you can push through to the other side, attach the wire (or a string) to and pull back through. For longer runs, you need a 'conduit snake' which is essentially a big roll of similar (though usually more flexible) material.

      Both these tools operate on the principle that you can force this semi-rigid thing through the conduit to the next entry point by pushing it. Depending on how sharp the turns in the conduit are, it can be difficult, but It's a whole lot easier than doing it without conduit. (and if you have ever run 14ga romex through conduit, compared to that, running a conduit snake or cat5 is pie)

    12. Re:Fibre only? by afidel · · Score: 1

      62.5nm fiber is pretty damn future proof. Considering they run terabits per second over it today I don't think any home network is going to outgrow it during my lifetime =)

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    13. Re:Fibre only? by Pinback · · Score: 3, Funny

      In your case, you really do get to the internet via tubes.

    14. Re:Fibre only? by MrMacman2u · · Score: 1
      --
      This signature is lame.
    15. Re:Fibre only? by Anonymous Coward · · Score: 0

      15 mega what?

    16. Re:Fibre only? by peragrin · · Score: 1

      When i build my home I am going to do just this. It doesn't make sense to put cat5, phone line or even cable TV coax inside a wall without being able to pull it back out later.

      Electric wiring is good for decades communication tech changes dramatically every 5-10 years.

      --
      i thought once I was found, but it was only a dream.
    17. Re:Fibre only? by QuantumRiff · · Score: 1

      What happens in a few years after Cat9 is no longer sufficient? Everyone knows cat's only have 9 lives..

      --

      What are we going to do tonight Brain?
    18. Re:Fibre only? by segfaultcoredump · · Score: 2, Informative

      I guess that is why we run 50/125 multimode everywhere. the 62.5 just didnt cut it anymore for higher bandwidth applications :-)

      Maybe you are thinking about 9micron singlemode fiber?

    19. Re:Fibre only? by afidel · · Score: 1

      MM is ok for short runs with a single wavelength but single mode is what's used for the truly high speed stuff =)

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    20. Re:Fibre only? by Anonymous Coward · · Score: 0

      Details? What kind of Conduit did you use? Plz!!!

    21. Re:Fibre only? by afidel · · Score: 1

      Wrong, Cat6 supports 10Gbit (10GBase-T) over reasonable distances and Cat6A supports it to 100m. If you have a small number of runs you should be able to run to 100m over Cat6 so long as you can physically separate the runs so there is no alien crosstalk (inter-cable crosstalk).

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    22. Re:Fibre only? by hosecoat · · Score: 1

      >62.5nm fiber is pretty damn future proof.

      so your saying that 62.5nm fiber ought to be enough for anyone

    23. Re:Fibre only? by Anonymous Coward · · Score: 0

      M as in meter, not Mega :) - CX4's max. length is about 15 meters ~= 50 feet

    24. Re:Fibre only? by idontgno · · Score: 1

      Trained weasels.

      No, really!

      --
      Welcome to the Panopticon. Used to be a prison, now it's your home.
    25. Re:Fibre only? by Guspaz · · Score: 1

      I don't know what they were smoking when they wrote that proposal, but there IS a copper spec. IEEE 802.3an-2006 (yes, 2006), or 10GBase-T.

      Rated for 55m over Cat6 cable, or the full 100m over Cat6a cable.

      Your Cat6 wiring is non-optimal for 10GigE, but will at least work.

    26. Re:Fibre only? by wtarreau · · Score: 1

      10 gig CX4 - which uses the same copper cables as Infiniband You can't use the same cables. The terminators are in the cable plugs in Infiniband,
      and on the NIC on 10GE.

      But I agree with your point, 15m is enough for most situations. It also has the
      extreme advantage of being affordable, as opposite to the fiber, where a 10 Gig
      XFP transceiver alone costs almost as much as a 10 Gig CX4 NIC.

      And damn that's fast !

      Willy
    27. Re:Fibre only? by NFN_NLN · · Score: 1

      Those copper CX4 style infiniband cables are horrible. They're heavy and they kink under their own weight which is especially bad considering they have a small bend radius. Also, 15m is not long enough. You can link adjacent racks but if you have a row you have to daisy chain everything together. At supercomputing Intel had CX4 IB compatible optical cables over 100 meters. They worked fine except there's a large electrical/optical dongle that sticks out the end. They should have a native optical IB cable and nothing else. One day, thanks to the price of copper, optical will be cheaper and therefore the only option.

    28. Re:Fibre only? by Belial6 · · Score: 1

      Fiber isn't even present proof. Thy hooking up your good old telephone to that fiber. I'm sure it can be done, but you are talking about thousands of dollars per phone. Plus, there is no guarantee that fiber will EVER be common or economical in the home.

    29. Re:Fibre only? by darkpixel2k · · Score: 2, Funny

      Unfortunately, you made a fundamental, but common mistake. You cannot future proof your home by running any kind of cable. You should have run conduit. That is the only way to future proof a home for data. When I renovated my last home, I ran conduit to every room. It was pretty cool in that I didn't run any data cables at all until the house was finished. When The house was done, I just pulled the phone, coax and Ethernet lines to the rooms I wanted. If and when fiber, or a higher quality copper is needed, it i will just be a matter of taping the new cable to the end of the old, and pulling it through.

      Unfortunately, you made a fundamental, but common mistake. You cannot future proof your home by running any kind of conduit. You should have run jefferies tubes. That is the only way to future proof a home for everything. When I renovated my last home, I ran jefferies tubes between every room. It was pretty cool in that I didn't run any data or power cables at all until the house was finished. When the house was done, I just crawled in a jefferies tube and pulled cables to the rooms I wanted. If and when my ODN needs a speed upgrade, it will just be a matter of entering the jefferies tube, reversing the polarity on some colerful widget embedded behind a panel and everything will be just fine.

      ...but seriously--wouldn't jefferies tubes make working on the utilities a million times easier? (And cost a million times more...)

      --
      There's no place like ::1 (I've completed my transition to IPv6)
    30. Re:Fibre only? by jabuzz · · Score: 1

      Assuming that your home does not need more than 55m cable runs (that's a big house if it does and you can probably afford to go on a holiday for a month while you have the house recabled), then Cat6 is good for at least 20 years. With 10GbE you can send 1080p uncompressed video at a whopping 200fps. You need serious high end server kit with mutli spindle SAS or FC RAID arrays to saturate a 10GbE link.

      Lets face it the original 10Mbps which is now 28 years old is still faster than the vast majority of peoples internet connections, and 10BaseT is around 20 years old now.

    31. Re:Fibre only? by oolon · · Score: 1

      It would not suprise me if it worked "ok" using a .5 metre cat 5 patch lead as it is electrically compatible, and I am sure the guy in the flat will probably the wiring works well enough, as I will with the stuff I am have in my house. That completely different getting a certified installation like I would in the datacentre I was in charge of. The company installing it certifies it will work at the rated speed over all the cables they lay. The standard is all about the complete wiring solution, not just the cable but also how it is run, for example they did not use standard cable ties as they tend crush the cable etc.

    32. Re:Fibre only? by Belial6 · · Score: 1

      Your install would not have suffice even at completion. Your install is only going to handle TCP/IP traffic. What about the telephone lines, audio lines, and video lines? Basically you have missed a huge amount of data that is being shuffled around your average home. You like most people pulling wire directly are too wrapped up in solving a small problem to see the big picture.

    33. Re:Fibre only? by Belial6 · · Score: 1

      I honestly cannot tell if you are trying to make a point, or just trying to make a joke. If you are trying to make a point, it doesn't make sense, since installing conduit is pretty much just as easy as installing wire directly, it doesn't take any more room, and is pretty darn cheap. Plus, the equivilent of a Jefferies tube already exists in most houses. It is called an "attic", and the "crawl space" under the house. Sometimes there are really big ones called "basements".

    34. Re:Fibre only? by MeepMeep · · Score: 1

      Don't worry, according to the task force for 10GBASE-T (IEEE 802.3an), cat6 can support 10Gbit up to 55m. The proposed cat6a will support it out to the usual 300m.


      Not to be a cabling nazi but I think you meant 328 feet (100 meters equivalent) not 300 meters, right?

      http://en.wikipedia.org/wiki/Category_6_cable
    35. Re:Fibre only? by masonc · · Score: 1

      Any professional installing Commercial Cabling Systems looks to the standards to define the acceptable systems. Sure, there are lots of solutions that might work over short distances, but the standards call for 90M cable lengths for UTP and that's what installers look for.
      AMP has a copper 10G solution using shielded pairs that works well. I just completed a training course on the installation of this system and it's impressive. Install this and the future is pretty much proof. It will be a long time before residential systems exceed 10G, I can't get commercial installs to consider it yet.

      --
      CM www.cometenergysystems.com Blog: http://caribbeanrenewable.blogspot.com/
    36. Re:Fibre only? by deamonpainter33 · · Score: 1

      that is incredibly thoughtful! i know a few people that think there the shit for wiring up there houses with cat5e or similar grade. they think there the shizz because they don't have to cut there walls up since they got it done when the house was built etc etc. well i'll be keeping this in mind. conduit will keep the rats and termites out!!!! HAHAHA!!!!

      --
      "In the kingdom where everything dies, the sky is mortal."
    37. Re:Fibre only? by darkpixel2k · · Score: 1

      If you are trying to make a point, it doesn't make sense, since installing conduit is pretty much just as easy as installing wire directly, it doesn't take any more room, and is pretty darn cheap. Plus, the equivilent of a Jefferies tube already exists in most houses. It is called an "attic", and the "crawl space" under the house. Sometimes there are really big ones called "basements".

      Both a point and a joke.

      Personally, I would love to have jefferies tubes in my house. An attic is used for storing your Christmas ornaments and lights for the 300 days of the year they aren't in-use. The crawl space is...well...good for nothing. And lastly, most cabling is run inside the walls. Yeah--some may be under the house or in the attic--but to get to the plugs, it has to go through the walls.

      The big difference between a a jefferies tube and a crawl space however, is that I've never seen a spider the size of a volkswagen in a jefferies tube...

      --
      There's no place like ::1 (I've completed my transition to IPv6)
    38. Re:Fibre only? by petermgreen · · Score: 1

      Surely the way to avoid daisy chaning racks is to have a switch in the rack with a load of copper ports and one fiber port and then a core switch with all fiber ports.

      That should certainly be possible with 10 gigabit ethernet, I dunno about infiniband.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    39. Re:Fibre only? by petermgreen · · Score: 1

      I'm sure they are usefull for large commercial/industrial ducts but there is no way they would get down normal building wiring conduit.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    40. Re:Fibre only? by iowannaski · · Score: 1

      millimeters

      --
      i forget
    41. Re:Fibre only? by afidel · · Score: 1

      Yes. My current broadband is measured in megabits per second and 62.5nm fiber is capable of terrabits per second. If Bill had said 640GB of ram should be enough for anyone he would have been right for a VERY damn long time =)

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    42. Re:Fibre only? by Anonymous Coward · · Score: 0

      Simple, we can put telephone/audio/video OVER tcp/ip. Ever heard of voip/audiostreaming/videostreaming?

    43. Re:Fibre only? by Shimbo · · Score: 1

      From their white paper,
      "The draft standard for 10 Gigabit Ethernet is significantly different in some respects from earlier Ethernet standards, primarily in that it will only function over optical fiber, and only operate in full-duplex mode" That's not their white paper; it's an old one by the 10Gb Ethernet Alliance, which was wound up in 2003. It predates the standards work on copper 10Gb Ethernet.

      This article is about the 10 Gigabit Ethernet *Storage* Alliance i.e. iSCSI vendors. Completely different animal: confusing article title.
    44. Re:Fibre only? by owndao · · Score: 1

      Or, blow a light weight string or yarn through it with your wet vac hooked up to blow. Of course you have to cap all open ends except the one you want to run the string to (also works for wild bends). Also, if you are pulling a large bundle of cables back there is a kind of cable lube for pulling through conduits. It looks like partially solid hand soap. Don't underestimate the tension needed to pull around lots of bends. In industrial settings I've seen crews use a crane to pull long runs of large bundles! Good luck.

      --
      Be as you would have the world become.
  3. How about hard drive speeds by Anonymous Coward · · Score: 0

    Networking is getting faster in leaps and bound, yet hard drives are still uber slow.

    1. Re:How about hard drive speeds by malinha · · Score: 1

      Networking is getting faster in leaps and bound, yet hard drives are still uber slow.
      But getting bigger and bigger
    2. Re:How about hard drive speeds by cshake · · Score: 2, Insightful

      That seems to be the idea behind this spec - a network interface that is as fast as a drive interface on a local machine, which would allow for nearly transparent remote drives, or even striped and mirrored raid across multiple machines to make it really fast. It really would be nice to see that.

    3. Re:How about hard drive speeds by Em+Ellel · · Score: 1

      Networking is getting faster in leaps and bound, yet hard drives are still uber slow. Yeah, but they are getting larger and cheaper. You can have drives in parallel (RAID) which will increase your throughput significantly. If you have $$ to spend on 10Gig link, you are unlikely to be hooking up a single drive.
      --
      RelevantElephants: A Somatic WebComic...
    4. Re:How about hard drive speeds by jimicus · · Score: 1

      By the time you account for seek time and latency, we've practically got that already with gigabit ethernet unless you're already running a reasonable RAID.

    5. Re:How about hard drive speeds by arivanov · · Score: 1

      Exactly, and I am not particularly sure about the benefits of 10GE vs Fiberchannel or Infiniband. Ethernet tends to have higher latency than these two and latency is what makes people go for SAN instead of NAS in the first place.

      --
      Baker's Law: Misery no longer loves company. Nowadays it insists on it
      http://www.sigsegv.cx/
    6. Re:How about hard drive speeds by Shadow-isoHunt · · Score: 1

      Unless your SAN is hitting solidstate disks to get it's data, or the data is in the diskcache, you're not going to notice the difference either way. Even then if the latency is more than 1 to 3ms for a LAN, you need to take a look at it.

      --
      www.isoHunt.com
  4. speed by Anonymous Coward · · Score: 0

    I bet in 8 years 1GB/sec will be normal. I'm already downloading at 2.5MB/sec from my ISP roadrunner.

  5. Awesome! by Anonymous Coward · · Score: 0

    Can't wait to wire my house with 10Gbit ethernet while my cable modem is lucky to get 10Mbit during off-peak hours.

    1. Re:Awesome! by moderatorrater · · Score: 2, Insightful

      You're not using your home network like you should be then. I often find myself transferring multiple gigabytes of information from one computer to another.

    2. Re:Awesome! by calebt3 · · Score: 2, Insightful

      Why not?

    3. Re:Awesome! by mrsteveman1 · · Score: 2, Funny

      BECAUSE silly, you are supposed to be using quantum drives by now. Flip a bit on drive 1 in room A, and drive 2 in room B flips the same way.

      Sheesh

  6. Misleading Title. by Anonymous Coward · · Score: 2, Insightful

    The 10GEA is not the same as the storage alliance mentioned in TFA.

  7. Block storage? by mosel-saar-ruwer · · Score: 2, Interesting


    By combining block and file storage on one network, they say, you can cut costs by 50% and simplify IT administration.

    What is "block" storage?

    1. Re:Block storage? by spun · · Score: 4, Informative

      SAN is block storage, NAS is file storage. Simply put, if you send packets requesting blocks of data, like you would send over your local bus to your local hard drive, it is block storage. If you send packets requesting whole files, it is file storage.

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    2. Re:Block storage? by QuantumRiff · · Score: 1

      The previous reply was good, just wanted to expand. File access is literally grabbing a file over the network. Like opening a word document. It pulls the entire file over the network, then opens it.

      Your hard drive is a block device. A SAN just uses some protocols to make the OS treat a remote storage as a local disk (Think of it as scsi going over the network, instead of a local cable, which is almost exactly what iSCSI is). You can format, defrag, etc. The OS does not know that the device isn't inside the box. Very usefull for things like databases, because you just modifiy the blocks (think of it as clusters/sectors on the hard drive that need to be changed.

      Also, block storage is usually dedicated to a particular server, so you don't need to worry about "locking" a file that is open for changing like with NFS to ensure integrity.

      --

      What are we going to do tonight Brain?
    3. Re:Block storage? by Guy+Harris · · Score: 2, Informative

      SAN is block storage, NAS is file storage. Simply put, if you send packets requesting blocks of data, like you would send over your local bus to your local hard drive, it is block storage. If you send packets requesting whole files, it is file storage.

      No. If you send packets requesting blocks of data on a region of disk space, without any indication of a file to which they belong, that's block storage. If you send packets opening (or otherwise getting a handle for) a file, packets to read particular regions from a file, packets to write particular regions to a file, packets to create, remove, rename files, etc. that's file storage.

      Most of the file access protocols out there (NFS, SMB/CIFS, AFP, NCP, etc.) permit you to read or write particular regions of a file (they don't even have to be aligned on block boundaries; they don't require whole file access. That's NAS, not SAN.

      There are protocols used on SANs that mix file and block access, e.g. the protocols used by Quantum's StorNext, where create, delete, rename, open, etc. operations go to a metadata server and involve files, but reads and writes are done directly to the disk blocks in question over the SAN (you ask the metadata server for information to let you know what blocks on the SAN corresponds to particular data within a file).

    4. Re:Block storage? by spun · · Score: 1

      Thanks for clarifying that. But I've never heard anyone refer to random access on a given file as 'block' storage.

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    5. Re:Block storage? by DamageLabs · · Score: 1

      I am sitting here just now contemplating should I go iSCSI over Ethernet or NFS (over the same gigabit Ethernet) for a small VMware Server (to cheap for ESX) deployment.

      My brain tells me that iSCSI should be faster and simpler - one filesystem layer less to translate - but it seems that NFS is simpler and not much slower; actually faster when sharing a datastore with multiple VMware physical servers.

      Has anybody got experience on how linux NFS deals with large (10-100GB) files being mounted as virtual file systems on remote end? And which local filesystem would be ideal for sharing vmdk files over NFS?

    6. Re:Block storage? by Anonymous Coward · · Score: 0

      may as well mention shared storage by servers using clustered filesystems is also block level. The open source products for such filesystems are lagging the proprietary ones in capability and performance.

    7. Re:Block storage? by pigs · · Score: 0
    8. Re:Block storage? by QuantumRiff · · Score: 1

      I don't know about VMWare server, but I do know that MS recommends Block storage over file storage for Virtual Server 2005. Ask VMWare.

      --

      What are we going to do tonight Brain?
    9. Re:Block storage? by DamageLabs · · Score: 1

      Their answer would be simple: buy ESX, buy FC storage, buy buy...
      I am trying to skip some of those buy steps for my small deployment.

    10. Re:Block storage? by Anonymous Coward · · Score: 0

      You should take a very careful look at what ESX can and can not do with both vmfs (on iSCSI or any other type of storage) and an NFS filesystem. If my memory serves, things like VMotion only works with a single vmfs pool on a SAN, so NFS isn't going to cut it.

    11. Re:Block storage? by QuantumRiff · · Score: 1

      You don't need to actually buy anything for block storage.. There are open source iscsi target drivers for most operating systems. Most also have initiator drivers.

      --

      What are we going to do tonight Brain?
    12. Re:Block storage? by petermgreen · · Score: 1

      The previous reply was good, just wanted to expand. File access is literally grabbing a file over the network. Like opening a word document. It pulls the entire file over the network, then opens it.
      hmm, i'm pretty sure both nfs and smb only transmit the bits of the file the app wants (and maybe a little bit extra to reduce network traffic) not the entire file.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    13. Re:Block storage? by turbidostato · · Score: 1

      "I am sitting here just now contemplating should I go iSCSI over Ethernet or NFS (over the same gigabit Ethernet) for a small VMware Server (to cheap for ESX) deployment."

      Then you should try a third way: ATAoE from Coraid. For your kind of deployment can be half the cost of iSCSI, outperforms NFS and it just works (and for a real cheap try you can go with your standard linux boxes first and, if convinced go then for a disk cabin and a gigabit switch for a SAN-only network).

    14. Re:Block storage? by atamido · · Score: 1

      I guess it depends on the quality of your iSCSI initiator. If you were going to use VMware ESX, then you would want to go NFS as the VMware iSCSI driver is terrible and slow. If your host environment is Windows and you are using the Windows iSCSI initiator, and then providing that storage to VMware Server, then you are probably better off with iSCSI. I don't know about the speed of current open source iSCSI initiators.

  8. SCI, Infiniband by Colin+Smith · · Score: 1

    etc.

    I can do this already. Up to 90 odd Gbit.

    Ethernet will have to be cheap.

    --
    Deleted
    1. Re:SCI, Infiniband by afidel · · Score: 2, Insightful

      There are literally several orders of magnitude more ports of ethernet sold per year than fiberchannel and there are about an order of magnitude more fiberchannel than infiniband. Most of the speakers at storage networking world last week think that it's inevitable that ethernet will take over storage, the ability to spread R&D out over that many ports is just too great of an advantage for it not to win in the long run.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    2. Re:SCI, Infiniband by jd · · Score: 1
      I thought the current limit on Infiniband was 12 channels in any given direction with 5 Gb/s per channel (and even then that only applies if you're using PCI 2.x with the upgraded bus speeds), giving you a peak of 60 Gb/s. Regardless, 60 Gb/s is still well over the 10 Gb/s of Ethernet. More to the point, latencies on Infiniband are around 2.5-8 us, whereas they can be 100 times as much over Ethernet. Kernel bypass is another factor. It exists for Ethernet, but it's rare, whereas it's standard for Infiniband. Remember, Linux has something like a 20 ms context switch time and that's low, so you really want to keep switching to and from the kernel to a minimum.

      SCI is definitely an interesting technology - I've seen several presentations from Dolphinics - and it would seem to be ideal for something like a storage system. Not sure what the current limitations are.

      There are certainly other networking technologies out there, and some of those may also compete in this market. Part of the problem, I think, is that there is a lot of scattered information out there and very little independent, organized collection and dissemination of it. You wouldn't need a consortium to promote 10 Gb Ethernet if it was already clear to people what 10 Gb Ethernet actually offered on a practical, day-to-day basis. Not ping-pong tests, not dodgy benchmarks, not marketspeak but actual practical information and some form of real-world guide on how to map user requirements onto what each network technology would realistically deliver.

      --
      It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
    3. Re:SCI, Infiniband by Anonymous Coward · · Score: 0

      Yes, and the problem with 10GB ethernet is that it is _not_ cheap... Still many people seem to buy it, which is rather weird.

      Infiniband cards and switches are dirt cheap in comparison, AND have much lower latency and higher bandwidth.

    4. Re:SCI, Infiniband by atamido · · Score: 1

      It's advantage is existing infrastructure. Heck, I could take a Cat6 cable from our current building and plug it into a 10GbE switch and be just fine. I could do that to connect it to one of our 1GbE switches and instantly expand our network. The supporting infrastructure is everywhere, and just about everyone understands it.

      It may not be anywhere near the technical best, but it has too much inertia at this point to be stopped, and the economies of scale with keep it cheap enough for people to keep using it.

    5. Re:SCI, Infiniband by Colin+Smith · · Score: 1

      Sometimes 10Gbit simply doesn't do the job.

      --
      Deleted
  9. bandwidth = performance ? by magarity · · Score: 4, Interesting

    So how will tcpip networking over this speed measure up to dedicated storage devices like SAN over fibre channel? I have to suspect not; existing iSCSI over 1GB tcpip is a lot less than 1/4 of 4GB fibre to a decent SAN. Sigh, I'm afraid even more of my databases will get hooked up to cheap iSCSI over this instead of SANs space that costs more dollars per capacity but delivers the speed when needed :( Reports coming up fast enough? Remember the planning phase when the iSCSI sales rep promised better performance per $ than SAN? It wasn't better overall performance, just better per $. There's a BIG difference.

    1. Re:bandwidth = performance ? by RingDev · · Score: 1

      If the practical bandwidth differences between bleeding edge SCSI and bleeding edge Ethernet over fiber between the physical storage of your data, the controller of the database, and the requester of the data, is the limiting factor of your "reports coming up", there is either a fundamental design issue going on, or your clients are sitting at the terminals with stop watches counting the milliseconds of difference.

      -Rick

      --
      "Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
    2. Re:bandwidth = performance ? by Znork · · Score: 2

      existing iSCSI over 1GB tcpip is a lot less than 1/4 of 4GB

      I'd have to wonder what kind of config you're running then. I've gotten 90MB per second over $15 RTL8169 cards and a $70 D-Link gigabit switch. Between consumer grade pc's running ietd on linux to a linux iscsi initiator. I have no doubt that 10GB ethernet will wipe the floor with FC.

      Remember the planning phase when the iSCSI sales rep promised better performance per $ than SAN?

      Remember the planning phase when the SAN vendor promised cheaper storage than disks in every server? I saw an article the other day about a SAN consultant who had helped companies cut storage costs by $75000 per terabyte. That's impressive for something that costs around $200...

      Your database servers may have some requirements (particularly if, as is so often the case, the application developers are using the database the wrong way), but the vast majority of servers can share SAN and NAS connection without a problem, even on 1Gb networks.

      So get the expensive option for your databases and let them carry the whole cost for the expensive infrastructure. Maybe it turns out you'd be better off distributing the databases and putting them on cheaper hardware too. Consolidation and expensive hardware isn't an end to itself (well, except for the ones actually selling the expensive hardware).

    3. Re:bandwidth = performance ? by oolon · · Score: 1

      It also means when the networking team does a bad firewall change, not only will prevent user access it will mess up the storage, requiring alot more work to get your databases up and filesystems running again or atleast a forced shutdown and reboot. Personally I would not want to share block storage on my public interface in any case, as iscsi is not designed as a highly secure standard, as this would impact performance, public interfaces generally has more sufficiated firewalling rules on the switch gear with slows performance. It also means your need yet another person available for storage changes. It might seem sensible at the low end to share switches, however the public switches need to route everywhere, storage it generally limited to within a building, and all special cases need to be considered to avoid over loading things.

    4. Re:bandwidth = performance ? by afidel · · Score: 1

      10Gb ethernet is 20% more bandwidth than bleeding edge 8Gb FC and it's a fraction of the cost per port. For new installations it's a no-brainer. For places with both infrastructures it's most likely to be evaluated on a per box basis.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    5. Re:bandwidth = performance ? by Sentry21 · · Score: 1

      One of the big issues you should consider is not just whether you are using jumbo framers or not. Some people claim a minimal performance increase, but jumbo frames can significantly reduce transmission/reception overhead on a gigabit network when doing block data transfers between 1500 and 9000 bytes.

      For a database server, it depends on your read/write patterns, but especially when doing large blocks of data, it can make a difference in both CPU use and throughput. Might be worth a look, but the NIC, switch, and iSCSI SAN all need to support it (which might not be the case).

    6. Re:bandwidth = performance ? by towelie-ban · · Score: 1

      25% more bandwidth, not 20%. And to think, the original poster was worried that the article had to explain how math works.

    7. Re:bandwidth = performance ? by QuantumRiff · · Score: 2

      It is usually recommended to run a seperate network for the storage network. It is possible to run storage over the same nic that the server uses for other network traffic, but is not recommened (but is used often as a "failover"). This also helps when you turn on Jumbo Frames, some servers just don't like to work correctly. Seperate network makes it better.

      However, the best advantage of iSCSI over FC is replication. How much extra infrastructure and technologies do you need to replicate to a site 1000 miles away? Compare that with FC, and its special software and hardware to do replication over TCP/IP.

      --

      What are we going to do tonight Brain?
    8. Re:bandwidth = performance ? by afidel · · Score: 1

      20% more usable, there's more overhead for ethernet =)

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    9. Re:bandwidth = performance ? by rathaven · · Score: 1

      8Gb FC v 10Gb iSCSI? Well thats 2Gb potential bandwidth more... Tcpip is not really a limiting factor. In test I've seen boxes demo'd using IOMeter and the like showing iSCSI boxes with 10Gb with throughputs of 800MBps or higher whilst 2 x 4Gb FC connected SANs of a higher price were only pushing 600MBps. As someone else here says - the disks are slow. The limitation in any well designed SAN is disk spindles and disk seek times and unfortunately most iSCSI boxes are SATA disks not SAS or Fibre Channel disk. SATA is big with higher seek times and it has approximately 1/4 the throughput of SAS or FC disk. If your disk is decent, the raid cards in the iSCSI boxes are up to the task (some are just plain cheap - if they aren't just cheap servers running Windows Storage Server) and you have the network bandwidth on the iSCSI boxes then you should look elsewhere for your performance limit not blame iSCSI or TCPIP in general. Q. Are your switches configured for Jumbo frames? Q. Is you bandwidth to the SAN through the same network cards as the normal network data? Q. Are your servers performance not able to load the iSCSI boxes? Any of these may be the other factors...

    10. Re:bandwidth = performance ? by Anonymous Coward · · Score: 0

      I have no doubt that 10GB ethernet will wipe the floor with FC.

      No, it won't. First off, 8 and 10 Gb FC are available now. FCoE, which isn't done yet, still has more overhead than FC by itself. If you're thinking iSCSI, then you're deluding yourself -- that's ethernet with TCP, which means even more overhead.

    11. Re:bandwidth = performance ? by oolon · · Score: 1

      Your quite right in your points, I would still see it as a entry level installation and not really in competition with the FC equipement. We had 3 fibre fabrics, two for data/failover one for backup, all with dedicated switches. We moved to a site which had dark fibre for remote storage connections, however if your running it over 1000 miles, using iscsi, and host based replication, thats got to have real latency/syncronisation problems and for that kind of setup a magic boxes like IBMs SVCs come into there own. The Tagmastore we were using cost 1 million pounds, all the nics we needed were bundled with it as were the switches! For jumbo frames you might be able to run different MTUs on different vlans, if you are nic sharing.

    12. Re:bandwidth = performance ? by Anonymous Coward · · Score: 0

      It is usually recommended to run a seperate network for the storage network.
      Which negates all the supposed cost savings of this unified ethernet storage propaganda. Hell, most of the FCoE equipment doesn't even exist yet.

      However, the best advantage of iSCSI over FC is replication.
      Incorrect. Try FCIP.

    13. Re:bandwidth = performance ? by gwk · · Score: 2, Insightful

      Your right iscsi does perform acceptably for a lot of uses but I feel I should point out that your results probably are a reflection of linux cacheing and read ahead performance, most gig-e gear now seems to do wire speed which is great but gig-e might not cut it. FC still provides considerably more bandwidth and lower latency. 10GBe will improve that but I read a benchmark of 10GBe HBAs (which I apologize but I can't find) that showed the maximum performance that could be got out of the operating system and HBA at 2.5Gb/s which is no where near advertised and lower than FC. Too me this isn't really surprising TCP/IP imposes quite a bit of overhead e.g. run bonnie++ on an NFS share over IB without any of the cute RDMA stuff and watch the machine spend 30% of its time in kernel.

    14. Re:bandwidth = performance ? by Limited+Vision · · Score: 1

      These guys are all pretty small networking are storage players. They're not getting anywhere in the enterprise with iSCSI and they think 10Gb ethernet is going to save them.

      Guess what -- I/O takes CPU. Somewhere you're doing about a GHz of CPU to drive a Gbit of I/O. This might be on a TOE, or an HBA, or on the CPU itself, but it's happening somewhere...

      Remember also that 10GbE is not lossless Data Center Ethernet / Converged Enhanced Ethernet. Storage does NOT like lost packets, and you have to compensate for that. That eats CPU cycles and takes away from bandwidth, or adds costs if you offload it.

      While I understand the desire to consolidate cables, think of what happens to your computer when you pull out your ethernet cable, and when you pull out your hard drive... It seems like the ethernet guys want to reinvent the ethernet and make it more like Fibre Channel (lossless, low latency, etc). So why not just use FC?

      Don't get me wrong, I definitely see network convergence over the next decade. I just think the Ethernet evangalists are naive in how quickly it will happen. Standards, interoperability, QoS and reliability are a bitch and take more time than you ever thought possible.

      I have a feeling storage over ethernet is going to look a lot like the issues we saw with backup-over-the-LAN, and how people were surprised how it affected their other network traffic...

    15. Re:bandwidth = performance ? by Anonymous Coward · · Score: 0

      I've just spent this week evaluating iSCSI for DB storage consolidation.

      The 1GbE testbed, open-iscsi software initiators on IBM pizza boxes, jumbo frames and 3com switches - 119MB/s sustained.

  10. Bonding for Unlimited Bandwidth by Doc+Ruby · · Score: 1

    If these new fast ethernet specs came with specs for plugging multiple parallel paths between machines all under the same host IP#s, so we just add extra HW ports and cables between them to multiply our bandwidth, ethernet would take over from most other interconnect protocols.

    Is there even a way to do this now with 1Gb-e, or even 100Mbps-e? So all I have to do is add daughtercards to an ethernet card, or multiple cards to a host bus, and let the kernel do the rest, with minimal extra latency?

    --

    --
    make install -not war

    1. Re:Bonding for Unlimited Bandwidth by Feyr · · Score: 1

      yes, look up "etherchannel" or "bonding"

    2. Re:Bonding for Unlimited Bandwidth by SpacePirate20X6 · · Score: 1

      The two gigabit ethernet plugs on the Mac Pro can do this; they can be combined to serve as one 2Gb connection.

    3. Re:Bonding for Unlimited Bandwidth by Phishcast · · Score: 1

      Kind of. This method of aggregating bandwidth by using multiple links does poor man's load balancing. The traffic between one source and one target will only traverse a single path until that path fails. If you have a lot of different sources on one side of an etherchannel going to a lot of different targets on the other side of the etherchannel, you get a relatively balanced workload. If you've got a smaller number of sources and targets it's easy to get uneven bandwidth utilization across those links. This is more common in networked storage than in everyday IP land.

    4. Re:Bonding for Unlimited Bandwidth by witherstaff · · Score: 1

      I don't believe anyone has done this as you'd need to make new switches as they use the MAC to route traffic. Potentially I could see this done between two machines with custom code, directly wired together, no need to get custom switches then.

      That is a neat idea, a real use for the extra 10/100 NICs that everyone has laying around, and for speed increases without rewiring.

    5. Re:Bonding for Unlimited Bandwidth by imbaczek · · Score: 2, Informative
    6. Re:Bonding for Unlimited Bandwidth by Em+Ellel · · Score: 2, Funny

      yes, look up "etherchannel" or "bonding" Wow, that takes me back years. A little over 10 years ago, straight out of college and not knowing any better I purchased the "cisco etherchannel interconnect" kit for their catalyst switches. I had to work hard to track down a cisco reseller that actually had it (which should have been a clue). When I finally got it, the entire "kit" contents were, I kid you not, two cross connect cat5 cables. I learned an important lesson about marketing that day.

      -Em

      P.S. In all fairness to Cisco the cost of the kit was about same as you would expect to pay for two cross connect cables in a retail store. Not that I would have bought cat5 cables at a store.
      --
      RelevantElephants: A Somatic WebComic...
    7. Re:Bonding for Unlimited Bandwidth by Doc+Ruby · · Score: 1

      No, it looks like what I asked about is standardized as IEEE 802.3ad, and even implemented in the Linux kernel. But does it actually work?

      --

      --
      make install -not war

    8. Re:Bonding for Unlimited Bandwidth by Doc+Ruby · · Score: 1

      It looks like that's even supported in the Linux kernel. But does it really work?

      --

      --
      make install -not war

    9. Re:Bonding for Unlimited Bandwidth by Dr.+Spork · · Score: 1

      Great story!

    10. Re:Bonding for Unlimited Bandwidth by Feyr · · Score: 1

      iirc, some switches provide more sorts of hashing than just src/dst mac. it's been a while, so i'd have to look it up again

    11. Re:Bonding for Unlimited Bandwidth by petermgreen · · Score: 1

      With the right NICs and switches you can already do this. You will probablly find it much easier if all the nics are from the same range.

      You may be able to get away with cheap nics if you are running linux (you won't be able to if you are running windows as bonding must be implemented at the device driver level). You will certainly need managed switches which explicitly support this feature.

      http://en.wikipedia.org/wiki/Link_aggregation seems like a good starting point for finding information on this topic.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    12. Re:Bonding for Unlimited Bandwidth by Anonymous Coward · · Score: 0

      That's a horrible idea. You'll hear it called trunking/bonding/802.3ad. I've used it in my ISP for almost ten years with our cisco switches. The problem is that it doesn't improve the speed of the connection at all. The latency is still the same. You get double (or triple if you have three connections, and so on) the capacity, but for most purposes the connection is no faster. However moving to a faster single connection will decrease the latency so that is why people are pushing a real solution rather than the cheap hack of bonding.

      If it is unclear why it is no faster, then think of it this way. Even if you have ten connections if you want to transmit a packet it still takes the same amount of time as it would with an unloaded single connection. Each of the ten paths is serial at the packet level.

      The reason I still use bonding is for redundancy. I have two NICs in each of my servers connected to two different cards on my big cisco switch. That has saved me several times when someone has unplugged a cable, a NIC quit, or (the most common scenario by far) one of the cisco ports quit working.

    13. Re:Bonding for Unlimited Bandwidth by Doc+Ruby · · Score: 1

      Actually, I don't see at all why having 10 serial 1Gb-e connections that are each serial at the packet level would not be something like 10x as fast as a single connection.

      No, not improved latency. But I'm not talking about latency, I never said anything but that I didn't want more than minimal extra latency, not that I want the parallelism to reduce latency.

      The question is whether 802.3ad multiplies bandwidth over multiple parallel connections. For decreasing latency, of course each of the parallel links has to be faster, lowered latency. But the problem of one ethernet link isn't its latency (not for me), but its bandwidth. You don't mention increased bandwidth, only reliability from redundancy. Don't you get more bandwidth?

      --

      --
      make install -not war

  11. Cost? by segfaultcoredump · · Score: 1

    Hope they fix the pricing issue, because I the FC network I just put in cost less per Gb than a 1 gig-E network. When compared to the cost per Gb of a 10G-E netowrk, the entire thing cost less than the optics on the 10G stuff, let alone the actual switch costs.

    I'm also noticing that most if not all of my systems never even tap the bandwidth available on a pair of 4Gb FC ports, let alone 10Gb. I'm sure there are folks out there who need it, but it aint us.

    Of course, our corporate standard is Cisco, so I'm sure that had something to do with the high cost of the ethernet equipment ;-)

    1. Re:Cost? by Phishcast · · Score: 1
      Didn't you buy Cisco Fibre Channel gear then? I don't recall their FC gear being inexpensive either!

      The 10Gb ports aren't really about the hosts (today anyhow). They're generally more useful for the connections to large storage arrays which can push that kind of bandwidth, you'd be able to fan in more hosts to each storage array port.

    2. Re:Cost? by segfaultcoredump · · Score: 1

      Our original SAN consisted of four MSS 9216's with the 32 port blades for a total of 48 ports per switch. Two at each site (stretched fabric, 3k of dark fiber between the two sites, dual fabric, bla, bla, bla) Every server has two ports and thus full redundancy in the event of a switch/cable/whatever failure. (ever try and bond two ethernet ports between two switches?).

      To replace the aging MDS switches we went with four Brocade 4900's. We have what I would call a medium sized environment, so the larger brocade switches did not make sense, but then again neither did the little things like the Brocade 200E. We have some hosts who will push 500+MB/s of traffic through our SAN, so trying to bond a ton of ethernet ports together did not make sense (think of the cables). Of course, bonding to a single switch is easy, ever try bonding between two physically different switches for redundancy? (to save money, our 6509's only have 1 sup, so we need switch diversity). Storage multipathing is easy with a dual port 4Gb hba. One could argue that a dual port or a 10G-E card allows for failover but removes the need for bonding. Of course, guess which one is cheaper :-)

      The nice thing about the 4900's vs the cisco MDS switches are as follows;
      1) Smaller (2RU for 64 ports)
      2) No oversubscription (64 ports, 4Gb any to any)
      3) Less power. (Not even close)
      4) Hitless firmware upgrades (cisco sorta has this with the MDS 9500, at 10x the cost)
      5) Ooooh, shiny box :-)

      At the same time our network folks were purchasing some 10G line cards for their Cat 6509's. Lets just say that the 8 port card with optics cost more than my 64 port 4900 with optics. They got 160Gb of bandwidth (assuming you never leave the card when it quickly falls to 40Gb), I got 512Gb.

      Just for kicks I also looked at their costs for the 48 port gig-e line cards. Loking at just the cost of the cisco line cards (6748's) and ignoring the cost of the chassis and supervisor, my cost per Gb was just a touch under theirs (and mine included the whole chassis). Of course, 256 ports of gig-E takes up a lot more space, more cables and I still have the bonding issue for redundancy.

      I'm sure some day we will see it all merge together. Whitness the Magnum switch from Sun: http://www.sun.com/products/networking/datacenter/ds3456/

      Until then, I'm sticking with ethernet for IP and FC for storage.

    3. Re:Cost? by Phishcast · · Score: 1
      It looks to me like everything on your list about the 4900s could be achieved using the stackable MDS 9134 switches. You get a 64 port switch in 2RUs, 4Gb line rate ports (no oversub), hitless firmware upgrades and less power than your old 9216s. There aren't two supervisors like in the director class MDS switches, but I suspect the same is true for your Brocade 4900s (I've never looked into them).

      Interesting you point out a Sun Infiniband switch as the a possible option to "merge it all together". Cisco's idea of a unified datacenter fabric is based on Ethernet, see Nexus. I dunno...Infiniband is certainly cool stuff, but could it ever overtake Ethernet in the datacenter?

      By the way, it may sound like I work for Cisco. I don't. I do, however, manage large Cisco MDS Fibre Channel SANs.

    4. Re:Cost? by evilbessie · · Score: 1

      This is only really for the datacentre for the moment, from servers to SAN. So yes if you have already paid for plenty of bandwidth using current tech then maybe you're not going to go this route, but those companies who are upgrading may go for this because of a possibility of lower TCO. That and 40/100GbE is on the way so there is an upgrade path for the future.

    5. Re:Cost? by segfaultcoredump · · Score: 1

      The 9134's look a lot like the Qlogic 5600's. They both have the same issue: Stacking two will oversubscribe the 10Gb FC interconnects. I know that there are ways to avoid this in theory, but luck always has it that I never have a port available where I need it for optinal performance. Thus things like the 4900 are so nice in that it is next to impossible to screw up :-)

      That said, I'm happy to see that cisco finally got their power consumption under control with the 9134's. The older models sucked so much power that it was not even funny (Like a lot of folks, I'm limited on 3 things in my datacenter right now: space, power and cooling)

      I did entertain quotes for a 9506 from the cisco folks. But when I compared the physical size, power requirements and cost (2x as much for 64 blocking ports), it did not make sense in my environment. The 9509 was just way too big to put anywhere after our network team droped in a pair of 6509's at each site and took most of the remaining power and space that we had. The cisco folks never even offered the 9134 as an option.

      As for the Nexus vs Sun magnum, Cisco's bread and butter is ethernet, so that made sense. Sun just went for whatever gave them the best performance, and the fact that the HPC world was already used to Infinitiband made sense for them. That said, sun moves 110Tb/s in their switch vs Cisco's "System designed for investment protection with a 15 Tbps highly scalable fabric". Ok, so the cisco one is a bit smaller, but still, its not that much smaller :-)

    6. Re:Cost? by segfaultcoredump · · Score: 1

      I have to make the decision with what is available now based on what my requirements are and what I estimate them to be over the next 3 to 4 years.

      In other words, my choices today were gig-e and 10G-E iSCSI, 4Gb and then 10Gb FC. Anything else was not an option since it simply does not exist outside of the lab (or at a resonable cost).

      At this time, 4Gb FC made the most sense for our storage network. It is well understood, everybody supports it and the things "just work". I didnt see any major storage vendors offering 10G-E iSCSI arrays just yet. Of course, those Force10 switches sure do look nice :-)

      I did pick up a a pair of 10Gb-E cards for our netbackup server since right now it is saturating the Gig-E network that we have for hours at a time trying to pull in data from all of the clients (fun with D2D2T backup solutions) (and for those wondering, off host backups are not an option for a bunch of individual app servers using internal storage) Of course, once the data is on the NBU server, it sends it to the array via 4Gb FC at a rate of 600MB/s. And then there is the part when it needs to move the data from the array to the LTO tape drives (a single lto-3 can move at 160MB/s, and they dont support iSCSI).

      I hope that in 4 years or so, I can stop running so many damn cables to the servers and get away with a pair of 10Gb or faster . Until then, I'm stuck running 2 fiber FC cables and 4 copper gig-e's (two public, 2 heartbeat for the cluster) to most of our major systems and ESX boxes.

    7. Re:Cost? by Phishcast · · Score: 1
      Maybe they are rebranded Qlogic switches...I recall the old 9020's were Qlogic underneath (Cisco's first 4Gbps FC switches). Those things didn't understand VSANs when they came out, which seemed like a strange thing to put into the MDS line. I guess they just wanted to get out there with a 4Gbps switch.

      There's something to be said for port density in large switches vs. small edges everywhere. Where I work we use Cisco 9513s in the cores and at the edges these days. We're migrating from 20+ physical McData switch fabrics down to 5 physical switch Cisco fabrics, and vastly increasing port count while we're at it. I never did the power calculations, I wonder what the difference per port would look like.

    8. Re:Cost? by segfaultcoredump · · Score: 1

      The 9132's run SAN-OS and offer all of the features one would expec, so I think they actuall did it themselves this time.

      I dont know what the old McData's were like in terms of power. Brocade was proud of their 1W / Gb power envelope (4w per port). The new 9132's look like they are close to that. For what is worth, brocade has a power calculator for the 9513 vs thier 4800 at http://www.brocade.com/products/directors/power_draw_density_calculator.jsp (the numbers look close to real world, so it is a good start). The new Brocade DCX pulls about the same power. More marketing dirt can be found here: http://www.brocade.com/products/competitive/directors.jsp and http://www.brocade.com/products/competitive/references.jsp (I'm sure cisco has a similar page for brocade equipment)

      Of course, only cisco offers 528 4G ports on a single chassis (as long as you cable stuff to avoid oversubscribing the backplane. Again, in theory this can be done but in practice one just needs to be a bit more careful).

      And lord help me if I ever manage an environemnt that requires that many san ports. I have enough to do now as it is. :-)

    9. Re:Cost? by evilbessie · · Score: 1

      Sounds like you made the right choice for you at the time. But this is newish tech with still some kinks to be worked out, this alliance will hopefully move things along in this front. But there may be better things to come from elsewhere.

    10. Re:Cost? by segfaultcoredump · · Score: 1

      I hope they do, the thing I hate doing the most is running all of the cables to the servers and storage. I know larger shops have 'people' who do this for them, but I'm not lucky enough to have a team of lackies to do the grunt work for me. :-)

    11. Re:Cost? by Anonymous Coward · · Score: 0

      I found a few interesting things about your post. And guess whom I work for? (and why I'm posting AC) However, I'm not in sales.

      The first thing I sense from your post is that you had an outdated MDS environment, and you went with Brocade either because they offered a lower price, or you didn't research the MDS products after hearing what Brocade told you. Hey, it happens. I've got nothing against you.

      The other thing I sense is that you've unwittingly mixed up ethernet vs. fibre channel as some Cisco MDS vs. Brocade issue. What I think you meant to say is that FCoE or iSCSI doesn't make sense for you whereas a separate, pure FC fabric does -- that's perfectly acceptable.

      Until then, I'm sticking with ethernet for IP and FC for storage.
      Agreed. Even though it isn't what marketing around here is saying.

      At the same time our network folks were purchasing some 10G line cards for their Cat 6509's. Lets just say that the 8 port card with optics cost more than my 64 port 4900 with optics. They got 160Gb of bandwidth (assuming you never leave the card when it quickly falls to 40Gb), I got 512Gb.
      I don't think you were clear in saying this was Cisco 10G ethernet vs. Brocade 4Gb fibre channel. An apples to oranges comparison. It makes sense with the statement I quoted above, but you left the IP vs FC (which should be Eth vs. FC) statement as your last sentence. Not a big deal...

      Just for kicks I also looked at their costs for the 48 port gig-e line cards. Loking at just the cost of the cisco line cards (6748's) and ignoring the cost of the chassis and supervisor, my cost per Gb was just a touch under theirs (and mine included the whole chassis). Of course, 256 ports of gig-E takes up a lot more space, more cables and I still have the bonding issue for redundancy.
      Again, comparing Cisco ethernet to Brocade FC.

      The nice thing about the 4900's vs the cisco MDS switches are as follows;
      You're comparing a new 4900 vs an oooold MDS 9216. Brocade loves to compare theirs vs. outdated equipment. But I'll go through line by line, and I'll be honest with you when Brocade does have an advantage.

      1) Smaller (2RU for 64 ports)
      As has been pointed out, 2 9132's fit this.

      2) No oversubscription (64 ports, 4Gb any to any)
      64 ports in one fabric switch. True, Cisco doesn't have a fabric switch with 64 ports. I don't have the 4900's internal specs, but in general I take Brocade's "not oversubscribed" claims with a grain of salt, from experience. The 9132 definitely isn't oversubscribed. Of course, just like Brocade's switches, you get oversubscription between switches. With a bit of planning, you should be able to use the 20Gb ISL on the 9132's without overloading it -- and those 2 10Gb ports are in addition to the 32 4Gb ports. You aren't likely putting 64 ports in the same zone. And those ports that will talk to each other the most would be put on the same switch and even the same port group. Interconnect two Brocade 4900's and you'll be losing your valuable fabric ports.

      3) Less power. (Not even close)
      The power issue has been over with for years now. Now, 1 4900 vs. 2 9132 switches, in principle that 1 switch better beat 2 switches marginally on power. If the 4900 doesn't, Brocade has serious problems.

      4) Hitless firmware upgrades (cisco sorta has this with the MDS 9500, at 10x the cost)
      Cisco's MDS has been hitless for a couple of years now. Once again, this is a comparison of Brocade's now vs Cisco's years past. All 4Gb MDS products are hitless, even with one supervisor. And even the older 2Gb are too. Firmware revisions are marching on... Oh hey, at least it's not as confused as Brocade's infamous continuous hodge-podge stream of versions. And don't confuse Cisco IOS with MDS.

      5) Ooooh, shiny box :-)
      Well, damn. Brocade does put more styling in their boxes. ;-)

    12. Re:Cost? by Limited+Vision · · Score: 1

      There's a newer power calculator at

      http://brocade.com/power

      The DCX is under half a watt per Gbit.

      The 9513 not 528 x 4Gb ports -- take another look at that web page. There's no local switching on Cisco FC linecards, so with 48Gb per slot, it's really just 528 x 1Gb ports. A port can do 4Gb, but it's stealing bandwidth from neighboring ports because of that 48Gb limit.

      Brocade can do more bandwidth per slot -- 64Gb in the 48000 (16 x 4Gb ports per slot), and you can get 4Gb on up to 48 ports per slot if you locally switch the other 32 ports.

      The DCX has 256Gb per slot and can do 4Gb on every port without needing to locally switch. It can do 256 ports of 8Gb, any to any, and 384 if you use local switching.

      The DCX also has a 1/2 terabit interconnect so you can attach two chassis and get 768 ports talking at a better subscription ratio that the 528 ports in the 9513.

    13. Re:Cost? by segfaultcoredump · · Score: 1

      I remember going through this with cisco the first time when they offered the 16 port or 32 port (2Gb) line cards, depending on what your access patterns were.

      Most corporate san and network environments are bursty in nature. Clients pull a bunch of data and then think about it for a long time. Not many go full bore all day long. If this is the access pattern, then the MDS systems are fine since the chances of everybody trying to pull 4Gb at the same time is nil.

      If, however, you are running a huge supercomputer site that must moves huge amounts of data around all the time as fast as possible... well, lets just say that the MDS is not what you want (unless you want a 132 port switch that takes up half a rack and doubles as a space heater)

      I'm sure the folks at cisco will eventually have local switching. They offer DFC's for their ethernet products that do just that, so they have the technology.

    14. Re:Cost? by Limited+Vision · · Score: 1

      All well and good if you can predict your data flow over the time you own the gear. (Riiight...) More often than not things change and it's critical to have the flexibility to use *any* port for *any* IO pattern due to things happening you didn't plan for, whether it be faster growth, mergers, consolidation, or worst of all being successful at your job and people discover the wonders of SAN backup, or VMware which consolidates IO big time.

      The MDS 9500 line will not have local switching unless they fundamentally redesign -- the central arbiter in the supervisors is the choke point and unless they figure out a way to distribute that all local IO will have to check there and pass through the crossbar. IOS â SAN-OS, and different chipsets. I'd be impressed if they could figure it out but it's been 4 years and if it hasn't happened yet, I don't think it will never happen in the MDS. Nexus -- we'll see, but that doesn't support FC.

      If you're in the single switch range, you're fine, but I know too many guys above 100 ports growing at 30-100% a year -- knowing neighboring ports can talk at line rate regardless of the other IO patterns makes for a great insurance policy and happy management.

    15. Re:Cost? by Anonymous Coward · · Score: 0

      Nexus -- we'll see, but that doesn't support FC.
      Incorrect. The Nexus does indeed support FC.

  12. The Galactic Token-Ring Empire by davidwr · · Score: 1

    I thought that war was over already.

    --
    Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
  13. Don't complain ... by BoredAtWorkWhatElse · · Score: 1

    At least they got it right !

    ... this time anyway.

  14. May I propose... by Anonymous Coward · · Score: 0

    ...the 20Gb per sec ethernet alliance, it has all the benefits of the 10Gb alliance, and more.

    Surely there'll be no way, that anyone can possibly think of, that'll somehow be better than that.

  15. Channel bonding by h.ross.perot · · Score: 3, Informative

    Sigh, Aggregating 2 or more 1 GIG adapters does not give you 2 GIG of throughput. It is a sliding scale; the more you add the less total bandwidth you see. The safest bonding scheme uses LACP; Link Aggregation Control Protocol. This protocol communicates member state and load balancing request to the link member. 10G over copper will be a good thing for VM's. Sad; that the current crop of 10G over copper adapters do not approach 5 gig throughput; raw. Give the industry time; this it just like the introduction of 1 GIG from fast Ethernet. It took 2 generations of ASICs to get to what we consider a GIG card today.

    --
    ... I'll have a Pan Galactic Gargle Blaster with a side of Plutonium Nyborg ...
    1. Re:Channel bonding by wtarreau · · Score: 1

      Sad; that the current crop of 10G over copper adapters do not approach 5 gig throughput; raw. Wrong, current NICs are quite able to saturate the wire, even more (I got Ingo's TuX to push 20 Gbps of data through two Myricom NICs, and HAProxy to forward 10 Gbps between two NICs). The problem is the chipset on your motherboard. I had to try a lot of motherboards to find a decent chipset which was able to saturate 10 Gbps. Right now, my best results are with intel's X38 (one NIC supports 10 Gbps in both directions, ie 20 Gbps), and two NICs can saturate the wire on output. The next good one is AMD's 790FX which can saturate 10 Gbps in either direction, but not both (limited to about 14 Gbps).

      Never forget that 10 Gbps translates into 1.25 GB/s the bus to the RAM in very small parts, and that's very hard to sustain. I can assure you that my motherboard heats up when it gets 800000 packets/s at 10 Gbps :-)

      If you're looking for a decent motherboard to support large network bandwidth, throw to the bin all the ones with integrated VGA, and buy one that supports 4*PCIe 8x or 3*PCIe 16x to get some margin (you only need 1*8x in both directions, but marketting it stronger than technical skills at motherboard makers).

      Willy
    2. Re:Channel bonding by this+great+guy · · Score: 1

      The parent is correct. Here is one example of a 10GbE card transmitting at wire speed (9.7 - 9.9 Gbps): http://www.myri.com/scs/performance/Myri10GE/

      The "5 Gbps" bottleneck mentioned by the grand parent is due to 10GbE NICs often being installed on a relatively slow 100MHz PCI-X bus, whose practical bandwidth is only: 100 (MT/s) * 64 (bits) * 0.8 (efficiency of a PCI bus) ~= 5.1 Gbps. Fully exploiting the throughput of 10GbE requires at minimum a (1) PCI-X 2.0 266MHz+, or (2) x8 PCI Express 1.0, or (3) x4 PCI Express 2.0 NIC.

  16. look forward to the new standard by v1 · · Score: 1

    in other news, ISO starts the process of ratifying the new MS10G(tm) specifications.

    --
    I work for the Department of Redundancy Department.
  17. Further blurring the distinction by FranTaylor · · Score: 1

    Between the network cable and the drive cable. USB subsumed many old-technology interconnects, perhaps 10 Gb Ethernet can replace SATA and continue the trend of decreasing the number of interfaces required on a computer.

    1. Re:Further blurring the distinction by Anonymous Coward · · Score: 0

      Sure, all we'd need is a NIC on every drive....

      Actually, that'd be a pretty neat feature. Good idea :D

  18. "Tolken-Ring Empire" by Anonymous Coward · · Score: 0

    hehehe, that makes marginally more sense with an "l" crammed in there ;)

  19. Yes and no... by Junta · · Score: 1

    Yes, it works to the extent reasonable/feasible.

    No, it isn't a robust scalable solution. To play nice with various standards and keep a given communication stream coherent, it has transmit hashes that pick the port to use based on packet criteria. If it tries to use criteria that would actually make the most level use of all member links, it would violate the aforementioned continuity criteria. I have seen all kinds of interesting behavior depending on the hashing algorithm employed. I have seen a place buy 500 servers from one vendor and have a 2 port aggregation to somewhere. The problem being, that vendor had all even mac addresses on the first ethernet port, and so one of the ports got to be virtually unused because it was a hash of the mac addresses mod the number of member ports. Since there were two ports, it was mod two and all evens went on one port, and only odds would have been the other.

    --
    XML is like violence. If it doesn't solve the problem, use more.
    1. Re:Yes and no... by Doc+Ruby · · Score: 1

      That sounds like a problem with some rare edge cases. Can't the admins test the deployment to ensure the traffic is maximally using the multiple channels in the actual installation configuration? Is it that complicated to test and reconfig until it works? Maybe with a mostly automated tool?

      --

      --
      make install -not war

  20. Will there ever be "enough" bandwidth to a home? by Dr.+Spork · · Score: 3, Interesting
    Stories like this always make me think of the following:

    I can't think of anyone who's longing to get a fatter gas pipe connected to their house, or a fatter pipe to municipal water, or a cable of higher capacity to bring in more electricity.

    But we're not like that with bandwidth. We always seem to want a fatter pipe of bandwidth! Will it forever be like that? Is the household bandwidth consumption ever going to plateau, like electric, gas and water consumption has in the US? (I know that global demand for these utilities is growing, but that's mainly because there are more people and a larger proportion are being hooked up to municipal utilities. The per-household numbers are not really changing very much, and in some cases decreasing.)

    Will there be a plateau in bandwidth demand? If so, when and at what level? Thoughts?

  21. Re:Will there ever be "enough" bandwidth to a home by RiotingPacifist · · Score: 1

    yes but this isnt for homes, this is for offices, no (normal person's) home has a 100MB internet. I think the limit for broadband will simply be when you can download a film in 5 minutes over BT, which basically depends on how fast the average is not just your connection.

    --
    IranAir Flight 655 never forget!
  22. Fibre-channel has significantly lower overhead by LukeCrawford · · Score: 1
    than ethernet alone- to even get the quoted ratio, the new ethernet standard will have to be much more efficent than the old ethernet standard. And if they use iscsi, then they are running storage over IP over ethernet, adding further overhead

    I work a lot with 1G fibre channel; it is worlds faster than 1G ethernet for storage applications, just 'cause fibre channel was designed, first and formost, to handle storage. Sure, you can run IP over it, but that's not what it's *for* - the problem is compounded by the fact that ethernet has a bunch of legacy baggage. so obviously, given equal speed, fibre is going to beat ethernet when it comes to storage.

    The big advantage of using bog-standard ethernet is scale. If everyone uses this connectivity method, it's gonna get cheap. Potentially you could have ethernet that is a lot faster than fibre channel for the same price, making up for the overhead. The Idea is sound, I think. Reality has yet to catch up, though.

    Good,managed name-brand 1G fibre-channel switches are almost free, and good (name brand, managed) 1G ethernet switches still cost real money. (the used market is what I'm familiar with for both)

  23. Re:Will there ever be "enough" bandwidth to a home by Anonymous Coward · · Score: 0

    If us in the US had to pay per mb, I'm sure it would plateau pretty darn quick.

  24. Re:Will there ever be "enough" bandwidth to a home by Dr.+Spork · · Score: 1

    Yea, I got that, but the fact that this is feasible for offices now means that homes could use the same technology in the near future. There are some "normal" homes in Sweden that already get 100MB internet, which is enough for streaming HD, but not enough for many other things we will eventually come up with. So I was wondering whether there will ever be an "enough" level to the home. For a business like Google, "enough" might only be: the sum of all user bandwidths in the world. But for users, will there be a plateau?

  25. Re:Will there ever be "enough" bandwidth to a home by Anonymous Coward · · Score: 0

    My parents installed a fatter pipe to the water system a few years ago. Before, if anyone flushed the toilet, the shower would be a scalding hot drip for the next 5 minutes. Now, they can run the shower, and the washing machine, and the dishwasher, and flush every toilet in the house with nary a dent in the flow.

    Just thought you should know that there ARE people who clammer for fatter pipes to other things.

  26. Re:Will there ever be "enough" bandwidth to a home by Anonymous Coward · · Score: 0

    My friend uses a doubly fat water hose to pump natural source water without losing pressure so quickly.

    He also needs to get a fatter municipal water pipe because he's adding a carriage house.

    Who'd have thought?

  27. Re:Will there ever be "enough" bandwidth to a home by patchvonbraun · · Score: 1

    I've always felt that there's a per-person
        bandwidth limit that's inherent in being human,
        and our ability to pay attention to more than
        one thing at once.

    I think that about 1Gbit *per-person* in your
        household is about right. A HDTV channel or two,
        some background audio. A telephone
        conversation. Some online gaming, and a few
        software/media downloads going in parallel is
        a reasonable model for someone with lots of
        "stuff" on the go.

    Obviously, if you run significant *services* out
        of your house, these numbers go up. So for an
        average house with 2.5 adults and 1.5 children,
        something like 3-4GB/sec should do just fine.

  28. Re:Will there ever be "enough" bandwidth to a home by MojoRilla · · Score: 1

    There has not yet been a plateau in how fast and how distant we want to communicate with each other, so there probably won't be a plateau in bandwidth demand. Communication started with speech in person (which wasn't very fast or could reach many people), and then moved to paper and ponies (as in the pony express, which could at least reach a larger audience), visual with signal towers (starting to speed up), telegraph, telephone, radio, TV, satellites, the internet...the pipe keeps expanding. It may never stop, because what we want to say to each other will keep getting more and more complex.

    Sure, our communication speed might plateau for a time: for a long time TV was limited to NTSC and PAL sizes, but unlike people's appetite for water (which is pretty much fixed, as there is only so much you can drink or shower in it), people's appetite to communicate is unbounded.

  29. xenaoe.org by Tracy+Reed · · Score: 1

    http://xenaoe.org/

    I'm way ahead of ya guys!

    A lot of details left to fill in but I have a few clusters up and running already. Working on documenting my setup so that others may duplicate it.

  30. Re:Will there ever be "enough" bandwidth to a home by AeroIllini · · Score: 1

    Yes, it will plateau, eventually.

    The per-capita demand of each of those other utilities grew at a large rate when they were first introduced. The first running water was for maybe a sink and a tub, then we started adding toilets, multiple bathrooms, then clothes washers, dishwashers, automatic sprinkler systems, etc. As technology progressed, the amount of water delivered to a given house each day increased dramatically, but the change happened over many decades, which allowed the infrastructure to be updated as new buildings were constructed. I've been in old buildings where flushing a toilet in one room causes the water to slow to a trickle in another room.

    The same happened with electricity: we went from a couple light bulbs to central heating to electric appliances to televisions to computers to all the various mobile gizmos and gadgets we plug in now. Lots of older houses have electrical wiring that is not up to the task of running all the lights, the dryer, three computers and the television all at once. Again, this infrastructure was updated gradually as new structures were built or old ones were renovated.

    We're seeing the same demand increase with the internet, as we go from BBS/Usenet to text www to images to games to video and beyond. The difference now is that this increase is happening at a rate that is orders of magnitude greater than the other utilities, so structures are having to be upgraded more than once as technology progresses.

    The other difference is that faster data bandwidth requires technological breakthroughs like more efficient packet switching software, faster hardware, etc. More electricity just requires more breakers and bigger wires. It scales linearly.

    --
    For security, the MD5 hash of this message and sig is 09f911029d74e35bd84156c5635688c0.
  31. this consortium is simply a front end for geeks by porky_pig_jr · · Score: 1

    who want to download more pr0n.

  32. Re:Will there ever be "enough" bandwidth to a home by arkhan_jg · · Score: 1

    That's mainly because the utility companies demand is increasing slower.

    Specifically, electricity - just compare the number of plug sockets in a house built today compared to a house built in your grandparents day, the number of ring mains, and the size of the breaker board.

    The demand for electricity in the home has soared since it was first implemented. It has plateau'd now somewhat in the west, but that's due to the cost of making it going up steeply, and more wareness of energy efficiency.

    Most people were pretty happy with basic broadband compared to dialup. Now streaming HD video is becoming commonplace, faster broadband is needed. I think 50Mb - 100Mb will be a decent plateau spot for a while. At least until the next bandwidth hungry app comes along.

    --
    Remember kids, it's all fun and games until someone commits wholesale galactic genocide.
  33. Re:Will there ever be "enough" bandwidth to a home by Anonymous Coward · · Score: 0

    You never think "there isn't enough gas to heat my house."
    Water is different: if you don't have enough water coming out of the shower, its because someone started the dishwasher or flushed the toilet. Its a local problem.
    When bandwidth gets to the point where you don't think about it, you'll stop wanting more. For people who browse the web & read email, current broadband is probably enough and they don't give it a second thought. For those that want to do everything they can do at the office over VPN, its not even close.

  34. Problem being... by Junta · · Score: 1

    While some pieces of equipment will allow configuration of the transmit hash, many will not.

    --
    XML is like violence. If it doesn't solve the problem, use more.