There tend to be a lot of misconceptions about network hardware and these misconceptions tend to be reinforced by vendor-sponsored tests that are far more marketing than analysis. I'd personally love to see reviews that included the following:
1.) *ALL* configurations used. This includes the specific configuration files of each device being tested as well as those surrounding it. Tell me exactly what versions of code were used on everything. I want to be able to completely reproduce your test should I be so inclined. Without full disclosure the validity and reliability of these sorts of review tend to be questionable.
2.) As someone who buys many, many millions of dollars of network equipment I honestly don't care all that actively about the raw throughput of a given box. Turn features on. Turn *lots* of features on. Then tell me how much traffic is passed. As mentioned in #1 define exactly what sort of traffic you pushed through and what the end points might be. Many major vendors' equipment tends to fall apart when ACL's are applied to interfaces and absolutely crumble when QoS features are turned up. Give me an idea of what (if any) features have an impact on performance, and how much of an impact they have.
3.) Reliability: Run through as many failure modes as you can. Pull linecards, fans, power supplies, CPU modules, switch fabric cards, etc.. More importantly, measure any traffic loss when new parts are inserted into a live box. Tell me how much of a hit I take when I reboot the box. For larger boxes tell me about features that let me gracefully take some- or all- of the unit offline for maintenance. Any software extensions (e.g. graceful restart for OSPF/BGP/IS-IS) should be tested. Observe the behavior of the box as routing protocol timers are tightened. Many larger boxes allow for degraded operation under extreme fault conditions. Are there mechanisms to assure that critical traffic makes it through or do faults hit everything. In short, I want to know what happens when things break. Tell me about those things that could interrupt production traffic.
4.) Management / Operation: You'd be surprised how badly some vendors fall down in this department. Tell me about craft/console interfaces and any caveats that might apply. Tell me about facilities for out-of-band management (..if there are any). Tell me about mechanisms to access the box - do ssh/telnet/rsh/whatever work correctly with all common desktop implementations (certain combinations of network hardware and ssh clients are notorious for problems). If the box supports SNMP, what version and what facilities are available to limit access to the box? Also verify that the MIB's are available and well documented. The same would apply for TL-1 on carrier equipment. Tell me about syslog facilities and the ease/consistency of configuration of same. Tell me about aaa support - RADIUS? TACACS+? Others? What happens when authentication servers go unavailable. What about debugs on the box? Are they potentially destructive to activate with heavy traffic on the box? Are they complete? Basically you just need to remember that visibility into the operation of the box is absolutely critical - both reactively (fault management) and proactively (trending performance/etc).
5.) Security: What mechanisms are available to secure the box and how well do they work? In some cases there may be both hardware and software measures the vendor has taken. Evaluate how well the vendor has kept up with security patches, how available said patches might be and how much impact introducing these patches might have. Can I apply updates to minor software components without interrupting traffic/rebooting the box? What sorts of services does the box offer to the outside world and how readily can they be disabled? What mechanisms are available to not only block traffic to the box but also rate-limit to deal with DoS conditions.
6.) Vendor Support: This is tough to quantify but is arguably among the most importa
I'm afraid this isn't right. Any modern, shipping Cisco platform supports some level of distributed switching (e.g. ASIC's on interfaces). In the higher-end boxes traffic literally cannot be switched through the CPU (GSR). Even common L3 switches (3550, 3750, 4500, 6500) normally always push their forwarding (L2 and L3) out to separate ASIC subsystems.
The above is also absolutely true for Juniper, Foundy, Extreme and Force10 - and, as you point out, Nortel. Switching packets in software hasn't been a standard practice (outside of bugs) in most modern platforms for many years.
CPU's are fast now. Heck, memory speeds are getting very fast. An Opteron might even be able to switch packets between a couple of 10G interfaces at- or near- line rate. Now extend that to a box with 32 10G interfaces in it. You now not only need 320G to the physical interfaces via some number of bus connections, you've also got to be able to move packets in- and out- of memory, maintain routing adjacencies and any other miscellaneous network management tasks... all in real time. PC's are not built to do this. Outside of real-time extensions Linux/BSD/et al are not built to do this.
Think of it this way - assume an average packet size of 300 bytes. On a one gigabit ethernet interface this represents something on the order of 40,000 packets per second... in one direction. Multiply this by 10. Now by 32 interfaces. What does an OS and PC platform look like that can malloc() 25.6 million times per *second* above and beyond any other OS processes? Oh - and don't forget the fancy queues, packet re-writing, CRC calculation and such that would necessarily follow each one of these memory operations.
This is obviously an extreme example, but it illustrates the point that everyone in the industry pretty much figured out a bunch of years ago that distributed forwarding via dedicated hardware was the only realistic answer to this problem. This is why the CPU in just about any Cisco platform you'll see in common is less capable than a lot of PDA's out there and also why the architecture of basically all of the major players is moving toward a condition where forwarding and network control are handled by roughly autonomous units.
You can run hybrid mode/CatOS on the sup720 (catos 8.x, I think). I also would not use the 6513 - only slots 9-13 actually run at full speed. You're limited to half the backplane bandwidth on slots 1-8.
The 6513 won't support 11 48-port 10/100/1000 linecards if you expect anything close to wire speed. The only blades that would even come up in that application would be 6148-GE-TX or 6548-GE-TX. In both cases the interconnect between the blade and the backplane is very oversubscribed. If you want to run at full speed, only a 6509 with supervisor 720's and the 6748 linecards will approach line rate at high density. The 6513 has serious limitations on how many high-speed blades can be employed. I would strongly recommend using a 6509 instead. The practical limit would then be 384 10/100/1000 ports.
That said, if cost is a factor and you really only need layer-2, we found the Nortel 5510 stackable to be extremely impressive. 48 ports of 10/100/1000, stackable to 8 - manages as a single unit, tons of capacity between switches, great redundancy features and a cost literally less than a quarter of the Cisco's.
I've been working on large Cisco deployments for around ten years and recently passed the six year anniversary of completing my CCIE (..complete with recertification). I've literally worked on everything from the old 6xx series through the GSR and Stratacom lines. I don't work for any equipment vendor and never have.
A good chunk of this thread seems to focus on the fact that Cisco gear is expensive. I don't think anyone (...Cisco included) would argue this point. The part that disturbs me is that there's this basic notion that somehow these costs yield boxes that are faster/better/more functional. At some point I think this might have had more of a ring of truth to it, but in the past 3-4 years the following has happened:
1.) The quality of support staff in the field has declined tremendously. The layoffs hurt the company and its culture a lot more than people might imagine.
2.) The quality control of software/hardware is absolutely horrific. From major, widespread issues with 4000- and 4500- series ASIC's to a stream of major security issues in IOS to an exceptionally poor history with the emergence of VOIP and the continued prevalence of bugs in fundamental parts of the IOS (...including EIGRP), I have now been in a number of large financial services and service provider shops that actively distrust anything from Cisco that they themselves haven't regression tested for an *extended* period of time (testing it yourself is always the right thing to do, but the extent of required checking here is in a different league).
3.) The performance of Cisco equipment has never been #1. I don't think a lot of people realize that they're often not even in the top 5. I know of a number of situations where 7200-class routers with the newest NPE's and moderate interface counts (1-2 DS3's worth of traffic) have either been completely ripped out or - worse yet - fronted with Juniper boxes in response to situations requiring mechanisms to respond to spiking traffic loads and/or DDoS attacks. As anyone who has ever worked with Cisco equipment to any depth can attest, the best possible throughput an IOS device can achieve is with virtually no configuration. Adding features (fancy queueing, security features, etc) will often dramatically reduce the capacity of the box.
I can buy a Juniper M7i for approximately the same price as a 7200, but the M7i can handle OC12 and GigE at line rate - with features enabled. This should be an ominous omen for Cisco, as is the new series edge routers that Juniper is bringing out (J series). I can buy Nortel L2 switches with radically better performance and density for a fraction of the price of equivalent Cisco gear - and still get very solid support.
Cisco is a huge company that is increasingly treating even its larger customers very poorly. Technologies like EIGRP provide a degree of lock-in (I'm actually surprised that more folks on this board don't have an issue with the proliferation of this closed standard) but many large organizations are deploying BGP or OSPF backbones to facilitate migration toward technologies that allow competition.
Having a single vendor is a nice thing and it's certainly an easy answer to automatically implement more Cisco boxes but I see a growing realization that there are other (potentially superior) ways to solve problems...
Slashdot Mirror
1.) *ALL* configurations used. This includes the specific configuration files of each device being tested as well as those surrounding it. Tell me exactly what versions of code were used on everything. I want to be able to completely reproduce your test should I be so inclined. Without full disclosure the validity and reliability of these sorts of review tend to be questionable.
2.) As someone who buys many, many millions of dollars of network equipment I honestly don't care all that actively about the raw throughput of a given box. Turn features on. Turn *lots* of features on. Then tell me how much traffic is passed. As mentioned in #1 define exactly what sort of traffic you pushed through and what the end points might be. Many major vendors' equipment tends to fall apart when ACL's are applied to interfaces and absolutely crumble when QoS features are turned up. Give me an idea of what (if any) features have an impact on performance, and how much of an impact they have.
3.) Reliability: Run through as many failure modes as you can. Pull linecards, fans, power supplies, CPU modules, switch fabric cards, etc.. More importantly, measure any traffic loss when new parts are inserted into a live box. Tell me how much of a hit I take when I reboot the box. For larger boxes tell me about features that let me gracefully take some- or all- of the unit offline for maintenance. Any software extensions (e.g. graceful restart for OSPF/BGP/IS-IS) should be tested. Observe the behavior of the box as routing protocol timers are tightened. Many larger boxes allow for degraded operation under extreme fault conditions. Are there mechanisms to assure that critical traffic makes it through or do faults hit everything. In short, I want to know what happens when things break. Tell me about those things that could interrupt production traffic.
4.) Management / Operation: You'd be surprised how badly some vendors fall down in this department. Tell me about craft/console interfaces and any caveats that might apply. Tell me about facilities for out-of-band management (..if there are any). Tell me about mechanisms to access the box - do ssh/telnet/rsh/whatever work correctly with all common desktop implementations (certain combinations of network hardware and ssh clients are notorious for problems). If the box supports SNMP, what version and what facilities are available to limit access to the box? Also verify that the MIB's are available and well documented. The same would apply for TL-1 on carrier equipment. Tell me about syslog facilities and the ease/consistency of configuration of same. Tell me about aaa support - RADIUS? TACACS+? Others? What happens when authentication servers go unavailable. What about debugs on the box? Are they potentially destructive to activate with heavy traffic on the box? Are they complete? Basically you just need to remember that visibility into the operation of the box is absolutely critical - both reactively (fault management) and proactively (trending performance/etc).
5.) Security: What mechanisms are available to secure the box and how well do they work? In some cases there may be both hardware and software measures the vendor has taken. Evaluate how well the vendor has kept up with security patches, how available said patches might be and how much impact introducing these patches might have. Can I apply updates to minor software components without interrupting traffic/rebooting the box? What sorts of services does the box offer to the outside world and how readily can they be disabled? What mechanisms are available to not only block traffic to the box but also rate-limit to deal with DoS conditions.
6.) Vendor Support: This is tough to quantify but is arguably among the most importa
The above is also absolutely true for Juniper, Foundy, Extreme and Force10 - and, as you point out, Nortel. Switching packets in software hasn't been a standard practice (outside of bugs) in most modern platforms for many years.
CPU's are fast now. Heck, memory speeds are getting very fast. An Opteron might even be able to switch packets between a couple of 10G interfaces at- or near- line rate. Now extend that to a box with 32 10G interfaces in it. You now not only need 320G to the physical interfaces via some number of bus connections, you've also got to be able to move packets in- and out- of memory, maintain routing adjacencies and any other miscellaneous network management tasks ... all in real time. PC's are not built to do this. Outside of real-time extensions Linux/BSD/et al are not built to do this.
Think of it this way - assume an average packet size of 300 bytes. On a one gigabit ethernet interface this represents something on the order of 40,000 packets per second ... in one direction. Multiply this by 10. Now by 32 interfaces. What does an OS and PC platform look like that can malloc() 25.6 million times per *second* above and beyond any other OS processes? Oh - and don't forget the fancy queues, packet re-writing, CRC calculation and such that would necessarily follow each one of these memory operations.
This is obviously an extreme example, but it illustrates the point that everyone in the industry pretty much figured out a bunch of years ago that distributed forwarding via dedicated hardware was the only realistic answer to this problem. This is why the CPU in just about any Cisco platform you'll see in common is less capable than a lot of PDA's out there and also why the architecture of basically all of the major players is moving toward a condition where forwarding and network control are handled by roughly autonomous units.
You can run hybrid mode/CatOS on the sup720 (catos 8.x, I think). I also would not use the 6513 - only slots 9-13 actually run at full speed. You're limited to half the backplane bandwidth on slots 1-8.
That said, if cost is a factor and you really only need layer-2, we found the Nortel 5510 stackable to be extremely impressive. 48 ports of 10/100/1000, stackable to 8 - manages as a single unit, tons of capacity between switches, great redundancy features and a cost literally less than a quarter of the Cisco's.
A good chunk of this thread seems to focus on the fact that Cisco gear is expensive. I don't think anyone (...Cisco included) would argue this point. The part that disturbs me is that there's this basic notion that somehow these costs yield boxes that are faster/better/more functional. At some point I think this might have had more of a ring of truth to it, but in the past 3-4 years the following has happened:
1.) The quality of support staff in the field has declined tremendously. The layoffs hurt the company and its culture a lot more than people might imagine.
2.) The quality control of software/hardware is absolutely horrific. From major, widespread issues with 4000- and 4500- series ASIC's to a stream of major security issues in IOS to an exceptionally poor history with the emergence of VOIP and the continued prevalence of bugs in fundamental parts of the IOS (...including EIGRP), I have now been in a number of large financial services and service provider shops that actively distrust anything from Cisco that they themselves haven't regression tested for an *extended* period of time (testing it yourself is always the right thing to do, but the extent of required checking here is in a different league).
3.) The performance of Cisco equipment has never been #1. I don't think a lot of people realize that they're often not even in the top 5. I know of a number of situations where 7200-class routers with the newest NPE's and moderate interface counts (1-2 DS3's worth of traffic) have either been completely ripped out or - worse yet - fronted with Juniper boxes in response to situations requiring mechanisms to respond to spiking traffic loads and/or DDoS attacks. As anyone who has ever worked with Cisco equipment to any depth can attest, the best possible throughput an IOS device can achieve is with virtually no configuration. Adding features (fancy queueing, security features, etc) will often dramatically reduce the capacity of the box.
I can buy a Juniper M7i for approximately the same price as a 7200, but the M7i can handle OC12 and GigE at line rate - with features enabled. This should be an ominous omen for Cisco, as is the new series edge routers that Juniper is bringing out (J series). I can buy Nortel L2 switches with radically better performance and density for a fraction of the price of equivalent Cisco gear - and still get very solid support.
Cisco is a huge company that is increasingly treating even its larger customers very poorly. Technologies like EIGRP provide a degree of lock-in (I'm actually surprised that more folks on this board don't have an issue with the proliferation of this closed standard) but many large organizations are deploying BGP or OSPF backbones to facilitate migration toward technologies that allow competition.
Having a single vendor is a nice thing and it's certainly an easy answer to automatically implement more Cisco boxes but I see a growing realization that there are other (potentially superior) ways to solve problems...