OS Router Challenges Proprietary Networking

Jane Walker writes "Dave Roberts talks about Vyatta's open source router and how open source technology may soon alter the landscape of enterprise networking." From the article: "Initially, we believe that the x86 PC running Vyatta -- given the range of hardware that's available in the PC world -- can basically replace the midrange of the router market; to use Cisco terminology and model numbers, simply because it's convenient shorthand, basically from the 2800 series to the 7200 series. There's a whole host of equivalent products from Nortel and Alcatel -- but essentially in that range. I wouldn't describe it as Cisco model numbers so much as T1 branch office to gigabit LAN product categories."

Comments

Sigh....

[222]

I love open source and all, but can a project like this really offer the same number of WIC modules?

I can plug damn near anything into a Cisco router....

Re:Sigh....

[Svartalf]

All depends on what they provide in the way of PCI/PCI-X cards- or whatever the future buses might be...

I'd say that odds are good you'd get about the same number of media interfaces and what you didn't
have would very probably have a media adapter or bridge that's standalone to take care of the gaps.

Re:Sigh....

[Nuclear Elephant]

I can plug damn near anything into a Cisco router....

Open source routers and pr0n sounds like a dangerous combination for you then.

Re:Sigh....

[ChaoticChowder]

This software would have to offer much more that just WIC modules to even have me consider using it. Cisco routers may have low clockspeeds on the core chip, but its the ASICs that give them value. Also, take the 6509 for instance, slap in a SUP720B and you now have a 720 GBps back plane. No PC could ever hope to do that. Also, configuring a Cisco router is pretty much the easiest thing ever. I haven't checked out the software yet, but it better be much easier. Maybe they should network with the Open Source chipset guys and design some ASICs and all the other niceties.

Re:Sigh....

[Thundersnatch]

Also, configuring a Cisco router is pretty much the easiest thing ever.

Trolling for a +1 funny mod, are we?

I don't remember who said it, but this is my favorite quote about Cisco software: "Cisco makes easy things difficult, but difficult things possible."

Re:Sigh....

[kindbud]

I can plug damn near anything into a Cisco router....

And if you disable autonegotiate and set speed and duplex at fixed values, you might even get link.

Re:Sigh....

[Jzor]

...but can a project like this really offer the same number of WIC modules?

What do foodstamps have to do with networking?

Re:Sigh....

[monsted]

> That's not quite true. There's no reason I can think of why you couldn't make a backplane for a PC that handles all the network traffic locally, without touching the PCI bus (or whatever bus). In fact, high speed interconnects used on clusters do that sort of thing already, and I suspect any high speed backplane for any platform would need to do the same.

I think you're missing the point. The backplane of the Cat6500 is pretty much what the PCI bus does for a PC. A 32bit/33mhz PCI bus gives you just about 1 Gbps while the Cat6500 backplane provides three buses of 256, 32 and 4 Gbps (not 720 GBps as the GP suggested - the "Sup720" refers to the 720 Mpps switching capacity). Switching to PCIE gives you 2.5 Gbps per lane, but how many motherboards provide the 100 PCIE lanes needed to compete?

> I'm not intimately familiar with ASICs, but if they add this type of functionality regardless of the clockspeed on the core chip, they probably handle all the traffic locally too.

ASICs offload the hard work from the CPU of the Cisco systems. Basically any kind of compute-intensive bulk work, switching (yes, it switches layer 3 too), filtering (access-lists) and so on, is handled by dedicated ASICs and require little or no CPU intervention. This enables the catalyst to handle high amounts of data with a quite small CPU. Things that do end up on the CPU is management work, route computation (BGP changes, for example) and logging. To handle 720 million packets per second, you'd need quite a lot of CPU in your PC.

> A backplane like that may not exist for a PC currently, but if their PC router is successful enough to get companies using it, somebody will create one. Engineering a high speed backplane for arbitrary hardware is a problem that has already been solved, all that's needed now is a market with enough demand to make it worthwhile to build.

Well, large Juniper routers are run by a FreeBSD service processor, but i doubt you can run your open source router on it... :)

One thing the GP doesn't mention is the availability of special service modules, which one again do their work mostly in hardware: firewalls, load balancing, intrusion detection, intrusion protection, voice gateways, etc. Also, it takes 4 port 10GigE and 48 port GigE blades, giving it up to 48 10GigE or 576 GigE ports in a 13 blade chassis (one slot goes to the supervisor), something you'd have a hard time stuffing into a PC :)

With regard to ease of use, within 15 minutes of powering it on for the first time, most Cisco admins could have it up and running, switching and routing - IOS is a fantastic OS for most things.

Either way, i doubt the catalyst 6500 is the intended target for an open source routing platform and i'm sure it'll do just fine competing with the 2800-sized routers.

You may commence flaming the Cisco fanboy now.

Re:Sigh....

[RareButSeriousSideEf]

I don't know what all the fuss is about Cisco routers. For my money, Black and Decker wins every time.

Re:Sigh....

[Thundersnatch]

I have configured many Cisco routers, switches, and VPN concentrators. None had anything close to an intuitive interface, and even standard operations differ from model to model. There's as much backward-compatibility cruft and illogical organization in IOS as in Windows. Cisco documentation is often just plain wrong or so poorly written as to have ambiguous meaning.

In fact I've never worked on another brand of router besides Cisco, but the CLIs and GUIs of other complex networking devices like Checkpoints & SonicWalls let me know that something more intiutive than IOS is definitely possible.

Re:Sigh....

[hal9000(jr)]

That's interesting. I know alot of people, myself included, that like the IOS command line not becuase it is intuitive (I haven't met a CLI that is "intuitive"), but because it is pretty easy to navigate once you learn a few tricks. Alot of other networking gear have IOS-like interfaces in some cases replicating the IOS structure, but none are exact.

Maybe the only other CLI that is easier to use is Junipers JunOS, but I haven't spent alot of time with it.

Oh, and the docs have gotten much, much better from say 6 years ago.

Re:Sigh....

[bdp]

I think you're missing the point. The backplane of the Cat6500 is pretty much what the PCI bus does for a PC. A 32bit/33mhz PCI bus gives you just about 1 Gbps while the Cat6500 backplane provides three buses of 256, 32 and 4 Gbps (not 720 GBps as the GP suggested - the "Sup720" refers to the 720 Mpps switching capacity). Switching to PCIE gives you 2.5 Gbps per lane, but how many motherboards provide the 100 PCIE lanes needed to compete?

I don't think I was entirely missing the point, but maybe I could have gone a little further in explaining myself. My poorly explained suggestion was that you can avoid the PCI bus entirely, thereby avoiding the bottleneck. When you have multiple cards installed in a single box you could just connect them together using some kind of high speed interconnect that goes directly between cards instead of using the system bus. That way you can have your high speed backplane to handle network switching/routing.

That said, I understand that this is a bit silly and completely misses the point of using a PC as a router instead of something designed to do the job, but it would hardly be the first time such a product came out of the tech industry. If somebody thinks there's a market for it, it will be built.

ASICs offload the hard work from the CPU of the Cisco systems. Basically any kind of compute-intensive bulk work, switching (yes, it switches layer 3 too), filtering (access-lists) and so on, is handled by dedicated ASICs and require little or no CPU intervention. This enables the catalyst to handle high amounts of data with a quite small CPU. Things that do end up on the CPU is management work, route computation (BGP changes, for example) and logging. To handle 720 million packets per second, you'd need quite a lot of CPU in your PC.

Is there any reason why you couldn't accomplish this using something besides an ASIC? Offloading the work to the card is what I suggested before, and if you want to switching or routing with any kind of low latency, you have to do that. If you used that strategy, the speed of the processor in the PC wouldn't matter any more than the speed of the processor in a Cisco. Again, it may defeat the point of using a PC for routing, but that doesn't mean nobody would build it if there was a market.

I think we're in total agreement that the high end of what Cisco does is not the target market. My biggest point that I was trying to make with my post was that there was no reason you couldn't stick a high speed backplane into a PC. It's just a matter of creativty and some engineering. Whether or not it makes sense is a whole different question.

You may commence flaming the Cisco fanboy now.

Can't see why I would do that. You managed to express a different point of view without a) going apopletic, b)insisting I'm an idiot for not sharing your view, and c) not insulting me and/or my lineage. If anything, your behavior should be applauded. I can see how you would expect different treatment on /. though.

Re:Sigh....

[Cramer]

It would've taken 3 minutes on the Cisco too if Cisco hadn't dropped QDM (QoS Device Manager) several years ago. QoS is never simple. On anything.

(Obviously, there are alot of people doing that on a SonicWall or there wouldn't be simple "click here" interfaces to set it up.)

BTW: you're issues with the 2800 could also simply be BUGS in IOS.

Good luck with that!

[winkydink]

Cisco and Juniper offer 24/7 worldwide support. Whether or not it sucks, this is the thing that keeps people cozily asleep at night, knowing that if they have a problem, they have an unchallengeable defense of having bought the best in class support solution (notice I avoid any discussion of h/w, because in the enterprise, h/w without support is worthless).

Yes, Vyatta talks a good game, but 24/7 worldwide support isn't something you build with a few million bucks in VC funding.

Re:Good luck with that!

[winkydink]

It is when you have shareholders. Like it or not.

Re:Good luck with that!

[winkydink]

Look at Red Hat. Why do you think Oracle is considering doing their own distro? Not because there's gobs of money in the distro space, but because RH can't support them well enough now and they have a significant RH installed base. One large enterprise customer would kill a startup with pre & post sales support requirements alone. This is one of the many reasons that startups have problems cracking the enterprise space.

Re:Good luck with that!

[Fordiman]

So it's a growth issue. I buy that.

Now, question: How hard would it be to solicit new VC funding if you've suddenly got a big name customer? Crank that couple million to a couple hundered million.

I realize there's training timeline issues along with it, but an appropriately motivated company should be able to handle it.

I think it's just an issue of knowing when to change leadership (e.g., the guy that motivates a couple hundered programmers isn't necessarily the same guy capable of motivating a couple thousand tech support monkeys)

Re:Good luck with that!

[timeOday]

notice I avoid any discussion of h/w, because in the enterprise, h/w without support is worthless.

Yes, Vyatta talks a good game, but 24/7 worldwide support isn't something you build with a few million bucks in VC funding.

This sounds eerily like old Sun talk. "We don't care if competing products can do it for less, we're [Sun | SGI | Cray]!! The low end will never catch up with us, because we have special pixie dust!"

Re:Good luck with that!

[Jason Earl]

Yes, but there is a large market of folks that either A) have dealt with Cisco's so-called support and aren't impressed or B) would rather simply have a preconfigured spare box for less price than a single Cisco.

If there is one thing that Linux has proved it is that you can't underestimate inexpensive and "good enough." You may not be interested in what Vyatta is selling right now, but I would bet that enough people are interested that the next gen Vyatta is even more competitive. In the long run, the low end of the technology spectrum tends to improve and gobble up marketshare from the guys asking for big margins.

Re:Good luck with that!

[vsavkin]

Whether or not it sucks, this is the thing that keeps people cozily asleep at night, knowing that if they have a problem, they have an unchallengeable defense of having bought the best in class support solution

Here in Moscow, Russia most ISPs buy Cisco gear w/o any kind of support. Not even usual warranty period on hardware is there (Cisco gives 2 or 3 months only). Not having their asses covered by that kind of paper works fine for them.

Also, when talking about hardware, off-the-shelf PC router can do 100..400 kpps, it is more than enough for small provider's core router, not even considering branches.

Siad the OS advocate...

[Duncan3]

Advocate 1: "I work at Oracle by day, but work on Vyatta by night."

Advocate 2: "Well, I work at Cisco by day, but work on PostgreSQL by night"

[awkward pause]

Advocate 1: "Pistols or swords?"

Re:Siad the OS advocate...

[Profane MuthaFucka]

Well I work on Microsoft Windows during the day, and SQL Server at night.

Advocate 1: "Pistols or swords?"

Both please. Right in my head.

No. You're not making a 1U into a $40K router

[postbigbang]

Here's why:

1) it takes an RTOS to make things work well. You can grind all the driver code you want, but an RTOS foundation is required with lots of cache
2) only PCI-X bus gets close, and most 1Us don't have it. That gives you a real ceiling in terms of port-port throughput; don't kid yourself
3) the algorithms needed to maintain cross-bar speed are gruesome. You don't find this kind of code in anything but sledge-hammered C and assembler, and code that only a mother (and an embedded systems engineer) could love. There is very little forgiveness here.

Yes, a 1U can make a decent router. But don't kid yourself into believing that you can beat F5, Cisco, Alcatel, etc.

You can certainly embarrass them, but on the high end, it doesn't work.

Re:No. You're not making a 1U into a $40K router

[twiddlingbits]

The RTOS doesn't use a lot of cache, It needs a fast CPU and tight code to handle the massive numbers of context switches. The code you mentioned isn't all running on a CPU either. A lot of it is on custom hardware to keep up those data rates. The PCI-X bus would work except very high end, and it IS available in current 1U servers from people like Sun and HP, but certainly not in that old 286 in the closet. You could turn an Opteron with the HyperChannel architecture into a pretty darn good router. But the Opterons cost quite a bit more than a 286 would (does any foundry still MAKE 286 chips?). It's a good project but I agree it's not ready for prime time in the corporate data center.

Re:No. You're not making a 1U into a $40K router

[Megane]

Two words: cut through.

With a software router (aka your typical Linux-nerd router), the entire packet has to be read before the routing decision can be made. Then it has to be sent out again.

With Cisco, what you are paying for isn't the routing, it's the low latency of hardware that can see the destination IP address in a packet header, then effortlessly shunt the bits off to another interface in real time. You're also paying for the hardware being designed with 24/7 operation in mind, with little extras like watchdog reset timers that you won't find in that seven-year-old beige box.

Re:No. You're not making a 1U into a $40K router

[Florian Weimer]

Perhaps not a $40K router, but a $15K one--for $3K (including the replacement, should the first unit fail).

1) it takes an RTOS to make things work well. You can grind all the driver code you want, but an RTOS foundation is required with lots of cache

IOS is not a real-time operating system, which nicely disproves your claim. 8-)

2) only PCI-X bus gets close, and most 1Us don't have it. That gives you a real ceiling in terms of port-port throughput; don't kid yourself

In some of the Cisco low-to-mid-range routers, the line cards are connected to PCI busses (not that what's usually called L3 switches, of course). In fact, these routers are basically PCs with a MIPS CPU.

3) the algorithms needed to maintain cross-bar speed are gruesome. You don't find this kind of code in anything but sledge-hammered C and assembler, and code that only a mother (and an embedded systems engineer) could love. There is very little forgiveness here.

Most vendors do not offer wirespeed performance in all cases, either. You usually do not need the last 10% or 20% in performance. Given the savings (both in terms of device price and training), I'm willing to sacrifice 50% of the performance and more.

The real issue is the feature set of the available software. For GNU/Linux or the BSDs, the basic protocol support is usually there (but perhaps just in a userspace implementation which sacrifices some performance), but even pretty common things are sometimes missing because those who sponsered the development didn't need it (and it's not required by the standards, either). In some case, the implemntation isn't very mature yet. For the price difference of a Cisco box, you could hire someone in Russia to implement the missing stuff, but this gets a bit complicated, especially if you just want a working router.

On the other hand, there are some things which are impossible to do with IOS (obscure policy-based routing, most kinds of payload analysis, prefix-based accounting, OpenVPN). But most people don't expect routers to do such things anyway.

The Dawn of Open Source Networking? wtf?

[Anonymous Coward]

If you go to Vyatta's website they claim that they are bringing in the "Dawn" of Open Source Networking.
Unfortunately these folks obviously were living under a ROCK for the past 8-10 years and never noticed
things like oh.. IPTABLES, and there has been WAN support in Linux for a long time. Great companies like
Sangoma offer T1 cards etc etc. This is just a bunch of folks trying to cash in on support contracts on
the backs of great open source projects and developers. We shouldn't even be giving them the press! They
are a bunch of HACKS!

Ah hem, OpenBSD.?.?

[Anonymous Coward]

You get OpenBGPD and OpenOSPFD all working in concert through the kernel. Oh and did I mention the price? $40.

Brilliant!

Open source and routing

[stox]

I guess those BSD guys have just been playing around all these years.

Hardware Components

[CelestialWizard]

While a company such as Vyatta may be able to deliver the software to actually do the routing, you still need hardware pieces to actually connect to your equipment.

There aren't many PCI (full or half height) cards that can do ATM with OC3, etc....

Then there is the size factor. Data centre space is sparse and expensive, cisco (and such) equipment is built for this space. x86 PCs also run hotter (and louder) than specifically designed hardware from vendors such as cisco, juniper and 3com. oh and they draw more power.

i just can't see how this will take off in the top end of the market.

sure, for a small branch office that connects to frame, isdn, dsl or pstn and runs a vpn it may be fine, but not in a data centre or racked environment.

Re:Hardware Components

[burne]

There aren't many PCI (full or half height) cards that can do ATM with OC3, etc....

I've been able to live in ISP-land for over ten years without ever coming close to ATM. Big exchanges like the AMS-IX (biggest public IX worldwide) have been pure ethernet since their inception. Getting ethernet in some form from a transit-provider is just a checkbox in the right place. Current commodity hardware will do linespeed GigE over PCI-X. Current high-end PC's have sufficient bandwidth available. 66MHz 64bits PCI-X might sound like 266MB/s, but keep in mind that equates to well over 2.5Gbit/s. The right hardware has 3 independant PCI busses and busmasters, so should be able to move 7.5Gbit/s of data via busmastering DMA, and thus with low CPU load. Keeping a full routing table and a bgp-daemon running doesn't require odd hardware. Juniper has been doing that on a Pentium MMX 333 with 768Mbyte since 2001, and a dual Xeon 2.4 will giggle at that 'workload'.

Combining the above will give you a 3U box (smaller than a 7200) which will route (not switch) 4-5Gbit/s reliable. A 7600 is a lot bigger and a serious sh*tload more expensive. You could buy several identical boxes for redundancy and still keep some change left.

Support is the only serious objection one could have in a FastEthernet-, GigE- or 10GE-world. Luckily I don't need support. I have been supporting stuff like above for ten years so I can manage. I can even support your Cisco and Juniper-platforms as well. I can handle my monthly exabyte by myself, thank you very much.

True...

[jd]

...they buy "world-class support", but having tried to use said support on occasion, I can say that I feel sorry for the world. Sure, it's better than a kick in the head, but not so much that it's worth the cost. I believe the record for longest repair ever was at the University of Manchester, in England, where a Cisco router corrupted the 1518th byte in every packet (thus only corrupting packets with a 1500 byte payload or 1496 bytes over 802.1q). Took them NINE MONTHS to fix. The first three of those, they denied there was even a problem.

Re:True...

[l0ki]

system mtu 1504

THEY didn't currupt your data- .1q adds extra bits onto a packet so that it can "tunnel" data from source to destination- with- you VLAN info etc intact... thus it has to add data to the packet- which can make it bigger (jumbo) than intermediate devices expect/allow... it's normal bevahior and you just need to tell those intermediate devices to allow bigger MTU size without dropping the frame as being too big... Maybe this was a while back or something.

You can't blame Cisco for a missing config line...

Re:True...

[nolife]

If the solution was really that simple, you just proved the parents post. The referenced Cisco world class support team took nine months to diagnose and fix a problem that a random person on /. could have fixed in 30 seconds.

Re:True...

[osbjmg]

Dude, they surely tried this. Don't assume you fixed the problem when in fact you weren't there! Depending on the platform and functions applied to this particular device, it could have been much more complicated. Usually they are running traffic tunnelled through the FWSM module and it forgets to take into account the .1q tag or they are using an encryption module which had calculated on pre encryption sizes.

Not just BSD.

[jd]

GateD used to be under a semi-open license. Then there was MRTD, Zebra and Quagga. XORP is said to be pretty good, too. MIT's Click is probably the most versatile, as you can just about script your own routing elements - very pluggable - with the added capability of routing between physical and simulated (eg: NS-2) networks.

This reminds me this...

[dark-br]

... interesting article on TechWorld: A reality check for open source routing.

7200? How about replacing big iron?

[burne]

My former employer is using three relatively simple Tyan dual Xeons with a couple of Syskonnekt cards to shove 4-5 gigabits per second of traffic over the internet (yes, full routing, and over 240 peers on AMS-IX and NL-IX). Most of that is usenet (http://www.top1000.org/top1000.current.txt look for 'tweaknews') but well over a gigabit is DSL end user traffic and some hosting. Those boxes cost in the order of 7000 euro's a piece, and are about as stable as a cisco running an current IOS (not as stable as you'd like). 7 grand buys me a single linecard for a 7200 on the secondhand market, and no 7200 will do as much traffic.

Cisco and Juniper: start getting scared *now*

Re:7200? How about replacing big iron?

[mlyle]

Right; the parent's point is that commodity hardware is even threatening the high end (e.g. above the Cisco 7200 mentioned in the summary).

Of course, lack of support and other issues will keep this away from the enterprise for the foreseeable future-- but this could make sense for a lot of startups with specialized needs or wishing to push a lot of traffic on the cheap.

Re:7200? How about replacing big iron?

[necrogram]

I got a few concerns about diting my cisco and juniper boxen. I've blown a sup on cat 6000. for those of you not up on your cisco slang, a sup is your supervisor engine, analogus to a motherboard/processor pair. two thumb screws, one console cable and 10 minutes later. i was back on line. I cant see swapping out x86 platform that quick. I dont see something like statefull switch over in an x86 style platform. the hardware/software intergration for that is absolutly sick. while on the topic of x86... I peer one of these guys up to a new AS or VRF and suck down a larger BGP table, is it going to pause forwarding and routing while it does a memory swap becuase its underlying os wants to do memory managment buisness as usual style? I'm not knocking open source, but wanting to set your sights on the upper end of the router segment, you need to look at why cisco and juniper are that segment. If you have a need to sustain 4 to 5 gig of routing performance, you're probably going to need more than a routing table

Oohh, Vyatta

[creepynut]

"Initially, we believe that the x86 PC running Vyatta -- given the range"

Reading from a distance, I thought that said VISTA, not Vyatta :)

I was starting to think that Vista had lost so many features that the only thing it was good for was for setting up a really, really expensive router.

Re:Netgear

[Thundersnatch]

FYI, we're talking about "real routers" here... routers that speak BGP and other dynamic routing protocols to link sprawling multi-site networks with leased lines and VPNs. Enterprise-class stuff.

By comparison, the Netgear, Linksys, D-Link, or whatver else you picked up at CompUSA are not "real routers" at all, as they only use simple NAT and static routes with 2-3 interfaces at most.

Can we slap ourselves?

[Triode]

We (by that I mean geeks in the networking world) have been doing this for years...

Why can't we think of ways to profit from this as these companies do??

Damn, should have gone back and gotten that MBA...

Re:Netgear

[ruckc]

I believe he was referring to netgear's "REAL" routers, they offer some small business routers that are bigger badder than the ones they sell for home use. http://www.netgear.com/products/business/prod_vpnr outer_wired_security_sb.php

Re:Yawn. Slow news day?

[El Torico]

PCs running tuned linux or BSD kernels work great for anything but ISP-grade stuff.

The first Juniper routers were "Olives", which were PC's running modified BSD. JUNOS is BSD based.
UUNET, IMHO the greatest ISP ever, first tested them in 1998 or 1999. CISCO had annoyed UUNET with poor service, so UUNET helped bring Juniper into the market. Yes, I am former UUNET and proud of it.
I found an interesting link to Olives at http://juniper.cluepon.net/index.php/Olive.

More FUD from someone pet project

[jbossvi]

This keeps coming up every 6 months or so. To rehash it for you:

1) performance wise a 6x PCI-X motherboard is rare and commodity computers are not built for the buses to independantly talk to each other without invoking cpu.
2) feature wise you Have to have a RTOS or bad things happen when you try to implement QOS. speaking of features they have libraries full of books that talk about the *thousands* of features technologies that real routers implement (its hard to do that most companies spend tens/hundreds of millions to do this). implementing a few protocols/nat/firewall does not a router make.
3) If you actually have been involved with these things you would know:
    -ds3/oc3/oc12's are not cheap... phone company bills of $100k a *month* is very common.
    -a couple network engineers $100k/year each
    -dedicated power/colo space/ups/generators $50+k/year
    -SLA's and peering arraingment... $$$
    -uptime to your customers measured in seconds of uptime (revenue $200+k/MONTH). ...... AND you want to save $30k by using a #@$%#$%#$% software router running on a DELL?????

really, try explaining that to the CEO after the site has lost $10k/HOUR because something wonky is going on with the cpu or the memory oorrr it could be the kernel, I dunno I just rebooted the thing "cuz that usually fixes MY problems"... bye bye SLA.

--jboss

*sigh* a solution in search of a problem.

[l33t-gu3lph1t3]

It'll never, EVER challenge Cisco in the big iron market. Why? Simple. No IT manager has EVER been fired for buying Sun servers, Cisco routers/switches, or IBM PCs. Big iron isn't about open source. Big iron is about triple-redundant reliability, service contracts, and brand trust.

niche

[Neuropol]

It's very parallel in it's nature in that a small networking company could present this as a cost effective option. I see how a small networking consultant company could actually push this towards the small business level. But I'm doubtful it could ever be presented at the public/community level for use in schools or public wi-fi rest areas when the state lays out stipulations regarding only accepting bids from Cisco based numbers and Cisco certified installers? More or less, mandating that tax payer investment for this massive scalability be present. Often times, in these areas, that is ten-fold overkill. So there is the need, but I'm afraid that Cisco's name is so far embedded in corporate and state america, it's going to be a tall order to ever penetrate that market. Bringing this comment back to the small business. If a consultant can convince the client that this is a viable router to placing thier 20-something client station network on, then, yes, it has a niche.

This isn't news.

[rnxrx]

I think we see some version of this article every few months - yet another revelation of an open source package that can turn PC's into routers. This isn't news. There have been various shapes and forms of routers on *NIX platforms for many, many years. Some of these platforms served (and still serve) as reference implementations of certain routing protocols.

The common responses on here seem to revolve around the inability of PC hardware to handle high bandwidth. To an extent this is necessarily true. A general purpose PC is going to rely on its CPU to handle each packet traversing the box. Processors are fast and cheap and becoming faster and cheaper as time passes. Most commercial router vendors realized quite a while ago that any architecture whose perforance is based on a single, centrl CPU inherently represents an eventual bottleneck and thus a serious challenge to scalability. As such, most commercial routers have moved to a model where forwarding is pushed as far as possible from a control plane that is as discrete as possible.

In other words, if we push the actual heavy lifting of forwarding out to distributed components (e.g. the interfaces themselves) then we're no longer left in a situation where our BGP process is vying for cycles and memory access with packets in transit. When properly implemented this means that I can be moving huge amounts of traffic through my router without interrupting network control traffic, management of the box, etc, etc.. It also means that by distributing packet switching they can hit massive performance levels with a comparitively modest CPU.

At the high end with Cisco and Juniper you're paying for the development of some exotic ASIC's and some even more exotic interface hardware. You're also paying for the capability to support high density - PC platforms aren't going to support tens of 10G or hundreds of 1G interfaces any time soon. The capacity for redundant CPU's, stateful failover, etc is also worth remembering.

At every level of Cisco and Juniper hardware you're paying for the ongoing development and maintenance of a highly complex codebase full of features that just aren't practical (or, in some situations, possible) for the OSS community to implement well. Implicit in this is a huge system test and regression faculty.

I've used and deployed open source routers up to OC3 bandwidth. They worked and, for the most part, worked well when faced with relatively simple networking tasks - multihoming enterprises to the Internet, basic WAN routing, etc. My observation has been that these platfoms start to fall apart when faced with requirements for complex routing policies, fancy QoS, MPLS, etc.

There's a definite place in the world for PC-based open source routing platforms - particularly at the edge of larger networks or in the midst of small and medium sized ones but I don't think Cisco and Juniper need to worry about being rendered completely obsolete any more than Oracle needs to worry about being driven completely out of business by MySQL or PG.

WideBand beats the crap out of Cisco

[Rabid Cougar]

Wideband makes Layer-3 switches that beat comparable Cisco routers hands down. With their nMU (pronounced "NetMU") it makes easy things easy and difficult things easy too. With their 28-port switches, you can get full-duplex, non-blocking Gigabit transfers on all ports simultaneously. And did I mention that they can even do Gigabit over CAT-3 and barbed wire? Also, if you use the nMU control your switches, none of them even need IP addresses. Good luck trying to hax0r a switch with no IP address. Throw in the fact that all their stuff is made in the USA (no off-shore customer support) and costs much less than comparable Cisco gear that doesn't perform nearly as well, and you have yourself a superior product. If you are expanding or replacing your network infrastructure, consider WideBand over Cisco. You'll be glad you did.

***Disclaimer***

I do not now, nor have I ever worked for WideBand, but we use their gear where I work. BTW, there were some guys who ran a Cisco shop in the training class I was in that WideBand offered. Last I heard, they were replacing all their switches with WideBand gear. IMNSHO, WideBand is the best kept secret in networking

Re:OSS is great, but not for everything...

[msimm]

FTR, if you can manage the support and deal with irregularities as they might come up, as it sounds like your company probably can, I totally agree. I'd even go so far as to recommend ClarkConnect, personally.

But these still don't deal with the issues of hardware/platform stability (yes, its a *lot* easier to design, troubleshoot and design driver modules if you control the platform first), QA (testing commercial *before* sending a product out the door), organized 'knowledge bases' (assuming your appliance has large enough penetration), commercial support because things *will* go wrong and if your running mission critical applications behind your 'appliance' you'd better be able to get fixed fast and have the CMOA part dealt with too (after all, the large the company the less forgiving they can be for mission critical application/server/network downtime).

So, ya, if you've got the wiggle room and need to allocate re$ources elsewhere and have someone onboard who's stable (hate to inherit someone elses 'customized' framework) I think its very useful.

But if your company/job/livelyhood/client-base depends on it I feel pretty strongly about using something start to finish purpose built.

As an aside I did a lot of research on firewall appliances before we purchased our own and of the sys admins I know Sonicwall was the one product that almost unanimously was not recommended. So its probably not just you, just bad luck. We've gone with Astaro, who aside from making a software distrobution also does build an appliance. Its Linux, so I know if things every really went south I could get my hands dirty and make things right, but I don't and shouldn't have to. I can dynamically update rules, add nodes, do hot/cold or hot/hot failover and I don't have to string together a bunch of software applications of varying quality and flexabilty.

And best of all, although possibly alarming, if I should ever leave the company whoever picks up my work will be able to quickly learn to manage the software. The network doesn't skip a beat.

Anyway, I'm not trying to argue against what your saying. If it suits your needs use it. You know your company better then I do. I work for a medium sized sompany and some large (fortune 500) sized clients. So we've got a little room in the budget (of course its always a fine line) and certainly a justified need. I don't know if you've ever had to sit in a meeting and explain your network topology and how you handle things like redundancy but when you start naming OSS products outside of say the top 10 you get some pretty disarming looks. :)

Enjoy IPCop. I'd say take a look a ClarkConnect but until they get the rules/insert method updated I won't touch it, they were using Shorewall and even a minor change (like say opening an FTP port for a new client) requires a Shorewall/IPTables restart (or a CLI insert, but I always though those were more prone to error...as in sleep deprived, up at the colo human error then a clean GUI) and that, at least in my case, is totally unacceptable. Maybe IPCop has dealt with this differently since I last used it, but on the fly changes should be the first priority of any serious firewall solution (well, after overall system security).

Anyway, I'm just throwing out my $.02. You certainly don't sound like an idiot.

Re:Yawn. Slow news day?

[jmilne]

I had some experience with Olives as well. However, their performance wasn't that great. Especially compared to a M10. ASICs made a huge difference.

Re:Yawn. Slow news day?

[Lumpy]

You can easily route 5 T1's on a Thrown away 586MMX at 266Mhz.

I did it for 6 years with a hand rolled linux install and ipchains. IT was faster than the HP 6 port router it replaced in both speed and network performance and adding in some filtering gave us a product that sould have cost $6000 at the time from the New Cisco company or then popular Colorado networking.

Every single one of these guys here claiming that no way a PC can route much traffic knows absolutely nothing about networking and routers. Hell a cheap P4-2.8 with the right hardware can route ATM speeds over ethernet easily (Yes kids, you can get ethernet termination on anything from your provider).

Hell a slow 386 can do a single T1 without getting about 5% processor loads.