x86 Commodity-Hardware Router?

neomage86 asks: "I recently had to set up a router for a small company, only five users at any given time, and the needed VPN capabilities are built in. So, instead of using a Cisco or other embedded router, I decided to just install Linux and IPTables on an old 200 MHz PII I had lying around. It's been working fine, and I'm thinking about doing something like this for a much larger network (3000+ users). Does anyone have suggestions on how much I will have to beef up the hardware to provide IP Masquerading for about 1000 users on a T3; provide network-layer filtering of the transmission; and route between 4-5 internal subnets?"

Comments

Upgrade? Hell, you're already massively over-spec!

[Finni]

You'll be fine with what you've got right there!

No seriously, you're going to swamp your PCI bus if you're doing routing between internal subnets. Goodbye, LAN throughput. Not to mention what merry hell you'll play with the CPU with VPN and firewall rules.

Your solution is great for a small place, or even a large place in a dedicated niche (like only VPN and/or firewall, or monitoring/IDS.) I wouldn't do something that ambitious with PC hardware though.

What's good for the customer

[duffbeer703]

If I was a potential customer of yours, red alarm bells would be going of in my head.

Instead of offering standardized equipment that can be managed via console, ssh or SNMP by any competent network engineer, you offer some customized linux router solution that will always need to be handled differently.

What advantage does your solution offer?

Is it worth "saving" a little money up front, only to need to seek out your consulting services later?

Re:What's good for the customer

[jhoger]

So you're saying that his customer should avoid vendor lock-in by locking in with a proprietary vendor?

Hmm... Linux routers and firewall rules are well described on the web. Any "competent network engineer" as you describe him/her is likely able to read...

Re:What's good for the customer

[Anonymous Coward]

I'll bite (the troll).

Instead of offering standardized equipment that can be managed via console, ssh or SNMP by any competent network engineer, you offer some customized linux router solution that will always need to be handled differently.

A customized linux router solution can be managed via console, ssh or SNMP by any competent network engineer.

What advantage does your solution offer?

1. More online documentation than every other router and firewall vendor combined. Docs ranging from step by step howtos to in-depth discussion of complicated setups.
2. An open system that is upgradable on your timeline not your vendors
3. An easy upgrade path. If you want IPv6 support (or some other feature) and have an old firewall you might have to purchase a whole new unit if a new firmware with those features isn't available for your unit.
4. An army of people who know how to use iptables

Is it worth "saving" a little money up front, only to need to seek out your consulting services later?

Since it'll be running on an open system, they can seek out anyone's consulting services they want including those that might be in their own organization.

1000+ Users????

[the eric conspiracy]

Do the math. If your homebrew system goes down, you will be burning the time of 1000+ people ($60,000) per hour. With those kind of numbers it doesn't pay to do it on the cheap. Get a redundant Cisco system with plenty of power backup.

Re:1000+ Users????

[ADRA]

Not that i'm arguing here, but a Cisco equiv. Is hella-bucks for what this guys is trying to do, and its only a Passive failover anyways. If you want a solution that is truly expensive, try any ACTIVE failover provider.

Anyways, I have been using netfilter/iptables for on my 30 user, >100mbs network, 6 active NIC's and I've never had a crash that I didn't cause!

no can do sorry

[nocomment]

It doesn't matter what sort of PC you are using...you simply cannot pump that much through a standard PC. 3000+ users? forget it. You are going to need a cisco my man. Unless anyone knows if those quad cards can route between connectors at faster (much much muuuuuch faster) than the PCI bus will allow.

Re:no can do sorry

[prisoner-of-enigma]

"true...sort of. if those 3000 users aren't doing much other than checking email and browsing the web. If they are doing some serious stuff; which they may not be who knows?; then chances are good that 3000 users means a heck of a lot of traffic."

You're just not grasping this concept very well, are you? Let me spell it out to you very slowly: the limiting rate here is his T3 connection! No matter what these 3000+ users are doing, they cannot generate more than 45Mbit/sec of traffic because that's the max the T3 will handle (actually it's slightly less than even that due to overhead). So, with a single 100Mbit Ethernet card for the internal net and a single 100Mbit Ethernet card for the external net (or a T3 PCI adapter, it doesn't matter which), what's the max traffic you're ever going to have to deal with? Bingo! 45Mbit/sec, which is well within the capabilities of a single 100Mbit Ethernet card. It sure as hell isn't a problem for the PCI bus, which maxes out at 133MB (bytes, not bits) per second. That's 1064Mbit/sec, compared to the T3's 45Mbit/sec.

So, in short, it doesn't matter whether you've got one user, 5,000 users, or 50,000 users -- they are restricted by the smallest pipe in the system, and that's the T3. This should be obvious, but for some reason you keep thinking that more users can somehow generate more than 45Mbit/sec of traffic through a T3. Sorry, it can't be done. Perhaps you're thinking about using a PC as a switch instead of a firewall or something, but as a firewall you are completely and totally wrong.

Re:Buying a service, not a router

[jhoger]

Okay, I'll bite. You're saying that if you have a magic nicely shaped appliance it somehow won't require security patches like a Linux box does?

All software has bugs. All software, particularly that which runs on the edge of the network, must be maintained with patches. All hardware networking solutions of any reasonable complexity like a router or firewall run software. Therefore they too must be patched from time to time.

At least with a Debian box you could put a cron job that automatically apt-get's latest patches for itself, if you wanted to have a box which maintains itself (I would rather have an admin maintaining it, but whatever...).

This may or may not be a feature of whatever appliance the parent thinks is a better alternative.

Don't use Linux for this

[phoenix_rizzen]

The packet filtering software on Linux is horrible. The syntax is just nasty. And there are no guarantees it won't change again with the next kernel release.

Use a BSD system, with a real packet filter. FreeBSD gives you the choice of IPFW, IPF, or PF. OpenBSD gives you PF. NetBSD gives you IPF or PF. All of those have much larger / better features sets than IPChains / IPTables, and work a *lot* better in NAT/PAT/MASQ situations. These packet filters are also truly stateful (last time I checked IPTables, it wasn't truly stateful without a bunch of extra patches).

Linux makes an OK home firewall. But I wouldn't use it anywhere near a business.

We use FreeBSD 4.9 on Pentium 166 MHz systems with 128 MB RAM using IPFW to server secondary schools with just under 300 student computers. Haven't had any problems yet with network slowdowns or dropoffs or anything. These are on T1s in the remote schools, and 8 Mbit cable in town.

(I had problems keeping a similar box running Linux and IPTables working on my home wireless T1-equiv link.)

Dont bother

[moosesocks]

If your company can afford to pay 1000 people and run a T3, they have the money to buy a PROPER Cisco-based setup.

Oh. And hire an experienced professional to install it (i don't dobut that you could manage it, though). I wouldn't trust a job of this size to someone who 'did it once at home and it worked'. The enterprise works much differently than your basement.

If you set it up and something goes wrong, you, my friend, are screwed.