Slashdot Mirror

← Back to Stories (view on slashdot.org)

x86 Commodity-Hardware Router?

Posted by Cliff on from the pushing-the-limts dept.

neomage86 asks: "I recently had to set up a router for a small company, only five users at any given time, and the needed VPN capabilities are built in. So, instead of using a Cisco or other embedded router, I decided to just install Linux and IPTables on an old 200 MHz PII I had lying around. It's been working fine, and I'm thinking about doing something like this for a much larger network (3000+ users). Does anyone have suggestions on how much I will have to beef up the hardware to provide IP Masquerading for about 1000 users on a T3; provide network-layer filtering of the transmission; and route between 4-5 internal subnets?"

80 of 102 comments (clear)

  1. VPN by aeakett · · Score: 3, Interesting

    VPN can be a real resource hog... word is though, that the Via C3 has some sort of processor level instructions to help accelerate this. Has anbody else heard of this?

    1. Re:VPN by aeakett · · Score: 4, Informative

      Ah! Here it is! It's the encryption that the C3 seems to rip through.

    2. Re:VPN by quinkin · · Score: 2, Informative
      VPN should be offloaded to a seperate box/boxes (NB: boxen is not the plural of box, just as foxen is not the plural of fox - although that does imply that bixen should be a female box...).

      Even with the higher end router/vpn embedded solutions there seems to be an appreciable slowdown in the other traffics response times and throughput when the VPN is being heavily used - and the hardware acceleration in these systems is liable to "better" than the C3 acceleration.

      Does anyone know if the C3 can do h/ware accelerated 3DES? AES is good, but compatability is better... I would assume the RNG could also be applied to DES/3DES to at least improve performance.

      Q.

      --
      Insert Signature Here
  2. Upgrade? Hell, you're already massively over-spec! by Finni · · Score: 4, Insightful
    You'll be fine with what you've got right there!

    No seriously, you're going to swamp your PCI bus if you're doing routing between internal subnets. Goodbye, LAN throughput. Not to mention what merry hell you'll play with the CPU with VPN and firewall rules.

    Your solution is great for a small place, or even a large place in a dedicated niche (like only VPN and/or firewall, or monitoring/IDS.) I wouldn't do something that ambitious with PC hardware though.

  3. Go BSD rather then Linux..... by jsimon12 · · Score: 5, Interesting

    I would personally go with a BSD flavor rather then Linux. Don't get me wrong Linux is great but BSD was designed with routing in mind. You will be able to get away with less hardware and out of box things like OpenBSD are going to be more secure then a commodity Linux.

    1. Re:Go BSD rather then Linux..... by frankm_slashdot · · Score: 2, Interesting

      i wanted to say that... but was feeling lazy.

      now that ive hit refresh a few times and have read your comment i might as well add my own $0.02

      openbsd with pf is, imho, 50x better (and easier to set up and manage rules for) than anything linux can offer.

    2. Re:Go BSD rather then Linux..... by fozzmeister · · Score: 1

      Don't know about speed, but the rule system is also waaay more readable

    3. Re:Go BSD rather then Linux..... by bluelip · · Score: 1

      More secure "out of the box"??? Who is going to install an OS w/o tuning it and doing updates anyhow? This whole "out of the box" saying is bogus. If you're running a default install, you're just asking for touble w/o doing the required updates, etc.

      --

      Yep, I never spell check.
      More incorrect spellings can be found he
  4. 1000+ Users???? by the+eric+conspiracy · · Score: 3, Insightful


    Do the math. If your homebrew system goes down, you will be burning the time of 1000+ people ($60,000) per hour. With those kind of numbers it doesn't pay to do it on the cheap. Get a redundant Cisco system with plenty of power backup.

    1. Re:1000+ Users???? by ADRA · · Score: 2, Insightful

      Not that i'm arguing here, but a Cisco equiv. Is hella-bucks for what this guys is trying to do, and its only a Passive failover anyways. If you want a solution that is truly expensive, try any ACTIVE failover provider.

      Anyways, I have been using netfilter/iptables for on my 30 user, >100mbs network, 6 active NIC's and I've never had a crash that I didn't cause!

      --
      Bye!
    2. Re:1000+ Users???? by ryanmoffett · · Score: 1

      The Cisco PIX OS 6+ and recent Cisco IOS code revs support stateful failover. We already do this and it works just fine. If you are looking at several interfaces and need to run at DS3 speeds, a PIX515E Unrestricted Licensed PIX and a PIX515E Failover licensed PIX will do the trick. True, they aren't price comparable to commodity x86 hardware (even though they are based on that architecture), if you have 3000+ users, you can most likely afford these.

    3. Re:1000+ Users???? by pete-classic · · Score: 1

      I have a nickel that says you compose your posts in a word processor.

      Am I right?

      -Peter

    4. Re:1000+ Users???? by ADRA · · Score: 1

      If you concider Mozilla forms a word processor, then YES!

      --
      Bye!
    5. Re:1000+ Users???? by darrelld2 · · Score: 1

      I must agree here. Working for a large company, 10,000+ users that have a 45 Meg Internet connection I have seen first hand even the most powerful Linux solution can not handle the load and log files that are needed. We ended up pulling out a cluster of 2 linux boxes load balancing NAT connections and replacing them with the PIX 535 firewalls. This was without this solution handling any of the VPN requirements. VPN is a whole other ball game. Cisco is in this business, and the stuff is priced reasonable enough.

    6. Re:1000+ Users???? by Anonymous Coward · · Score: 1, Informative

      OpenBSD has state synchronization, which could be used to implement failover. But I don't know how this would compare with Cisco's failover support.

    7. Re:1000+ Users???? by dubl-u · · Score: 1

      Do the math. If your homebrew system goes down, you will be burning the time of 1000+ people ($60,000) per hour. With those kind of numbers it doesn't pay to do it on the cheap. Get a redundant Cisco system with plenty of power backup.

      Or you could use something like this to provide redundant Linux routers on cheap commodity hardware and spend the money saved on getting more backup power.

  5. All things considered, spend a couple hundred. by WolfWings · · Score: 5, Informative

    And for said couple-hundred, you're looking to pick a secondary network card, along with a 2Ghz or so Athlon or P4 of your choice with a motherboard with a built-in network card. The built-in network card is important for a router.

    An Athlon-64 or above would be ideal, simply because you'd be able to mount ludicrous amounts of memory on the box, which is pretty much all that could ever matter for a router/firewall app, as Linux can easilly support logging anything you want to a remote boxen.

    Realistically though, I've routed 8 T1's at 80+% capacity in both directions among 650 laptops before, including 3 seperated subnets, all routed through one box.

    The box was a Celeron (P2) 800Mhz we'd downclocked to 633Mhz (standard practice at my company, downclock everything for live events for stability) and it used around 10% of the CPU at peak once configured correctly.

    By 'correctly' I mean having the T1's all coming in on a seperate PCI bus from the actual network cards for the subnets. Specifically, the built-in ethernet turned out to be on a seperate PCI bus from the actual PCI slots in the case. Configuring the box to take advantage of this dropped CPU load from 80+% to ~10%.

    So... for a T3 fully loaded? I'd say get a 2.0Ghz machine just for breathing room, and give it at least 2GB of memory, as neither is that expensive and will leave plenty of breathing room for things like IPSec or other fancier options down the road without any problems.

    1. Re:All things considered, spend a couple hundred. by ComputerSlicer23 · · Score: 1
      Hmmm, I've got a few questions. First off, they never made a P2 that went 800Mhz. Those stopped at the 500Mhz mark or so. Next, your telling me you cracked open a case on a working machine, fiddled with jumpers (not many P2/3 era MoBo's had clock speed settings via the BIOS). I'm not sure, but you probably had to switch the clock rate of the FSB, which was bad in and of itself. Why on earth would you switch the configuration on a known good machine? That's among the more assine things I've heard done in the name of stability. Next you'll tell me you kept machines in the bathroom, so you could ground the cases directly to the pipes. I've been told that downclocking can lead to its own set of instabilities (you didn't downclock enough from what I've heard, you'd have to downclock from about 800 to 200 or so. I've considered picking up a 2.4Ghz chip and running it at 400Mhz so I wouldn't need a fan, instead I picked up a Via CPU).

      Now, I'm also curious about the machine you had. It had to have two PCI buses for the configuration you describe. At least 4 PCI slots (assuming, you got 8 T1's in a single card). I'm not aware of too many PCI cards that can do a T1. I sure don't know of one that can do 8 T1's on a single card (maybe you had a T3 interface card and only 8T1's worth of bandwidth). Probably you had to have either 5 or 9 slots (3 Network, and 2 or 4 T1 cards). Possible 11 slots. The bus on that had to be pretty damn impressive, that's a whole lot of data to be pumping thru a PCI bus of that era. Probably only 33/66Mhz bus on that bus.

      Pretty impressive that you could move that much data thru a CPU with too small a cache, and too slow a cache, on a Motherboard that couldn't have dealt that that much data. Somehow the details aren't adding up.

      Kirby

    2. Re:All things considered, spend a couple hundred. by WolfWings · · Score: 3, Interesting

      First off, the case itself was one of the 'all in one' deals, simple one-5.25 bay, one-HD bay, one-floppy, half-height PCI cards only, etc.

      The P2 was a typo, and one I appologize for. P3 would be much more accurate, and overlooking the typo is inexcusable as I was simply typing quietly before I hit post, and didn't read the entire post from the beginning before hitting post.

      As for the T1's, we didn't use any PCI T1 cards. We used an external 10/100/1000 switch with all 8 T1's plugged into it via normal T110/100 converters as a concentrator, with the uplink port plugged into the computer. Four 10/100 PCI half-height network cards + onboard, three + onboard used. Onboard led to the switch with the T1's on it, the individual network cards all led to individual subnets.

      As for the downclocking, yes, we had to throw jumpers. And as I said, it was policy at the time, and one I didn't completely agree with but it did noticably lower the heat output on the CPU's, which was often a problem when we had to install these things under bleachers or in other areas with absolutely zero ventilation and little access. In one case, we had to repurpose a bathroom actually, speaking of those. For that specific reason, the downclocking made sense.

      The configuration of the multiple T1's on one ethernet port was fairly simple, using the Aliasing features of Linux to pretend to be 8 seperate ethernet cards plugged into that one switch, leading to each of the 8 T1 cards.

      And yes, the CPU had little cache, and slow cache to boot, but lots of memory, and with that configuration it wasn't dealing with much data, barely a fraction of the actual network traffic, because all the network cards we'd installed could copy data directly from their own buffers to other network cards. The fastcopy option under Linux Networking in the kernel IIRC.

      If you have any more questions, feel free to post again though. :-)

    3. Re:All things considered, spend a couple hundred. by WolfWings · · Score: 2, Informative

      The T110/100 was supposed to be "T1 to/from 10/100" with arrows pointing both ways. Slashdot ate the greater-than/less-than signs, along with the hyphen.

    4. Re:All things considered, spend a couple hundred. by ComputerSlicer23 · · Score: 1
      Hmmm, never heard of the T1 to 10/100 converters (I was unaware the framing was the same, or that the addressing could be fiddled with so that you could use a transceiver to change from one to the other). I'll have to look at the physical framing to see if that makes any sense at all. Got a link to one? I could use one where I work at points.

      While you can do the direct copying from one care to another (I didn't know it could jump PCI buses, but it makes sense), that eliminates all of the packet processing. In fact, I thought it made Linux act like a bridge. I'm not that familiar with the issue, but I've always been told it is an odd feature, that eliminates a ton of the processing done on a packet. So it's only good on a straight up router/bridge type setup.

      All firewalling and policy routing is skipped if I remember correctly. Which makes it a good idea if you split the traffic into small enough streams that a single machine could then firewall/IPSec the stream (it also means that do egress/spoof checking isn't possible via firewall rules, however I think Linux has on option to do that via a sysctl, presumable that could do it for you while dong fastcopy).

      Finally, how the hell did you get a phone company to bring in T1's in such a quick manner that you couldn't get some type of ventilation. That or bring a box fan.

      Kirby

    5. Re:All things considered, spend a couple hundred. by WolfWings · · Score: 2, Interesting

      Okay, point-by-point again.

      The 't1 to 10/100 converters' are just common T1 interface boxes that output ethernet instead of 24 voice/data jacks. Data-only T1 interfaces, essentially. Unfortunately, that was one aspect I had zero to do with, the site provided them and I haven't had a reason to use them since (we usually do satellite T1 links for remote sites, or use sDSL for medium-term fixed emplacements), so other than saying Netopia was branded all over the boxes, I can't help further than a Google search would.

      And the direct copying can change the addresses, so MASQ can still function as I understand it. To be honest, the direct copying of packets didn't drop the CPU load anywhere NEAR as much as simply having the cards seperated across seperate PCI busses, so the CPU could talk to each of the groups at the same time, instead of having to shout down the same piece of tin-can-and-string to everyone at once.

      We did do what you described though, all the firewalling/IPsec/what-have-you was a seperate set of rules between a pair of virtual ethernet devices.

      The overall layout was this:

      Arbitrary subnet gets VPNed/MASQed/etc to a virtual ethernet address. Virtual ethernet gets firewalled to another virtual ethernet. Second virtual ethernet gets dynamically MASQed with connection-tracking to the 8 T1's to send the traffic to the lowest-usage T1 over the last minute or so using QoS rules.

      Most of that's just shuffling headers around, which are tiny, and the final copy boiled down to a single MASQ and either getting passed on or dropped on the floor, which still works with fastcopy.

      And yes, tracking a couple thousand concurrent connections did eat up the memory. (2-4 per laptop, LONG story, client was using multiple bidirectional realmedia streams to push an IRC-like live QA session at the Detroid Auto Show one year for vendors, so the presenter could ask questions and get realtime answers back without having to resort to a 'show of hands' count. Yes, we told them it was a bad design.)

      As for cooling... At detroit we had plenty of space, plenty of cooling, etc, etc. But to be quite honest we've literally shown up at a site, and been informed they 'repurposed' our space for storage, and found we can barely squeeze a folding chair and a laptop into the space left for us, even with setting things on shipping crates. We gave up complaining and learned to expect (and equip ourselves) to be crammed in the equivilant of a furnace room with zero ventilation and space for one person to stand unseen as our minimal requirements for getting a live press event running for up to 12 hours at a stretch. Live press-style events are a bitch, but we do fairly well at supporting them.

    6. Re:All things considered, spend a couple hundred. by ksheff · · Score: 1

      I have at least two, maybe three slot-1 motherboards from different manufacturers that allow you to change the FSB via the bios. they were around.

      --
      the good ground has been paved over by suicidal maniacs
    7. Re:All things considered, spend a couple hundred. by John+Harrison · · Score: 1
      I have a Celeron 300a and I can change the clock, FSB, and voltage in the bios. The machine is now 5 years old and has the original mobo.

      I used to overclock it to 475 but it was unstable.

      --
      Lasers Controlled Games!
    8. Re:All things considered, spend a couple hundred. by asdfghjklqwertyuiop · · Score: 1

      The 't1 to 10/100 converters' are just common T1 interface boxes that output ethernet instead of 24 voice/data jacks. Data-only T1 interfaces, essentially.

      There's no such thing as a 't1 to 10/100 converter'... you must be talking about something which bridges two ethernets over a t1 line.

      Anyway, so you had all these 'converters' plus this linux router plugged into one ethernet switch? I hope you weren't relying on the linux router to enforce any kind of security among these remote networks. Assuming they're bridged like I described above, ethernet frames can be sent directly from one remote network to another bypassing your router/firewall.

    9. Re:All things considered, spend a couple hundred. by WolfWings · · Score: 1

      I'd pull up proper terms, but we don't actually deal with physical T1's from the Telco often enough for me to have bothered memorizing the correct terms, manufacturers, or anything else about them. Even in phone work we usually find ourselves dealing with PRI at the fanciest.

      The T1's were all used for combined bandwidth, as the event organizers dropped their order for a fractional T3 and got eight T1's at the last moment. We had no say in that aspect. And each T1 was plugged directly from the box-with-a-card-in-it from the Telco directly into the Netopia box, which had two other ports on it. One for power, and one for 10/100 Ethernet. As far as we were concerned, using the Netopia box was the same as using a 10/100 fiber-optic converter for extending an ethernet run, hence the terminology I've been using here.

      And the T1-side didn't need any security. They were all going to the same ISP (that didn't support equalizing or banding or any other form of merging multiple T1's, yes, we asked) so it made sense to simply use a high-speed switch as a concentrator to our router.

      Internally, the room was wired in a two-level switch tree for the main 'pool' of 500+ laptops. One 32-port switch feeding seperate 32-port switches for every eight tables of three laptops. Those laptops were fully locked down unless you went out of your way to blatantly physically tamper with the laptop. Since this was a private sub-event just for PHB's, we weren't too concerned about aggressive network intrusion from the pool of laptops, and didn't roll out a fully-secured solution.

      The remaining subnets were small single-switch affairs for the 'master control' and presenter areas, respectively. Each of those subnets were free to crosstalk internally, for obvious reasons.

    10. Re:All things considered, spend a couple hundred. by WolfWings · · Score: 1

      First off, thanks for pointing out the typo.

      Second, that depends on what you define by professional. We can and have been called with travel-time notice only (as in, under 2 hours), and provided a 2Mbit link when a T1 went down.

      Pretty? No.
      Tidying up for another hour? Yup.
      Did it work? Hell yes.
      Was the client happy? Yes.
      Does anything else matter to me, a field grunt that doesn't deal with marketing or any other aspect of the company except making the tech work on-site? No.

      But thanks, I passed along the typo to the person that does care about that part of the company. And she thanks you for the notification. =^.^=

  6. no can do sorry by nocomment · · Score: 1, Insightful

    It doesn't matter what sort of PC you are using...you simply cannot pump that much through a standard PC. 3000+ users? forget it. You are going to need a cisco my man. Unless anyone knows if those quad cards can route between connectors at faster (much much muuuuuch faster) than the PCI bus will allow.

    --
    /* oops I accidentally made a comment, sorry */
    /* http://allyourbasearebelongto.us */
    1. Re:no can do sorry by Paul+Jakma · · Score: 2, Interesting

      you simply cannot pump that much through a standard PC. .... Unless anyone knows if those quad cards can route between connectors at faster (much much muuuuuch faster) than the PCI bus will allow

      If its 100baseT, 4x12.5MB/s = 50MB/s is easily within the capabilities of a standard 32bit/33MHz PCI bus (100MB/s sustained), at least in terms of transfer rate. Make sure to use a card that has drivers which support polling (aka NAPI on linux).

      --
      I use Friend/Foe + mod-point modifiers as a karma/reputation system.
    2. Re:no can do sorry by nocomment · · Score: 1

      you're talking max bandwidth there. Would you actually try to route 3000+ users through that?

      That's just the max that is _theoretically possible. The PCI bus (32 bit) is capable of a (again) _theoretical 127MB. Would you stake your job on those numbers? I sure as hell wouldn't. I'd divide all numbers by 5 and you will see a more likely transfer rate plus have room to grow a little. The asker didn't say what type of business it is, but I'd bet at 3000 users a lot of those are transferring some big files. Does this company have it's own advertising department? If so, they'd be spiking the T3 by themselves all day long.

      You _could_ set it up, and it _would_ work, but you'd be the guy that gets blamed for making a a crappy router (even if it's the best thing since RAID 5). You're better off going with gigabit and a cisco router.

      --
      /* oops I accidentally made a comment, sorry */
      /* http://allyourbasearebelongto.us */
    3. Re:no can do sorry by Paul+Jakma · · Score: 1

      you're talking max bandwidth there. Would you actually try to route 3000+ users through that?

      4x100Mbit is 4x100MBit.. what in gods name does the number of users have to do with it? If you have 400Mbit/s, is that 400MBit/s "bigger" in some way because its generated by 3000 users instead of, eg, 1000 or 500 or even just 1? It isnt.

      That's just the max that is _theoretically possible.

      Look, if the machine has a quad fast ethernet card then the max that box will have to route is 4x100Mbit/s. No amount of users is going to be able to push more at that box than that, and it's well within the capabilities of PCI. So... ?? what is your point?

      The PCI bus (32 bit) is capable of a (again) _theoretical 127MB.

      Of raw bandwidth yes, but one out of every 4 cycles is an address cycle, so the amount of sustainable throughput is ~100MB/s.

      The asker didn't say what type of business it is, but I'd bet at 3000 users a lot of those are transferring some big files.

      Ah yes, and those big files will clog up the PCI bus wont they, even though the bandwidth of 4 100BaseT NICs is well within bandwidth of the lowest-end PCI. Those big files will magically somehow make more than 100MBs go through those 4 12.5MB/s * 2 (in and out) ethernet interfaces. Yes of course, how silly of me.

      You are just plumb plain wrong. Indeed, if people were transferring big files all day long, that would actually be the ideal situation for a PC router. Provided that your bus bandwidth is sufficient (and with 66MHz/64bit PCI it is even for multiple GigE) it is the packet rate that is the obstacle. Polling mode drivers helps for this.

      You're better off going with gigabit and a cisco router.

      Maybe, if you have the money and you actually need high-end performance. The cheap (ie 5 to 10k) low-end Cisco's actually perform no better, sometimes worse, than PCs, and wont handle GigE at linerate either. Also, how will GigE make their T3 any faster? Yay - i've got GigE connectivity to my 43MB/s internet connection.

      --
      I use Friend/Foe + mod-point modifiers as a karma/reputation system.
    4. Re:no can do sorry by nocomment · · Score: 1

      4x100Mbit is 4x100MBit.. what in gods name does the number of users have to do with it? If you have 400Mbit/s, is that 400MBit/s "bigger" in some way because its generated by 3000 users instead of, eg, 1000 or 500 or even just 1? It isnt.

      true...sort of. if those 3000 users aren't doing much other than checking email and browsing the web. If they are doing some serious stuff; which they may not be who knows?; then chances are good that 3000 users means a heck of a lot of traffic. For example, I have an FTP server that you can get a good 20MB sustained (through a gateway), if there was just 4 people downloading at that rate then that's 80M that doesn't leave a lot of room for the other 3 networks that I have connected to that same router. 5 users downloading at that rate? oh, well that the entire PCI bus that's flooded, unless you have 64bit PCI. _MY_ router is OpenBSD and I do have 64 bit PCI, but the asker is saying he wants to do this with a PII.

      Also, how will GigE make their T3 any faster? Yay - i've got GigE connectivity to my 43MB/s internet connection.

      it won't, but chances are good that he needs to route between the interfaces to the other networks. He did say _4_ interfaces remember. In my example above, if there's an FTP server on network 1, and 5 users on network 2 downloading from that server, ALL of the other interfaces will notice severe lag (it will stil work because the packets will get queued but it would be sloooooow), even if their traffic has nothing in the world to do with the interfaces that are bogged down.

      If you do decide to do this with a PC, make sure it has a MINIMUM of a 64 bit PCI (507MB theoretical IIRC) and 100baseT (the norm these days) 1000baseT is even better as the cards have a bigger buffer that will help you even if you aren't routing at that speed. Use OpenBSD (if you can) the altq functionality of PF will help you to eleviate many of the bandwidth problems, so that none of the other interfaces can completely wipe another off the map if a few people are using some big time bandwidth.

      --
      /* oops I accidentally made a comment, sorry */
      /* http://allyourbasearebelongto.us */
    5. Re:no can do sorry by Paul+Jakma · · Score: 1

      true...sort of.

      There's no sort of about it, sorry.

      then chances are good that 3000 users means a heck of a lot of traffic.

      Again, no amount of users will be able to get more than 100Mb/s of data through any of those 4 or 5 interfaces. That's 12.5MB/s * 4 * 2 = 100MB/s - absolute worst case, which PCI can do. However, you're unlikely to get 100Mb/s of multi-stream traffic through a 100BaseT network to the box, never mind into this box itself. So that's 100MB/s of bus bandwidth is an absolute max.

      I have an FTP server that you can get a good 20MB sustained (through a gateway), if there was just 4 people downloading at that rate then that's 80M that doesn't leave a lot of room for the other 3 networks that I have connected to that same router.

      Firstly, he isnt running an FTP server, he wants to route between 4 or 5 subnets and a T3. Secondly you're saying that this is a limitation of the PC? That a Cisco would magically alleviate those limits?

      He did say _4_ interfaces remember.

      So you did, what an amazing coincidence then that I was multiplying by four in my previous post.

      ALL of the other interfaces will notice severe lag

      Not really. If anything the FTP traffic from the box itself will suffer before the forwarded traffic does - but anyway, he only wants to _route_ - he is _not_ running an FTP server. He wants to forward between 4 or 5 subnets, a T3, and oh yes, VPNs. And the (much) faster CPUs in PCs help with that versus Ciscos (unless you get the 3k+ crypto modules, in addition to the extra-cost IOS modules needed to even do anything crypto related, hardware-assisted or not).

      If you do decide to do this with a PC, make sure it has a MINIMUM of a 64 bit PCI (507MB theoretical IIRC)

      Errr, no. If he wants 4 100Mb/s interfaces a single quad-ethernet card on a low-end PC will most likely do (with polling drivers). Or a bit better (esp for 5 subnets) a machine with 2 32/33 PCI buses - eg if you have an old Proliant lying around that will do perfectly. 64bit/32MHz PCI is 260MB/s burst bandwidth and hence ~200MB/s real, PCI-X @ 66MHz is double that again, PCI-X @ 64/100 double that again. But that only means something if the cards on the bus support it. (esp for the cycle-rate, bus frequency must be lowest common denominator).

      1000baseT is even better as the cards have a bigger buffer that will help you even if you aren't routing at that speed.

      Right.

      Use OpenBSD (if you can)

      No, use linux or freebsd, as they both support polling mode for some of their network drivers, the most critical factor if you want to forward lots of traffic on a PC.

      the altq functionality of PF will help you to eleviate many of the bandwidth problems

      Err, no. You're presuming from the start there will be bandwidth problems, and rate-limiting to begin with? That is a silly approach. The shaping will, for most bursty office subnets, be an unneccessary extra overhead and injection of latency. And all in attempt to solve a bandwidth problem that need not exist in the first place, even on an old PC (presuming 4x FE and half-decent cards), when many of the bottlenecks and bandwidth impedements are most likely elsewhere - eg the packets from these 3000 users most _definitely_ have to be going through a switch or two before being forwarded to one of the relevant links on the PC router.

      so that none of the other interfaces can completely wipe another off the map if a few people are using some big time bandwidth.

      An interface isnt going to wipe another interface off the map. (where are you getting this form?). The problem is _not_ bandwidth - if you have enough, you have enough, if you dont, that's tough. And the real problem is packets/sec, not bandwidth, be it PC or Cisco. (hence use NICs whose drivers support polling mode operation.)

      --
      I use Friend/Foe + mod-point modifiers as a karma/reputation system.
    6. Re:no can do sorry by prisoner-of-enigma · · Score: 2, Insightful

      "true...sort of. if those 3000 users aren't doing much other than checking email and browsing the web. If they are doing some serious stuff; which they may not be who knows?; then chances are good that 3000 users means a heck of a lot of traffic."

      You're just not grasping this concept very well, are you? Let me spell it out to you very slowly: the limiting rate here is his T3 connection! No matter what these 3000+ users are doing, they cannot generate more than 45Mbit/sec of traffic because that's the max the T3 will handle (actually it's slightly less than even that due to overhead). So, with a single 100Mbit Ethernet card for the internal net and a single 100Mbit Ethernet card for the external net (or a T3 PCI adapter, it doesn't matter which), what's the max traffic you're ever going to have to deal with? Bingo! 45Mbit/sec, which is well within the capabilities of a single 100Mbit Ethernet card. It sure as hell isn't a problem for the PCI bus, which maxes out at 133MB (bytes, not bits) per second. That's 1064Mbit/sec, compared to the T3's 45Mbit/sec.

      So, in short, it doesn't matter whether you've got one user, 5,000 users, or 50,000 users -- they are restricted by the smallest pipe in the system, and that's the T3. This should be obvious, but for some reason you keep thinking that more users can somehow generate more than 45Mbit/sec of traffic through a T3. Sorry, it can't be done. Perhaps you're thinking about using a PC as a switch instead of a firewall or something, but as a firewall you are completely and totally wrong.

      --
      In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
    7. Re:no can do sorry by nocomment · · Score: 1

      You're just not grasping this concept very well, are you? Let me spell it out to you very slowly: the limiting rate here is his T3 connection! No matter what these 3000+ users are doing, they cannot generate more than 45Mbit/sec of traffic because that's the max the T3 will handle (actually it's slightly less than even that due to overhead).

      *Sigh* I grasp that concept very nicely, my point is being misunderstood. Not all traffic is going out the T3. Yes I'm aware that you will never go above the 45MB. But, You can easily get beyond that traversing from one internal network to another internal network. It _almost_ happens here where I work all the time. A couple of users from our ad design department will download something from our FTP server, and the bandwidth MRTG reports is around 80Mb/s. A few more of those and you'd start to notice sever lag on an older system with 32bit PCI. That is the entirety of my point. The T3, has never been a limiting factor in my argument.

      --
      /* oops I accidentally made a comment, sorry */
      /* http://allyourbasearebelongto.us */
    8. Re:no can do sorry by prisoner-of-enigma · · Score: 1

      Fine, but it still doesn't remove the basic issue that the PC is certainly capable of doing the job. 100Mbit being saturated? Get a Gigabit card, or load-share across multiple 100Mbit cards. There are multiport Ethernet cards that do internal port-to-port switching as well, completely bypassing the PCI bus limitations.

      As one of the other posters pointed out, it's not bandwidth that's your problem, it's pps (packets per second). The limiting factor there is going to be how quickly your system can handle interrupts, which is where polling driver support comes in.

      --
      In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
  7. Re:What's good for the customer by jhoger · · Score: 4, Insightful

    So you're saying that his customer should avoid vendor lock-in by locking in with a proprietary vendor?

    Hmm... Linux routers and firewall rules are well described on the web. Any "competent network engineer" as you describe him/her is likely able to read...

  8. It's do-able but... by dcowart · · Score: 3, Informative

    It's do-able but segment out the functions at that point. Do you really want to try to route between subnets as iptables is traversing the masq table? Get three boxes; one box for routing, one for vpn traffic and one for actual firewall/masqing. IBM has crypto boards for accelerating SSL/IPSec stuff with linux drivers IIRC for your vpn box. Also, with three boxes you can take down the vpn without taking down the internet connection.

    I would suggest getting PIII's instead of PII's though, but check where bottle neck's may be PCI bus, CPU processing packets, NIC not doing so well... etc. Plus if one box is connected to multiple subnets, it can be dhcp and/or dns and/or wins for them (if you do DNS please use the forwarder's option to forward dns requests to an upstream DNS server if possible).

    --
    www.rdex.net
  9. Buying a service, not a router by dpilot · · Score: 2, Informative

    Especially with a PC-based router the customer needs to understand that he is now buying a *service* instead of a machine. It's not too smart to leave *any* box live on the Internet, or even in a customer's office without some sort of maintenance, but for a Linux (or Windows, any flavor) box it's potentially dangerous.

    The number of exposures for Linux doesn't particularly bother me, for a box that's being actively maintained. For a generally non-service box you don't even need to be paranoically prompt about getting fixes applied. But I'd get worried about an *appliance* PC.

    --
    The living have better things to do than to continue hating the dead.
    1. Re:Buying a service, not a router by jhoger · · Score: 2, Insightful

      Okay, I'll bite. You're saying that if you have a magic nicely shaped appliance it somehow won't require security patches like a Linux box does?

      All software has bugs. All software, particularly that which runs on the edge of the network, must be maintained with patches. All hardware networking solutions of any reasonable complexity like a router or firewall run software. Therefore they too must be patched from time to time.

      At least with a Debian box you could put a cron job that automatically apt-get's latest patches for itself, if you wanted to have a box which maintains itself (I would rather have an admin maintaining it, but whatever...).

      This may or may not be a feature of whatever appliance the parent thinks is a better alternative.

    2. Re:Buying a service, not a router by dpilot · · Score: 1

      Nothing is completely free from needing patches, not the least my little blue Netear on my Home LAN. (I've flashed it for updates)

      But appliances have one big advantage in this respect - less. No hard drive, little RAM, well minimized software set. Much of this can be done with Linux as well, but a generic PC makes it all harder to do. (Even if an appliance could be r00ted, it probably doesn't have enough 'spare' resource to do anything useful with, especially without compromising its base function in a user-obvious (reboot) way.)

      I like auto-patching, but retaining an admin. For one thing, it's pretty tough to replace a kernel automatically.

      --
      The living have better things to do than to continue hating the dead.
    3. Re:Buying a service, not a router by potat0man · · Score: 1
      Why can't someone build computers and software, someway, somehow, that don't need an army of men to maintain it?

      Why can I have a machine as intricate as my internal combustion engine car that could likely go 30-40,000 miles without me doing any more than putting gasoline in it. I don't need a team of mechanics constantly following me around. Yet no one can devise a system that can deliver web pages upon request without a team of IT guys monitoring it 24/7.

      What's wrong with the analogy?

    4. Re:Buying a service, not a router by jhoger · · Score: 1

      1) Software is more complex than a car.
      2) Your car *does* needs more men to maintain it than a computer system does. It's just designed to wear out equipment/ material at regular intervals so the maintenance can be done periodically instead of intermittently
      3) No one is actively working to break-in/destroy your car from a remote location... and those that try to do it locally will succeed. Computers on the network have to be safe from break-in in all cases (but they too are usually vulnerable if thief is at the keyboard).

      And probably some other reasons...

  10. T3 only? by ADRA · · Score: 2, Informative

    If you're just powering a T3 and 6 10/100 subnets, you could get by on

    P4 2.xxGhz (assuming moderate VPN usage)
    512MB-1GB RAM depending on how many simultaious connections you're working with. The more connections the more memory eaten up
    Hard drives: minimal config.
    Motherboard & NIC's: Depending on how much you're 10/100's saturate, you may want to get some 66Mhz 64bit PCI cards instead of regular 33/32's. Eg:
    http://www.cisco.com/en/US/products/hw/vpndev c/ps2 030/products_data_sheet09186a0080189f0a.html
    It all depends on how much simultanious traffic you're looking at. You can use the analogy that the PCI bus is a network switch's backplain. 66/64's can transmit a theoretical maximum of 4gbits/sec. so it should be enough for anything you throw at it. 33/32's maximum theoretical is 1gbits/sec. but in reality expect for much less.

    --
    Bye!
    1. Re:T3 only? by Loualbano2 · · Score: 1



      If you are looking for cheap 64bit PCI NICs go here

    2. Re:T3 only? by prisoner-of-enigma · · Score: 1

      Interesting idea: have you heard of floppyfw? I've used it in a variety of small-office locations and found it to be a fantastic little one-floppy firewall. You can totally dispense with the need for a hard drive, which removes the possibility of mechanical failure. Now the only mechanical things left to fail are fans.

      The other nice thing about using a floppy instead of a hard drive? Just write protect the floppy when you're done building the firewall. If someone ever "breaks into" your firewall, you can simply reboot the firewall knowing that there's no way the floppy's been muddled with. Takes a lot of the guesswork out of recovering from a break in.

      Just my $0.02 worth, adjusted for inflation.

      --
      In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
  11. Re:What's good for the customer by Anonymous Coward · · Score: 2, Insightful
    I'll bite (the troll).
    Instead of offering standardized equipment that can be managed via console, ssh or SNMP by any competent network engineer, you offer some customized linux router solution that will always need to be handled differently.
    A customized linux router solution can be managed via console, ssh or SNMP by any competent network engineer.
    What advantage does your solution offer?
    1. More online documentation than every other router and firewall vendor combined. Docs ranging from step by step howtos to in-depth discussion of complicated setups.
    2. An open system that is upgradable on your timeline not your vendors
    3. An easy upgrade path. If you want IPv6 support (or some other feature) and have an old firewall you might have to purchase a whole new unit if a new firmware with those features isn't available for your unit.
    4. An army of people who know how to use iptables
    Is it worth "saving" a little money up front, only to need to seek out your consulting services later?
    Since it'll be running on an open system, they can seek out anyone's consulting services they want including those that might be in their own organization.
  12. Wrong Answer by MerlynEmrys67 · · Score: 2, Informative
    PCI bus = 127Mbps = 15.8mbps
    T3 = ~50mbps

    Wrong - you got the division wrong
    PCI Bus 127 MBytes = ~1Gbit/sec
    T3 = ~45 Mbits/sec

    Are you telling me the fastest a PC bus can go is 15 MBits a second ??? I know of Intel class hardware that can keep 100 MByte going over a Gbit NIC. Lets not even go into shipping PCI-X busses and soon to be shipping PCI-Express busses that are significantly higher throughput than this.

    Now that we have that problem solved, what you will run across with multiple 100Mbit network cards running into your system is a higher latency than your low end cisco router, and lower reliability potentially (all though in both cases, I suspect software reliability is orders of magnitude lower than hardware reliability). If you can live with the higher latency going through a PC based router - go for it, you might save a few bucks...

    --
    I have mod points and I am not afraid to use them
  13. Re:Upgrade? Hell, you're already massively over-sp by LunaticLeo · · Score: 2, Informative

    Uh, PCI bus is 128 Mega-BYTES per second maximum thruput. That is 1 Giga-BIT per second. And that is just for the standard 32bit at 33MHz speeds. There are plenty of Intel based servers with 64bit and 66Mhz PCI variations.

    --
    -- I am not a fanatic, I am a true believer.
  14. PCI-X: more bandwidth! by �berhund · · Score: 1

    If you get something with PCI-X, instead of standard PCI, you'll have a lot more bandwidth.

    PCI-X is 64-bit, and with multiple cards, they'll probably be running at 100MHz. Vs. standard PCI at 32-bit, 33MHz, that's 6X the bandwidth, or about 90mbs, more than enough.

    Just make sure you get one with enough 100MHz PCI-X slots for all your NICs. Many boards come with, say, 2 100MHz PCI-X, 2 66MHz PCI-X, then some standard slots. (Note that it's 2 slots per bus, and for more slots, the mobo will have multiple buses.)

    Of course you'll need PCI-X network cards to handle that. Does anyone make those?

    This is starting to sound expensive. Not sure it would actually save you any money. It would be cool, though.

    --
    -Uberhund
    1. Re:PCI-X: more bandwidth! by �berhund · · Score: 1

      Oops, except the grandparent had the math wrong, as others have pointed out. 127.2MB/s is not 15.8mbps, it's more like 1Gbps.

      But PCI-X at 100MHz is still 6X the bandwidth of standard PCI.

      --
      -Uberhund
  15. Nice idea, but the hardware won't cope by Masarand · · Score: 2, Informative
    Buying enterprise-class network equipment for the first time round is scary (it's a strange and complex world.)

    If you're inexperienced, try to get everything from one vendor so that getting it all working together is their problem, not yours.

    You could do worse than a http://www.nortelnetworks.com/products/01/passport /lan/.

  16. Apple Airport Base stations by adzoox · · Score: 2, Informative
    The original Apple Airport Base Stations called "Graphite v1.0" actually had a 486DX100 AMD Equivalent - if I'm not mistaken these were called "Dave Processors".

    The Airport Base Station (original) is a very good "take apart" to learn how to build your own router.They couldn't be more simplistic in design and implementation.

    --
    Yell & scream & rant & rave... it's no use... you need a shaaaave ~ Bugs Bunny
    1. Re:Apple Airport Base stations by mcbridematt · · Score: 1

      Pratically every Wireless AP has a 486/Elan or simular. It's amazing that they can pump so much traffic through with that low speed.

  17. What are you people talking about? by Gilk180 · · Score: 2, Informative

    Admittedly, the pci bus will probably be the first absolute roadblock with a good machine, but I think you are all underestimating it's ability.

    I did a quick test on my home network to make sure. I easily got 97 Mbps using NFS to transfer (multiple simultaneous) files between 2 machines on 100 Mb ethernet. I think that is pretty conclusive evidence that the PCI bus will not be a limit even on a DS3(T3), which only goes 51 Mbps. One of these machines even has the video card on the pci bus.

    Anecdotally, why would we even bother building Gigabit ethernet if unable to read the data (given, the bandwidth is shared, but anyway).

    For stabilities sake, you should probably not try to do the routing among internal subnets with this box. However, if most of the internal traffic is accessing external hosts, this would also be possible, since most of the traffic will be crossing this box anyway.

    1. Re:What are you people talking about? by Atzanteol · · Score: 1

      Arp lookups, and caches have nothing to do with PCI bus speeds though (what the grandparent poster was talking about). Many people posting here are vastly underestimating how fast a PCI bus runs at. One poster even mentions that it won't keep up with 10Mbit!

      But even still, a sufficiently fast PC should be able to keep up with a Cisco switch. Optimized code can be 'brute forced' with higher class hardware, yes?

      I may be wrong, but most of the answers here are conjecture. I'd love to do some real tests. I've got a P120 at home that I'd love to find the limits of...

      --
      "Ignorance more frequently begets confidence than does knowledge"

      - Charles Darwin
    2. Re:What are you people talking about? by harryk · · Score: 1

      So you're saying that you could maintain a transfer rate of 97mbit between two boxen. I'm curious to see your test environment. I've done some samples accross my local network, and at best, I've only been able to sustain 40mbit. I'd like to know a couple of things. What protocol were you using, what NIC, etc hardware config, and again what was your test files.

      I'm not doubting you, just really skeptical!

      harryk

      --
      think before you write, it'll save me moderator points.
    3. Re:What are you people talking about? by Gilk180 · · Score: 1

      Disclaimer:
      Original experiment is not scientific!! I am not legally responsible if someone dies because you cited my post that 97 Mbps is possible over the PCI bus.

      It would probably have been a better idea to use netcat to dump packets to /dev/null on one end and have it send an endless stream of them on the other, but I didn't have the time or interest. My method was quick.

      Actual response

      Using bargain basement NIC's, I can't remember manufacturer or model.

      Netgear 8 port switch.

      800 MB file

      I copied part of the nfs mounted file from the remote box to the local disk in 3 terminals. The multiple terminals are required because only 1 does not produce peak usage (I'm pretty sure nfs uses RPC, which blocks waiting for the data, so you don't get peak usage).

      Used iptraf to measure throughput.

      One box has 3 NIC's and an IDE controller(not the one accessing the file) on the PCI bus. The other has a single NIC and the video card on the PCI bus.

      I would think that the remainder of the bandwidth (about 3Mbps) is probably lost due to packet collisions and ethernet layer overhead (iptraf measures IP traffic after the physical layer stuff has been stripped.)

    4. Re:What are you people talking about? by pyite · · Score: 1

      Switching is layer 2, routing is layer 3. If you are, in fact, saying that a PC can perform layer 2 operations faster than a switch, you're wrong. Layer 2 switching is done in hardware (custom ASICs). The beautiful thing about Cisco is that with MLS (Multi Layer Switching) and CEF (Cisco Express Forwarding), layer 3 operations can be performed in hardware, never seeing the CPU. You cannot do that on a PC.

      --

      "Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman

    5. Re:What are you people talking about? by harryk · · Score: 1

      Understood about the disclaimer.

      My real reasoning for questing is because I want to do video streaming on a local network, with the potential for as many as 4 video streams. Currently I'm not able to sustain that kind of bandwidth, but then again, I'm streaming UDP, which I figured would be more effective, seeing as TCP would cause skipping.

      Was the three NIC's acting as one, like in line balancing, or where they seperate IP addresses as well?

      Still interesting though.

      --
      think before you write, it'll save me moderator points.
    6. Re:What are you people talking about? by Gilk180 · · Score: 1

      You definitely want to use udp for video streaming, even on a fast lan. You might also want to look at RTP (real-time protocol). It is used for voip and some video applications.

      The three NICs in one box are all on seperate networks. One directly to a cable modem, another directly to a WAP, the other to an ethernet. (I live with techie roommates, so we have lot's of boxen).

      If all four of these streams are the same, you should definitely look into multicasting. It really sucks over the real internet (read: doesn't work), but on a lan it can be done effectively.

  18. Re:Go BSD rather than Linux..... by Anonymous Coward · · Score: 2, Funny

    Linux wins the speed race, hands down.

    You will get 0wned much, much faster with Linux than with OpenBSD.

  19. Don't use Linux for this by phoenix_rizzen · · Score: 4, Insightful

    The packet filtering software on Linux is horrible. The syntax is just nasty. And there are no guarantees it won't change again with the next kernel release.

    Use a BSD system, with a real packet filter. FreeBSD gives you the choice of IPFW, IPF, or PF. OpenBSD gives you PF. NetBSD gives you IPF or PF. All of those have much larger / better features sets than IPChains / IPTables, and work a *lot* better in NAT/PAT/MASQ situations. These packet filters are also truly stateful (last time I checked IPTables, it wasn't truly stateful without a bunch of extra patches).

    Linux makes an OK home firewall. But I wouldn't use it anywhere near a business.

    We use FreeBSD 4.9 on Pentium 166 MHz systems with 128 MB RAM using IPFW to server secondary schools with just under 300 student computers. Haven't had any problems yet with network slowdowns or dropoffs or anything. These are on T1s in the remote schools, and 8 Mbit cable in town.

    (I had problems keeping a similar box running Linux and IPTables working on my home wireless T1-equiv link.)

    1. Re:Don't use Linux for this by Atzanteol · · Score: 1

      These packet filters are also truly stateful (last time I checked IPTables, it wasn't truly stateful without a bunch of extra patches).

      What do you mean by 'truly stateful'? AFAIK iptables is stateful.

      I've got a little diskless P120 that does just fine with DevilLinux on a 1.5Mbit cable connection at home. Even does VPN. Not the fastest for VPN, but I've never seen it not keep up with my non-VPN traffic.

      --
      "Ignorance more frequently begets confidence than does knowledge"

      - Charles Darwin
    2. Re:Don't use Linux for this by hbackert · · Score: 2

      Linux makes an OK home firewall. But I wouldn't use it anywhere near a business.

      I have to question this. Given the few arguments named, it's easy to do so. I built lots of routers with and without VPN (FreeS/WAN and recently Racoon), proxy services (for http, ftp, mail), firewalls doing NAT, VPN and anything else you can imagine. Customers read about a feature on Cisco routers/PIX and they want to have this suddenly. It often does not make much sense, but the customer is king. (I don't connect to T1s directly though, always Ethernet-only, avoiding T1/T3 with provider supplied routers which output Ethernet and don't do anything else, and everything is data-only, no voice (VoIP counts as data)).

      Problems so far: 0, except hardware failures. That's my about 5 years of experience in this business.

      No router was 0wn3d, we patch them regularily when there is a security patch needed, the firewall is pretty safe and closed, no outages, even when hundreds of notebooks connect to them. Collegues have way more problems with Cisco PIX/Checkpoint firewalls. So if Cisco & Co is ready for business, then Linux is ready too.

      I do know that *BSD is working well too and I very much like the ports system, but there is nothing wrong with Linux and e.g. Gentoo. Both run stable and outperform all but the most expensive Cisco/Checkpoint gears I know.

    3. Re:Don't use Linux for this by Danny+Rathjens · · Score: 1

      I recommend ClarkConnect for a firewall for those who can't figure out iptables. It's got a nice default config and a web gui to tweak it. Plus it has other nifty stuff setup; like snort for intrusion detection and gives some nice stats with mrtg.

    4. Re:Don't use Linux for this by ksheff · · Score: 1

      hmmm...this guy didn't have a problem:

      Somewhere in 1997 I downloaded and configured the linuxrouter.org projects mini-distro (called LRP). It was based on Debian 'some vegtable' 2.0.36 kernel. It was put on a Zenith Z-select Workstation with a 486 DX with 12 Megs of Ram and two identical 3Com 3c509 ISA parallel taking cards 10Mbit only. From the time it went to production til its retirement it served solid. It has only one failure, the floppy drive. I can say it NEVER failed because of OS. It did act squirrely during the last big worm we had because a bunch of workstations tried to ping 'SCO' and it kept running out of NAT ports, BUT it did not fail. Its longest uptime was 530 days, It would have been longer but people kicking power plugs out and other external events. On march 4, 2004 at 7:45am 'shutdown -h now' was performed and the box went to sleep.

      The box serviced some 450 people on 25 different subnets. It was fed directly from the Internet on a T1. It put up with streaming video, audio and many, many windows update sessions.

      --
      the good ground has been paved over by suicidal maniacs
  20. etc by XO · · Score: 1

    my network is basically served by a Tandy Sensation 2, a 486sx/33 with a 487slc/33 coprocessor installed. 40MB RAM, 2GB hard disk. It runs router services for .. uh.. 4 computers currently, and has run services for 10-12 computers. It also sports the network's email server, for three domains that I receive mail at. And a MySQL server, that I haven't had much use for lately, but it used to gain a few thousand SQL requests a day.

    --
    "Champagne for my real friends - and real pain for my sham friends!" http://ericblade.postalboard.com/
  21. Dont bother by moosesocks · · Score: 3, Insightful

    If your company can afford to pay 1000 people and run a T3, they have the money to buy a PROPER Cisco-based setup.

    Oh. And hire an experienced professional to install it (i don't dobut that you could manage it, though). I wouldn't trust a job of this size to someone who 'did it once at home and it worked'. The enterprise works much differently than your basement.

    If you set it up and something goes wrong, you, my friend, are screwed.

    --
    -- If you try to fail and succeed, which have you done? - Uli's moose
  22. I'm already doing something similar by David+E.+Smith · · Score: 2, Interesting

    There's a whole niche market for "stripped-down versions of Linux" that handle things like this.

    Currently, I'm using Mikrotik RouterOS as a core router. It's at a small ISP -- 400 or so high-speed customers, 3000 dialup customers (400-500 of which are connected during peak times). Standard routing stuff (30 or so internal static routes, big deal). Couple hundred firewall rules (some for stopping Windows worms from spreading, some for general network security, some to help keep the nastier spammers in check). And BGP, taking a full BGP feed from our upstream, plus a couple multihops from places like Cymru's bogons project. And it doubles as a PPTP server so I can securely work from home (in a gesture of supreme irony, I can't get Internet connectivity from the company I work at).

    And some other stuff I can't think of right now.

    All this is running in a 1U system I got from eRacks (they make good cheap stuff), except for the hard drive, which I yanked and replaced with a 64MB IDE-flash drive from these guys. Celeron 1.3GHz, 512MB RAM. The system never ever, even during peak times, goes over 10% CPU load.

    This isn't quite up to the specs the original author was looking for, mainly because this hardware isn't also doing the T1 stuff. (It's got plain old boring Ethernet to an older Cisco router, to which our four T1s are connected, but the Cisco is basically just a really big media converter.) But given how low the hardware utilization is on this unit, and how underpowered this system is as compared to current hardware, I think it shows that the notion is quite feasible.

  23. Re:What's good for the customer by pyite · · Score: 1

    Cisco is a standard. Period.

    1. Cisco has plenty of documentation, online and otherwise.
    2. No matter whether you run Linux, OpenBSD, or IOS on a Cisco box, if a vulnerability comes up, unless you're a fluent coder, you're not patching it until someone else fixes it. Cisco is generally very good about fixing critical problems.
    3. Considering 10 year old Cisco equipment is still in use in many places, I don't think that you have to worry about purchasing "a whole new unit."
    4. An army of people who know IOS. Need I remind you of the evolution of Linux routing? 2.0.x: ipfwadm. 2.2.x: ipchains. 2.(4|6).x: iptables. Things change a lot in Linux it seems.

    Now that I've defended Cisco, let me throw some points towards Linux. Linux is great for networking, I use it a lot. I wouldn't buy a Cisco router for my house, it's just not practical. If up front cost is that much of an issue, use Linux. I've been in the situation, and I still have [Linux] boxes routing at places I haven't worked at in a long time without rebooting since I've left. That said, the networks I work with now could never be run on PC hardware. It's just not possible. The beautiful thing about dedicated networking hardware (i.e. Cisco) is that it has the ability of doing operations via custom ASICs. Therefore, the CPU load stays super low, latency goes way down. That doesn't make a difference on a T1, but it sure makes a difference on a Gigabit connection. Whenever the money is there these days, I recommend Cisco.

    --

    "Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman

  24. Folks are doing this commercially by Myself · · Score: 1

    and they seem to be doing pretty well. I went looking for weird NIC hardware and came across Imagestream. They make big routers with Linux at the core, on x86 hardware in industrial form factors. Definitely worth a look.

    Also on the thread of interface cards, try Mikrotik. If you're doing wireless, the MiniPCI carrier boards will make your day.

    Full disclosure: I'm not related to or affiliated with either of those companies in any way. I've never even bought anything from either of them. I just came across them while searching and thought they were bookmark-worthy.

  25. No Problem by Tip · · Score: 1

    Our main firewall for our hosting company is a 2Ghz P4. We are not doing vpn, which would be the most resource intensive, but our T-3 line comes directly into it and we have a ton of firewall rules. There is never a load on the box, except when nimda hit :).

    And with bridging you can have two transparent firewalls (no ips) that are redundant, using Spanning Tree Protocol. Pretty cool.

  26. Re:Upgrade? Hell, you're already massively over-sp by drsmithy · · Score: 1
    You'll be fine with what you've got right there!

    As long as he;s not trying to do VPN encryption on it, he probably will. Personal experience tells me a P100 (running FreeBSD, not Linux) can easily firewall a 100Mb network link for a few dozen users, so anything P2 class shouldn't have any trouble at all.

    No seriously, you're going to swamp your PCI bus if you're doing routing between internal subnets. Goodbye, LAN throughput. Not to mention what merry hell you'll play with the CPU with VPN and firewall rules.

    The PCI bus is not an issue. PCI is ~120M/sec and even 100T ethernet is only 10M/sec.

    Any P2 class PC with some decent network cards (I personally recommend Intel EtherExpress cards) should handle his setup _easily_. You'll probably run out of (ie: have to manually tweak) OS resources long before the hardware is stressed. The bottleneck is not going to be raw throughput per se, it's going to be how fast the machine can process individual packets.

  27. Re:Upgrade? Hell, you're already massively over-sp by SouLShadow · · Score: 1

    well, to give you an idea of what can be done, i'm running a slackware based p100 as a gateway/firewall/router/name server for my entire home network. there are 12 computers between a bunch of people. it sits between the cable modem and two switches with 2 10/100 $10 ethernet cards in it. of course this is not a large scale network, but it shows you what a p100 can handle with ease. for security i disallow all incoming connections except ssh. and if you think it doesn't get much traffic, i keep gnutella running on one of my computers 24/7 with about 100 active downloads at any time.

  28. Re:Go BSD rather than Linux..... by Yottabyte84 · · Score: 1

    Interesting, nobody has 0wned any of the linux systems I admin at work in the year I've been working there.

  29. more specs by neomage86 · · Score: 1

    Just to clarify, this is a project for my High School. They are upgrading the network infrastructure, and I work with the tech-ed department through an internship class. I just wanted to make sure this was reasnoble, before I suggested it to my own bosses.

    1. Re:more specs by WolfWings · · Score: 1

      I would say yes. If you're not dealing with all the connections needing to be encrypted or something else that requires every single packet to be fully modified by the CPU of the router in question, a medium-low-end ($200-$500 bought piecemeal at Fry's or similair) PC should do the job just fine.

      And good luck to you. :-)

  30. Re:Go BSD rather than Linux..... by Rysc · · Score: 1

    Pfft, I'll bet you just didn't notice. That's how 0wnz0rd you are.

    --
    I want my Cowboyneal
  31. pc Hardware? by sydres · · Score: 1

    Love the thought but pc hardware is hardly up to mission critical status even with a stable OS on it ata drives fail cpu's overheat junk ram corrupts data a company of 3000+ people cant afford to have downtime from that crap chipset or failed ram and can afford to by something that is more likely to last