Slashdot Mirror


Powerful Linux ISP Router Distribution?

fibrewire writes "I'm building a Wireless ISP using commercial grade, low cost equipment. My main stumbling block is that I cannot find a decent open source ISP class routing distribution. Closest thing to even a decent tool is Ubiquiti's AIRControl — but even it doesn't play well with other network monitoring software. I've used Mikrotik's RouterOS for five years, but it just isn't built for what I need. I don't mind paying licensing fees, but $300K for a Cisco Universal Broadband Router is out of my budget. Has anyone seen any good open-source/cheap hardware/software systems that will scale to several thousand users?"

268 comments

  1. Just use any Linux distro by ls671 · · Score: 5, Interesting

    Just pick up your favorite Linux distribution and get back to me with your requirements. I think Linux can easily do what you need almost out of the box. It is only a matter of configuring it. I bet some would recommend looking at OpenBSD or FreeBSD as well.

    Either way, you would definitely have a more flexible solution that any canned product will provide you with.

    --
    Everything I write is lies, read between the lines.
    1. Re:Just use any Linux distro by grub · · Score: 5, Informative


      Does it have to be Linux?

      Why not try OpenBSD and its excellent BGP implementation OpenBGP! It powers some pretty hefty businesses and ISPs.

      -

      --
      Trolling is a art,
    2. Re:Just use any Linux distro by Threni · · Score: 1

      You asked for it!

      I have a acer one with a broken screen. Currently i'm using it as a (120gig) file server, but I'm quite interested in using it as a firewall, and/or to monitor the traffic on my wireless router. My current router works fine, but I have absolutely no idea who is connected to it wirelessly, nor do I know how much data is being up/downloaded through it. Is there some Linuxy solution where the PC either becomes the router, or at least can intelligently talk to it? How much of this depends on which router I have, and how much is the least I can get away with spending for one which would work with such a system?

    3. Re:Just use any Linux distro by Anonymous Coward · · Score: 0

      OpenBGPD also includes OpenOSPFD - from my experience *way* more intuitive than Zebra/Quagga. There's a port for FreeBSD

      Also pf kicks some *serious* butt.

    4. Re:Just use any Linux distro by ls671 · · Score: 2, Funny

      The title in the question was asking for a Linux distro.

      Anyway, you have proven me right, if you read my OP very carefully, it states:

      > I bet some would recommend looking at OpenBSD or FreeBSD as well. ;-))

      --
      Everything I write is lies, read between the lines.
    5. Re:Just use any Linux distro by Anonymous Coward · · Score: 1, Interesting

      PFsense. BSD, though.

    6. Re:Just use any Linux distro by Enuratique · · Score: 1

      I'm pretty sure Linux can do this but I am by no means a guru. If your router is a Linksys WRT54G, I highly recommend installing the 3rd party Tomato firmware. All the features you require are there. I personally find the Quality of Service packet prioritzation an absolute must in a household of bachelors all running BitTorrent and Xbox 360s...

      --
      A black hole is where God divided by 0
    7. Re:Just use any Linux distro by grub · · Score: 2, Informative

      Yeah, I meant to reply to the story, not your comment. :)

      --
      Trolling is a art,
    8. Re:Just use any Linux distro by ls671 · · Score: 1

      I have all my wireless IP addresses on a different subnet. The wireless router connects directly into the Linux router with its own interface. Thus it is easy to setup firewall rules specific to the wireless network and to monitor it for bandwidth usage and what not. Then connect your linux router to the ISP link.

      To provide even more monitoring and trafic control capabilities from the Linux router, I do not use the DHCP server in the router but instead, wireless machines query the DHCP server on the Linux server.

      All the required programs to accomplish this already come with most Linux distributions.

      --
      Everything I write is lies, read between the lines.
    9. Re:Just use any Linux distro by clarkn0va · · Score: 1

      Definitely PFSense. I prefer the traffic shaping in Linux (can't speak for the traffic shaping in BSD), but PFSense is sufficient in that regard, and excels at everything else. You can't beat the interface for visual presentation and ease of management.

      If it absolutely has to be linux though, I love Tomato. It's mostly aimed at less-powerful hardware though, so I'm not sure how much you could scale it up.

      --
      I am literally 3000 tokens away from the chaotic crossbow --Stephen
    10. Re:Just use any Linux distro by udippel · · Score: 1

      Tomato is great, really.
      On 40$-plastic-boxes.

      But if you read the story, he is looking for a commercial grade system for an ISP. Then you have easily 1000 times the throughput of a little Broadcom box.
      And its limitation of 2 physical interfaces doesn't help neither.

    11. Re:Just use any Linux distro by Anonymous Coward · · Score: 0

      OpenBSD for sure! I use Linux too, where/when needed, if needed, but OpenBSD is most definitively the best solution for such a thing and I've done the same in Wirless ISP environments in the same way too.

      OpenBSD, no doubt is your best option here and can do everything you need and more.

    12. Re:Just use any Linux distro by Spit · · Score: 1

      I agree, OpenBSD seems to have bottomless performance in my installations and the configuration is so easy.

      --
      POKE 36879,8
    13. Re:Just use any Linux distro by jonbusey · · Score: 1

      OpenBSD is the best way I know of to do this, hands down. It'll save you more money and headaches than you can imagine. The only thing FreeBSD has on it for this application (i.e. domUs, virtualbox, zfs not applicable) is easier Sguil http://sguil.sourceforge.net/ installation, but OpenBSD has a tighter security profile, easier maintenance, and narrower attack profile than any other OS or distro.

    14. Re:Just use any Linux distro by tkjtkj · · Score: 0, Offtopic

      in your comment: " Either way, you would definitely have a more flexible solution that any canned product will provide you with." it seems clear that you would better have used the correct english: eg: "you would definitely have a more flexible solution that any canned product " ought to be replaced with: " you would definitely have a more flexible THAN any canned ...".. ie, 'that' is not used in any comparative sense .. rather: .... THAN .. is proper..and consequently comprehensible... Of course, your error might have been a consequence of 'expeditious need' , in which case my comment was merely to enlighten those who might have accepted your construction ... If so, do forgive me .. but.. at the same time, thank me for contributing to the erudition of our peoples. j. anderson tkjtkj@gmail.com and btw, what is the significance of the otherwise UNdefined lil 'checkbox' just below my 'subject line' ??? ehhh????? what gives????

      --
      "There are 11 kinds of people: those who know binary, those who don't, and those who could not care less!"
    15. Re:Just use any Linux distro by Anonymous Coward · · Score: 0

      I've used pfsense for quite a while. It's an embedded version of FreeBSD, which uses OpenBSD's pf. It works pretty well, and supports CARP (patent free VRRP) so you can have redundant routers (even with NAT -- it will sync firewall/nat state). It handles firewall rule changes gracefully by not destroying existing firewall states.

      pfSense evolved from monowall and has more features at the cost of being a bit more heavyweight. It's a larger install, but supports more recent hardware. Monowall might work how you want.

      In all honesty, if you need to spend $300k on a router, you're way past what something off the shelf hardware can do (in terms of performance) and you're looking at something that does the routing in an ASIC/FPGA.

      Cisco will support the most features (doing most of it in hardware, emulating everything else in software). Foundry (now Brocade) and Force10 will do some insane routing throughput, and at line speed. I want to say the Force10 E600 we have at work will do 50gbit/sec of throughput and our Foundry RX16 will support around 640gbit. Both Foundry' and Force10's philosophy is, for the most part, "if we can't do it in hardware, we don't." They'll do things like spanning tree and BGP on the CPU, but anything related to getting a frame/frame from A-to-B is in an ASIC.

      So yeah, if you're looking for a cheap embedded solution, you're golden. Get up to 10 gigabit and you start running into latency problems and I/O issues that you can't solve by swapping out an OS.

    16. Re:Just use any Linux distro by ls671 · · Score: 2, Funny

      blah, blah, that was just a typo.

      Go fuck yourself !

      Cheers,

      --
      Everything I write is lies, read between the lines.
    17. Re:Just use any Linux distro by itwerx · · Score: 1

      Yep, pfSense fer shure. It's being used in environments a lot bigger than 1000 users. (Without naming names, think major news sites :).

    18. Re:Just use any Linux distro by Jaruzel · · Score: 1

      I have all my wireless IP addresses on a different subnet.

      I do not use the DHCP server in the router but instead, wireless machines query the DHCP server on the Linux server.

      Is this do-able on Windows DHCP ? I'm using pfSense, and have it serving out 192.168.2.x for my wireless, against 192.168.1.x for my wired (OPT1 and LAN in pfSense language), but I'm having DNS issues on the wireless where they either resolve to internet or LAN domain, but never both. I'm rambling but in essence can I get Windows DHCP to 'know' that the requesting client is on wireless, and serve out an IP from the appropriate scope ?

      -Jar

      --
      Together, We Can Make Slashdot Better. I Do NOT Mod ACs. - Check Me Out
    19. Re:Just use any Linux distro by robinstar1574 · · Score: 0

      All you really need for routing service is IpConfig and a properly configured config file to provide service to everyone you wish to. This is no difficulty. After that, you can just use any router with the most users able to use it at a time.

    20. Re:Just use any Linux distro by Fez · · Score: 1

      Jar,

      The DNS issue sounds like a good question for the pfSense forums (http://forum.pfsense.org) or if you are on freenode, try ##pfSense.

      A little more information about your setup would be needed to say much of anything for certain (e.g. DNS configuration on pfSense, use of the DNS forwarder, DNS servers specified in the DHCP config, etc.)

    21. Re:Just use any Linux distro by robinstar1574 · · Score: 0

      The way you say that makes it sound like there is no limit to how bad it is.

    22. Re:Just use any Linux distro by theyulman · · Score: 1

      I agree ...
      OpenBSD + OpenBGPD + OpenOSPFD + PF + CONFIGURE ALL OF THIS PROPERLY = Rock solid secured ISP

    23. Re:Just use any Linux distro by Unequivocal · · Score: 1

      Probably you'd need to put the wireless network onto a different interface to the Windows DHCP server and serve a different set of data to that interface. I haven't used Win DHCP server in a while though, so it's possible it can handle other strategies too.

    24. Re:Just use any Linux distro by BitZtream · · Score: 1

      Either way, you would definitely have a more flexible solution that any canned product will provide you with.

      Citation needed.

      Just because its open source doesn't automagically make it more flexible. You're going to have to pay someone to do the work, and contrary to popular belief, if you're willing to pay for the development almost anyone will write custom software for their hardware, including the giants like Cisco.

      So ... you're going to pay someone to do the work anyway you look at it. The idea that OS == MORE FLEXIBLE just because is retarded and is generally only quoted by fanboys and people without a clue.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    25. Re:Just use any Linux distro by awpoopy · · Score: 1

      I second pfsense. It will do what you need. As your load requirements go up, just beef up your hardware and restore the backup config from the older box. It works great and you can get support from the dev team. Your costs for support (if you want/need it) will be significantly lower as you're not paying for all the middle management like you would going with cisco. If you don't want/need technical support, the software is free and the forum is pretty good about timely responses.

      --
      I say things which affects my Karma negatively. (and I don't care) For instance; All religion is false.
  2. DD WRT by Anonymous Coward · · Score: 1, Informative

    http://www.dd-wrt.com/site/index

    It's Linux on low cost wireless routers.

    1. Re:DD WRT by pak9rabid · · Score: 4, Insightful

      http://www.dd-wrt.com/site/index

      It's Linux on low cost wireless routers.

      Yeah, that's just what I'd want my ISP to run as a core router.

    2. Re:DD WRT by Anonymous Coward · · Score: 0

      My telco blocked our network access because somebody in the corporate LAN downloaded a copyrighted movie. Kindly they left port 80 & 21 open, but not 53, so despite trying to be kind, the idiot showed his true colours. This guy suggested DD-WRT on a LinkSys router to manage all our corporate traffic and be able to block BitTorrent. I nearly fell off my chair laughing and resisted the temptation to ask him to go back to network school. Instead I suggested he make a proposal for how I could use our professional Cisco equipment to do that. Never heard anything back.

    3. Re:DD WRT by Hydroksyde · · Score: 1

      No. Just no. Why the fuck was this modded informative?

    4. Re:DD WRT by ntk · · Score: 2, Informative

      They cut off your network access because of a report of infringement? Are you in the US? Do you think you could mail me at danny@eff.org with more info? We're always interested in the details of these incidents.

    5. Re:DD WRT by ParanoidJanitor · · Score: 1

      Like others have said, DD-WRT is for consumer grade hardware. Since DD-WRT is closed-source, you can't recompile it for anything else. However, nothing is stopping anyone from taking the OpenWRT source and compiling it for a more powerful router.

    6. Re:DD WRT by Anonymous Coward · · Score: 0

      http://www.dd-wrt.com/site/index

      It's Linux on low cost wireless routers.

      Yeah, that's just what I'd want my ISP to run as a core router.

      That's funny, I run the whole interwebz on mine!

    7. Re:DD WRT by Lisandro · · Score: 1

      Don't diss DD-WRT so quickly. The software running it is damn solid and feature rich - most of it's limitations are caused by the relatively weak hardware low cost routers use. I think there was some beefier hardware available on the DD-WRT shop.

    8. Re:DD WRT by duanes1967 · · Score: 1

      I know an ISP that has run DD-wrt on its entire wireless infrastructure (500+ users up to 7 miles from 20 different towers spread all over four counties) with very good results. Heck, my router has been running DD for five years and I reboot it maybe five times. Twice, we moved to a new house, and twice were firmware upgrades. Linux in general has VERY nice routing features, you just have to hunt for Interface cards that will keep up. The old problem was simply inter-card throughput of the PCI bus, but with PCI express, you can handle ridiculous traffic loads. If you need more than a linux PC router can handle, then you probably has a router budget to match.

  3. Vyatta by Anonymous Coward · · Score: 3, Informative

    http://www.vyatta.com/about/press_releases.php?id=75

    try the beta v6

    1. Re:Vyatta by dixon1e · · Score: 1

      It's good stuff, try it.

    2. Re:Vyatta by Anonymous Coward · · Score: 0

      Hell yes!

      I just stood up a whole building network with a lot of obnoxiousness recently, and Vyatta saved me a ton of money and time. Though admittedly, an ISP network is a different ball of wax. Vyatta is closer to a router though, so wireless management may not be as good as other distros.

    3. Re:Vyatta by Anonymous Coward · · Score: 0

      vyatta is great kit

    4. Re:Vyatta by Anonymous Coward · · Score: 0

      I was going to mention this one.

    5. Re:Vyatta by Anonymous Coward · · Score: 0

      The community version 6 is still in alpha. And the community 5 version is stable.
      Subscription version which is based on 2.6.30 is based on a version in between
      the two.

  4. http://www.vyatta.com/ by Anonymous Coward · · Score: 0

    http://www.vyatta.com/

  5. Erm... Requirements? by teqo · · Score: 2, Insightful

    So AirControl "doesn't play well with other network monitoring software" (which one, and why?), and MikroTik "isn't built for what [you] need" (what's that?) - other than that, you don't give us any idea what you really expect. What are your requirements? Suggestions out of the blue: OpenWRT with quagga/zebra, hostapd, radius, olsrd, b.a.t.m.a.n. etc. etc, or you might want to have a look at Vyatta (no affiliation).

    1. Re:Erm... Requirements? by BitZtream · · Score: 1

      I know this is blasphemy but ...

      Instead of pulling together a bunch of half assed incomplete kludges of projects and spending a fortune getting a bunch of dirty socially inept hippies from slashdot making it work right and then keeping it working, he could just get a clue and buy something ready made for his purpose.

      Hardware and software are such a tiny amount of the overhead of running an ISP that you are in fact a complete fucking idiot if you're making your choices on hardware and software based on cost. Any ISP worth its salt will pay for its hardware and software with a months gross income, even if you buy the most expensive item suited to that purpose. Customer support and bandwidth is where you pay. The land for your towers and the buildings for your data centers are your costs.

      Equipment isn't shit in your budget.

      As I stated before, the best thing this guy can do is find someone who actually know what the hell they are doing to build it out, otherwise he's going to spend 2 or 3 times as much to use 'cheap' alternatives.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    2. Re:Erm... Requirements? by Cramer · · Score: 1

      Equipment isn't shit in your budget.

      Bull. One well loaded Cisco 10k costs more than the 5 year contract lease for our (class A) office space.

      Google is the only organization I'm aware of that actually owns and builds their own facilities -- buys land, builds buildings, puts computers in them, and hires people to manage everything. Even major ISPs (level3, MCI/Worldcom/UUNet/et.al.) and data center providers around here (RTP NC) don't own their facilities. The smart ones are in converted warehouses. Others are in flex office space.

    3. Re:Erm... Requirements? by jeffstar · · Score: 1

      mikrotik are a bunch of latvian asshats. Their software has lots of features that are half implemented and partially tested. I'd wager they are GPL violators as well. The mikrotik routerboards are pretty solid though. can't say the same for their wireless cards, get ubiquity wireless cards.

      That being said mikrotik routerboards hit the price point. That being said I can't wait for ubiquity products on OpenWRT (I think airos is based on openwrt?) to overtake them for usable features.

      If linux can do it, so can all these devices that run linux.

  6. Screw Linux by TheRealMindChild · · Score: 1

    Why does it have to be linux? Use pfSense

    --

    "When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
    1. Re:Screw Linux by JSG · · Score: 1

      Seconded and my idea of fun is running 50 odd Gentoo based systems around the UK. I probably wont try and screw them though.

      For me the multi link routing ie load balancing/failover gateways is the key feature (I have 6 ADSL lines - my office is a bit rural). Add to that a good list of add ons, eg ntop, OpenVPN and IPSEC, WiFi with mesh and captive portal etc etc etc and its a bit of a winner.

    2. Re:Screw Linux by Ritz_Just_Ritz · · Score: 1

      PFsense has been OK for me in a small business environment, but it's nowhere near robust enough for ISP duty. For one, the multiwan implementation has been somewhat troublesome (mostly working, but occasional glitches) and traffic shaping doesn't work at all with multiwan. If you can do your multiwan stuff with an appliance, then perhaps that's not an issue, but my assumption was that you wanted something to act as your "core" using commodity hardware.

      Best,

    3. Re:Screw Linux by Fez · · Score: 2, Informative

      pfSense 2.0 will solve the multi-wan traffic shaping limitation, and it's in beta right now. As for the multi-wan glitches, I'm not sure when the last time you tried it was, but the outbound load balancer was redone in 1.2.3 and 2.0 will have even more changes as well.

      I run an ISP and we use a pfSense CARP cluster in front of our servers and it's worked great for us, but admittedly we are a small ISP. We also use it at more than a dozen customer sites. Everyone loves it.

    4. Re:Screw Linux by Anonymous Coward · · Score: 0

      I'm also using PFSense on my small business (20-30 users + VPN) and it performs very well with quite modest hardware (Athlon XP). I haven't used the multi-wan feature yet (plan to) but from what I can see at the forums, it seems to work descently. Also, since he mentions that he has no problem paying a modest fee, he could have the Commercial support and perhaps even throw a bounty on a feature he misses or a bug that needs fixing.

    5. Re:Screw Linux by Anonymous Coward · · Score: 1, Funny

      You say you're running 50 odd Gentoo systems. That implies there's such a thing as a non-odd Gentoo system. I thought the whole point of Gentoo was to be odd...

    6. Re:Screw Linux by Ritz_Just_Ritz · · Score: 1

      I've been using 1.2.3 since it was released and while some of the problems have gotten better it still isn't nearly stable enough to be a core component of an ISP's mix of gear. Also, 2.0 has been in alpha for ages and only JUST went to beta and has a prominent warning in their support forums about not trusting it for production use.

      I've heard things are MUCH better when using it as an inbound loadbalancer, but the outbound stuff is troublesome and doesn't scale well (at least for me).

      That said, for a SOHO environment, it's a pretty good solution and the price is right. :)

  7. Are you serious, or just killing time? by jeffmeden · · Score: 4, Insightful

    So Cisco makes billions of dollars a year selling some ungodly expensive, ungodly powerful head end router like devices (not even routers in the IP sense) and somehow you suspect a Linux distribution with the same features is going to unpack itself and be everything you want it to be? You need to tell us what the rest of your platform looks like if you expect any answers that go beyond 'any linux distribution can act like a router!'. What subscriber equipment is in use? How much user control do you need (access on/off vs. bandwidth filtering, etc.) Details, details, details.

    1. Re:Are you serious, or just killing time? by Meshach · · Score: 0, Offtopic

      Just like Microsoft has spent billions of dollars a year building their operating system. I cannot predict any one doing the same thing for free and then just giving it away.

      --
      "Maybe this world is another planet's hell"
      Aldous Huxley
    2. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      So Cisco makes billions of dollars a year selling some ungodly expensive, ungodly powerful head end router like devices (not even routers in the IP sense) and somehow you suspect a Linux distribution with the same features is going to unpack itself and be everything you want it to be? You need to tell us what the rest of your platform looks like if you expect any answers that go beyond 'any linux distribution can act like a router!'. What subscriber equipment is in use? How much user control do you need (access on/off vs. bandwidth filtering, etc.) Details, details, details.

      Why, yes. And?

    3. Re:Are you serious, or just killing time? by jeffmeden · · Score: 1

      When someone is giving away exceedingly powerful, linux-equipped servers for free, let me know where and when. Until then, software (as in Microsoft vs. Linux) is *not* equal to software/hardware (as in Cisco). Plain and simple.

    4. Re:Are you serious, or just killing time? by dave562 · · Score: 4, Insightful

      And beyond that, just because a Linux box might support all of the protocols and implementations that Cisco has leveraged in their own products, it does not mean that the Linux box is going to configure itself. A lot of the reason that Cisco makes money is because they provide solutions. The solutions themselves leverage established technologies in many cases (RFCs are in the public domain), but Cisco makes them work together. It's the old discussion about Open Source vendors. They aren't making money selling people Linux because Linux is free. They are making money selling people Linux configured to perform specific tasks, and then selling support to keep the solution functioning and up to date.

    5. Re:Are you serious, or just killing time? by b1t+r0t · · Score: 3, Insightful

      The "same features"? You mean like ASICs that forward the data with low latency once the route is established? Yep, Linux is going to somehow magically add those to your computer, and that's one of the reasons people pay the extra money for Cisco over some old P3 tower PC and a CD-ROM with a penguin on it. Another is that they fit nicely in a rack.

      The submitter apparently has his own unique idea of what "ISP class" means. Admittedly, this is for a wireless network, so there is already a bit of latency expected and maybe not as much total bandwidth as a wired ISP, but you can never remove latency, only add less. And as you have pointed out, "ISP class" should include things like metrics and controls for users.

      --

      --
      "Open source is good." - Steve Jobs
      "Open source is evil." - Microsoft
    6. Re:Are you serious, or just killing time? by BobMcD · · Score: 2, Interesting

      When someone is giving away exceedingly powerful, linux-equipped servers for free, let me know where and when. Until then, software (as in Microsoft vs. Linux) is *not* equal to software/hardware (as in Cisco). Plain and simple.

      So you've never heard of the FrankenPix, I take it?

      I'd buy a claim of 'more stable', 'customized', or something similar, but 'exceedingly powerful' probably just isn't true. Barring some evidence to the contrary, I see a given Cisco device as about on par with an extremely weak desktop computer, in terms of pure 'power'.

      I'm not undervaluing the total package. I just think you may have gone a tad too far with the Kool Aid.

      Further, I don't think the question was asking about hardware anyway, so I'm not certain why you'd be muddying the waters with that part of it. Software to software to software, the comparison remains valid.

    7. Re:Are you serious, or just killing time? by JWSmythe · · Score: 1

          Cisco (and others) make ungodly money because they are perceived as the "best". I won't argue that too much though.

          For low end stuff, there are cheaper options. Do you need a Cisco Catalyst to handle 3 desks on a fairly slow DSL line, who aren't doing outrageous sharing between each other? No. Do you have 100 desks, then sure. Could you do the 3 desk operation with a Linux machine and 4 network cards? Sure. In this example, it's cheaper to pick up a cheap hub, than to take even a salvage machine and put 4 network cards in it.

          What I've seen is bandwidth constraints on the bus. What can pass more traffic, purpose built high end networking hardware, or a PC based machine? If he's only passing 80Mb/s through to his upstream, then hey, go with Linux. If he's passing 800Mb/s, then he needs serious equipment and shouldn't even consider going with a PC based Linux machine.

          I've done some really neat stuff in Linux that I couldn't do anywhere near as easily on Cisco equipment. But, it depends on the purpose. I really do love Cisco gear. :)

          I do have to wonder about his infrastructure though. If he's setting up a WISP, does he have little Linux boxes strapped to towers?

          I will agree with you, just about any Linux distro may do what he wants, and there are only the rare exceptions where things won't just work across distros. Without knowing more about the business, there's no way to guess at what his business requirements are.

      --
      Serious? Seriousness is well above my pay grade.
    8. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      install Cisco software on a LinkSys router.

      As I understand it, Cisco markets consumer grade routers under the LinkSys brand name. The software (and aspects of the hardware) on the consumer products is a crippled version of the commercial products.

      See: http://arstechnica.com/open-source/news/2009/05/cisco-settles-fsf-gpl-lawsuit-appoints-compliance-officer.ars

    9. Re:Are you serious, or just killing time? by rantingkitten · · Score: 4, Insightful

      Do you need a Cisco Catalyst to handle 3 desks on a fairly slow DSL line, who aren't doing outrageous sharing between each other? No.

      Sheesh. I wish someone would tell that to our clients. My company provides service to (mostly) small businesses, and half of these little five-man operations have some totally over-engineered Cisco gear acting as their network edge because some smartass, self-styled "IT Guy" told them it was the best. Surprise, he vanishes after plugging it in and collecting his fee, and now the client has all these problems with our SIP service and of course they have no idea how to manage their own equipment, and WE end up looking like jerks because our stuff won't work out of the box with whatever equipment the client has.

      Could you do the 3 desk operation with a Linux machine and 4 network cards? Sure. In this example, it's cheaper to pick up a cheap hub, than to take even a salvage machine and put 4 network cards in it.

      Here, though, I disagree. At the same company I mentioned, when I joined, we were a three-person operation, and we used a Linux machine with two network cards and a switch as our router. It worked great as we scaled up in staff numbers, particularly when tools like ntop and tcpdump existed to let me see when some joker was ruining it for everyone by torrenting the entire internet. If you never plan to expand, then sure, some cheap little router toy from Dlink or Linksys will do fine, but if you intend to grow, may as well do things right the first time than have to re-engineer your network down the road.

      Also, a hub? Who the hell uses hubs anymore? I can't even think of a use for them these days other than packet sniffing, and an inexpensive managed switch will let you do that.

      --
      mirrorshades radio -- darkwave, industrial, futurepop, ebm.
    10. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      Ha ha, you think Cisco makes things work together.

      Obviously you have never been given two different bits of cisco equipment and told to make them talk to each other. If you had you would still be reading cryptic IOS manuals full of details about features one of the two boxes doesn't have, trying out random commands found on blog posting in the hope it might just start working, and eventually giving up and paying Cisco very large sums of money to tell you that you, in fact, have to buy a completely different bit of kit.

      Ciso is the hardware version of Oracle. Their products are not complex because they have to be, they are complex so that it's hard to learn how to administer it. That complexity generates revenue in the form of training and consulting, and lock-in in the form of institutionalised knowledge and pain of replacement.

    11. Re:Are you serious, or just killing time? by Dadoo · · Score: 0

      If he's only passing 80Mb/s through to his upstream, then hey, go with Linux. If he's passing 800Mb/s, then he needs serious equipment and shouldn't even consider going with a PC based Linux machine.

      I'll agree with you, in principle, but not on the numbers. You can get a decent machine (~2GHz, dual-core, 4GB memory) for less than $1000, today. I'd be surprised if that couldn't handle at least 3-4 gigabits, total throughput. Heck, I'm using a 10-year-old, 350Mhz, Pentium 2 workstation as our Internet router, here at work, and the latency it adds to ping times isn't measurable.

      --
      Sit, Ubuntu, sit. Good dog.
    12. Re:Are you serious, or just killing time? by dave562 · · Score: 1

      I worked on enough Cisco hardware in the late 1990s to not only know that what you say is true, but to also decide that I didn't particularly want to be a CCIE anytime soon. Slight differences in IOS releases can lead to serious headaches. I've heard tales that the Cisco CCIE test is basically being thrown into a locked room with a bunch of misconfigured gear and told to make it work within a certain time frame. Thanks, but no thanks. My philosophy is that I'll let the ISPs and telcos handle bringing the circuit in and configuring their premise equipment. Give me a RJ45 jack to plug and a phone number to call if the line goes down.

      As complex as Cisco configurations can be, it has been my experience that once they are configured, they are rock solid. The initial complexity seems to come with the trade off in reliability. It may be difficult to get up and running in the first place, but once it's running, you can focus on other things. My experience is limited to setting up ISDN and T1 point to point circuits, but at that level, I haven't run into any problems once the circuit is up.

    13. Re:Are you serious, or just killing time? by nabsltd · · Score: 1

      You can get a decent machine (~2GHz, dual-core, 4GB memory) for less than $1000, today. I'd be surprised if that couldn't handle at least 3-4 gigabits, total throughput.

      Agreed. I tested a 2GHz single core and reached over 1000Mbps when using pfSense, and that's without optimization.

      Unfortunately, with the traffic shaper enabled, pfSense gets only about half the throughput on the same hardware.

    14. Re:Are you serious, or just killing time? by mattsday · · Score: 2, Informative
      The RFCs may be in the public domain, but it's companies like Cisco that champion them. some examples of common RFCs Cisco has been involved with.

      Thus, these guys are setting most of the major network standards, as well as implementing them.

      --
      Now there's one hoopy frood who really knows where his towel is!
    15. Re:Are you serious, or just killing time? by JWSmythe · · Score: 3, Interesting

          I don't believe in overselling customers. I believe customers appreciate the fact that I'm looking to milk them for extra money. Really, I can score one big scale, or I can build a relationship and continue with them as needed. I've had customers not call for years because they didn't need anything, but the minute they do, I'm there for them.

          Growth is a funny thing. A lot of places I've seen have had 4 desks with the intention of growing, and years later they still have exactly 4 desks. One place had a dozen or so servers with high hopes for the future. Those high hopes were a serious understatement. Their partial T3 became multiple GigE circuits, and their dozen server became over 100. Even the first big growth spurt overgrew the agreed upon server naming convention and it had to be changed after two years.

          One place I worked at, which was growing rapidly, they were set up with a bunch of hubs (I'll explain the hubs thing in a moment), and terrible links between the suites (multiple suites in a complex). It was terrible. Literally, it was normal to have >100ms pings between suites on a good day. I got 6 Cisco Catalyst 2924XL-EN's with 4 port 100baseFX cards, deployed one switch per suite, and ran fiber between all the suites. Total expense was about $600. Then the economy took a dump. They started downsizing, and I believe they were down to something like 5 desks and 3 servers (don't ask).

          Ok, now the hubs thing. I say "hubs" for any low end consumer grade unmanaged "switch". For some manufacturers, it was a marketing ploy to say "switch", which just meant "auto speed switching", where it would handle 10baseT/100baseT/100baseTX, but was still a hub (you could see all traffic on all ports). Some really are switches, but usually not at the level of a real managed switch. If you can get 5 ports for $20, it's a hub. :) I have seen some recently that act like a hub, which is really sad. Well, not just act. They'll even have a single collision light on the front. Oh, there's a big hint. :)

      --
      Serious? Seriousness is well above my pay grade.
    16. Re:Are you serious, or just killing time? by mysidia · · Score: 3, Informative

      Show me the Franken' Catalyst 2950/6500 Sup720 3BXL, Franken Cisco 12006, or Franken Juniper M7i/M320, and then I'll be impressed. Your desktop PC will not contain TCAM or other components required for a minimal level of forwarding performance needed by an ISP.

      After all these years, a desktop PC still cannot perform the task of a simple 8 port switch, at nearly the same packet rates as the switch. The packet rates that can occur on an Ethernet network easily overwhelm the desktop PC's limited interrupt capacity and memory I/O bus bottlenecks.

      For an Enterprise branch office edge a desktop router is fine. Because Enterprises only buy a limited amount of capacity from an ISP. Also, Enterprise branch offices have only clients, not servers, so they aren't really subject to a DoS (rejecting unwanted packets is half as expensive as fully forwarding normal packets).

      Of course, Enterprise server farms never use a firewall at the edge on the path into the servers, unless the periodic unavailability due to DoS attack taking out the firewall is not an issue.

      But for an ISP, if you are planning on being a serious ISP, your core business is providing a professional service. Use a well-designed solution, not something you've cobbled together from off-the-shelf parts. You get real value buying gear that performs forwarding in hardware

      In the long run, one 24 hour outage or service degradation, can cost more than engineering the network properly, and using good managed pieces.

      The fact of the matter is the FrankenPIX was based on the original PIX platform, and Enterprise firewall, that used to be just a PC with some fancy packaging and a proprietary flash card. That platform has been obsolete for many years, and is not suitable for an ISP, anyways.

      In case you didn't know, Firewalls like the original PIX can't handle that much traffic, and they are easily DoSed into oblivion by a simple flood.

      Anyways, decent gear for service providers these days offloads work to hardware. And runs on a real-time OS that can provide something closer to a service level guarantee than a commodity OS can.

      In case you didn't know... Linux is not a real-time OS, and cannot provide timing guarantees a RTOS can.

      Generic Linux running on commodity hardware cannot provide proper separation between control plane and forwarding plane.

      For certain very important functions, a commodity PC simply can't match the performance of a dedicated ASIC.

      You can talk BGP all you want, but you can't reliably forward 30,000 pps through a commodity PC, or push speeds higher than approximately 200megs, due to interrupt contention.

      There is also the matter of reliability of the hardware...

      Commodity desktop parts are not designed to run 24x7, and they fail frequently. Physical failure in routers is rarer, unless there are environmental issues, or the equipment is old.

    17. Re:Are you serious, or just killing time? by mysidia · · Score: 2, Insightful

      Another is that they fit nicely in a rack.

      And they provide a packaged solution, that most network engineers recognize and know how to manage, troubleshoot... meaning it will be easier to find/hire people to help manage it, than some custom home-brewed solution?

      Lower long-term operational expenses, hardware is darn proven (fewer operational risks than you have buying commodity desktop parts), and you can get a support contract, usually (or opt to save money upfront by finding equipment and replacement parts in the aftermarket).

      Many of the low-end routers 26xx are pure software switching. But they can still perform better than Linux, because the OS is designed solely for that purpose, which means performance optimizations too.

      Linux is more of a jack of all trades. Forwarding performance and operation as a network device isn't a central design goal in the linux Kernel.

    18. Re:Are you serious, or just killing time? by networkBoy · · Score: 1

      FWIW:

      PCI / PCIe x1 are both ~1Gbps max throughput (not counting overhead, that's raw bus speed).
      All the other PCIe's scale linerly, thus a PCIe x4 is 4Gbps bus speed.

      After communications protocol over the bus that speed drops (not sure how much). There are other factors as well but what it all comes down to is PCI or PCIe can really handle only about 500Mbps per link.

      -nB

      --
      whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
    19. Re:Are you serious, or just killing time? by Mad+Merlin · · Score: 2, Insightful

      FWIW:

      PCI / PCIe x1 are both ~1Gbps max throughput (not counting overhead, that's raw bus speed). All the other PCIe's scale linerly, thus a PCIe x4 is 4Gbps bus speed.

      After communications protocol over the bus that speed drops (not sure how much). There are other factors as well but what it all comes down to is PCI or PCIe can really handle only about 500Mbps per link.

      -nB

      Not quite. One lane of PCIe v1 is 250M/s, double that of PCI. One lane of PCIe v2 is 500M/s, double that of PCIe v1. So, a PCIe v2 4x slot would be able to push around 2G/s, or 16 Gbit/s, which is slightly more than the 500 Mbit/s you state.

      Furthermore, given that built in gigabit ethernet ports on any motherboard built in the last 5 years or so are connected via PCIe, and I've never had an issue saturating the whole gigabit, it doesn't make a lot of sense to say that PCIe is limited to half a gigabit.

    20. Re:Are you serious, or just killing time? by fluffy99 · · Score: 1

      Other way around. Cisco bought out LinkSys and is selling their stuff with a fancy Cisco label on it. It's definitely not the same as the normal Cisco stuff. If it's like other acquisitions, Cisco will eventually start producing them and what's under the hood will change. Another good example of this is when Cisco bought out Komodo and rebadged their voip box into the ATA-18x series. Cisco rewrote the software and made it a nice unit.

    21. Re:Are you serious, or just killing time? by fibrewire · · Score: 1

      Well, i am trying to build an ISP, so... all of the above?

      Subscriber equipment - Ubiquiti NanoStation M5

      Access Point Equipment - three Ubiquiti Rocket M5 on 19db 120 sectors -> Mikrotik RB1000 for routing -> Rocket M5 for backhaul

      Distribution facility - bonded FiOS links 100Mbit down / 60 Mbit up into ? (trying lots of stuff)

      only 130 customers at this point - free access until i get the bugs worked out

      FiOS will be upgraded to multiple DS3 as soon as i have something workable

    22. Re:Are you serious, or just killing time? by fibrewire · · Score: 1

      I'm thinking along the lines of a Nvidia Tesla running http://www.xorp.org/ - any ideas?

    23. Re:Are you serious, or just killing time? by nhytefall · · Score: 1

      True enough... as an employee of one of those ISP-types you mention, I will absolutely say our Cisco-certifed engineers are worth their weight in gold. Why? 'cause when something breaks, or when some copyright holder comes to us and says some dumbass kid broke copyright by downloading a movie, those guys can tell me who, what, when, where, and how ... right before they make sure it never happens again.

      And that, to me, is worth every bloody cent they get paid. Every. Bloody. Cent.

      --
      0100010001101001011001 0100100000011010010110 1110001000000110000100 1000000110011001101001 0111001001100101
    24. Re:Are you serious, or just killing time? by atamido · · Score: 3, Insightful

      The "same features"? You mean like ASICs that forward the data with low latency once the route is established? Yep, Linux is going to somehow magically add those to your computer, and that's one of the reasons people pay the extra money for Cisco over some old P3 tower PC and a CD-ROM with a penguin on it. Another is that they fit nicely in a rack.

      A lot of router equipment is essentially an x86 PC. Add on cards are often just PCI or PCIe cards. You'd be surprised how commodity a lot of that equipment is. At least, for a big part of the mid range stuff.

      Granted it's all specially chosen hardware and custom firmwared, plus Cisco IOS is a heavily developed and mature OS specifically written for routing, so you're not going to see anywhere near the same performance with some random Linux whitebox system.

    25. Re:Are you serious, or just killing time? by atamido · · Score: 1

      I've picked up a 5 port gigabit switch for $25, which certainly doesn't act like a hub. I just now quickly verified that it will push 890Mbps between two desktop onboard NICs. I've had the switch for over a year and it's never locked up on me. Cheap switches are here.

    26. Re:Are you serious, or just killing time? by shaitand · · Score: 2, Insightful

      ASICs offload processing power from the cpu as dedicated hardware. But the PC you are running linux on likely has a CPU that is at least 10x as fast as that in the CISCO routers (and use a lot more power) so they can keep up.

      This is one of those things that is as much about marketing as reality. There are no shortage of hardware appliance network boxes like BIG-IP LTM/GTM and Bluecoat ProxySG's that cost tens of thousands of dollars and are nothing more than BSD/Linux rack mounted PC's in a fancy case. These devices have no trouble handling enterprise loads (which is about the only place $50k+ pieces of equipment will be found in the racks).

      The linux box does consume much more power to accomplish the task than the cisco with its ASICs but the raw power is definitely there. A more significant concern than the processing is the bus speed. I doubt that is a problem if he is concerned about an $800 software license (mentioned in a suggestion earlier) in that kind of budget range he isn't going to have links that could tax the bus.

    27. Re:Are you serious, or just killing time? by shaitand · · Score: 3, Funny

      'Ok, now the hubs thing. I say "hubs" for any low end consumer grade unmanaged "switch". For some manufacturers, it was a marketing ploy to say "switch", which just meant "auto speed switching", where it would handle 10baseT/100baseT/100baseTX, but was still a hub (you could see all traffic on all ports).'

      Your showing your age here my friend. This hasn't been true for many years.

    28. Re:Are you serious, or just killing time? by clare-ents · · Score: 1

      http://www.vyatta.com/about/press_releases.php?id=78

      20Gbps and 3,000,000 pps on commodity hardware with CPU to spare.

      The parent is clearly unaware of interrupt coalescing or PCI-E.

      --
      Only two things are infinite, the universe and human stupidity, and I'm not sure about the former. (Einstein)
    29. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      My company provides service to (mostly) small businesses, and half of these little five-man operations have some totally over-engineered Cisco gear acting as their network edge because some smartass, self-styled "IT Guy" told them it was the best.

      Remember the old saying? "Nobody ever got fired for buying IBM."

    30. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      It's not so much offloading the processing power, it's avoid the processing entirely and reducing it to a basic table lookup.

      The basic process is something along the lines of:
      - Packet arrives.
      - Destination IP looked up in a forwarding table.
      - This contains a line saying what to do with the MAC/IP headers and what port to send the packet.
      - Packet exits box.

      The only processing that happens is to recalculate the checksums.

      Networks run at ever increasing speeds by avoiding the need of processing packets at all rather than by increasing the bus & CPU speeds.

    31. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      Sorry, the moment you used the word 'leverage' you blew it.

    32. Re:Are you serious, or just killing time? by cheekyboy · · Score: 1

      You gota admit though, any $300 PC running linux can outdo a $3000 cisco router.

      --
      Liberty freedom are no1, not dicks in suits.
    33. Re:Are you serious, or just killing time? by Jaruzel · · Score: 1

      But, I don't want to die in a fire. :(

      When I see binary, I just HAVE to decode it. Now you'll have to change it ;)

      --
      Together, We Can Make Slashdot Better. I Do NOT Mod ACs. - Check Me Out
    34. Re:Are you serious, or just killing time? by JWSmythe · · Score: 1

          Now you kids hush up, and get off my lawn! :)

      --
      Serious? Seriousness is well above my pay grade.
    35. Re:Are you serious, or just killing time? by JWSmythe · · Score: 1

          I don't deny that they exist, but your sample group was insufficient to determine if all cheap switches are really switches.

          Damn, I can't think of the name of the graph. It's too early in the morning. But think of the overlapping circle chart. Some Slashdot users are men. Some Slashdot users use Linux. Therefore all men use Linux. :)

      --
      Serious? Seriousness is well above my pay grade.
    36. Re:Are you serious, or just killing time? by Hel+Toupee · · Score: 1

      A Venn diagram?

      --
      PERL:
      All of the power of Voodoo with most of the understandibility!
    37. Re:Are you serious, or just killing time? by atamido · · Score: 1

      I don't deny that they exist, but your sample group was insufficient to determine if all cheap switches are really switches.

      I never made that assertion. I was merely offering a single data point to disprove your assertion.

      If you can get 5 ports for $20, it's a hub.

      My data point showed that this statement is not true in all cases.

      I could make a chart for you if you like. :)

    38. Re:Are you serious, or just killing time? by JWSmythe · · Score: 1

          Yes! Thank you. :) I had just woken up when I wrote that, and couldn't think of the term.

      --
      Serious? Seriousness is well above my pay grade.
    39. Re:Are you serious, or just killing time? by mysidia · · Score: 1

      Yeah I know about interrupt coalescing and PCIE. That speed is still unrealistic, it's quite obviously manufactured by marketing in non-real-world conditions, if it refers to desktop hardware.

      Can you get that speed when you have a full routing table with several peers, and must send the packet to the right destination, and implement ACLs as well?

      No way.. Keep in mind, you have a table with 500,000 entries (the full BGP table), plus all your local routes, and have to perform a lookup within 300 nanoseconds, to be able to reach a 300 million pps forwarding rate. The very best DDR3 RAM (DDR3-1600) has a minimum best-case cycle time of 5ns. You get no more than 1 RAM access to check your routing table, before you have spent much more than 300ns in memory access time alone.

      With such a large table, and access lists to process a well, it's inconceivable that a desktop meets the 300ns latency requirement.

      Routing tables and ACLs are much larger than anything that will fit in your 1MB CPU cache. Nope.

      Not even close. 3 million pps is 4 orders of magnitude greater than the best possible forwarding performance than you get in real-world conditions, using the best commodity hardware possible.

      Not even server-grade platforms based on Nehalem / Intel 5400 and 5500 architecture can pull that off.

    40. Re:Are you serious, or just killing time? by BitZtream · · Score: 1

      Those 'hubs' are 'switches', just not full switching on every port. And you can't see traffic on every port.

      From a logical perspective:

      They contain 2 hubs. One on the 10mb side, one on the 100mb side for 10/100. There is a switch that connects the two. You'll see all the 10mb traffic on the 10mb ports, and all of the 100mb ports traffic on all the 100mb ports. You will not see all traffic from 100MB on the 10MB side.

      This has always been the case for 10/100 hubs since the 100mb link doing full traffic can not possibly send all of its data down the 10mb side.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    41. Re:Are you serious, or just killing time? by Cramer · · Score: 1

      Yes, I'm aware of the Pix. Are you aware the pix is not a Cisco creation, but an acquisition? The only thing that makes a Pix a "pix" is the flash card. Almost any PC hardware can run the software.

      It wasn't until the (cisco designed) 525/530 that specialized hardware started being integrated. All models of the modern ASA's have assisting hardware -- internal switch(s), crypto chip(s), ASIC(s), etc.

      (BTW, the ASA runs Linux. The part you're paying for is the application running within that Linux.)

    42. Re:Are you serious, or just killing time? by nhytefall · · Score: 1

      But I don't wanna... :)

      --
      0100010001101001011001 0100100000011010010110 1110001000000110000100 1000000110011001101001 0111001001100101
    43. Re:Are you serious, or just killing time? by Cramer · · Score: 1

      Actually, no one has manufactured a true "hub" in nearly a decade. Today, "hub" is just another name for "cheap, unmanaged switch."

    44. Re:Are you serious, or just killing time? by mindstrm · · Score: 1

      I'd have to agree... there are very few hubs left out there, even fakes masquerading as switches. Everything is a switch nowadays - nobody's even bothering to make hub gear.

      There was a short time when this wasn't true, and there was confusion - but that was, like, 10 years ago.

    45. Re:Are you serious, or just killing time? by Anonymous Coward · · Score: 0

      What about simply putting the ACL on a few separate machines? It might cost a few miliseconds, but costs less than what you'd need from CISCO/Juniper/Force10.

    46. Re:Are you serious, or just killing time? by dickens · · Score: 1

      yeah PITA that.. I carry a real 10baseT hub (with thinwire and AUI ports even) in my bag in case I have to sniff a network that has an un-managed switch.

    47. Re:Are you serious, or just killing time? by Cramer · · Score: 1

      Buy a network tap. Yes, they are insanely expensive, but they make taps for everything these days.

    48. Re:Are you serious, or just killing time? by clare-ents · · Score: 1

      I have a Core 2 Duo based router, it has 100 peers with 8 full routing tables. It's currently passing around 300,000 packets per second with, about a gigabit of traffic at 10% CPU load.

      That's within one order of magnitude of the Vyatta result, on much slower hardware. It's within one order of magnitude of the 3mpps performance you quote is '4 orders of magnitude greater'.

      By the way, the Core 2 Duo had 4MB of cache, Nehalem comes with 8MB. The nature of cache is such you don't have to get your entire data set in it, if memory access is correlated it will speed up access. You'd get correlated requests in the unlikely scenario that a tcp packet destined for a given IP address was followed shortly afterwards by another one going to the same IP address.

      --
      Only two things are infinite, the universe and human stupidity, and I'm not sure about the former. (Einstein)
  8. Be more specific! by dokebi · · Score: 2, Interesting

    Without more performance and cost requirements, it's really hard to figure out what would work for you.

    Are your users all in one building? Over a large area? Are you talking about a dozen access points or hundreds?

    Without some more specific information, only advice I can give is:
    Soekris boxes with FreeBSD.

    Good luck.

    --
    In Soviet Russia, articles before post read *you*!
    1. Re:Be more specific! by TheRaven64 · · Score: 1

      Why would you go with FreeBSD over OpenBSD? It has an older port of pf, and the networking infrastructure in both kernels has changed enough since the port was done that it will be tricky to update the FreeBSD version. The OpenBSD version more than doubled the throughput in Soekris systems since the FreeBSD version was branched.

      --
      I am TheRaven on Soylent News
    2. Re:Be more specific! by Fez · · Score: 1

      There are several reasons to go with FreeBSD (Though OpenBSD is great in its own regard).

      The reasons given by the pfSense project are here:

      http://doc.pfsense.org/index.php/Why_did_you_choose_FreeBSD_instead_of_%27insert_OS_here%27%3F

    3. Re:Be more specific! by Anonymous Coward · · Score: 0

      Supporting several thousand users in a single building? Remind me not to jump off the 34th floor.

    4. Re:Be more specific! by fibrewire · · Score: 2, Informative

      The Coachella Valley is the area - all of it. A large area.

      A dozen to start but hundreds in the near future - i'm going to provide high bandwidth service for next to nothing. So the routing HAS to work for minimal bucks.

    5. Re:Be more specific! by dokebi · · Score: 1

      Huh, I didn't know that FreeBSD pf is not up to date. FreeBSD does have multiple firewall options, though.

      But since OP asked about Wireless support, does OpenBSD have good wireless support now?

      --
      In Soviet Russia, articles before post read *you*!
    6. Re:Be more specific! by Anonymous Coward · · Score: 0

      Couple of us take care about wireless network in one county, and we have some 30 nodes, and about 100 members. Most of the routers do run Mikrotik, while there is also significant number of monowall/pfsense (BSD) and linux (customized) boxes, depending on the role of the router.

      We also designed a whole network for one high school, with several pfsense and linux routers, using 4-6 network cards in router (only new hardware, mind you). The school has a number of work places for adminstration, at least one computer per teacher and 10 classroms with 20+ computers for students. There is also a number of video cameras for security purpouses.

    7. Re:Be more specific! by Anonymous Coward · · Score: 0

      I was looking at the Soekris boxes. Ick. For approximately 200 dollars (that's for their 300-500mhz bare bones box) you could buy a bunch of old pentiums and slap a bunch of pci network cards in them. Soekris boxes are overpriced if you are going for bang for buck. Nice plug though.

    8. Re:Be more specific! by TheRaven64 · · Score: 1

      Yes, in my experience (and the opinion of one Linux kernel developer) OpenBSD has better wireless support than any other free *NIX now, although this is not true when it comes to 802.11n. They've massively rewritten the WPA2 stuff so it's actually easy to use and they've reverse engineered a load of wireless chipsets that FreeBSD needs a blob driver for.

      --
      I am TheRaven on Soylent News
    9. Re:Be more specific! by Bert64 · · Score: 1

      Depending how many network ports you need, you could buy an openrd box, which has a 1.2ghz ARM cpu and 2 gigabit ports onboard, and consumes less power than a soekris box..
      It also has hardware SSL acceleration for certain ciphers.

      They also have PCIe, tho the slot doesn't have a connector attached (its just pins on the board), not sure if this can be made into a usable port for adding more network cards or not.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  9. Vyatta? by Anonymous Coward · · Score: 0

    I use it, it works really well.

    http://www.vyatta.org/

  10. Cisco uBR7111 Universal Broadband Router by Anonymous Coward · · Score: 1, Interesting

    http://www.provantage.com/cisco-systems-ubr7111~7CSCR275.htm
    up to 2000 users for $8942.32
    just buy a couple of them with a bunch of linksys wrt54gl's running tomato or tomatovpn and you will be all set.

  11. Vyatta or pfSense by ipstatic · · Score: 1

    Vyatta is pretty good, although the firewall rules can get overly verbose quickly. Also pfSense might be good to look at.

  12. Try Vyatta by macintard · · Score: 1, Redundant
    1. Re:Try Vyatta by Anonymous Coward · · Score: 0

      Vyatta seem to have a few nice solutions. They can also provide you with the box to run the OS should you prefer that. If you want to try it there's an open source version on their site you can use (you can also use it in production if you want)

    2. Re:Try Vyatta by Anonymous Coward · · Score: 0

      I am running Vyatta and love it. It has a very easy to use CL interface and is very quick and resource light.

  13. Mutually exclusive by vawarayer · · Score: 2, Insightful

    I'm building a Wireless ISP using commercial grade, low cost equipment.

    To me, some words in this sentence seem to be mutually exclusive.

    To my humble opinion, a good ISP needs to have good reliable equipement. Sometimes, out of the box routers are better because they don't have moving parts and their firmware could be more stable than a full-blown OS (even if it is Linux).

    Disclaimer: Not that I don't like Linux, I use it all the time.

    1. Re:Mutually exclusive by ls671 · · Score: 1

      OpenBSD packet filter supports transparent router redundancy pretty well I think. Used by pretty large corporations.

      --
      Everything I write is lies, read between the lines.
    2. Re:Mutually exclusive by clarkn0va · · Score: 1

      No reason a person can't run linux (or a half-dozen other unix-derived router platforms) on good quality, no moving parts hardware.

      --
      I am literally 3000 tokens away from the chaotic crossbow --Stephen
    3. Re:Mutually exclusive by BobMcD · · Score: 2, Insightful

      To my humble opinion, a good ISP needs to have good reliable equipement. Sometimes, out of the box routers are better because they don't have moving parts and their firmware could be more stable than a full-blown OS (even if it is Linux).

      If not for this reason, why do you suppose the question got asked?

    4. Re:Mutually exclusive by Fez · · Score: 5, Informative

      You can have low-cost commercial grade services run using off-the-shelf hardware.

      pfSense includes support for CARP, which lets you build high-availablity failover clusters. You can have two (or three or four...) cheap systems and if one dies, just fix/replace it as needed. The backup system(s) automatically take over and nobody would likely even notice the changeover.

      When it's cheap, that is much easier to consider.

      If you want no moving parts, you can use an ALIX box, Soekris, or perhaps even some atom-based boards. If you want to use server-grade boxes to make yourself feel warm and fuzzy, you can do that too. Supermicro even has a server-class atom board in a 1U rack which runs pfSense very well for us.

    5. Re:Mutually exclusive by Fez · · Score: 0, Offtopic

      I should add this:

      Disclaimer: I am a pfSense developer, documentation writer, and co-author of pfSense: The Definitive Guide. :-)

    6. Re:Mutually exclusive by peektwice · · Score: 0, Troll

      They are mutually exclusive. The OP is taking the Wal-Mart approach. He's getting some of the cheapest shit possible, and then telling himself that it's just as good as "commercial grade" equipment.

      --
      Other than this text, there is no discernible information contained in this sig.
    7. Re:Mutually exclusive by MrKaos · · Score: 1

      To my humble opinion, a good ISP needs to have good reliable equipement.

      I don't think that is the point. Motorola's commercial gear does not support nearly the functionality AirOS and MicroTik do. It's great gear - you just can't make it do some of the stuff you need to do.

      --
      My ism, it's full of beliefs.
    8. Re:Mutually exclusive by Anonymous Coward · · Score: 1, Informative

      An important point of note here: pfSense is a firewall, not a router. Yes, it has routing functionality, but it is designed to be a firewall and doesn't have support for the kinds of routing functionality that the original poster probably needs.

      pfSense, however, rocks as a firewall.

    9. Re:Mutually exclusive by turbidostato · · Score: 2, Insightful

      "To my humble opinion, a good ISP needs to have good reliable equipement."

      To my humble opinion, a good ISP needs to have good reliable *service*.

      Ask i.e. Google to learn the difference.

    10. Re:Mutually exclusive by fibrewire · · Score: 1

      Mikrotik Routers are solid routers, but they choke on bandwidth intensive applications - or maybe my routing tables choke. How would i know? What tools would i use?

      The equipment being used is commercial grade and low cost compared to Motorola and Cisco alternatives. Other wireless equipment manufactures used licensed frequencies and it's still horrible to use or featureless. Then said same companies charge a 3000% markup for an unlock code to access additional bandwidth on their devices. I can't tolerate that. How am i supposed to pass on the savings to the end user without fair pricing?

    11. Re:Mutually exclusive by Xenna · · Score: 2, Informative

      These guys:

      http://www.applianceshop.eu/

      Sell embedded systems with monowall/pfsense preloaded.

      Extremely easy to use and reliable.
      I use a pfsense one at home, no idea how things would scale...

    12. Re:Mutually exclusive by Anonymous Coward · · Score: 0

      I second pfSense, awesome.....

    13. Re:Mutually exclusive by Mista2 · · Score: 1

      In my opionion, using reliable equipment means I have to work less frequently at 3:00am when something screws up or dies. The Cisco routers we use arn't cheap, but they are scaleble, run a familiar OS and integrate well with our security systems and monitoring, and they are basicly plug and play - we have config and complaince template, new ger is deployed, config uploaded to the router and that's it. Just rack it and plug in the cables.

      Much less sweat than dealing with configuring a linux router from scratch.

    14. Re:Mutually exclusive by Fez · · Score: 1

      They're good in the EU, and if you are in the US, http://www.netgate.com/ also sells systems pre-loaded with pfSense or m0n0wall.

      I typically prefer the build-it-myself path for the larger systems, but we've bought several ALIX kits from Netgate. Their ALIX cases are nice (reversible lid that can hide/expose antenna holes for wifi is a nice touch)

    15. Re:Mutually exclusive by Bert64 · · Score: 1

      What you consider "commercial grade" isn't what it used to be... Much of the commercial offerings are based around generic x86 hardware, and sometimes the hardware is of a lower standard than you'd get by buying a regular server from one of the well known vendors...
      Look at the current Cisco ASA firewalls - the lower end models use celeron cpus, processors which are designed for use in cheap desktops.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  14. Hire someone who knows what they are doing. by BitZtream · · Score: 1, Insightful

    Sorry to be blunt, but you're asking the wrong question.

    The proper question is: How do I find someone qualified to do this for me?

    The fact that you are asking on slashdot shows that you are not qualified, and what you're going to get back is a bunch of others, who aren't qualified, suggesting all sorts of half assed hacks to do it which will just result in a utterly shitty service overall.

    You could get by with this in the late 90s, but when you're going to compete with cell phone companies, cable companies and standard POTS companies, you probably need to have a bit of a clue.

    --
    Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    1. Re:Hire someone who knows what they are doing. by Anonymous Coward · · Score: 0

      Sometimes you just have to fake it until you make it... telling them they're unqualified isn't going to help at all: either they already know and don't care, and if they don't know they're too stupid to care what you think. at least they're trying to do things better

    2. Re:Hire someone who knows what they are doing. by Em+Emalb · · Score: 2, Funny

      The fact that you are asking on slashdot shows that you are not qualified, and what you're going to get back is a bunch of others, who aren't qualified, suggesting all sorts of half assed hacks to do it which will just result in a utterly shitty service overall.

      Dude, don't shit on a well-known slashdot tradition! How dare you!

      --
      Sent from your iPad.
    3. Re:Hire someone who knows what they are doing. by lymond01 · · Score: 5, Insightful

      The fact that you are asking on slashdot shows that you are not qualified, and what you're going to get back is a bunch of others, who aren't qualified, suggesting all sorts of half assed hacks to do it which will just result in a utterly shitty service overall.

      I disagree. The Open Source community has a thousand hidden gems that a person might not have heard about. Proxmox VE for one: virtualization, with a GUI, with live migration, and if 2.0 turns out, with heartbeat and failover (high availability). Most people have never heard of this where I work even though half the place is virtualized with KVM, VMWare, Hyper-V, etc. I would think the Slashdot, with its plethora of experiences, might come up with a little-known or workable solution in an already developed product that you haven't heard of yet.

    4. Re:Hire someone who knows what they are doing. by Jeng · · Score: 4, Funny

      Wait, isn't shitting on topics a well-known slashdot tradition?

      --
      Don't know something? Look it up. Still don't know? Then ask.
    5. Re:Hire someone who knows what they are doing. by Em+Emalb · · Score: 2, Funny

      Nobody expects to get shit on!

      (To answer your question, yes. Slashdotters shit on anything and everything. We're like a pack of wild pigeons when it comes to that.)

      --
      Sent from your iPad.
    6. Re:Hire someone who knows what they are doing. by Ichijo · · Score: 2, Insightful

      The proper question is: How do I find someone qualified to do this for me?

      The problem is, if you ask a Cisco person to do it, you'll get a Cisco solution, even if it isn't the best solution for the task.

      --
      Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
    7. Re:Hire someone who knows what they are doing. by nine-times · · Score: 3, Informative

      I think you have a good point, but I don't necessarily agree. First, we don't know what market the submitter plans on operating in or who his clientele are. We don't know what his experience is, how much resources he has, or exactly what level of service he intends to offer. Like the guy who criticized the submitter for refusing to buy a $300k Cisco router, I think you committed a common mistake in thinking that IT is just a series of 1-size-fits-all solutions, and that if you going to use the "right" solution to each problem, you shouldn't bother.

      The era of entrepreneurship and hacking things together isn't over, and it probably never will be. Our tools and hacks may become more advanced, but hopefully there will always be people trying out new techniques and business models, testing new start-up technology, and finding different ways of accomplishing the same goals. The answer isn't always to pay an expensive expert or to use established tech.

      As for this:

      You could get by with this in the late 90s, but when you're going to compete with cell phone companies, cable companies and standard POTS companies, you probably need to have a bit of a clue.

      That's true, but neither my phone company nor my cable company provide wireless access where I live. Cell phone companies provide wireless, but it's pretty spotty and slow, and I live in NYC. There are plenty of areas in the US where no service is available except through dialup. Obviously these large companies aren't interested in competing in all markets, so if you come up with a business model and think you can make it work, then I say go for it.

    8. Re:Hire someone who knows what they are doing. by GooberToo · · Score: 4, Interesting

      The proper question is: How do I find someone qualified to do this for me?

      You mean because he's humble enough to realize he doesn't know every thing, you believe he's unqualified anything. I suggest you look hard in the mirror and read what you just wrote to yourself.

    9. Re:Hire someone who knows what they are doing. by TerribleNews · · Score: 2, Insightful

      I disagree, wholeheartedly. The secret ingredient to a successful business is elbow grease. The fact that this person has asked slashdot this question is not a good indicator of success one way or the other. The important thing is whether this person will be able to take a significant number of the suggestions provided and give'em the old college try.

    10. Re:Hire someone who knows what they are doing. by Anonymous Coward · · Score: 2, Insightful

      Have you worked at a cell phone, cable, or standard POTS company lately?
      What exactly do you think you are going to get there besides a bunch of unqualified, "half assed hacks to do it which will just result in a utterly shitty service overall."

      Besides, many of the folks posting in this thread are probably those same unqualified, half-assed hacks who work at such companies. Corporations don't have any corporate voodoo that makes them special any more than someone working for the government makes them any smarter or able to perform miracles (free healthcare, news cars, and money for everyone, YAY!) Put down the kool-aid and open your eyes.

    11. Re:Hire someone who knows what they are doing. by rjstanford · · Score: 1

      and if 2.0 turns out

      And that, my friends, is the Open Source quote of the day, right there.

      --
      You're special forces then? That's great! I just love your olympics!
    12. Re:Hire someone who knows what they are doing. by fm6 · · Score: 1

      The Open Source community has a thousand hidden gems that a person might not have heard about.

      True. But no "software gem" can do the human-level problem solving their guy needs.

    13. Re:Hire someone who knows what they are doing. by lymond01 · · Score: 1

      Meh. That applies to all software: Microsoft, open source, games, etc. Proxmox has drdb planned for integration in 2.0. That's about it.

    14. Re:Hire someone who knows what they are doing. by tkjtkj · · Score: 2, Funny

      Rude, but true.

      --
      "There are 11 kinds of people: those who know binary, those who don't, and those who could not care less!"
    15. Re:Hire someone who knows what they are doing. by uvajed_ekil · · Score: 1

      The fact that you are asking on slashdot shows that you are not qualified, and what you're going to get back is a bunch of others, who aren't qualified, suggesting all sorts of half assed hacks to do it which will just result in a utterly shitty service overall.

      Seriously? That's your answer? You think this is that hard, and you presume that no experts or professionals read and contribute to slashdot? There is plenty of good advice in this thread, though this is not it. My area of expertise is not ISP infrastructure, but I know a project of this size and scope is not necessarily impossible for someone who doesn't do it all the time. My advice for the original poster is to formulate a complete plan before beginning work - complete as in have every bit of hardware figured out, know exactly what software will be running where, allow twice as much time as you think you will need, test and test some more before going live, and plan for budgetary contingencies. Throwing ungodly sums of money at a problem isn't the only way to do it full-assed.

      --
      This is a hacked account, for which the owner can not be held responsible.
    16. Re:Hire someone who knows what they are doing. by Asian+Freud · · Score: 1

      "Have you worked at a cell phone, cable, or standard POTS company lately?"

      So you have?

      --
      Excellence is an attitude.
    17. Re:Hire someone who knows what they are doing. by BitZtream · · Score: 1

      and if

      And that my friend is where you utterly failed to get the point.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    18. Re:Hire someone who knows what they are doing. by BitZtream · · Score: 1

      No, he isn't trying to do better. He's trying to half ass hack something together cheap that he doesn't know anything about in order to save money. If he was trying to do better he'd hire someone with a clue and learn from them while not wasting a shit load of money learning crap that anyone with even the most limited experience will avoid.

      This is called being greedy. Its not about being cost effective, its about spending as little as possible and keeping as much is possible. That I get, thats business, the problem is that his complete lack of business knowledge or technical knowledge in this area means that he has almost 0 chance to succeed. He wants to use OSS to get by on others work without spending money, and expects slashdot to give him the free knowledge to do it.

      Lets for a moment assume that the perfect answer is given here on slashdot, its OSS, it works great, perfectly integrated, better than any possible commercial product out there ...

      He'll still likely screw it up because he has no knowledge to weed out the wrong from the right.

      Anyone with half a clue knows when they are in over their head and isn't afraid to go find someone who can do it properly.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    19. Re:Hire someone who knows what they are doing. by BitZtream · · Score: 1

      No, I believe he's unqualified because EVERYTHING in this thread that he has posted tells me he doesn't know anything about what he's doing.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    20. Re:Hire someone who knows what they are doing. by BitZtream · · Score: 1

      First, we don't know what market the submitter plans on operating in or who his clientele are.

      And the fact that he's asking without providing that information shows he doesn't understand what he's getting into, at all.

      common mistake in thinking that IT is just a series of 1-size-fits-all solutions, and that if you going to use the "right" solution to each problem, you shouldn't bother.

      You mean kind of like the retarded assumption that someone without any experience or subject area knowledge can whip up a Linux based network on some large scale that performs better or as well as the properly selected commercial equivalent, for less money? Its not a one size fits all solution, yet he's assuming that you can just throw Linux at it at make it work by pulling together a bunch of other packages and integrating them. You realize all of that costs money right? It isn't free. If someone is configuring a system you are spending money on it. Anyone who has 'built an ISP' knows that your hardware and software costs are nothing compared to the other operational expenses. Go ahead, build all your custom solutions, go out of business while you wait to get it working because you didn't want to buy the 300k router, but instead spent that much hiring a bunch of guys to get something slapped together well enough to almost go live with.

      The answer isn't always to pay an expensive expert or to use established tech.

      Experts don't have to be expensive, they just need to be experts. And not using established tech without a SOLID reason IS WRONG. He has no clue if it'll be cheaper to roll his own or just buy something, but he's running heard first into rolling his own. Again, shows utter lack of knowledge about what he needs AND what can fill those needs

      That's true, but neither my phone company nor my cable company provide wireless access where I live.

      Because its not profitable to do so. If you come into an area, and show them you can make it profitable, or even pretend you're going to do so, they'll have data available over the airwaves so fast you'll have thought they'd been selling it for 20 years. I've BEEN in that situation. You come up with a good plan, can beat their prices and still turn a profit and you know what they do? The same thing you did, only faster because they have budgets well beyond anything you could hope for. But lets pretend you get everything going and your business is making money at say $20/month/user, and that nets you $5/month. Your phone and cable companies will just charge $15/month and you'll go out of business because they end up providing better service at cost since they can still make money off the rest of their network as well.

      Does that mean you don't do it? That depends, are you expecting that your entire business is going to revolve around basic service at cheap rates? Then no, you don't do it, you'll get your ass kicked by the encumbants. If you're going to compete with them you have to have something they don't and you have to have it better and before they can get it. He's going to spend all his time and money just recreating something they can just go buy off the shelf, and he'll more than likely spend more trying to do so. In the mean time, they'll spend a tiny amount of money buying something new from someone to add a new feature that people will love (useful or not, marketing is a powerful thing) and now before he's even caught up, the target has moved.

      Automobile manufactures don't make their own bolts, they buy them, for a reason, MAKING BOLTS ISN'T THEIR BUSINESS. Had he started out saying 'I want to start a software company that makes a Linux based ISP-in-a-box solution' then that particular part of the story would be different, but thats not what he's asking for.

      Rolling your own is only useful when you are
      * The first to do it

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    21. Re:Hire someone who knows what they are doing. by nine-times · · Score: 1

      And the fact that he's asking without providing that information shows he doesn't understand what he's getting into, at all.

      Fair enough.

      You mean kind of like the retarded assumption that someone without any experience or subject area knowledge can whip up a Linux based network on some large scale that performs better or as well as the properly selected commercial equivalent, for less money?

      I didn't say it would scale or perform better. I'm suggesting that it might scale or perform well enough for their needs. However, depending on a person's particular needs, it may be that a custom solution will "perform" better, especially if your concept of performance isn't just raw throughput. It may be that they want customization or some particular whacky feature that can be hacked into Linux more easily than into a Cisco router.

      Regardless, it's not as though every person and business always needs the absolute best-performing solution in every case. Often you just need something that performs well enough for now and the short-term future, ideally with some kind of upgrade path should upgrades be necessary.

      Experts don't have to be expensive, they just need to be experts. And not using established tech without a SOLID reason IS WRONG.

      What I'm trying to point out is that businesses rarely have unlimited cash or unlimited resources. Often, we have to make due with what we can afford and whatever solutions we can hack together. Somewhere down that road of hacking things together, you reach the point of being reckless and irresponsible, but where that point lies really depends on what you're doing. If your business is life-or-death, then your threshold for errors and down-time should be a lot lower. A lot of our jobs really aren't life or death, though.

      Because its not profitable to do so.

      No, it's not because it's not profitable, it's because they believe (rightly or wrongly) that it's not the greatest return on their investment. If they have a billion dollars to spend, they'll spend it where it will make them the most money. Every company does not enter every profitable market.

      Whether something is profitable depends on a lot of things. Yes, it's absolutely true that, if you're a small business entering a industry that's dominated by large businesses, you'll be in danger of being run out of the industry. There's a whole other step before you get to the point of "it's so impossible to compete that it's not even worth trying."

      Now maybe you're right about it, and in this particular instance, the telephone company and cable company have such a duopoly everywhere in the country that no one can hope to provide any kind of Internet service anywhere without being run out of business. Well ok then, that's a whole other problem. In that case, why even bother telling the guy to hire someone who knows what they're doing? Just tell him to get out of the business.

      However, I still disagree with you about homegrown solutions. My company (I won't get into what exactly we do) according to your metaphor, we're making some of our own bolts without being in the bolt-making business. We do it because we can make better quality bolts more cheaply than existing commercial bolt-manufacturers provide.

      In our case, our bolts are software created out of hacked-together open source solutions. We don't sell the software and it doesn't get distributed outside of the company, we just built and maintain it for our own purposes. We provide a service that's also provided by some VERY large and very evil companies, and we scrape by because we aren't competing with them directly. There are also commercial packages that do what our software does, but ours does it better, and by "better", I mean it better serves the particular needs of our particular clients. We've even had a large competitor try to put us out of business with an expensive commercial pack

    22. Re:Hire someone who knows what they are doing. by lymond01 · · Score: 1
  15. Few requirements given but... Vyatta? by backtick · · Score: 2, Insightful

    Maybe Vyatta @ http://www.vyatta.org/ does what you want. I really don't have any idea what that is from the actual post, tho. You need some routing for thousands of users, and can't afford a Cisco UBR. I'm not sure exactly if you wanted to use the UBR for DOCSIS type support for some reason (a la cable modem) but the fact it'll be wireless leads me to believe it won't be. I'm assuming you don't need a lot of physical ports, just something to manage your VLANS, some routed subnets, a bit of BGP, etc. Maybe XORP is what you want, tho @ http://www.xorp.org/ so you may want to look there. IHeck, 'm not even sure if you want to take a server with a bunch of PCIe ports and slam multiport switchable fabric cards in there like the ones DSS @ http://www.dssnetworks.com/v3/gigabit_pcie_6468.asp makes, or do something else. Maybe these links will help, and hopefully there'll be a detailed followup so we can aim at the real target :)

    1. Re:Few requirements given but... Vyatta? by DarkOx · · Score: 1

      Where Cisco is a good value is enterprise licensed switching. You could buy 37XX, 4XXX, or 65XX gear depending on the level of residency you need and do lots of your heavy lifting there; BGP learning and advertisement and port access control and basic ACLs; you might then put some Linux servers behind some of that to do some of the really complex routing jobs (things with lots of rewrites and NAT operations; process authentication information, provide DHCP with dynamic DNS updates etc. You might save some serious scratch over the high end router like hardware from Cisco. It would be worth your time to look at Enterasys and Extreme Networks too. They have pretty competitive product catalogs in terms of features and reliability and its usually a bit cheaper than Cisco, Juniper, and Nortel.

      I don't you want to try and run a carrier grade service with just Linux on what I assume is going to be a bunch of x86 boxen (hopefully server class at least). There is just too much to go wrong; I am not saying it does often but this is a job for a thin software stack not a general use OS no matter how good it is.

      --
      Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
    2. Re:Few requirements given but... Vyatta? by fibrewire · · Score: 1

      not DOCSIS - i was just making a point

      i meant i need some sort of easy way to terminate, load balance, QoS, and manage all of my wireless clients.

      XORP looks awesome - maybe putting it on a Nvidia Tesla box will help with routing speed?

  16. What on earth are you trying to actually do? by sirket · · Score: 4, Insightful

    Routing and ISP's are huge topics- what are you trying to do?

    The main problem with routing isn't bandwidth- anyone can pump enough 1500 or 9000 byte frames per second to fill a gigabit pipe. The problem is when you have lots of small packets. At that point, dedicated routing hardware with a high-speed TCAM becomes really important.

    What kind of line cards do you need? ADSL? Ethernet? OC12?

    What kind of services do you need to run? BGP? OSPF?

    What kind of bandwidth are you going to be pushing?

    1. Re:What on earth are you trying to actually do? by fibrewire · · Score: 2, Informative

      Tons of multicast video data will eat up 1/4 to half of my last mile bandwidth, followed by voip and data.

      I'm trying to balance Access point range to around 1/2 mile without dropping bandwidth, so Ubiquiti AirMAX equipment seems to work in trial runs.

      i don't want to drop below 100Mbit lan speeds, rates are fixed so if a customer can't connect they won't kill all the bandwidth for everyone else.

      Client's actual throughput will be about 10Mbit down / 2Mbit up + about 45Mbit of Multicast video overhead - 100 clients will share about 50Mbit of bandwidth, if it scales out ok then 300 clients will share 50Mbit

  17. Not speaking from experience... by Anonymous Coward · · Score: 0

    Maybe this will work for you. On my linux box, I entered `yum search ospf` and it came back with a package called 'quagga'. I did a Google search and found they have a website. According to the website, they support OSPF and BGP.

  18. Ebay is your friend. by jjeffries · · Score: 4, Insightful

    Start off small. Pick up some used Cisco stuff off Ebay at 1% list. Maybe a 6500 with a couple of SUP2s for your core switch, a couple or four 7200s for the upstreams/customer facing bits. Make lots of money, upgrade to newer stuff as needed.

    1. Re:Ebay is your friend. by gobbligook · · Score: 1

      I second this

    2. Re:Ebay is your friend. by Anonymous Coward · · Score: 0

      you are aware that Cisco IOS is non-transferable so while you can certainly pick up the HW at a cheap cost your going to either a) run it illegally or b)take it up the ass to get it inspected, licensed and covered by smartnet which by the way are 3 things that are mutually tied together. You cant order the software without ordering smartnet and you cant do that until you get the box inspected.

      If you want to go a cheaper route then get Cisco Authorized referb. Its cheaper than new and legal. Save your company from doing something stupid...

    3. Re:Ebay is your friend. by duguk · · Score: 1

      you are aware that Cisco IOS is non-transferable so while you can certainly pick up the HW at a cheap cost your going to either a) run it illegally or b)take it up the ass to get it inspected, licensed and covered by smartnet

      Citation? Surely the software that's on the device is all that's needed. Negating support of course. If you're suggesting thats non-transferable, that's like suggesting you can't legally buy an old PC or any mobile phone with similar licences second-hand.

    4. Re:Ebay is your friend. by emt377 · · Score: 0

      Totally. Used Cisco gear is cheap on ebay.

  19. Mikrotik by VonSkippy · · Score: 1

    You say: " I've used Mikrotik's RouterOS for five years, but it just isn't built for what I need."

    What exactly isn't it built for?

    Mikrotik has numerous large WISP's (+5000 seats) running on Mikrotik Software and hardware.

    Have you contacted Mikrotik's engineers with your "problem"?

    1. Re:Mikrotik by fibrewire · · Score: 1

      I have not contacted Mikrotik engineers with my problem, sometime things like that slip my mind.

      I need Mikrotik's "The Dude" and Ubiquiti's "AirControl" to work together.

    2. Re:Mikrotik by jeffstar · · Score: 1

      you don't need mikrotik's dude for anything. use nagios, opennms, cacti, whatever.

  20. pfSense by mhab12 · · Score: 2, Informative

    Give pfSense a try. http://www.pfsense.org/ Also a VERY active user forum at http://forum.pfsense.org/

    1. Re:PFSense by Fez · · Score: 1

      crunchgen is not used anywhere in pfSense (in fact the crunchgen binary is removed as part of its build process).

      Yes, you can get the same functionality by manually installing all of the included software on a bare OS, but you lose the GUI, configuration code, backup system, ease of use, extra patches used by pfSense, and lots of other functionality. Incidentally you also gain other functionality by using the base OS, but it's always a trade-off.

      I'm not saying pfSense is the answer to everything, but it's been more than capable of anything I've tossed at it from lots of wacky scenarios, and then some.

      (As stated elsewhere in this topic, I am one of many pfSense developers, so I am a little biased :-)

    2. Re:PFSense by SgtChaireBourne · · Score: 1

      Cool. Thanks for the clarification and for the work on PFsense. Where can we read about the build process? How is it done, if not crunchgen? The "extra functionality" and, more importantly the extra field testing are useful. However, I myself am a minimalist, when I can get away with it.

      --
      Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
    3. Re:PFSense by Fez · · Score: 1

      The act of building your own CD or install image is covered here:

      http://devwiki.pfsense.org/DevelopersBootStrapAndDevIso

      If you're just interested in the tools, patches, and scripts that build the system, they can be found in the pfSense "tools" repo here:

      https://rcs.pfsense.org/

      The code for the different pfSense branches is also there, as well as the code for the livecd repo based on freesbie2.

      If you have a spare FreeBSD box (or a VM) it isn't too hard to follow the how-to and make an image, but the instructions only cover a fraction of what it is capable of doing. That one tools repo contains the scripts to build everything: LiveCD ISOs, Firmware update files, Embedded images, you name it.

      If you want to know more, check out the forums or ask on freenode, someone is usually around who is familiar with the process.

  21. Re:m0n0wall is a great BSD distro by clarkn0va · · Score: 3, Interesting

    I have to agree, although I registered a vote for PFSense above. PFS is based on m0n0wall and both are excellent routers filling slightly different niches. I currently use PFS at home for its packages (freeswitch, squid), but I recently worked for a growing WISP and got them onto m0n0wall, now serving something in the neighbourhood of a thousand customers.

    If you want pure simplicity, go m0n0wall. Otherwise, I strongly recommend looking at PFSense for the squid caching and adjust-on-the-fly connection table size.

    --
    I am literally 3000 tokens away from the chaotic crossbow --Stephen
  22. Big Sur Wireless by north.coaster · · Score: 2, Informative

    There's a small wireless ISP located in the Big Sur area of California that seems to have been up and running for a few years now. Maybe the OP wants to build a system like Big Sur Wireless. Their web site includes a lot of details about their homebrew system.

    1. Re:Big Sur Wireless by Fallen+Kell · · Score: 1

      How do you know the poster isn't from Big Sur Wireless :D

      --
      We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
  23. ClearOS by darp · · Score: 1

    I use pfSense and it works reasonably well. I was looking also for something more sophisticated. ClearOS (http://www.clearfoundation.com/) looks like nice judging the screen shots but I haven't had a chance to try it. Did anyone tried ClearOS? feedback?

  24. routing solution by freddieb · · Score: 1

    I am not sure what you are asking. I have used a linux or freebsd home router for years. You can configure either rather simply with the information available on the net including firewall filtering. I am sure you could use 1000mb ethernet cards and make a super fast router with either OS. Linux is a little more hardware friendly than freebsd. As others have said, more information on specifics will get you specific answers.

    1. Re:routing solution by Arimus · · Score: 1

      Most home hardware will not get you 5 9's availabilty and if the OP is aiming to be an ISP then you need as close to 5 9's as possible.

      --
      --- Users are like bacteria -> Each one causing a thousand tiny crises until the host finally gives up and dies.
    2. Re:routing solution by h4rr4r · · Score: 1

      Not according to every home ISP ever. I highly doubt comcast and TWC and COX even come close to two nines. Heck, their ntp servers are probably not even one nine.

    3. Re:routing solution by h4rr4r · · Score: 1

      I meant dns, I don't think they have customer facing ntp servers at all.

    4. Re:routing solution by Anonymous Coward · · Score: 0

      What the hell ISPs have you been using? Around here it's Telus and Shaw, both of which *might* be close to two nines.

    5. Re:routing solution by Anonymous Coward · · Score: 0

      Anyone that uses consumer grade hardware for big commercial applications is destined to get 9 5's instead.

    6. Re:routing solution by Arimus · · Score: 1

      Sorry; should have prefaced my remark with any COMPETENT isp ;) And to be fair probably more the buisness/premium price end of the market.

      Infact now I think of it... I'm just baised to good quality,reliable, hotswappable etc kit after working in the teleco industry and working for an isp who did aim for 5 9's - which we didn't always meet for all customers as BT have a habit of screwing their end up.

      --
      --- Users are like bacteria -> Each one causing a thousand tiny crises until the host finally gives up and dies.
  25. Support? by travisb828 · · Score: 1

    Part of the reason why you pay for Cisco is for support. If something fails you can get a replacement quick. If you dont mind spending a little more you can get a replacement delivered to you while TAC sits on a bridge with you. You also have the option of delivering it on a silver platter.

    1. Re:Support? by h4rr4r · · Score: 1

      So take some of the money you save and buy spare whatever you are using. No delivery is faster than one sitting on site in the closet and another at your second site in case of damage to the building.

      We have some used cisco stuff and that is what we do. I could get 9 spares with every unit and it would still be cheaper than the retail pricing.

  26. Mesh technology by Anonymous Coward · · Score: 1, Interesting

    If your seriously looking at going the ISP level, you shouldn't be messing around with second-hand or non-isp class hardware.

    <slashvertisment>

    You could always try contacting a company that actually does this for themselves and provides hardware/software for others to do the same thing. I'd -highly- recommend going with a mesh-based technology to add redundancy to your infrastructure. Cambridge Matrix has some pretty good kit.

    </slashvertisment>

  27. As others have said... by KiwiGod · · Score: 4, Insightful

    What's your interface to the net, line cards, bandwidth expectations, etc. I spent 5 years building a fairly heavy duty wISP network on a stupid low budget from my boss. You can obtain used cisco stuff for cheap. For instance, you can get your hands on a 7206vxr with a NPE-G1 for $10k or less nowadays... If you need something with high redundancy do do less intensive switching, you can pick up a 6509 with a pair of SUP2-MFSC2 cards for less than $2k. As far as support contracts go, I can't imagine that you need the latest and greatest IOS, let alone a support contract that costs more than the replacement of a piece of hardware. On a side note... why are you asking about the uBR series? Are you not running an ethernet network? Last I checked, there's no such thing as "low cost commercial grade." Depending on where you are, unlicensed stuff may not cut it, dealing with interference etc. And licensed hardware is certainly not cheap. With wireless, as well as so many other areas, you get what you pay for.

    --
    Macs, Linux, Windows... who cares, they all suck at something.
  28. Are you a business ? by Bork · · Score: 1

    If you are going to be a business, I think your going to need a better business plan first.

    It almost feels like you have this great idea but have not sat down and wrote down exactly what you are going to do and how are you going to get there. Talk to the small business administration, they have people there that you should talk to first.

    You have selected the hardware before finding the software that will accomplish the task.

  29. OpenBSD 4.7 by Anonymous Coward · · Score: 0

    OpenBSD-current is the way to go. Excellent routing performance, very strong BGP and OSPF implementations, and BGP MPLS VPN support is almost complete in current too.

    OpenBSD 4.6 has a few significant OSPF bugs that are resolved in current. Also slightly lower routing performance.

    What are you trying to do that Mikrotik can't do ?

  30. Clearly no idea what you're talking about by dbarclay10 · · Score: 1

    Okay, clearly you have no idea what you're talking about, because a Cisco Universal Broadband Router is a bit of kit used to terminate DOCSIS lines. In other words, it's for cable-modem broadband, not wireless. It would be useless to you.

    That said, for others who're reading and who might be interested in some high-end, Linux-based packet-processing kit (because really, the prices Cisco and Juniper and the rest of them charge really are past the ass-raping point of the screw-me spectrum), you could check out Vyatta: http://www.vyatta.com/

    Enjoy. HTH.

    --

    Barclay family motto:
    Aut agere aut mori.
    (Either action or death.)
  31. just a thought by khelix · · Score: 2, Informative

    I did not see anyone suggest http://www.untangle.com/ . i have only played with it for a short time, but it might be worth checking out!

  32. Anonymous Coward by Anonymous Coward · · Score: 0

    I'm gonna go with "zeroshell" zeroshell.org. It appears to clearly favor security, configurability and ease of use. Additionally, the developers have a clear understanding of networks and what is really necessary to get things done.

  33. I vote for IPCop by Anonymous Coward · · Score: 0

    http://ipcop.org

  34. Go 2nd Hand Cisco by Niobe · · Score: 1

    Why bother with a high-maintainance OS system for a router?? Just buy a refurbed Cisco from a reseller. You won't get support from Cisco but you can buy the router and a spare second-hand for 5% of the original cost.

    1. Re:Go 2nd Hand Cisco by POTSandPANS · · Score: 2, Informative

      If you are just starting up, I'd suggest a couple of Cisco 3550 layer 3 switches with the IP Services image. They don't have all the features of the big routers, but they can handle a huge amount of traffic. I doubt you could build a linux router that would handle as much traffic for the same price as a 3550.

    2. Re:Go 2nd Hand Cisco by wulfhere · · Score: 1

      That's just how we started. They're great little layer-3 switches, and can be had for cheap ($200-300) on the used market. Check out Network Hardware Resale, PICS, or Network Liquidators. We've gotten 6503's with a sup720 for less than $13k.

      --
      -- Sent from a computer.
  35. OBSD or pfSense by Anonymous Coward · · Score: 2, Informative

    My history is: started on OBSD (due to hardware support, ironically); played w/ FBSD; ended up on pfSense.

    My observations:
    OBSD is absurdly security conscious... for ISPs especially, this is a good thing.
    OBSD tends to have a lot of focus on new network features (pf, carp)
    most OBSD features get ported to FBSD... but take time (look into carpdev)

    pfSense (built on FBSD) has some overhead vs FBSD raw (obviously), but has *nice* management UI, package support, etc
    customizations are easy for pfSense (I added some features to dhcpd a while back)... easier than generating the diff and submitting it ;)
    pfSense is more focused on network features (they're working on fixing carpdev for FBSD)

    I like pfSense a lot... I use it for routing between ~6 VLANs, IPSec tunnels with another pfSense, PPTP server, *tight* firewall rules (given 6 VLANs).

    pfSense 2 will be adding a lot of nice features for businesses (multiple admin accounts, different permission levels, etc)

  36. Vyatta by Anonymous Coward · · Score: 2, Informative

    What about Vyatta? It's a good router based on linux and you can install it on any old box you want or buy their hardware for it. Even has a cisco like interface if you want.

  37. Buy used. by roc97007 · · Score: 1

    Why not buy used Cisco routers? In the current economy, you should be able to make some pretty sweet deals.

    --
    Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
  38. Nagios, ssh, airOS by MrKaos · · Score: 1

    You can make ssh plugins with Nagios, AirOS supports ssh and key exchange. You should be able to achieve most things with that combination, what is it you are trying to do?

    MicroTik has a strong API, have you tried doing what you need to do by using that?

    --
    My ism, it's full of beliefs.
  39. A suggestion by scottraynel · · Score: 2, Informative

    RuralLink Ltd (yes, I work for them) does what you want, linux-based wireless network management. Get in touch with us at http://www.rurallink.co.nz/contact-us

    There's not a lot of info about that side of things on the website, but if you contact us we'll be happy to chat - and don't worry, we're all techs, there's no sales droids here.

    Cheers,

    Scott.

  40. been there done that bought the tee shirt by Anonymous Coward · · Score: 5, Interesting

    I founded and operate a wireless ISP serving about 1000 wireless subscribers, and have my own embedded linux distro inside just about everything. It would be a fair statement to say that linux literally saved our business on more than one occasion, by giving us the tools to overcome manufacturer software bugs, by establishing 'known good' systems of various types, by enabling read-only compact flash based systems running on solar power, by bringing a high level of utility and reliability into the critical parts of the network, by allowing us to make it anything it needed to be.

    As a CPE, my linux distro never lets me down and never puts customers of at risk of 'stone dead - lights on but nobody home', like linksys/netgear/etc always seem to. Never having to tell someone 'just pull the power and plug it back in' for their connectivity is a real saving grace. And when in a business situation, I can equip these customers with connectivity devices that _do not fail_ and make us look stupid, while at the same time giving them useful feature sets unavailable in higher end router manufacturer gear (cisco 2621 - excellent hardware with great stabillity, just weak on features I get with dnsmasq, openvpn, tcpdump and others.. trying to diagnose network connectivity issues without tcpdump is just dumb.). Its also never choked and zeroed out it's own flash config for no goddam rason, unlike the previously mentioned low-end consumer devices frequently do. Basically, that consumer stuff puts you at risk and is suicide.

    As a network appliance, linux flings packets just fine and gives you great tools to filer, mangle and generally control how and what it does. The ebtables code is awesome, the iptables stuff is killer, openvpn rocks asses, dnsmasq kills, there's just too many useful and cool things just go right. I have a pppoe server running rp-pppoe + my patches and userspace tools, running for years now and hit with every kind of client side bug and malfunction imaginable, and just keeps trucking along. Freeradius backed up with mysql is sweet as can be, and quagga for distributing my routes internally is just a dream. I have it all on read-only compact flash, so they never write and basiclaly will run until there is a show stopper hardware problem, at which point I will more than likely be able to remove the flash and put it into another machine and away I go.

    There is a lack of management interface, and there is a learning curve to this route, but the upside is very low dollar cost and an attainable level of flexibillity, reliabillity and stabillity you are unlikely to find in any commercial solution anywhere. Cisco IOS is awesome, but you won't power anything that runs it off a 12v battery and solar panel on the side of a mountain and flinging/filtering 20mbps of traffic.

    Good luck.

    1. Re:been there done that bought the tee shirt by fibrewire · · Score: 1

      As i recall, i worked with a brillant guy on a similar project. Truth is it would have all worked if the guy hadn't had a clue of his brillance. Everyone suffered in the end, including the guy who wrote this "Ask Slashdot" article. I've never seen such a smart person ready to burn every bridge on his way to success, especially when carrying 35lb batteries up the side of a mountain to replace the ones that aren't being charged by an underpowered solar panel for an access point. But i did see the light, and made sure that i am always open to suggestion. So please, if anyone has ideas on how to make this project a success, i'm ready for input, and ready to work together to make this cost effective wireless ISP a reality.

    2. Re:been there done that bought the tee shirt by Anonymous Coward · · Score: 0

      I played this game once too. Off the shelf boxes like bandwidth control units (BCU's) actually work the *opposite* of what you want, which is to saturate the pipe you are paying for and make your customers happy. The key to doing that is using HTB from devik, which has since been incorporated into the main kernel. BCU's will rate-limit each connection and no one will ever get better than your setting using a regular token bucket. HTB turns this on its head and always gives out all the b/w you have available.

      Multi-hop all the T1s (or E1s/whatever) you have available and enjoy all the happy emails you will get. HTB provides a guaranteed *minimum* b/w satisfying your need not to over saturate the b/w you have, but when you have idle b/w it will be doled out to whomever is on at the time. You can also give certain customers priority to over-ride common users and grab their own, special minimum b/w from the pool. It's perfect for a wisp.

      Obviously, this is a bad idea if you have any xfer limits, but if you have unlimited guaranteed b/w, HTB is the way to go.

      http://luxik.cdi.cz/~devik/qos/htb/

  41. Simon Lok's AirLok by demerson3 · · Score: 1

    The need reminds me of this guy: http://www.forbes.com/forbes/2004/1101/064.html But... never heard anything about him since 2004.

  42. Step in the right direction by fibrewire · · Score: 2, Informative

    I guess i'm looking for a scalable ISP-in-a-box solution. And if it doesn't exist, then let's build one. But Proxmox VE looks like it will fit well with managing computer resources between the handful of Dell 2950s slated for Zimbra, FreeIPA (Active Directory for Linux), Nagios, Cacti, and AIRControl. Still looking for a good FreeRADIUS server i can tie into FreeIPA - but i need lots of other stuff than just a router-in-a-box. A balance between smartest / practicality / economical directly translates into cost savings of the end user. Someday i will be able to provide free internet, but for now i am targeting $20-$40 a month for data, voice, video, and multicast TV. Some features of a good OSS router needing attention are:

    * PowerPC vs X86 vs GPU - does routing perform better on PowerPC (Mikrotik / Vyatta / Cisco)? would an Nvidia Tesla solution work well?
    * Easy to manage large scale routing implementations - speed of deployment, discovery of devices, failover, centrally monitored?
    * Weatherproof - power outages, network hiccups, etc. nothing more irritating than going on-site to an AP to reboot in the middle of a storm

    For more details about a specific area please ask.

    1. Re:Step in the right direction by tacotony · · Score: 1

      I work for a Wireless ISP in the midwest. We have quite a few towers and APs in a few different towns. We use MikroTik, Nagios, and FreeRADIUS. We have QoS setup through our MikroTiks and have power outage solutions for those nice storms :) but sometimes we just can't avoid the lightning. Haha. Hit me up and we can talk about everything you need help with.

    2. Re:Step in the right direction by halltk1983 · · Score: 1

      I work in a wISP in Texas, and we do similar to this. Don't rule out Mikrotik unless there are specific things you need, and then you may be able to get these in the form of a module.

      --
      Watch for Penguins, they eat Apples and throw rocks at Windows.
    3. Re:Step in the right direction by Bert64 · · Score: 1

      Proxmox is good, i run a small hosting environment and have a pair of HP servers running proxmox, which has clustered storage using DRBD. Normally both servers can load balance, but if one fails the other can take up the load (with resulting performance hit).

      PowerPC handles networking slightly better because it's a little endian architecture, and network packets are little endian encoded... x86 hardware has to reverse the byte order on every send/receive... Also, embedded PPC hardware consumes less power which is why cisco and others use them... Cisco have also used m68k and mips processors in their devices over the years, wouldn't be surprised to see arm processors on their lower end kit.

      Where are you looking to deploy routers? I envisage your setup (in its simplest form) involving a number of AP devices sitting on towers, connected back to a central point where servers and an internet connection reside... Are you talking about the individual access point devices, or the central core routers? And how much traffic do you envisage pushing?

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
    4. Re:Step in the right direction by BitZtream · · Score: 1

      I guess i'm looking for a scalable ISP-in-a-box solution. And if it doesn't exist, then let's build one.

      You are going to fail. Your core business isn't building a open source ISP-in-a-box solution is it? You'll spend far less money if you get over that thought and use something already packaged and integrated and working together. You are trying to using something open source to save money, but in the process you're going to spend a freaking fortune building and maintaining the open source package. This is not a logical way to go about business unless you plan on making that your core business.

      Someday i will be able to provide free internet

      So you're going to build a business and then give it away? How do you expect to pay for this stuff, donations? I guess you intend to subsidize the internet access by fees from the video/telephony side? If so thats not really free, thats called bundling, and personally I call it lying. If its something like 'I can get data for free but I have to buy something else' than its not free.

      * PowerPC vs X86 vs GPU - does routing perform better on PowerPC (Mikrotik / Vyatta / Cisco)? would an Nvidia Tesla solution work well?

      If you're packets are regularly making it to the CPU than you've already fucked up. Packet routing on any sort of scale thats useful is done on custom firmware AND hardware designed to do so. No one uses any of these to route packets. They may use them to control the system and handle routing table changes and figuring which interfaces to use, but in a real ISP only the first packet of a session is likely to ever hit the CPU. Firewalls are a little different, but if you're an ISP and you're shuttling all your traffic through a firewall you aren't much of an ISP in the first place.

      Again, you are not qualified for this. You are going to waste money and probably fail. You aren't even qualified to lead the development of the ISP-in-a-box solution because you know absolutely nothing about it and have 0 experience. How do I know this? Because you still haven't asked any of the right questions or provided ANY details that would help anyone make a even slightly educated guess about what would help you. You're still at the point where you need someone to actually TELL YOU what you need.

      Sorry to be a dick about it, but you're going to waste a lot of someone elses money and that pisses me off. You're also, assuming you ever manage to get it to the point where you have customers are likely going to provide really shitty service. Even if you got the project going and financed, you're just going to end up leading a project that doesn't know where its going because the leader DOESN'T KNOW WHAT HE'S DOING.

      I know PLENTY of shitty ISPs that did EXACTLY the crap you are doing in the late 90s/early 2000s. Back then it was mostly done by highschool/collage grads who could use a PC so someone thought they knew what they were doing and hired them to 'start an ISP' for them, and 6 months to a year later hundreds of thousands and in one case I know of personally, 10s of millions of dollars were gone and the company was bankrupt. All of them because some jackass thought because he read about something on the Internet and heard of Linux that they could make it all work.

      You'll be a much better leader when you realize where to draw the line and find someone who knows about the subject matter at hand. First hint: Slashdot isn't where you look.

      --
      Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
    5. Re:Step in the right direction by fibrewire · · Score: 1

      Instead of crying over aforementioned posts, I ask, where would one find "the right person" because as I see it, there are no "these are the thousands of steps needed to become an ISP" or "Collection of tools and resources every ISP should have" location on the internet. I know that I am being over generalized, but I don't see you saying "I'm the right guy but busy at the moment, talk to these guys" or "here is some information to help you achieve what you want" and if I had been very specific in my request, it would have never been posted front page /. in the first place, and I wouldn't get the chance to be flamed by you, and I would have never had the opportunity to turn the situation around and ask for your help, would I?

      So for the sake of moving forward, lets pretend that I really didn't need a Powerful Linux ISP Router - instead I want to build a cost effective Wireless ISP that can deliver Multicast IPTV, Video on Demand, VoIP, and data (10Mbit down/1Mbit up), and still be able to feed and shelter myself. As it grows, I'll be able to pay better people to engineer this system in place of me, but me is all i've got.

      Right now, I have Ubiquiti AirMAX equipment with Mikrotik Routerboards on a few towers that I own, and about 250 customers that get free access while I figure out how to do this - they just foot the bill for the equipment and installation. I've got AirControl monitoring my last mile equipment, and The Dude monitoring my network. All of this is out of my own pocket. I need an AAA system, and something other than this stupid Cisco 1811 ISR connected to FIOS. I use all kinds of tools. People want email, I give em Zimbra. People want Storage space, I give em OpenFiler. I'm struggling with FreeIPA at the moment to authenticate everything together, and you know what? It's a learning experience.

      If you aren't too busy playing WOW or just pulling on your own taffy, do you think you could help me out? Everything helps, including your condescending attitude towards my efforts at trying to make my world just a little bit better place to live in. If nothing else, you've fueled the flames that drive me to succeed.

  43. His belief that Cisco UBRs are $300K is a give by anti-NAT · · Score: 1

    away.

    CRS1 maybe, but not CMTS routers.

    --
    The Internet's nature is peer to peer - 20050301_cs_profs.pdf
  44. Free - sure! by Gr8Apes · · Score: 1

    Heck, that's the "not so open source model"! Build crap and give it away for free, then charge out the wazoo for "support".

    Well, it's a good business model anyways. :)

    --
    The cesspool just got a check and balance.
  45. Get Real by Anonymous Coward · · Score: 0

    You mention the UBR (Universal Broadband Router) specifically in your remarks. The UBR 10K which the poster refers to is not only a router but a CMTS (Cable Modem Termination System) as well. I guarantee you will not be able to turn a Linux box with some PCI cards into a DOCSIS head-end serving one cable modem let alone thousands of cable subscribers. You are way out of your league if you need service provider grade gear and 300k scares you.

  46. Correct question? by Runaway1956 · · Score: 2, Informative

    Alright - I read your question, then a couple responses - but it isn't clear here that you're asking the question correctly. Humor me for a moment, then decide whether you asked the right question.

    You have access to the web, with a hardware router behind the modem. That hardware router services both wireless and wired LANs, right?

    You want to set up a router behind that router? You still won't be able to monitor traffic going through that hardware router. You need to put your *nix router between the modem and the hardware router, so that you become the gateway for all traffic going to and from the internet.

    Of course, that is still not satisfactory if you wish to monitor traffic within the LAN. For that, you want to eliminate the hardware router entirely. Install the hardware to make your *nix router serve the WIFI and the wired LAN, and eliminate that hardware router entirely.

    You can only monitor and control traffic that is being gated through your router, so you want it ALL to be routed through your box!

    --
    "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
  47. OpenBGP with management interface? by fibrewire · · Score: 1

    Are there any OpenBGP that i could tie into a web management interface like Ubiquiti's AirControl?

  48. no DD WRT by fibrewire · · Score: 1

    Actually this isn't a bad solution, the interface is acceptable - but no real routing power behind it. Am i mistaken?

    1. Re:no DD WRT by bartwol · · Score: 3, Informative

      In my experience, I think there's something to what you say. The DD-WRT software is quite capable, but the CPUs in consumer routers are relatively slow and get bogged down when you fire up a bunch of chatty sessions, a good load of firewall rules, and try to pound data through too. Add monitoring of the router (which DD-WRT doesn't do much to support) and it doesn't take much to make the router start lagging and gasping for air. I've experienced such limitations in an office environment.

    2. Re:no DD WRT by operator_error · · Score: 2, Interesting

      The dd-wrt shop does have more powerful CPUs/throughput-hardware than is afforded by common WRT-class home routers. HOW much more powerful, or more throughput I do not know. Maybe someone else can comment, given the hardware available.

      The prices are reasonable; it seems for about $75 you can buy a outdoor-unit that will blanket an area better than a home router.

      http://www.dd-wrt.com/shop/catalog/

  49. Vyatta has licensing issues by fibrewire · · Score: 1

    Great model, but i need expensive licenses to do any real routing.

    1. Re:Vyatta has licensing issues by sleeper0 · · Score: 2, Informative

      Community Edition is free comes as a binary or full git tree, lags a revision behind. You can't buy support or prof services for it, but I'm assuming you aren't in the market for those if you can't/won't spend $800

    2. Re:Vyatta has licensing issues by fibrewire · · Score: 1

      Thanks!

    3. Re:Vyatta has licensing issues by jeffstar · · Score: 1

      wierd, don't community versions normally lead rather than lag? and the community tests and gets out all the bugs for the big paying customers?

  50. Exactly by fibrewire · · Score: 1

    How about a list of components that would be needed to, let say, take on time warner with a wireless ISP? Lets make it as detailed as possible.

    1. Re:Exactly by jeffstar · · Score: 1

      ubnt.com
      billing system
      presto!

  51. Possible Cisco option he was referring to by TheBrez · · Score: 1

    http://www.cisco.com/en/US/docs/ios/12_1/12_1xm/feature/guide/ftwrlsmc.html http://www.cisco.com/en/US/products/hw/wireless/ps2360/prod_installation_guide09186a00800d9d79.html AKA known as the Cisco WT2700 Wireless system. Which was end-of-lifed almost 3.5 years ago, so I wouldn't see why anybody would be putting in one of these systems anymore.

  52. Thanks by fibrewire · · Score: 1

    Valuable input - if you have any other ISP experience please post here. I'm going to put together a collection of ideas from this slashdot article and try to build a routing system using this information.

    Thank You

  53. Have you seen wal-mart lately? by fibrewire · · Score: 1

    Everyone seems satisified with Wal-Mart. But where is Service Merchandise nowadays? Or more recently, Circuit City?

  54. Heh... by fibrewire · · Score: 1

    Exactly.

    Thanks :-)

  55. Any advice? by fibrewire · · Score: 1

    Anything that could steer me in the right direction would be greatly appreciated
    --
    Original Poster

  56. Need help by Goody · · Score: 2, Funny

    I'm building a WISP, too. Do you think I should get a T-1 or a DS-3 for Internet? I haven't been able to decide between BSD and Linux for my router operating system. I think I'm going to go with Linux because I think the penguin mascot is cuter than that Satan mascot, but it's easier to get BSD to run on a 486 these days.

    --
    Tired of being "punished" by the Slashdot $rtbl since 2002. I'm now over at http://soylentnews.org/ .
  57. Meraki or open-mesh by witherstaff · · Score: 1

    Have you tried Meraki? Google bought into the company awhile ago and it all runs on Linux. There are proprietary bits nowadays so you can't put your own distro in place of the original code. However less than $200 for solid, lifetime warranty, outdoor gear is nice. The built in meshing control is impressive. The ranges with omni antennas are great. Also millions of users have connected to the 'net via meraki equipment according to the website. I'm currently writing this on a meraki mesh, 4th hop from the gateway, without a hiccup.

    I know it's fun to roll your own solution. If it's for your own personal needs I'd say go ahead with any of the variety of open source projects doing this. If you absolutely don't want a closed source then look at http://www.open-mesh.com/. It took the concept of Meraki and went totally open source. It's a neat idea but having transferred over a terabyte on meraki gear I'm completely happy and wouldn't want the headaches of hardware and software not backed by a commercial company.

    Good luck on your WISP venture. As anyone in the ISP field will tell you - you're gonna need it!

  58. IMHO by Anonymous Coward · · Score: 0

    Security as always as an after thought. This should be a foundation for your plans. If you don't care about exposing your customers to every half competent wardriver or script kiddy wifi cracker going then any distro and hardware can potentially do. If you wan to instill your customers with a sense of security while on your network you need to lay out the proper security framework first. We have a local wisp who is using OOB mid grade hardware and leasing tower space from local towers. One issue a client of mine recently had was mac address spoofing. They found out after bringing in the proper people that someone down the road was using a yagi and about $500 worth of hardware to get free internet. It could have been worse but it could have been avoided. Please try to think about getting the right security [i]and[/i] "good open-source/cheap hardware/software " in place if you haven't already

  59. FreeBSD, BGP, OSPF, pf by itzdandy · · Score: 2, Insightful

    Seriously, learn to love FreeBSD.

    I am assuming that you will be doing a tree style network with a central location providing you bandwidth on a fiber link or T1/T3 etc.

    Get a PAIR(at least, add more as necessary) of nice, quad core Dell Poweredge or HP DL series servers. FreeBSD+CARP them giving you as seamless load balancing/fail over as you can realistically get.
    at each hub consider either buying commercial wireless routers or build your own. If you build just keep everything fanless as that is where your equipment will fail you.
    Use OSPF on branches while being aware of scaling issues and where OSPF isnt ideal, kick in the BGP and you can link your OSPF clusters together giving an extra level on branch redundancy because traffic can hop to another branch if necessary.

    OLSR in mesh cells, OSPF on the cells backhaul router linking these cells and providing multiple route options for redundancy, and BGP between groups of cells and between you and other ISPs etc etc.

    You dont need to take the Mesh down to the client, only to the neighborhood AP level. The idea of mesh per client creates too many hopps and clients have too much latency. Ideally, you are no more that a 2-4 hops from the backbone, any more and you are going to be adding too much latency from the hops. When a backhaul link goes down and the OSPF saves your butt by routing traffic through a neighboring cell, you are already going to add latency and you dont need that complicated by 6 hops in the neighborhood and 5 more to the backbone (11 hops over wireless is just too many for broadband).

  60. Exactly what i wanted to hear! by fibrewire · · Score: 1

    I believe you may be onto something that would suit my needs for this Wireless ISP perfectly.

    Do you know of any commodity TCAM hardware?

    Maybe i can run RTLinuxFree on it?

    http://www.rtlinuxfree.com/

    and then i could put XORP on top of it

    http://www.xorp.org/

    or... i answered my own question...

    intel xscale cpus and powerpc based routing platforms (most single board computers) are all i really need.

    http://gateworks.com/
    http://routerboard.com/
    http://www.adiengineering.com/

    RB1000 from Mikrotik supports 400,000 pps and 4 gigabit ports for $700

    Take that Cisco

    1. Re:Exactly what i wanted to hear! by mysidia · · Score: 1

      If you just need to point default, a few routes for your networks and go, then great.. otherwise...

      quite a bit better than using commodity desktop components..

      However not without issues

      Depending on the nature of the ISP... most will want multi-homing, and that ultimately means taking full routing tables.

      So forwarding at a max of 400,000pps alone is not enough.

      There's also a need to take and have full routing tables at the same time as forwarding at that rate, at the same time as providing things like redundancy.

      If taking the table alone brings CPU to 100%, on an ISP border router (not that Cisco gear is entirely free of that either -- esp. when it comes to old/low-end), then.. houston we have a problem...

    2. Re:Exactly what i wanted to hear! by fibrewire · · Score: 1

      Do you know of Cisco gear that can handle full tables? prices? When I solve my routing issues i'll peel away the old private CPE addresses and get a block of public IP addresses and an AS number to go with a few DS3 connections. Then i'll be a real boy!

    3. Re:Exactly what i wanted to hear! by Glendale2x · · Score: 2, Informative

      Sure, the 2800 and 3800 ISR series can take full tables easily. You can get a 3845 starting at $10k. NM-1T3/E3 module is about $6k. Both the 2800 and 3800 take DDR-266 ECC SDRAM (except the 2801); don't feel the need to pay Cisco's prices for commodity RAM if you really don't want to. The 3845 is recommended to handle up to 2 DS3's. According to people I've asked, you can push a 3845 to 100-150 Mb/s. You can go as low as a 2811 ($2k) and still take full tables, but only at fractional DS3 speeds. I would guesstimate a 2811 is good to 10-20 Mb/s, Cisco recommends it for 4xT1.

      Also, consider that some ISP's will include equipment bundles with circuit orders if you haven't already explored that angle.

      --
      this is my sig
    4. Re:Exactly what i wanted to hear! by Anonymous Coward · · Score: 0

      For a couple of DS3s, you have lots of Cisco options.

      But (IMO) all the 'good' options at that level suck. Personally I'd search eBay for an old Juniper M7i or ancient M10 with an RE-2 or RE-3, or something, make sure OS version is old enough or new enough not to be subject to recent TCP issue.

      But my preference is because the Juni models are essentially a BSD kernel, with a hardware forwarding platform.

      The downside is very limited QoS.. but then, that goes on customer aggregation routers, anyways.

    5. Re:Exactly what i wanted to hear! by Cramer · · Score: 1

      We use a 2851 with a full DS3, and it'll move full rate traffic without a problem. However, if I config it do everything the pix does (i.e. NAT), it'll fall over and die at about 20mbps-ish. Pure routing, it'll sling line rate between it's gig interfaces. Anything that requires CPU will suffer. (it's designed to move packets, not fiddle with them in the process.)

  61. OpenBSD by DaMattster · · Score: 1

    OpenBSD has been used as a router in enterprise environments. Check out http://www.openbsd.org/ or their OpenOSPF and OpenBGP implementations. They strive to be lean, standards compliant, and meet the broadest set of routing criteria. Coincidentally, OpenBSD has an incredibly easy to configure IPSEC stack as well as tools for router redundancy called CARP.

  62. Business case! by Anonymous Coward · · Score: 0

    Your questions reveals why Cisco is still in business.

  63. Zebos from IP Infusion by Anonymous Coward · · Score: 0

    If you want a great platform that is portable check out Zebos from IP Infusion. I run it on several systems and find its features great. I honestly have no idea what it costs though but I'm sure it is less than that switch from Cisco.
    Runs on Linux Kernel 2.6 or several other OSes.

    Have fun!

  64. Anonymous by Anonymous Coward · · Score: 0

    Try Pfsense

  65. Alpine Linux by Anonymous Coward · · Score: 0

    Alpine Linux, a small, secure and powerful distro with good quagga/bgpd support.

  66. vyatta -- what is it worth? by Anonymous Coward · · Score: 0

    I looked at Vyatta, but didn't see what it offered that quagga didn't. There was also no way to install it from within an existing debian distro, even though it's based on debian. Vyatta corporate sales people couldn't provide me any benchmarks against other routing solutions, saying they don't believe in benchmarks. Not really the best way to make a sale. So they didn't. But perhaps someone here can offer some reason to use it instead of deb/quagga.

    As far as hardware goes, when you are building an x86 based router, make sure you have good ethernet cards. Run GigE if possible, even if you aren't running that much data through it, you'll appreciate the lower latency. Use reliable equipment - parity ram and lots of it, raid, well-tested, server grade boards (and not some cheapo $100 desktop board). Better to use an old xeon over a newer CPU that doesn't have much cache. Run as little extra software as possible on the router...i.e. have another server for firewall / IDS / monitoring. Keep it simp,e ya know? Of course, if rack space is costly you might want to reconsider that special-purpose setup. Document your config files by using comments, with your initials and the date. Every route should have comments indicating what it is for.

  67. Linux based routers by ImageStream by Anonymous Coward · · Score: 0

    I work for an ISP and we are using linux based routers from ImageStream (http://www.imagestream.com) or you can simply use any distro and configure it according to your needs.

  68. astaro by Anonymous Coward · · Score: 0

    Astaro Community is free, linux based, germab developers, it's great and it checks for updates. the limit is only on the number of hosts.

  69. Astaro Security Gateway is what you need by Apple+God · · Score: 1

    FAR cheaper than Cisco and not a self sustaining ecosystem like Cisco. (Cisco charges tech/company to learn, tech/company charges you for the device that you cannot configure, tech/compay charges you etc, it never ends cause you can't configure them) The reason that companies LOVE to sell Cisco gear is they will have someone on staff to configure it, and that is where they make their money. Not many people have the ability to configure Cisco routers, web interface on them or not.
    SO, if you purchase one, you will be forever paying when something needs to be changed or upgraded. The only people that buy Cisco gear anymore are those that are afraid to be responsible for their choice. As a consultant I have heard the speech several times, CIsco must be the best, its what everyone uses.
    Let those that truly understand their job be the ones to lead the way out of the world we live in where companies like Cisco get you from every end after the purchase.
    From a technical standpoint, they do have a good product that functions well, there is no reason for it to be so difficult to manage though.
    I do miss the days of Novell when a GUI was not on every computer, but thats not where the industry is now, everything has a GUI to configure it
    Yes, I can trudge my way through a Cisco, and yes it takes me much longer than someone who is a CCNA or just had training on a specific router.
    I can do it much faster in an interface like an Astaro.
    I also put in a vote for FreeBSD if Astaro is not an option for you.

    --
    Women and Alcohol are good seperatly, but mix 'em and they turn you into a dumbass
  70. PFSense by SgtChaireBourne · · Score: 1

    You are aware that PFsense is 'just' a crunchgen binary. It's a great distro and even has a packaging system, but the routing question could be answered with putting PF, an sftp subsystem like from dropbear, and BGP into a crunchgen binary using NetBSD or OpenBSD rather than FreeBSD. It's only hard the first time, if you keep notes. Even then its not so daunting.

    --
    Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
  71. Astaro by Anonymous Coward · · Score: 0

    Try Astaro

    http://www.astaro.com/

    It's not free but they offer hardware and software solutions. They are open source based and are a good FOSS player. They seem to have products that should be able to scale up to what you want. I got a copy running at home and it works great.

  72. VYATTA by Anonymous Coward · · Score: 0

    Try vyatta, it can be usefull.

    Regards

  73. Potential solution in an iso called IPCOP by Anonymous Coward · · Score: 0

    Have you tried IPCOP? http://www.ipcop.org

    The web site isn't flashy, but stick the disk onto a machine with 2 network cards and you've got yourself something that can handle a hell of a lot more than dd-wrt on a Linksys!

    The GUI isn't bad either.

  74. SNMP Nagios Cacti by Anonymous Coward · · Score: 0

    If monitoring is what you want, why don't you use SNMP, Nagios, & Cacti on a Linux server attached to your system? SNMP is supported by Microtik and Ubiquiti. Setting up SNMP and Nagios isn't the easiest thing in the world but they're free and work very well.

  75. Have you looked into Vyatta yet? by bmullan · · Score: 1
  76. You're missing the point. by cbbs70a2 · · Score: 1

    You're missing the point of what the purpose of a router is. A router is supposed to copy a packet from one interface to another based on certain routing criteria as quickly as possible. That's it. Period. Nothing else. All other services need to be placed elsewhere in the network based on the tiered network model of core, distribution, and edge. Otherwise, you will never have a scalable, redundant network.

  77. Check out Vyatta dot com by Anonymous Coward · · Score: 0

    Check out Vyatta dot com at http://www.vyatta.com/

    1. Re:Check out Vyatta dot com by Anonymous Coward · · Score: 0

      Vyatta.org is the correct link.

  78. The real question by rootforce · · Score: 1

    The real question you should be asking yourself is if you really want to get into the wireless ISP game. It is pretty late in the game at this point with CLEAR rolling out WiMax service(yes I have heard the negative reviews), docsis 3 and fiber to the home solutions becoming more prevalent. It is a rough time to be getting started. That being said I agree with some of the previous posters that your core equipment should really be Cisco. Think used 6500s and 7200s. I know that a 6500 with sup32s can handle multiple hundreds of mbps of traffic for thousands of users, but it depends on your usage scenario. In this case I would keep the Mikrotiks at the edge as it is quite the mature platform(not that I haven't spent countless nights trying to figure out why it behaves the way it does). If you absolutely must run open source for your core, investigate Vyatta on high end hardware. It is not as mature, but the community is very good and helpful. You might also consider the zebra project for just straight routing.