Slashdot Mirror
Can You Purchase Switch Hardware Without an OS?
dhahn asks: "I have a project where I'm building a large Linux router (about 40 ports or so). At this point, my only hardware solution is to purchase a box with lots of PCI-ish slots and fill them multi-port ethernet cards. I've looked into currently available solutions and haven't found anything that gives me the control I want. Does anyone know of where I could purchase a 'naked switch?' I just want the switch hardware with enough guts to allow me to customize a Linux OS and load it up." If anyone else has been in this situation, what did you do?
Why not use any managable switch, configure each port into its own vlan, hook up the Linux box to a trunk port and use Linux's vlan support, like anyone else does?
if you would like i can buy a switch and flash it with nothing and sell it to you.. but money frist and no returns.
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
I'm not sure if I'm familiar with the 'ish' variant of the PCI slot. Could you please elaborate?
This guy's the limit!
Take a look at the local computer stores, see if any resell used computers. A router doesn't need much power usually, so you might be able to find a bare bones system that fits your needs. Sometimes you can find old server hardware for sale: lots of space for memory/drives/cards.
'Sensible' is a curse word.
Limitations:
p lane/backplane.HTM
- PCI bus bandwidth is going to hurt you hard. 32-bit PCI @ 33Mhz = 127Mbyte/sec. 64-bit PCI-X @ 66Mhz = 508Mbyte/sec.
- 100Mbit ethernet = ~10Mbyte/sec (assume 10b8 encoding, easier numbers).
- 127Mbyte/sec / ~10Mbyte/sec = 12 100Mbit ports only.
If you aren't deterred by this:
1. Get a motherboard.
2. Get a decent PCI backplane. A quick Google search brings this company:
http://www.commell.com.tw/Product/Peripheral/Back
and they have a backplane with 17 PCI slots.
3. Buy 4-port PCI 100mbit network cards (http://www.americanpredator.com they don't list it on their site, but I'm certain they do custom quad port cards, or can point you to somebody that can, $500/card for industrial grade hardware).
4. 17*4 = 68x 100Mbit ethernet ports.
ICQ# : 30269588
"I used to be an idealist, but I got mugged by reality."
Even if you could buy just an "OS-less" switch, I don't think it would do you any good. Most switches have hardware fast-paths for switching packets. You'd have to completely disable the switching logic, and redirect all frames through the host CPU.
Some of the switch ASICs I'm familiar with [medium range broadcom, vitesse] are in fact slower at sending a packet through the host control interface, than at simply switching it to a port on which a host cpu might be connected. [Reference designs from the above have the host CPU connected to the host interface, and control packets, ssh, telnet, http, depending on the design captured and sent through it]. In that case, you'd need your host CPU to be connected to one of the ports of the switch, and then of course your routing speed is limited to the maximum speed that can be sent through a single port.
One of the posters above me mentioned buying a managed switch and using VLAN's, thats what I woulda suggested had he not beat me to it.
Good Luck!
Forget your hackish idea. Get a Cisco 3750-48 style switch and all will be well.
Trolling is a art,
Silicom http://www.silicom-usa.com/ has 6 port gigabit cards in PCI-E (PCI Express) and PCI-X variants. Try to find a motherboard with as many PCI-X slots and PCI-E slots that are x4 or greater; the cards do not fit in PCI-E x1 slots. You may not find a motherboard with enough slots, or enough fast slots. Most motherboards now come with 2 on-board ethernet ports which will help. And if you find you need more cards in your 'switch' then try this PCI expansion System I found online: http://www.mobl.com/expansion/products/pcie_expans ion/6slot/index.html it only needs one PCI-E x4 slot on your motherboard and it gives you 6 PCI-X slots
The poster leaves a lot of stuff out - like what's the bandwidth per port? Are they routing analog dial-up lines or gigabit ethernet? What protocols are they routing? Do they need ACLs? How many? Other filtering? Proxying? And if the installation is really that big, just lay out the $jack for a layer-3 or higher switch or router or firewall or whatever.
Not to be a dick, but if the poster has to Ask Slashdot about this sort of configuration, he or she has no business messing with this and should leave the design and configuration to grown ups (unless it's a lab experiment or something).
Help save the critically endangered Blue Iguana
You want to build a router, but you are looking for switch hardware.
You are asking for independent ports. If you need to route through each port seperately and not 'switch' data between ports but 'route' it among them then you need router hardware not switch.
See the thing with switches is that chips are available with 4 ports or 8 ports and it automatically switches data in ASIC between ports. Usually these chips cannot be interfaced to a microcontroller and almost never have PCI interfaces.
You do need individual ports, not a switched collection of ports. So you need something with 7 PCI ports (7x4=28). There are plenty of 4-port PCI cards out there, but there must be 8-port cards too. I have seen plenty of 6-port motherboards. You will have to use PCI extension devices to get to 7 or 8 ports unless you find those 4+ port ethernet cards. Do keep in mind you cannot switch between all ports at wire speed. You'll need faster busses and powerful processors. At this point you're looking at highly specialized hardware like cisco juniper etc.
It is inefficient to route between that many ports on a single CPU. Its better to cascade entire routers if your design allows it or add switches to routers with fewer ports. Unless you are a telco providing high speed connections to ISPs or a central location breaking the bandwidth for many branches, I dont see why anyone will need a router with 30 ports. In any application when you need more processing power, you'd divide the algorithm and use multiple CPUs or multiple computers. In this case you can almost definitely use cascaded routers if you need that many ports in the first place.
I have a Cisco 4700M router with 12 10-mbit ethernet ports. Never needed more than 3.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
Are you routing or switching? What purpose do you have that requires linux specifically? Are you just anti-Cisco, anti-brand name?
:)
You specify that you are building a router then request a switch hardware. What level are you managing your network at? What kind of throughput are you trying to get (10/100, 100 full, gigabit?) Are you managing at the port level or at the IP level (switching vs routing)?
Elaborate and maybe then we can get you some answers
Browsing with classic discussion, noscript, at -1 and nested
no hidden comments and I only mod UP
If anyone else has been in this situation, what did you do?
It is hard to understand exactly what the poster needs since he is leaving a lot of crucial information out of his setup, but in most cases it would be prudent to buy a product that is designed for that job, such as a layer 3 switch. If he wants to run Linux just for the sake of running Linux, he is going to have problems, and needs to find a product to get the job done. I would suggest investing time looking into something such as Cisco 3750 series switches.
Build the router on a nice 1u dev platform with maybe 8 ports on it and then use cisco gear for the switches. (you can find several in Linux Journal pages for cheap.
Why in the world would you want a router + 48 port switch all in one? so when you take the router offline the whole network crashes?
Do not look at laser with remaining good eye.
I am in about the same situation. We ended up going with the RPI-1245 for 12 ports in 1U (by adding a 4-port PCI ethernet card). The solution isn't ideal because only the 4 ports on PCI Express are actually provided with enough bus bandwidth, but it's close enough for us. More ports would be nice, of course. The remote management and the BIOS in general isn't as polished as the HP DL140GL2 we use for everything else.
For our next deployment we will likely go with a 1U switch combined with a 1U HP server, and use VLAN's. The problem is that VLAN's then have to be unique across all ports. If someone decides to make a PC in at most 2U with 24 ethernet ports all on PCI Express, we would certainly go for that. At 4U or bigger you can fit more cards in the machines, but bus bandwidth is a problem if you stick with PCI. PC's with 6 PCI-Express x 4 or 6 PCI-X busses are kind of rare.
Finally! A year of moderation! Ready for 2019?
Informative. Just a back of the napkin calculation and a few seconds thought says this will not be nearly as easy as just throwing a bunch of multi port ethernet cards into a PCI bus, nor just naively running linux on some random switch hardware. When you are dealing with that much data, you need tight integration of the hardware and OS. A managed switch with VLANs is definitely the way to go.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Get a normal Linux box with 1 or maybe a few ethernet ports (you can bond them together, if you like), and then connect a 48-port switch that supports VLANing. Set up the ports to the Linux box to pass all 40+ VLANs tagged, and then set the ports that are not connected to the Linux box as untagged ports. You now have each of the switch ports effectively as an interface on the Linux machine.
I've done this for cases where I needed a small machine to run with more ethernet ports than it's actually got. Works great.
Sean
Depending on how adventurous you feel about getting linux running on strange hardware, you could buy a used cisco or foundry L2/L3 box for cheap. I know there are a lot of linux projects at cisco, there must be something you can google about how they went about compiling for the platform. The CPUs inside will be something non-mainstream, i.e. not a pentium, but chances are there is a linux distro for it. It shouldn't be that hard to find some archived info on how to bootstrap a linux distro onto one of those boxes.
Used cisco 3500s or 2950s with 24 or 48 ports are on the market for a few hundred (dollars or euros) each. Foundry workgroup switches are less than 100 euros right now. Cisco 7200s are just PCs inside, but their PCI buses are a different layout to allow hot swapping. Cisco Pix 515s are just commodity 1U intel pc motherboards, cisco didn't even bother doing a redesign to remove the superfluous connectors.
If you have enough money for a PCI-ish box and many quad ethernet NICs, then you probably could afford a used Juniper M5. It already runs BSD, and pretty much looks like standard PC hardware inside. A used M5 without any interface cards should be had for less than a new PC, its the interface cards that will cost you dearly.
If you follow my advice, then with any luck you will document everything you did along the way, and release a linux distro for some otherwise proprietary hardware. I'd like to see a cisco 2950 turned into a linux box with all kinds of extra linuxey features. What I'd love to see is openBSD's pf on a switch, so I could set per port ACLs and bandwidth shaping.
the AC
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
Hardware switches can look straight at layer 2 encapsulation and shuffle packets based on that information. The switch's main CPU never sees the packet. The "naked" version of the switch still has enough main CPU in it to program the switch controller chip(s). More "clothed" switches have the spanning tree protocol, SNMP bits, and any other cool features you might have.
Routers have to look at layer 3. Back when I was writing code for a major switch/router manufacturer, most switch chips didn't pass the (de-encapsulated) packet up to the main CPU (or back down). The chip only gave notification that a packet arrived (etc) in the form of an incremented register. This meant the chips were unsuitable for routing because the main CPU had no visibility into the packet whatsoever.
Either of these designs generally require a separate Ethernet NIC for the main CPU, as the switch ports are too busy with external connections. That NIC might be connected to the switch chip on the switch's main board, or it might appear externally as a "management interface port." You'll be programming this NIC, too-- but hopefully, just with ifconfig(1M)
Suggestion: Look for commercial switch chips that can pass the packet to and from the main CPU. Find a company which has a COTS switch with the combination of your favorite switch chip and a CPU that will run your Linux version (or uCLinux). Make sure the implementation hardware is wired properly to be able to get the packets to and from the main CPU-- your favorite chip might have a separate HW interface for communicating with the main CPU that could be unconnected in the implementation hardware. At this point, you essentially have the naked switch that can route. Learn how to boot Linux on the implementation hardware and build a flash filesystem that the switch's bootloader will read. Then start writing code to add the capability you require (to routed?).
Suggestion: Once it works, sell the thing. Or open the source up for others. You went through a lot of trouble to get that capability in the system, and it's got to be so cool because it isn't already in a commercial router. You might consider selling it on the open market yourself, or finding someone to sell it to. Or drop it on Sourceforge for others to upgrade.
Suggestion: Or, you can get your cool feature embodied as an RFC and get the main router vendors to build it in. Or contribute the code to routed(or whatever). You can probably skip the effort of building this one-off switch/router.
[Error 407: No signature found]
Many people have pointed out reasons why this kind of thing is probably a bad idea, but if you still want to do it, ImageStream sells hardware platforms that can include several multi-port ethernet boards. There are reasons why they are marketed as routers and not switches, but they might be useful as switches for some unusual purpose.
You could buy a reference platform kit from network ASIC manufacturers. I know of the Broadcom XGS ones (chips that do L3 routing, L2 switching and ACLs in hardware), as my day job is at a company that uses these to do switch/router application software. The software's proprietary, of course, but Linux does run on those boxes.
That's probably many kilobucks, though, and you'd face the task of dealing with the awfully complex chip to get it to do what you'd want.
Another option would be to buy/license LVL7's software, of course :-), then build on top of that whatever custom application you'd need, to run alongside the software doing the switch/router stuff. Also many kilobucks, so not the sort of thing you'd do to save money, but the sort of thing you'd do to start a company selling boxes that do whatever it is you need to do that commercial router boxes don't.
AFAIK Apple only sells their hardware with OS X. And if you bought it without an OS, you probably wouldn't be switching. Also, the "Switch" ad campaign has been replaced with the "Get a Mac" ad campaign:
http://www.apple.com/getamac/
Sent from my iPhone
If you go to www.linuxdevices.com and look around, there are several vendors who sell sbc's that have ethernet switches as backplanes, you can add as many as you want. The underlying cpu is either a strong arm, or x86 compat cpu, some have mini-pci slots, etc. And they all run linux!
A 40-port router or switch that has typical office workstations with typical "office" use doesn't require a HUGE amount of power. Most users will drive minimal traffic during office hours except when accessing network resources, which will probably be bursty and random in nature. Realisticly, an office like this would probably have a 48- or two 24-port switches costing a few hundred dollars each, with a router to manage external traffic.
Now, throw half a dozen servers on there along with a few machines that are accessing the outside world 24x7, and now you need some beef and the corresponding dollars.
If it's 40 servers in a server farm, then you are absolutely correct, the user should be prepared to spend big because a cheap router or switch or PC-based solution just won't do the job.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Yeah, unless you really have a need to - customizing hardware is expensive, difficult, and prone to failure. Usually when I see questions like this, I assume they are from someone that really doesn't understand a problem and has decided to go down a path. Once you look inside and realize what the REAL problem that they are trying to solve these are pointless excersizes (why do you have to run Linux on a switch - many of them all ready run Linux). What are you really trying to do - why, and what are your limitations
I have mod points and I am not afraid to use them
This is vastly different than what you need/want, but you might find it interesting, I got nine 10/100 ports on a Sun Ultra 1, and the bandwidth is very nice:
http://lfnet.net/blog/?p=41
I've been using these for a couple months(released in july), they are pretty neat. Runs linux(though you can't get to it. .), 1U formfactor, line rate all ports, but the best part is it offers "core" features like BGP, OSPF, hardware IPv6, VRRP etc in the 1U formfactor, it's nice that I can finally use identical hardware for both the 'edge' and the 'core'. I haven't seen/heard/read of any other product on the market that comes close(performance, features, formfactor, price). my own network designs utilize their ESRP redudancy protocol, while extreme specific it works great with other products as well(have several cisco 10/100 switches and some F5 layer7 switches, failover within a couple seconds, both layer 2 and layer 3).
they list for about $9k(48-ports, 256Gbs switching fabric), though even in small quantities you should be able to chop 30% off that from a reseller. the price is the same as extreme's more mainline basic layer 3 switch the summit 400(which is comparable to some cisco, foundry and force10 products). redundant power will set you back another ~$900(list).
you don't give a clear indication of what your end goal is, but I really refuse to buy any network equipment unless it heavily utilizes ASICs. The leaders in this space to my knowledge are extreme, foundry, and force10. with foundry having the fastest 'raw speed'(on paper anyways), force10 having the highest density per chassis, and extreme having the best feature set(IMO of course, in the high end - layer 3 virtual routers, clearflow, ease of management).
these switches do draw a fair amount of power though I was kind of suprised, more than 3x the power of the more basic Summit 400. Probably more power than any other 1U switch on the market (~400 watts). But it's worth it for me anyways.
They basically took a 'blade' from their blackdiamond 8800 series chassis and slapped some management chips on it and put it in a 1U chassis.
(posting as AC since I don't have an account, I read slashdot daily but post maybe once every 2 years - aphro ^at^ aphroland ^dot^ org if you want more info)
(offtopic) Yeah, it's AC, but this is good info. :)
Dude, get an account... you've got nothing to lose, and it's free. Plus, if you read often and meta-moderate, you can get mod points and mod up those useful posts
-1 raving lunatic; +6 subGenius... Things even out...