Slashdot Mirror
Policy-Based Routing Using Software Firewalls?
Bios_Hakr asks: "My local computer group meets for monthly LAN parties. The location that hosts the parties also has a small internet cafe. After the cafe closes, they allow us to connect to their T-1 line. They supply us with a single IP address which we NAT/(PAT?) via a Linksys DSL router. We also have a second T-1 supplied by one of the more gracious members of our group. He agreed to supply this T-1 after experiencing abysmal ping rates with 30 people sharing the bandwidth. Herein lies the quandry: How can we implement Policy-Based routing for our LAN? I'd like all HTTP and FTP to be directed out one line while popular gaming ports are directed over the second line. All file-sharing traffic should be killed. I know how to do this via Cisco IOS and policy-route-mapping, but I'm at a loss when it comes to doing it via software firewall solutions. We have several Linux-familiar people in the group and lots of Windows geeks; but the solution should be simple and require zero brainpower to set up after the initial implementation. How would you split your LAN traffic across two T-1 lines?"
Especially in kernel 2.6 there are tons of funky stuff you can do in iptables.
Just reroute the traffic based on port number. You can direct it to a specific interface. And just drop the packets to certain file-sharing ports.
"man iptables" and other RTFM shit.
shorewall firewall can handle DNAT requests, as for packet shaping, there are several solutions out there. a friend of mine runs a router like this: Red Hat 8 Shorewall traffic shaping routing a T1 and wireless T1 equiv. to a single lan. it requires a bit of thought, but you can setup the router to forward everything below port 1024 requests out connection A and everything above out connection B (that keeps the web/ftp/pop/ssh/telnet stuff on one line, games on the other, though you might want to route some of the less popular/bandwidth intensive stuff (RTS's for example) out A as well. btw: this pal of mine runs the local LAN/computer group around these parts, i'll alert him to this thread, who knows, you may get some help.
Logistical Chaos Officer http://www.slagg.org - LAN Gaming in Sarasota FL,USA
You can do static routes to particular networks, but what you REALLY need is BGP routing to your two ISPs. Count on a router with AT LEAST 128 megs of memory to do this, and a high end CPU on the router to manage all the BGP tables. Even this will not give you load balancing, just best path routing.
pf w/integrated altq. setup authpf for those users that need to bend your rules a little
it's magical. it "just works"
lots of good examples in the man pages too
throw it on a soekris and toss the linksys =)
vodka, straight up, thank you!
Please visit http://linux-ip.net and more specifically for your problem: http://linux-ip.net/html/ch-advanced-routing.html
That should get you started. It's be no means simple, but my understanding is that once you get it up, it works.
-jay
Linux can handle this with little problems these days. Its a little technical, but you can basically do it with a combination of:
iproute2
iptables & Patch-o-matic
netfilter CONNMARK extension
You have the matching power of iptables to implement any sort of policy routing that you could ever dream of!
Bye!
PS: if you need help on actually how to implement this, see the archives from either LARTC &| Netfilter
Bye!
OpenBSD comes out of the box with a great firewall (that will also handle your NAT). The firewall can easily handle packet queueing and prioritization. Tell the firewall how much bandwidth you have to work with, set your host up with priority over your traffic, even break it down by protocol if you want.
Basically, you'd be looking at doing the following things. Multiple outbound providers, which will need another routing table built for the second link. Then you'll need to dive into QoS to split up your traffic into your definitions of bulk (HTTP, FTP), priority (Gaming), and drop (P2P). I notice that you have no default set up, but I leave that up to you. Finally, you can use iptables to mark and NAT your traffic out the right interface.
Under Windows, you would need some advanced routing software I think. ISA may do it, but I doubt your budget allows it. By default, Windows does have the ability to enforce QoS terms, but you'd need something to apply those QoS marks (I doubt that games commonly mark their packets with ToS)...which means a bridge in front of the Windows router. Might as well use a Linux router instead.
If anybody knows of a way to get a Windows box to route based on ports, I'd love to hear it.
Oh, and a simple solution for the exact problem you describe (which I don't think is what you really want) would be a proxy for the HTTP and FTP link, and a router for the other link. All HTTP and FTP requests would be sent out the proxy, everything else would go the default route (to the router) which could be configured to drop P2P and route everything else. Optionally, you could do QoS on the router to prioritize certain traffic. If you go that route, I'm fond of AnalogX Proxy (for Windows) because it's free and simple. Of course, that does require client configuration....unless you use Transparent Proxying.
There are a bunch of areas in Linux that can help you. Only some of them are routing based.
The first thing I would try would be to setup one of your lines with 'tc' and bandwidth shape the line with CBQ and SFQ. CBQ will let you set the outbound "rate" for the line, and SFQ will enforce "fairness" between different "connections". This should keep ftp uploads from swamping upstream traffic and pushing your ping times thru the sky. You can do some similar things with 'tc' ingress policies to shape the incoming traffic, but this is less effective.
If you still want to try two lines, here is the basic setup.
You need a Linux box that has three network interfaces. One for each of your T-1s, and one for your local LAN. The Linux box's IP address is the default gateway for everyone on your local LAN.
You setup a firewall on the Linux box with something like:
LAN on eth0
T1 on eth1
T1 on eth2
iptables -i eth0 --dport 80 --state NEW,ESTABLISHED --set-mark 1
iptables -t nat -o eth1 -j MASQUERADE
iptables -t nat -o eth2 -j MASQUERADE
ip ru add fwmark 1 table 10
ip route add default via IP_ADDRESS_OF_T1#1 dev eth1
ip route add default via IP_ADDRESS_OF_T1#2 dev eth2 table 10
This is far from complete (and I haven't tested it), but it should set "fwmark" to 1 for HTTP traffic. The router table should then take traffic with FWMARK set to 1 and use routing table 10 instead of the default table, which can have a different default route. In that both eth1 and eth2 are MASQed, both will NAT.
You will need a lot more here to be fully functional. You need to completely filter the traffic you don't want, and probably classify a bunch more stuff along the way.
Good luck.
See:
/etc/iproute2/rt_realms)
http://hibernia.jakma.org/~paul/rc.iprules
For a script that does something similar to what you want, policy routing to route based on source IP. It should be easy enough to add an additional 'firewall mark' field to the table and policy route based on that (i'm on holiday, otherwise i might have done that for you). The listed "intranets" will use the main table.
Basically, all you need is:
1. create a table for each policy (edit
2. use iptables to add arbitrary 'fwmarks' to incoming packets based on whatever criteria you have
3. use the 'ip rule' command to direct routing for packets with specific fwmarks to specific routing tables.
4. direct other traffic to the default 'main' table.
Finally, see the Linux Advanced Routing & Traffic Control site for further information.
I use Friend/Foe + mod-point modifiers as a karma/reputation system.
if the linksys itself is running linux,
and you can replace the system image
(tutorials exist for linksys routers)
then you can easily write a shell script
to select the outgoing connection by
destination port using a couple of iptable commands. No additional hardware is required.
-I like my women like I like my tea: green-
Search sveasoft on google and look up their aftermarket firmware..
A lot of people got down to the nitty gritty technical details, but as I understand you want something simple that just works. Well, I use a Linux Firewall distro to do the routing in combination with a small script to configure the QoS.
Try Clarkconnect in combination with Wondershaper. Wondershaper uses some basic input parameter to configure the kernel to traffic prioritization. I found it very easy to define my available bandwidth, what services require a higher or lower priority.
You will need to modify it to suit your particular set up, but it should be fairly obvious even to people new to perl.
.sjoins('s',s@ARGV);ssesGetopt::Long;sGetOptions(s "log"s=>s\$log_opts"help"s=>s\$help,s"ip=f"s=>s\$f irewallip,s"nat=s"s=>s\$nat,s"reserved"s=>s\$reser ved,s"executable"s=>s\$executable,s"queue"s=>s\$qu eue,s"paranoid"s=>s\$paranoid,s"trusted=s"s=>s\$tr usted,s"match"s=>s\$match_opts);sifs(s$#ARGVs<s0s) s{s&usage();sexit;s}swarns"WARNING:sIscan'tsfindst hatsinterface\n"sunlesss(greps/$ARGV[0]/,s`/sbin/i fconfigs-a`);s$EXTERNAL_INTERFACE=$ARGV[0];sdies"C annotsdeterminespathstosiptables:ssetsmanually"sun lesss(chomp($iptabless=s`whichsiptables`));s$logs= s();s$portmatchs=s();s$SERVICES="/etc/services";s% seens=s();s$DATE=localtime;sprints"$match_opt\n";s $logs=s"1"sunlesss!s$log_opt;s$portmatchs=s"1"sunl esss!s$match_opt;sifs($firewallip)s{s$DITTY="-ds$f irewallip";s#ssetsfirewallsipsifsgivens}selses{s$D ITTY="-is$EXTERNAL_INTERFACE";s}sifs($help)s{s&usa ge;s#sdumpsusagesdittysifs-hsexits1s}sprints"#!/bi n/sh\n"sunlesss!s$executable;s@RESERVED_IANAs=sqws (0.0.0.0/8s1.0.0.0/8s2.0.0.0/8s5.0.0.0/8s7.0.0.0/8 s10.0.0.0/8s23.0.0.0/8s27.0.0.0/8s31.0.0.0/8s36.0. 0.0/8s39.0.0.0/8s41.0.0.0/8s42.0.0.0/8s58.0.0.0/8s 59.0.0.0/8s60.0.0.0/8s127.0.0.0/8s169.254.0.0/16s1 72.16.0.0/12s192.168.0.0/16s197.0.0.0/8s224.0.0.0/ 3s240.0.0.0/8);s@RESERVED_MINIMALs=sqws(127.0.0.0/ 8s192.168.0.0/16s172.16.0.0/12s10.0.0.0/8);sprints "\n#schainspolicies\n",s"#ssetsdefaultspolicies\n" ,s"$iptabless-PsINPUTsDROP\n",s"$iptabless-PsOUTPU TsACCEPT\n",s"$iptabless-PsFORWARDsDROP\n",s"\n#sf lushstables\n",s"$iptabless-F\n",s"$iptabless-FsIN PUT\n",s"$iptabless-FsOUTPUT\n",s"$iptabless-FsFOR WARD\n",s"$iptabless-Fs-tsmangle\n",s"$iptabless-X \n",s"$iptabless-Fs-tsnat\n";sprints"\n#screatesDU MPstable\n",s"$iptabless-NsDUMPs>s/dev/null\n",s"$ iptabless-FsDUMP\n";sifs($log)s{sifs($queue)s{spri nts"$iptabless-AsDUMPs-jsQUEUE\n",s"$iptabless-AsD UMPs-pstcps-jsREJECTs--reject-withstcp-reset\n",s" $iptabless-AsDUMPs-psudps-jsREJECTs--reject-withsi cmp-port-unreachable\n",s"$iptabless-AsDUMPs-jsDRO P\n";s}selses{sprints"$iptabless-AsDUMPs-pstcps-js LOG\n",s"$iptabless-AsDUMPs-psudps-jsLOG\n",s"$ipt abless-AsDUMPs-pstcps-jsREJECTs--reject-withstcp-r eset\n",s"$iptabless-AsDUMPs-psudps-jsREJECTs--rej ect-withsicmp-port-unreachable\n",s"$iptabless-AsD UMPs-jsDROP\n";s}s}selses{sprints"$iptabless-AsDUM Ps-pstcps-jsREJECTs--reject-withstcp-reset\n",s"$i ptabless-AsDUMPs-psudps-jsREJECTs--reject-withsicm p-port-unreachable\n",s"$iptabless-AsDUMPs-jsDROP\ n";s}sprints"\n#sStatefulstable\n",s"$iptabless-Ns STATEFULs>s/dev/null\n",s"$iptabless-FsSTATEFUL\n" ,s"$iptabless-IsSTATEFULs-msstates--statesESTABLIS HED,RELATEDs-jsACCEPT\n",s"$iptabless-AsSTATEFULs- msstates--statesNEWs-is!s$EXTERNAL_INTERFACEs-jsAC CEPT\n",s"$iptabless-AsSTATEFULs-jsDUMP\n";sprints "\n#sloopbacksrules\n",s"$iptabless-AsINPU
#!/usr/bin/perls-Ws$command_lines=s"$0s"s
dumbass
Set up Squid (or something else if you prefer) on a box that faces the first T1. Block port 80 going out on the second. Tell everyone that they have to set up their browsers to use the proxy or it won't work. For bonus points, do some transparent proxy stuff on the gateway system and just force their web hits to go to the Squid box.
Squid doesn't help for raw FTP, but you can still use it as a FTP proxy if you access it through a web browser. You could also some some masquerading and route mangling to send port 21 stuff out through the first T1. You need iptables to masquerade things to the gateway's address, plus iproute2 to make it use a specific router. Unfortunately, iproute2 has the syntax from hell, so it'll take a while to figure that out.
All the talk about BGP and filtering and so forth is technically correct, but it's seriously complicated for what you want to do. Handle your biggest troublemaking protocols with some application level proxies and you can spend the rest of your time playing games instead of pulling your hair out with EBGP, getting the ISPs to play nicely, dealing with an AS number of your own, flapping routes, and so on.
If your users are being bastards and are going around your port blocking, consider blocking everything but your gaming ports on the second T1. That will force them to use the proxy to get to the "other" services.
Yes, that's going to work, but for the purposes of the guy who asked the question, it really is overkill.
:)
The problem is that they get shitty pings when gaming because web, ftp and p2p traffic is swamping the link. The simplest solution is simply to route all gaming traffic out one T1 (by IP) and route everything to every other IP out the other T1 and block p2p with iptables.
The way I see it, you're solution really doesn't accomplish anything greater in terms of more bandwidth for gaming (gaming traffic is only ever going out the one T1 anyway) and requires more processing power on the router. As long as you assume that any traffic to the IP address of the gaming server is going to be gaming traffic, then you maximise the whole T1 using this simpler method.
On the other 'miscellaneous' T1 all you need is iptables to block anything p2p related (and of course anything else undersirable) and you're set.
So simply using IP based routing, you can ensure that only gaming traffic goes out one link and everything else goes out the other. You can even start looking at some traffic control on the miscellaneous T1 if you want to make the sharing of that link fairer and knock up a quick script to easily permit the addition of new IPs to the gaming server routing list.
I'd certainly try it and see if it achieves the goal rather than possibly going for the overkill of traffic control. I mean even if people do start hitting the game server IPs on other than gaming ports, you just block that in your iptables script.
I'd be interested to hear your analysis. I haven't been in the field for a couple of years, and can't see why it wouldn't but would like to be corrected if I'm wrong
Also, as one of your T1s is only available at certain times of the day, a *nix solution has another advantage. Once you get your configuration working, establish cron jobs on the router to stop and start the routing over the time-restricted T1.
He who laughs last is stuck in a time dilation bubble.
http://www.openbsd.org/faq/pf/pools.html#outgoing
1% APY, No fees, Online Bank https://captl1.co/2uIErYq Don't let your $$$ sit in a no-interest acct.
Time to let your fingers do the walking...
Linux Advanced Routing and Traffic Control
I know this stuff is dense, but I happen to think it's stuff that any serious Linux admin should know about eventually, so I spread the word. If you want some pointers on where to start, send me an IM. I'll be at work all day today more-or-less.
"The Devil does not know a lot because He's the Devil, He knows a lot because he's old." -- unknown
How would you split your LAN traffic across two T-1 lines?
Very happily!
-------------------
First, split the traffic up with iptables, tagging "interface 2" traffic with a fwmark.
/etc/iproute2/rt_tables for each table)
/etc/iproute2/rt_tables /etc/iproute2/rt_tables
$IPTABLES -t mangle -A PREROUTING -p tcp --dport
80 -j MARK --set-mark 0x05
$IPTABLES -t mangle -A PREROUTING -j MARK --set-mark 0x06
Then, make sure you have used iproute to select the routing table you use based on the fwmark (requires entries in
echo 100 tone >>
echo 200 ttwo >>
ip route add $router1 dev eth1 src $eth1_ip table tone
ip route add default via $router1 table tone
ip route add $router2 dev eth2 src $eth2_ip table toneip route add default via $router2 table ttwo
ip rule add fwmark 5 table tone
ip rule add fwmark 6 table ttwo
There you go - stuff going out on port 80 goes out eth1 on T1 number 1, everything else goes out eth2 on T1 number 2. You may wanna SNAT in iptables -t nat (POSTROUTING) based on the fwmark, too - I don't remember if ip route will handle that (probably not).
GNU Zebra is a cisco IOS clone for linux. i think its what you're looking for.
Somewhere on this page I have hidden my signature.