Slashdot Mirror
Application Layer Packet Shaping on Linux
sommere writes "We have added application layer (layer-7) filtering to Linux. That means that you can set up your linux-router/linux-switch to prioritize mail over the web over kazaa or gnutella regardless of what port each program is using. Colleges have been paying thousands of dollars for packet shapers to prioritize their networks, now you can do it for free. Get your kernel patch at l7-filter.sourceforge.net."
This really helps networks that have smaller circuits and lots of clients doing various tasks on them. Not such a big help for a home user but great for corporations.
It's looking more and more like commodity linux boxen, with the right software, can do what your average pricey cisco box is renowned for.
If you can't beat them, arrange to have them beaten. -George Carlin
Hmm.. packet shaping.. can't wait to merge this in with the rest of my kernel and give it a whirl.. although, I do have to admit that some of the packets I've been getting are pretty nicely shaped.. there's the Ana packets, and the Kim packets.. but if this patch can help shape some of those no-so-well-shaped ones, I'm all for it!
---
Refusing to be a karma hore! Score: +5 Funny, -1 Karma Hore
In one hand, >I can prioritize what I want how I want. And it was good.
In the other hand, my ISP may downgrade my Quake performance or my school may block telnetting to my home box completely (no matter which port I put the demon on). And this was bad.
The idea is good but I'm worried it will be heavily abused and that worries me. In the other hand, it may mean a neat security tool...
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
It cost my school 15 grand for 100mbit of shaping to be exact. Try using Kazaa when there are 4 huge dorms full of students trying to access kazaa, irc, ftp, hotline and some other protcols on 150k. Not fun
Tim Smith - Ramblings from Nerd Land
This type of thing has been in OpenBSD long time now (altq) but it nice to see that this type of thing is done in linux.
The problem in the world today is communication. Too much communication - Homer Simpson
you can set up your linux-router/linux-switch to prioritize mail over the web over kazaa or gnutella
I vote for more kazaa than mail. Unless someone sends me movies by mail.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
If an attacker knows that you prioritize a certain service, wouldn't he cause a greater disruption with his DOS with this?
Another thing: couldn't the ??AA get ISPs to use this feature, not to kill P2P sharing, but to reduce its priority (perhaps as a compromise from not being able to kill P2P outright)?
Of course, there are many benefits to this as well, I'm just pointing out possiblities.
It is obvious to anyone that you could not possibly have developed such an advanced feature for the Linux kernel on your own or with the help of the community. This feature has obviously been lifted verbatim form the proprietary Unix code owned by SCO. I expect you to pay our standard SCOSource licensing fee of $150US per processor running this code, IMMEDIATELY. Failure to pay for this license within the hour is a violation of SCO's Intellectual Property rights and WE WILL SUE YOUR ASS OFF!!!!!!!!!!!!!
Darl "Sue em" McBride
How does a router know what the intended purpose/application a packet is destined for? Does not only the receiving computer actually know what applications have bound what ports?
Some drink at the fountain of knowledge. Others just gargle.
Yes! Hopefully my college's sysadmin will be nice enough to make Kazaa so slow that people will stop installing that spyware-infested, OS-breaking POS software, so that I (being a dorm's paid computer janitor) won't have to fix their computer later on :P
Now, if something could be done about stopping those fine young college girls inadvertantly running attacks on their campus's servers? :P
(Now that I think about it, I don't mind the girls needing help so much as the dumb college guys spilling beer on their laptop's keyboard...)
Why isn't anyone trying to make a home-server linux distro? "just put the cd in and wait, in half a hour you will have a printer-sharing, file-sharing server that will greatly enhance your internet experience! Now you and your family can download, surf and game without any problems in the bandwidth!" If Linux is going to break into home of joe average that might very well be the way. As a black box that does wonders for you. No learning, no configuring, just advantages.
For those of us practicing for our CCNA exams... packets are at layer 3, its known as data at layer 7.
FLR
This packet shaping software must be watching for embedded packet headers within the stream.
Suppose you have a Kazaa packet that is tunneling through HTTP. The shaper notes the HTTP header and passes the data according to HTTP rules until the embedded Kazaa packet is found. Now the shaper switches to Kazaa mode and shaping changes accordingly.
Now, if you want to defeat the shaper, tar and compress your kazaa files, then uuencode them and embed them inside html files. To the packet shaper, it looks like you are transfering some very large web pages. Alternately, drop your uuencoded text into mail messages, instant messages, etc.
Does anyone else use Wondershaper? It works very well for my cable modem and is extremely easy to set up and use. Any comments on how it compares to this one?
I've been doing traffic shaping based on port policies for months using the CBQ.init Script.
What's the advantage of using Layer-7 shaping, when CBQ does it quite efficiently?
Your ISP may tell SSL transfers are minority, waste bandwidth, are uncontrollable (and whatever your ISP marketing drones can think of) and downgrade any SSL transfers till you switch back to plaintext.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
This only works until the protocols become smarter. An encrypted IPIP (or SSH, or IPSec, et. al) stream carrying kazaa traffice looks the same to a packet inspection system as an encrypted IPIP tunnel carrying data from your rotodynamics sensors. There will come a point when bandwidth usage will be dealt with at the social level because all technical solutions have been obsoleted by encryption and tunnelling.
Thankfully, once your packets get routed onto the backbone, you shouldn't have to worry about this. Why? Because your data is packetized, and the internet is best effort. That means that your packets may travel over several sources to get to the destination. Thus, it would be possible to fragment your packets locally to a very high degree so that a router in the backbone would never be able to tell what protocol is in use because the packets would be sent via various hosts. So, the MPAA can't go an install this in the backbone of the net to stop your l33t divx pirating.
On a local network, well that's another story. There will always be ways around stuff like this though. It wouldn't be hard to get another link (cellphone?) and send just enough packets over that to make stuff confusing.
My Slashdot account is old enough to drink...
Now that would be useful to have in the kernel.
I know you can do a certain amount with Apache, but to be able to slot a nice little Linux box in where an Alteon would normally sit would be a)cool and b)cheap.
oh brave new world, that has such people in it!
For those not ready to upgrade to Linux 2.5, and for those on other platforms, there is Trickle, a userland traffic shaper for Linux, *BSD and Solaris. It works on a per-process basis (or on groups of processes to limit aggregate traffic consumption), does not require root-level access nor kernel patches, and is, of course, open source.
own the OSI model? =-).
My bro is an avid Kazaa/WinMX Pr0n colletor, and I'll come home and find 25 people downloading from him and his HUGE collection of trashy pr0n. :)
I'd like to be able to leave it running in a weighted environment without having to manually decide what share he should get or kill all the downloads
errr....umm...*whooosh* *whoosh* Is this thing on ?
FreeBSD has had this for years. Why keep on reinventing the wheel? Fight NIH!
It was just a few months ago that i needed a solution like this but had to bite the bullet for one of those $15,000 packetShaper routers. This is great and it sucks at the same time ;(
The computing service (who're responsible for the university and student networks) monitor general levels of traffic; if you've been using a lot of bandwidth for extended periods of time, they'll contact you, ask you what your excuse is, and tell you to slow down. The idea is that after a few warnings they'll disconnect your network socket, but most people take the hint.
Just looking at the stats rather than the protocol is also good for plausible deniability, since they don't particularly want to know the specifics of illegal file sharing and the like; they have been known to specifically stop a Direct Connect hub, but IIRC that was after another student had a private feud with the hub operator and decided to report them, after which the computing service had little choice.
They also occasionally scan random IPs for common server and trojan ports, then connect to some servers to see what banners etc. they produces, but this is more an anti-h4x0r thing than anything else; they don't even seem to mind students running low-traffic web servers on port 80, but they're likely to contact the student and verbally cluebat them if the server says it's IIS.
+/* XXX Is it ok to do nothing here? This gets called each time a filter
+is added (not sure why). */
This ain't touching my kernel...
in the kernel? layer 7 is for APPLICATIONS. your kernel should know about ethernet and ip and tcp. above that, it's up to the client processes to figure out what to do with the data.
if you want layer 7 shaping, that's easy. it's called a PROXY SERVER. having it in the kernel is bloat of the worst kind.
I'm curious about this... how much luck would the traffic shaper have telling apart, for example, an SSL-encrypted IMAP session, HTTP session, or Jabber session. If they were going to arbitrary ports how would it tell them apart?
Does it need to perform its own man-in-the-middle attack to get at the transmitted data?
Guys, the XMMS team has been busy with exactly what these L7 guys are trying. Check out http://www.packetlogic.com No wonder XMMS is stuck at 1.2.7 :)
It runs on Linux and blows the doors of anything Cisco, Allot, anybody else can do with Layer 7 protocol shaping/firewalling and better yet, you even get real-time surveillance.
Don't tell my boss; he might make me put this on the router so his EverQuest sessions don't start lagging when some secretary starts doing useful work online...
You're ridiculous. You have no idea what you're talking about. Really. Let me talk some sense into you, slappy.
Let's look at why this is important. Imagine someone wanted to use an inexpensive PC as their router? They can do a whole lot with this router, but up until now, it lacked being able to do layer 7 shaping and switching. Applications like Gnutella don't use any specific port, so you have to look into the packet to find out what kind of packet it is. This feature was previously only available in super-expensive "layer 7 switches". Now, it's freely available to everyone. It really increases the value of a linux router to people who want this type of shaping.
Don't spout off before you understand the subject, ok? Promise? Good.
It doesn't even see the code anymore, just - redhead - blonde...
Any technology distinguishable from magic, is insufficiently advanced.
The American government is officially totalitarian
This is not a nightmare
It really is this bad
Please don't insult the suffering of all those who have actually lived under totalitarian rule.
So, if you happen to act like a terrorist the government will treat you one. They might even be blatantly racist and overzealous. But they're not totalitarian.
Dissent is still very much a part of America--and no one, yet, has been punished just for speaking out against the government. (Well, not citizens by the government. A few university professors and private citizens have lost their jobs, and a few immigrants have been forcefully emmigrated, but you get the point.)
(Not that Republican domination isn't that scary--[just what we need, tax cuts in wartime]--but it's not quite totalitarian. Might as well call Canada Communist.)
The Cisco equivalent of this is called Network-Based Application Recognition (NBAR). Rather than use regular expressions, Cisco ship PDLMs (Packet Description Language Modules) that can loaded and unloaded whilst IOS is running, much like you'd get by combining Netfilter's ip_conntrack_helper modules with the ideas these guys have.
(I still think they should be doing this inside Netfilter rather than qdisc)
NBAR can also be - and is - used to filter network worms at ISP borders, by matching the specially-crafted URLs used to compromise vulnerable systems. For example, here's the Cisco config to catch the Nimda worm.
You complain about the current bandwidth usage of your brothers pr0n collection, but when asked, you provide his KaZaA username on slashdot. That's like putting a gun to your head, pull the trigger and blame the bullet for harming you.
Kjella
Live today, because you never know what tomorrow brings
He wasn't being ridiculous.
Both of you are approaching the same problem from different ends.
You are talking about filtering an existing open configuration, he was talking about opening access to a miminmal access system via the use of proxies.
Both are valid, though it is pretty obvious which one is more secure.
There is a whole class of firewalls that are proxy only (There is no ip_forwarding between interfaces, all access to internal or external is done via proxies) (See fwtk, or Symantec Velociraptors).
He was just referring to the fact that this type of packet shaping, is available by other means. (Though he was being trollish with the kernel quip)
In your attempt to be nasty to a stranger, you have only shown that it is you that:
"have no idea what you're talking about. Really. Let me talk some sense into you, slappy."
Maybe you should be more polite next time, just on the off chance you don't know everything. To do otherwise only makes you look foolish.
It's not like the kernel is caching data and maintaining a huge database. All it is doing is simple pattern matching on a session and attaching an identifier on it so the traffic shaper can identify it. Nothing more, nothing less. Simple pattern matching and id'ing sessions. If it is bloating the kernel, many of the kernel developers will realize this and it won't get merged. What's the worry?