Application Layer Packet Shaping on Linux

sommere writes "We have added application layer (layer-7) filtering to Linux. That means that you can set up your linux-router/linux-switch to prioritize mail over the web over kazaa or gnutella regardless of what port each program is using. Colleges have been paying thousands of dollars for packet shapers to prioritize their networks, now you can do it for free. Get your kernel patch at l7-filter.sourceforge.net."

Comments

Amazing enhancement

[mao che minh]

Wow, that's one massively powerful and useful enhancement that I had no idea anyone was working on. This makes Linux, like, 2 times more likely to be used in a datacenter. This is really cool.

PS: fist post fools

Re:Amazing enhancement

[Tyrdium]

Whoa... It's a non off-topic or troll first post... Ummm... Guys? Satan's yelling at me to get him a sweater...

Re:Amazing enhancement

[overbom]

why this got modded down is beyond me. how can the first post be redundant? c'mon mods, check his post history.

Re:Amazing enhancement

[Anonymous Coward]

Mod Parent Up, there is nothing wrong with claiming a 1st when making a valid point, which this poster does. Meta Mod Moderator down.

Re:Amazing enhancement

[joe_bruin]

in the kernel? layer 7 is for APPLICATIONS. your kernel should know about ethernet and ip and tcp. above that, it's up to the client processes to figure out what to do with the data.

if you want layer 7 shaping, that's easy. it's called a PROXY SERVER. having it in the kernel is bloat of the worst kind.

Re:Amazing enhancement

[op00to]

You're ridiculous. You have no idea what you're talking about. Really. Let me talk some sense into you, slappy.

Let's look at why this is important. Imagine someone wanted to use an inexpensive PC as their router? They can do a whole lot with this router, but up until now, it lacked being able to do layer 7 shaping and switching. Applications like Gnutella don't use any specific port, so you have to look into the packet to find out what kind of packet it is. This feature was previously only available in super-expensive "layer 7 switches". Now, it's freely available to everyone. It really increases the value of a linux router to people who want this type of shaping.

Don't spout off before you understand the subject, ok? Promise? Good.

Re:Amazing enhancement

[Mark Bainter]

if you want layer 7 shaping, that's easy. it's called a PROXY SERVER. having it in the kernel is bloat of the worst kind.

Ok, so what do you put on that proxy server? hrm....maybe...a kernel???

Re:Amazing enhancement

[Anonymous Coward]

He wasn't being ridiculous.

Both of you are approaching the same problem from different ends.

You are talking about filtering an existing open configuration, he was talking about opening access to a miminmal access system via the use of proxies.

Both are valid, though it is pretty obvious which one is more secure.

There is a whole class of firewalls that are proxy only (There is no ip_forwarding between interfaces, all access to internal or external is done via proxies) (See fwtk, or Symantec Velociraptors).

He was just referring to the fact that this type of packet shaping, is available by other means. (Though he was being trollish with the kernel quip)

In your attempt to be nasty to a stranger, you have only shown that it is you that:
"have no idea what you're talking about. Really. Let me talk some sense into you, slappy."

Maybe you should be more polite next time, just on the off chance you don't know everything. To do otherwise only makes you look foolish.

Re:Amazing enhancement

[HeX86]

It's not like the kernel is caching data and maintaining a huge database. All it is doing is simple pattern matching on a session and attaching an identifier on it so the traffic shaper can identify it. Nothing more, nothing less. Simple pattern matching and id'ing sessions. If it is bloating the kernel, many of the kernel developers will realize this and it won't get merged. What's the worry?

cool

[papasui]

This really helps networks that have smaller circuits and lots of clients doing various tasks on them. Not such a big help for a home user but great for corporations.

Re:cool

[Jedi Alec]

it isn't for a home user? I for one am quite tired of my roommate's kazaa lite leeching all the upload away, causing me huge delays in regular browsing. Using this on the router would make a simple home network a lot easier to regulate, and face it, the way things are going, pretty soon there'll be a pc per person, not per family.

Re:cool

[Zork the Almighty]

Re sig: YES I KNOW WHAT EFFECTED MEANS IT'S A JOKE

Good God, I almost fell out of my chair laughing at this one.

Re:cool

[stinky wizzleteats]

Not such a big help for a home user but great for corporations.

Well, I packet shape like a son of a bitch on my home network. (it lets me pump out gigs of mutella traffic while still getting speedy response to ssh) But, let me cite a few more conventional uses for this in the home:

• A VOIP phone brought home from the office.
• VPN
• Teenage kids using the home network

Re:cool

[SpaceLifeForm]

There is a setting in KaZaA to prevent that. You need to educate your roomie so (s)he is not running as server.

BTW, pc per person make sense,
since it does stand for 'Personal Computer'.

Re:cool

[Jedi Alec]

I'm fine with him sharing his files. After all that's what P2P networks are about, and I personally can't stand leeches. But my whole point was that with a server like this you don't need to educate him, the router box does the work for you...

Re:cool

[Sabalon]

This can help for home users as well. I mean single users, not people sharing their networks with a roommate.

Using this, you could allow some p2p type of app running in the background, but limit it to x% of your bandwidth, or make sure it always has the lowest priority, so interactive/web/whatever is always snappy.

I'll be interested to see how much of a run-for-the money this'll give Packeteer.

This will be nice

[mrjive]

It's looking more and more like commodity linux boxen, with the right software, can do what your average pricey cisco box is renowned for.

Re:This will be nice

[AndrewNelson]

As long as you don't care about performance.

(Seriously. A modified PC is more flexible, but it isn't going to beat custom hardware of the same generation. In a few years, though...)

Re:This will be nice

[mrjive]

Well to be fair, you probably wouldn't consider doing something like this for high-volume deployment (ie corporate/enterprise level). Chances are, they already have some kind of Cisco or other big box in place anyways.

However, for SOHO applications, this could save people thousands of dollars (especially small-to-medium businesses).

Re:This will be nice

[DShard]

For WAN connectivity to OC3 levels I seriously doubt this would be an issue. I don't think you would use it as internet backbone router, but that is not what this would be used for anyway.

Re:This will be nice

[Telastyn]

Except that small-medium businesses don't need big cisco routers. The little ones aren't even $1k these days.

Re:This will be nice

[AndrewNelson]

Certainly, and that's where being able to do this kind of thing in general (Linux routers, packet forwarders, and now level 7 switching) provides an option for people who would like these capabilities but don't want to/can't spring for the high end Cisco/etc gear.

My comment wasn't intended to be derogatory - this is a nifty project and I'm glad to see it. But I've already seen a few comments (and there will likely be more) talking about how this is going to "kill Cisco" or "pave the way for a linux only datacenter". Such talk is just silly :)

Re:This will be nice

[oldcowhand]

Performance isn't an issue--ImageStream has a full line of commercial Linux-based routers in use in over 70 countries worldwide. They offer wirespeed performance and interfaces from T1/E1 to DS3/E3 through OC12 and OC48.

http://www.imagestream.com/

Don't take my word for it, either. ImageStream's Rebel Router with a DS3 interface was reviewed in Linux Journal and Network Computing last year. Both publications confirmed the wirespeed specification.

Re:This will be nice

[mossmann]

Cisco-like functionality is old hat. Cisco doesn't do any traffic classification this sophisticated. This is along the lines of what Packeteer does.

Re:This will be nice

[AndrewNelson]

Interesting. It still appears to be custom hardware, which is where you get the real advantage.

Re:This will be nice

[DShard]

Yes if by custom you mean interface cards... I guess my linksys NIC's and video cards would make my PCs custom too.

Re:This will be nice

[Forge]

That's not entirely acurate.

The Fact is that a properly configured PC router is going to be faster than a special purpose cisco box simply beause you can throw more hardware at the problem for less money.

I.e. A PC with 3x 1 Gig NICs on a 64 bit PCI bus with 2GB ram, 3 disc raid 0, 2.4 GH CPU and prperly tuned kernel will still cost $1200 or so. Far less than any cisco box that even aproches the performance it will deliver under high loads.

($1200 Cisco boxes don't even do layer 7 filtering. So performance dosn't even matter until you enter the high priced stuff)

Re:This will be nice

[AndrewNelson]

Heh. I just glanced. If it's commodity stuff, then more power to them.

Re:This will be nice

[filledwithloathing]

As long as you don't care about performance.(Seriously. A modified PC is more flexible, but it isn't going to beat custom hardware of the same generation.)

You'd be suprised how many of those "custom hardware boxes" are really just K6's with 32-64 MB's of ram running custom software.

Re:This will be nice

[DShard]

Any comment stating that this is going to kill cisco should be marked -1 (Lack of Clue). You buy gear from Cisco, Nortel or Lucent for support (read: sue potential). If you put your ISP on a commodity linux box and expect five 9's you need to back off the medication.

Re:This will be nice

[DShard]

There are many suppliers of WAN gear for x86 as seen here.

Re:This will be nice

[afidel]

actually with Cisco it has almost nothing to do with sue potential. The TAC really is genuine good support that it fast to get past the BS and on to helping the customer. When I worked as a contractor at Cisco I got to know some of the third and forth level tech guys for the Cisco/Aironet division and these were some smart cookies! And when I talk about responsivness I mean it, one large customer was having a problem that was taking down their wireless network and the first three levels of support couldn't figure it out so the senior support guy got a call at 6am from his boss asking if he had his passport, three hours later he was on a plane headed for Norway! Cisco boxes won't always have the super duper ultimate featureset or best available throughput, but they have fast enough throughput for 99.9+% of installations and have the featureset that almost everyone needs.

Re:This will be nice

[Anonymous Coward]

*ERRRKK* Survey says, your a Moron. Cisco and the big boys are in business because most companies that need Mission Critical Network services need the support and level of service that those companies provide. As well the configuration tools that they offer dont require Uber Kernel hacker on staff to have a decently configured network. Uptime and reliabilty are of utmost importance for companies that require that much network speed. Joe Crackbox computer builder's home built computer probably wont offer the level of reliablity that a integrated hardware software solution will provide. The Extra services that Cisco and other big network companies offer offset the speed that most companies require.

Re:This will be nice

[DShard]

Even better is the fact that when a Telecom or Large ISP hits CAP A, they take developers off of new dev and apply them to fix issues. I have witnessed this, and It's quite amazing and reassuring to their customers.

Re:This will be nice

[Zugot]

Slow down here buddy....

The good thing about the l7-filter and similar software such as zebra is the chance for an alternative. There is nothing stopping some enterprising invidual from supporting this software for a fee. Just because it isn't created by a so-called "Big Name", doesn't mean it is not a feasible alternative.

Re:This will be nice

[way2trivial]

((seriously, in a few years tho, the custom hardware will be of a different generation))

Re:This will be nice

[GauteL]

$1200? That is a bit steep for that hardware. Do they throw in a limo and some nice champagne as well?

Re:This will be nice

[Xerithane]

However, for SOHO applications, this could save people thousands of dollars (especially small-to-medium businesses).

You do realize that since all the .com's went out of business you can buy used Cisco equipment for dirt cheap? You can even buy their low-grade equipment for pretty cheap.

For example: a Cisco 2501 you can find for probably $400 - $500. I'd rather have one of those than a Linux box, just for that whole "Best tool for the job" bit...

Re:This will be nice

[mrmeval]

Yes, but the user interface for all of this in Linux, etal is very difficult. I need a good, flexible, point and click with lots of twiddlies and blinking lights and hand holding or I CANNOT get this to the people/companies/etc who can benefit from it.

I'd like to know of an interface that is machine independant and can be operated FULLY within a browser and has the hand holding, help, diagnostics, etc.

Re:This will be nice

[nettdata]

I think you may be mixing up "performance" and "cost-effectiveness".

Performance is usually related to cost, but just because you can "throw more hardware at the problem for less money" doesn't mean that it has superior performance. It MAY work out that it is more cost-effective, or even more affordable, but that does not mean it is better performing.

I'll still put my money on a big (and expensive) Cisco or Extreme box when it comes to performance, as long as cost is not a factor or issue.

Besides, in such a (Corporate) situation, seemless and transparent fault-tolerance is usually an issue as well, and it's VERY difficult (if not impossible) to supply the same level of redundancy and failover with commodity PC gear.

Re:This will be nice

[cowbutt]

(Seriously. A modified PC is more flexible, but it isn't going to beat custom hardware of the same generation. In a few years, though...)

I didn't take a close look at the specifics, but a low-end Cisco box I glanced the innards of appeared powered by a mere M68030, and a SecureIDS box I looked at was definitely a Dell PowerEdge with a sticker covering the Dell logo. Given Cisco's markup, you could buy a kickass PeeCee for the same price. I call this the "US automobile" approach to performance; why bother solving the problem elegantly by building a light, but stiff chassis with a 2.0L engine, when you can put a 5.0L V8 in a heavy chassis?

Of course, this doesn't necessarily apply to Cisco's high-end gear, and certainly doesn't help with some users' support requirements...

--

Re:This will be nice

[NotAnotherReboot]

And if they don't need a big Cisco router, they could easily just build a cheap computer and run this for well under 1k.

Re:This will be nice

[cheetah]

It is clear that you have never done much with real routers. IOS (Cisco routers) is not user friendly. I don't know of one real router that has a point and click interface. Some routers have limited web interfaces but you almost always have to get to the real interface to use the advanced features.

Re:This will be nice

[geekoid]

If its liscensing/upgrade time they may consider switching. However, there will HAVE to be a company that has support equal to or better then Cisco.

And ig you are a medium company and you are using it, you'll probably keep using it when you become a big company.

Re:This will be nice

[Torrenc]

I've used a lot of network gear for various vendors over the past several years, ranging from your average Linksys to the big Cisco stuff. While performance and features can be issues, you also need to take a serious look at the support and reliability of the product. To me, this includes things like the "debug" features of all Cisco boxen. When things just aren't working as they're supposed to, debug is a bacon-saver. You can find all kinds of other gear whose speeds and feeds are similar to Cisco gear, but with lower cost, and without debugging and support features. Depending on your environment, your mileage will vary.

Re:This will be nice

[tzanger]

If you put your ISP on a commodity linux box and expect five 9's you need to back off the medication.

While not five 9's, I do run an ISP off of commodity Linux boxes and achieve three 9's (8.77 hours out of the year downtime) -- we're a commercial ISP and frankly, if that's not good enough for you, go buy someone else's service. I can't get three 9's downtime out of my upstream ISP if you count the scheduled downtime (which my three 9's figure does count).

Re:This will be nice

[tzanger]

If you'd rather have a 2500 series router over a $500 Linux box, you're on crack. There's no comparison. Those pieces of shit (the 2500s) can't handle more than about a T1's worth of traffic, can't do any kind of CAR or shaping, and can barely handle a few dozen ACLs. And yes, Virginia, you need to use telnet to get at them, because they don't support any kind of encryption.

No thanks. Now if you were talking a 2600-series for a small business or ISP, you're still on drugs, but it's only weed. I'd have to take a good look at what I need if presented with a stock Slackware install vs. a Cisco 2600-series router.

Re:This will be nice

[Anonymous Coward]

You don't know how a router works do you? It's all about the switching fabic, PCs suck balls at it.

Re:This will be nice

[JLester]

2500s are dead reliable though, which means tons of them are still in use. For remote sites with T1s or frame-relay circuits, they are cheap, dependable, and fast enough. I'll take one any day over a Linux box with some weird WAN card in it.

Jason

Re:This will be nice

[GiMP]

What the hell does a router need with a 3 disk raid 0? *maybe* raid 5, but even that is useless. Just put in a $30 IDE flash disk, keep one spare with a live system.

Re:This will be nice

[Mattsson]

Mmm... But a small Cisco router or firewall can't do advanced packetshaping.
Not even the large ones can do really advanced shaping.
You'll need specialised boxes that *aren't* routers or firewalls at all but only do packetshaping.
They're usually totaly transparent to the network, except that they shape the traffic.
The best product I know in this field is the Packeteer Packetshaper, but there might be other products that are as good or even better out there...

Re:This will be nice

[Triumph The Insult C]

rofl

64-bit bus, lots of ram, hw raid, fast cpu ... and the best part, a tuned kernel.

$1200 ... perhaps those could keep up with a decent router ... if you bought 4 of them and somehow magically combined them into one

routers are designed from the very beginning to do routing. the asics do it. the processors are designed for it. the os is designed for it.

Re:This will be nice

[mrmeval]

Internal to the router yes. I was unclear, the software I've seen controls from one router to hundreds.

There are several by several companies as addons. Some are a specific program, some operate in a browser. There are limitations in each, the more they do the more they cost. Some control just a companies routers, some control many brands.

Direct to a router is slow and difficult, about on par with manually setting up iptables. What I need is addon software for remote router management that is available commercially. It has to be very simple and non-threatening, especially at first, no CLI, good tool tips, a massive searchable help guide, some presets, etc. It needs to offer access to every thing that a router/firewall can do, everything, the user must never be forced to use a cli ever. It has to offer rock solid security as all operations are remote, even though they will be on the intranet.

It's not for me, it is to make using any free OS as a router+firewall as appealing as possible.

Re:This will be nice

[fishbowl]

On a Cisco, "user friendly" means having a backspace key.

Re:This will be nice

[hitmark]

"I can't get three 9's downtime out of my upstream ISP" isnt that suppose to be 3 9's uptime? 3 9's downtime is like 99.9% downtime of the year:) soulds like you want you ISP to be running windows:)

Re:This will be nice

[ball-lightning]

How good to you figure the performance would be? I remmember I once set up a WinXP computer as an internet gateway once, and I lost about 50% of my bandwidth. (Set up was a PIII 450 w/ 256mb RAM) This actually would be something nice, but if the slow performance was due to just hardware (as opposed to windows just being slow) then I'd still personally go with dedicated hardware.

Re:This will be nice

[Anonymous Coward]

linux boxen

what the hell is wrong with people? "boxen"??? it's boxes you friggin dimwit. people who use "boxen" are just trying to sound like they know something when in fact they don't -- *BOXES*!!!! where did all these st00bs come from?!

Re:This will be nice

[stinky wizzleteats]

(Seriously. A modified PC is more flexible, but it isn't going to beat custom hardware of the same generation. In a few years, though...)

I'm confused. Most of (Cisco/Nortel/Alteon etc. etc. etc.)'s shit is modified PCs, and those whose kernels are not based on Linux are based on BSD.

I started working recently with the packet shaping options in Linux. A modern Linux box can shape easily at line rate on a 100 mbps LAN. You have to get into carrier class routers to do that in "hardware". And the flexibility of Linux's filter technology puts it in a different universe of practicality as more networks are implementing QoS to deal with VOIP.

Since minor changes in configuration can actually cause a reversal of QoS effect under certain circumstances, and because VOIP is damn near impossible to get right anyway, this flexibility will be a life saver for anyone actually implementing QoS/shaping.

I'll look for this to become more mature before considering it for my customers, but it is difficult to overstate the significance of this advance in Linux technology. Way to go, guys!

Re:This will be nice

[Darby]

I once set up a WinXP computer as an internet gateway once, and I lost about 50% of my bandwidth.

Knowing practically nothing about XP, I'd guess you used home when you should have used pro.

Re:This will be nice

[ball-lightning]

No, I have the Pro edition actually (Actually I have both of 'em, but used Pro in this case, home for one of the clients) I suppose it could have been just a poor networking card (I know it was an Intel, stolen out of an old Dell, can't remmember the exact model though) Anyway, I decided in the end to go wireless, bought a USR 8022, (which I'm very happy with, by the way) so now I can roam my house with my laptop (at full speed in some cases)

Re:This will be nice

[wsloand]

routers are designed from the very beginning to do routing. the asics do it. the processors are designed for it. the os is designed for it.

Actually many medium sized routers use intel processors.

Re:This will be nice

[tzanger]

hahahaha... yes you're absolutely right; I did mean that I can't get three 9's uptime out of my upstream provider, including scheduled downtime.

Re:This will be nice

[weston]

This will be nice.

Literally, perhaps. This is ' nice ' for networks, so to speak, right?

Re:This will be nice

[Triumph The Insult C]

intel x86s? i wouldn't be suprised to see stuff like the i960s or whatever ... but x86?

Re:This will be nice

[arkanes]

QOS is enabled by default in XP, and thats 20% of your bandwidth right there.

Re:This will be nice

[Igmuth]

Just quick off-topic question:
How can you achive 3 9's uptime when your upstream can not supply you with that?

Shouldn't what you supply to you customers be limited to what your upstream provides?

Re:This will be nice

[SpaceLifeForm]

and use this GPL software:
http://lartc.org/wondershaper/.

Re:This will be nice

[Anonymous Coward]

How the hell am I supposed to play Quake on a $30 IDE flash disk, idiot!

Re:This will be nice

[Yottabyte84]

I work for an ISP, and we use a modified version of the wondershaper for our two of our wireless internet access deployments (30-50 users), and it works great for keeping things responsive when morons are on kazza.

Re:This will be nice

[Yottabyte84]

SBC? They can't even get us 3 9s on the T1 we have at work. Bastards.

Re:This will be nice

[Yottabyte84]

We recently had a 4 year old Equalizer Load balancer from Coyote Point Systems fail on us. (hard drive died) When I looked at the manual and the hardware, I found it was simply a Pentium II with 32 MB ram running some form of BSD.

Re:This will be nice

[Yottabyte84]

oh, and my boss said they'd paid $4000 for it, with a support contract.

Re:This will be nice

[Yottabyte84]

/bok'sn/ (By analogy with VAXen) A fanciful plural of box
often encountered in the phrase "Unix boxen", used to describe
commodity Unix hardware. The connotation is that any two
Unix boxen are interchangeable.

--FOLDOC

Re:This will be nice

[wsloand]

Pentium II and III are mentioned here.

Bill

Re:This will be nice

[Anonymous Coward]

Swap space.

Re:This will be nice

[Anonymous Coward]

Cisco PIXen are typically P3s. The original PIX was capable of running OpenBSD. Much of their router gear is embedded PPC.

Re:This will be nice

[Florian Weimer]

The Fact is that a properly configured PC router is going to be faster than a special purpose cisco box simply beause you can throw more hardware at the problem for less money.

Given that there a Cisco boxes that can switch IP traffic at line rates that exceed the PCI bus bandwidth, I seriously doubt that PCs win at this end of the spectrum.

However, PCs typically have significantly more CPU power than low-end Cisco routers (which haven't got ASICs and other stuff to speed up routing decisions, either).

Re:This will be nice

[Anonymous Coward]

Have you ever checked the CPU performance of a Cisco router in the range where you would use a PC?
(e.g. a 2600, a 3700)

Have you not wondered why such a supposedly high performance box needs a "coprocessor" option if you want to do things like compression or encryption?

It is because in reality the CPU power of todays PC is 20-40 times better than that of such a Cisco router. They may have some low-level fast switching but it is useless once you try to do advanced things like discussed here.

Re:This will be nice

[pe1chl]

When you want two or three alternative connections, or want to bundle two lines, the small ones often are unusable because of their hardware inflexibility.

The problem with Cisco is that they don't offer low-end models with a sufficient number of interface slots (NM, WIC). E.g. when you want a router that bundles two ADSL lines and has capability for dial backup (ISDN), you quickly leave the low-end range and end up with a box that is way above what you need w.r.t. performance.

With a PC you have much less of that problem, as systems with 6 PCI slots are easy to get, even in the price category you mention.
(PCI cards are also less expensive than NM or WIC modules of similar functionality)

Re:This will be nice

[Angry White Guy]

Feeds from different providers?

Re:This will be nice

[Lennie]

Well, in the future you can use one or more PCI-card (combo6) to use as ASIC, see: http://www.liberouter.org/.

It uses an FPGA, which is programmed at startup, so you can reprogram it, when they release a new version.

That will make it a lot faster, it's pretty much the same principle Juniper uses (they use Pentium 3-CPU's for the PC-part to run the controlling software on).

I'm sure it will give a big performance improvement.

Re:This will be nice

[Anonymous Coward]

They just released the patch. Someone will have it baked into a webmin module in a month.

Re:This will be nice

[GodOfNothing]

Is it as much as 20%? I seem to recall that it was a much more reasonable figure of ~5%. Back when I was trying XP Pro out it did not like my hardware and wouldn't run stably (precipitating many reinstalls) and altering this setting was on a list as long as my arm of tweaks to apply.

Re:This will be nice

[cduffy]

Swap space.

Why?

If all you're doing is routing, your RAM requirements will be quite static, to the point that you can put in enough RAM and not need to swap at all. Turn off the OOM killer in the kernel and you'll stop having apps die when it thinks it's getting low (we did this at MontaVista for some customers who knew exactly how much RAM they needed and didn't want the kernel second-guessing).

For most embedded system applications, swap is unnecessary.

Re:This will be nice

[cduffy]

...and it's VERY difficult (if not impossible) to supply the same level of redundancy and failover with commodity PC gear.

I'm not so sure about that. Lots of solutions for transparent fault-tolerance have come out for Linux in the last few years; see the Linux-HA project for more info.

And presuming one *does* have a good failover solution in place, the commodity-PC route becomes that much more desirable, as there's no longer a hard (and low) upper bound on the reliability of the system as a whole.

FWIW, I'm {system,network} admin for a tiny little startup with precious little cash to speak of. We absolutely can't afford to spend more on hardware than we must, particularly if we can throw more man-hours (of people who're mostly working for stock) at the problem. Perhaps my viewpoint would be different if I were working somewhere with more cash than man-hours to spend.

Re:This will be nice

[Anonymous Coward]

Turn off the OOM killer in the kernel and you'll stop having apps die when it thinks it's getting low (we did this at MontaVista for some customers who knew exactly how much RAM they needed and didn't want the kernel second-guessing).

If the OOM killer is kicking in when you're not actually completely out of memory then it's a bug (might try the -aa, -rmap, -ck patch sets).

Re:This will be nice

[1g$man]

This is incorrect. The 20% is reserved for use by QoS only if it actually needs the bandwidth. In other words, unless you have an application that is reserving bandwidth with the QoS service, then all your bandwidth is still available.

Re:This will be nice

[Anonymous Coward]

yeah, how dare they use the access they have paid for. Fucking bastards trying to get the most out of their purchase. You should cap them all at 56k modem speed just to teach them a lesson. I mean, no one should use all their bandwidth. Seeing the sarcasm yet?

Re:This will be nice

[nettdata]

I totally agree with what you are saying... it's all about the size/complexity of the configuration and the amount of resources available. Being CTO of 2 new startups, I know ALL about the (lack of) cash thing, and linux routers/firewalls are what we're using for our own installations because we don't have the cash, and it's cheaper for us to use our time instead of spend the cash on hardware.

At the same time, for some of our larger clients (banks, etc.) we'll spec out and install proprietary high-end gear.

It's all about the right tool for the job.

Re:This will be nice

[Forge]

I did rough calculations in my head without looking at my reseler price list. I stated that high a price to be on the safe side.

Re:This will be nice

[Yottabyte84]

They NOT capped. They are allowed to use as much bandwidth as the network can handle. Get on at 3 am, and you'll be able to download at a couple megabits. Interactive traffic is simply given prioritiy, and we apply fair queueing to make sure everyone gets thier fair share of bandwidth. Actualy, our lines are rarely anywhere near full, except for kazza uploads, which do seem to manage to fill up our outgoing bandwidth. If we do not shape traffic, network proformance will suck becaues TCP ACKs cannot get out.

Re:This will be nice

[LarsG]

intel x86s? i wouldn't be suprised to see stuff like the i960s or whatever ... but x86?

It is not uncommon for the lower end routers to contain an x86 or an embedded version of the Moto 68k or PPC. You also find general CPUs in higher end gear, but they also contain special ASICS handling (most of) the routing and forwarding work.

The venerable Cisco 760/770 series contained an 80386.

The Cisco 1600 series contained an embedded version of the Motorola 68020 - 68360@33MHz.

The Cisco 800 series contains an embedded version of the Motorola PPC.

Shape them right!

[Anonymous Coward]

Hmm.. packet shaping.. can't wait to merge this in with the rest of my kernel and give it a whirl.. although, I do have to admit that some of the packets I've been getting are pretty nicely shaped.. there's the Ana packets, and the Kim packets.. but if this patch can help shape some of those no-so-well-shaped ones, I'm all for it!

---
Refusing to be a karma hore! Score: +5 Funny, -1 Karma Hore

Re:Shape them right!

[ArsonSmith]

This is great I wont be embarased to send my picture to the chicks I meet in chat rooms now. run it through my packet shapper and have it take care of it all for her.

I hope they don't get them.

Re:Shape them right!

[ramdac]

don't you mean "whore" ?

Re:Shape them right!

[Anonymous Coward]

Refusing to be a karma hore! Score: +5 Funny, -1 Karma Hore

That's right, put your sig in your anonymous posts. slashdot is attracting more bright minds each and every day!

Re:Shape them right!

[Anonymous Coward]

Ah yes, but if I were you, I would be "embarrassed" to show off my writing skills. Do your pictures, by chance, resemble 'Cleetus' from the Simpsons?

Re:Shape them right!

[Anonymous Coward]

-1, doesn't know how to spell "whore." Twit.

Good or bad?

[SharpFang]

In one hand, >I can prioritize what I want how I want. And it was good.
In the other hand, my ISP may downgrade my Quake performance or my school may block telnetting to my home box completely (no matter which port I put the demon on). And this was bad.

The idea is good but I'm worried it will be heavily abused and that worries me. In the other hand, it may mean a neat security tool...

Re:Good or bad?

[Telastyn]

It will not be so bad.... People will devise a tool to get around it, just like they got around layer 4 filtering. Soon there will be * over HTTP, followed by layer 9 filtering, and so on and so forth until the end of the world.

Re:Good or bad?

[vadim_t]

As if they couldn't do this already. Some imaginative packet filtering can do wonders, and IIRC, Linux has some support in iptables for examining the content of the packets. I'm also sure that it's not too hard for an ISP to pay a programmer to create a Linux module like this. Cisco probably has that as a feature too in some expensive routers.

Re:Good or bad?

[Exiler]

From personal experience, people who run school networks are twits. I explained to the systems admin at my school that their netware restrictions on the startbar and hotkeys were totally ineffective becuase she left IE able to access local drives, and she stared at me blankly, so I doubt they would be running linux in the first place let alone know how to configure advanced packet filters. (this is at a public school btw, I'm still too young to be at uni =P)

On an upnote, I have been playing tetris on my shell account at sdf.lonestar.org when I was done with my work ever since =P

Re:Good or bad?

[vadim_t]

Forgot to say that it's not that hard to work around it, anyway. I already use SSL everywhere I can. It'll be just a matter of adding SSL support to programs, or writing some kind of proxy, which would add some compression as a side effect.

BTW, your ISP would do well by blocking telnet. There's SSH, and it's not that hard to use.

Re:Good or bad?

[SharpFang]

1) I'd need to carry some funny modified program versions with me to get around this. 2) A small ISP with 500-1000 customers won't pay anyone to write kernel modules (neither, luckily, they could afford a sysadmin who would be able to configure that correctly) 3) if my school admins blocked the RMB context menu and command prompt+"Run..." (but you can still create a .bat with "command.com" in it to get a shell), I wouldn't be surprised if they put really VERY strange rules - they like to monkey around with options they don't quite understand and make very bad mess...

Re:Good or bad?

[Anonymous Coward]

wtf layer 9 , since when did they add 2 more? Since you got them outta your ass?

Re:Good or bad?

[Anonymous Coward]

yes. joke; hah hah?

just saying that by filtering like this it's inviting more and more encapsulation (layers) to avoid it.

Re:Good or bad?

[Exiler]

yea, I just load up IE and hit C:\command.com in the address bar to get into the shell. Don't even need to create a batch file.

Re:Good or bad?

[Namaseit]

Your shitting me right? Your school actually implements the newest stuff? Probably not. If its not a problem they wont do it. Plus why are you using telnet? Oh well i guess some people like sending their stuff through plaintext.

Re:Good or bad?

[Anonymous Coward]

Encrypt. No more port blocks, no more selective service degradation, no more broken transparent proxies.

Re:Good or bad?

[Anonymous Coward]

Why am I using telnet? Maybe because ssh doesn't come with Windows by default and my dear admins won't ever install anything that isn't required by the school and doesn't lessen chance of students breaking stuff, and they block everything, including downloads - I'd have to carry a SSH binary floppy with me to school. (I do at times...)

Re:Good or bad?

[SharpFang]

Encryption/tunelling will help you as long as 2 conditions are satisfied:
1) The other end supports it. (HTTP? Kazaa?? Multiplayer games???)
2) The admin doesn't know/notice and doesn't downgrade all -your- encrypted transfers...

Re:Good or bad?

[Paradise Pete]

Oh well i guess some people like sending their stuff through plaintext.

While your post is not quite plaintext, its encryption is not very good. I was able to quickly determine that "your" = you're, and "wont" = won't. Next time try a more complex scheme.

Re:Good or bad?

[TheNetAvenger]

In one hand, >I can prioritize what I want how I want. And it was good.
In the other hand, my ISP may downgrade my Quake performance or my school may block telnetting to my home box completely (no matter which port I put the demon on). And this was bad.

The idea is good but I'm worried it will be heavily abused and that worries me. In the other hand, it may mean a neat security tool...

Fret not, if the network providers wanted to do this, there are already packages to do layer 7 filtering that they could have already purchased and put on the network.

This is just one of the first free implementations for Linux.

There are several products they could already be using if your provider or school wanted to lock down the networks that bad.

Shoot even MS ISA has been doing this for several years, as well as many other solutions in the *nix markets already.

And a lot of providers and networks use products like ISA for caching already, they just don't turn on the layer 7 filtering features.

So basically don't worry about it, it isn't adding any more level of control they what has already been available if they wanted to use it...

Re:Good or bad?

[Anonymous Coward]

well at least this was an almost clever grammar nazi.

Re:Good or bad?

[Anonymous Coward]

Idiot. "Your" was actually used correctly, you fucknozzle.

Re:Good or bad?

[Zarquon]

My method was to associate files with command.com or Fileman.exe in Netscape (Old, 3.1 boxes). Associate .gif, 'open location' any .gif, *poof* command line.

Re:Good or bad?

[Paradise Pete]

Idiot. "Your" was actually used correctly, you fucknozzle.

For sufficiently small values of correct. The post starts out "Your shitting me." Even a fucknoozle such as myself can see that's not right.

Re:Good or bad?

[deander2]

if you're _telneting_ into your home box from school, they'd be doing you a favor to block you. ;-P

use ssh!

Re:Good or bad?

[SmittyTheBold]

HTTP is a variant of Telnet, the packets all look identical (IIRC) except for content - so I really wonder how they distinguish between the various telnet-derived protocols.

Re:Good or bad?

[Kjella]

In the other hand, my ISP may downgrade my Quake performance or my school may block telnetting to my home box completely (no matter which port I put the demon on). And this was bad.

If anything, they are likely to downprioritize behemoth P2P apps such that lightweight apps such as browsing and Quake run *better*. Also, I doubt they'll block telnet, at least some MUDs are still using it so they can't claim it's completely dead. But why not go SSH? Telnet is a deprecated protocol.

Unless you're talking about traffic trying to "hide" in other traffic *cough*KaZaA 2 in HTTP*cough*, it's actually a more user-friendly solution than blocking it outright, which has been the old way of doing it. For one, you don't block anything actually useful, should there be any legitimate use. Second, people may accept the "performance" as it is, and not go looking for ten thousand ways to circumvent the scheme. And like I said, it could improve the performance in those apps classified as "important". I'm actually considering doing this myself, just to have my slashdot reloads go faster ;)

Kjella

Re:Good or bad?

[Anonymous Coward]

It's worth it. Just save a tcpdump output (port 23) to a file and inspect with Ethereal when you get home. That should scare you away from telnet for a while. If there are CD drives, make a mini-CD (or business card CD) with programs that will run without installation (e.g. PuTTY, vncviewer, Mozilla Firebird, gpg, etc.). With DSL at home, the only programs I now use on the CD are PuTTY+TightVNC to connect to home.

Re:Good or bad?

[Yottabyte84]

HTTP based on telnet? I'd think no more then SMTP, POP3, IRC, or most other TCP protocals....

Re:Good or bad?

[SmittyTheBold]

I suppose now that I think of it, yeah, pretty much *everything* is based on telnet. That implies that they don't examine individual packets, they filter individual TCP connections. If you don't keep track of the first few packets of the connection, there's not really any way to tell all these protocols apart from each other - they could (in theory) all carry the same data on a packet-by-packet basis.

Re:Good or bad?

[Yottabyte84]

Are you claiming telnet was the first TCP based service? If that's true you have a vaild point, but otherwise it's quite a stretch to say all TCP protocals are based on telnet.

Re:Good or bad?

[SmittyTheBold]

I'm not saying all, but the control connection in FTP, HTTP, IRC, and many others use the same basic methods as telnet, because, hey, why reinvent the wheel?

Re:Good or bad?

[LarsG]

I'm not saying all, but the control connection in FTP, HTTP, IRC, and many others use the same basic methods as telnet, because, hey, why reinvent the wheel?

Human readable commands over a TCP connection isn't telnet. However, because many Internet protocols use a plain text control connection, you can use a telnet client to talk to a FTP/HTTP/IRC/SMTP/POP3/NNTP/whatever server.

Re:Good or bad?

[LarsG]

If you control one computer on each side of a firewall/gateway, you will almost always find a way to get through.

Generic HTTP tunnels have been available for some time. Some people are even selling them.

15 grand for 100mbit to be exact

[York the Mysterious]

It cost my school 15 grand for 100mbit of shaping to be exact. Try using Kazaa when there are 4 huge dorms full of students trying to access kazaa, irc, ftp, hotline and some other protcols on 150k. Not fun

Re:15 grand for 100mbit to be exact

[mrmag00]

It'd be nice if there was an efficient way of sharing files in the local network first... i'm sure 4 dorms have about everything you can find on kazaa, on a much better network.

Re:15 grand for 100mbit to be exact

[OMEGA Power]

There is. It's called Direct Connect and ever since someone at my school setup a hub I hardly ever use Kazaa anymore

Re:15 grand for 100mbit to be exact

[anonymous cupboard]

How about just running a local Kazaa supernode (or ed2K server)? It has go to be cheaper than the bandwidth (untill you get caugh by the RIAA/MPAA).

Re:15 grand for 100mbit to be exact

[Sabalon]

15G is probably just the 45Mbit - if not you got a better deal than me. :)

Our problem was the students using all the bandwidth for p2p when some WAN apps like payroll (yeah...sql*net over the WAN - there's a good idea.) were not getting the time they needed.

Before Kazaa morphed into the port-switching thing it is now, blocking port 1214 showed a hug dropoff in traffic. Now all the p2p is limited, and everyone (except the leaches) are happy now.

Of course, not BitTorrent and DC are on the rise.

OpenBSD

[Penguuu]

This type of thing has been in OpenBSD long time now (altq) but it nice to see that this type of thing is done in linux.

Re:OpenBSD

[Otterley]

...except that ALTQ handles layer 3 of the protocol stack, not layer 7. ALTQ is incapable of recognizing the difference between an HTTP session and an SSH session if such a session were established on an arbitrary port.

ALTQ relies on the fact that well-known services are traditionally bound to assigned ports. The new layer 7 code allows the administrator to eliminate such an assumption.

Re:OpenBSD

[evilviper]

Actually, no. To the best of my knowledge (none of the info I've read on altq has contradicted this) ALTQ only filters based on port... While it may be a good system for SSH, HTTP, etc., with protocols like Gnutella where the traffic could be on any port, you need something like this patch to recognize Gnutella traffic, and limit it, no matter what port is being used.

Personally, I hope to see this kind of thing in OpenBSD soon myself. However, all the guys working on PF don't seem to be too interested in reaching out into newer territories. I would prefer to see deep packet inspection above all else, but the response I got essentially said they are not interested in working on it.

In their defense though, they have been making progress in other advanced fields, such as PF-Auth (which is very cool and not available in any other firewall/routing package), and merging Altq into PF.

Re:OpenBSD

[Anonymous Coward]

OS-X can do this too. Nice to see all these lame pee cee os's finally come up with support.

Re:OpenBSD

[metlin]

Hmmm, you're quite wrong there.

The differences would be:

ALTQ does not recognize if my sessions are on arbitary ports

This is for the application layer (which is why its called layer 7 packet filter), while ALTQ is for Layer 3.

And more than that, ALTQ controls only outgoing traffic.

I have not seen it mentioned anywhere that hints that L-7 Filter does the same. Since it is at L7, I guess it would be both incoming and outgoing.

(I could be wrong, I've not tried it, atleast not yet :-)

Re:OpenBSD

[metlin]

Oops! My other post was a reply to the parent's parent (Penguuu), not to the parent (Otterley).

Sorry! :-)

Re:OpenBSD

[schon]

ALTQ controls only outgoing traffic.

Can you explain how this Linux patch is different?

I always though it was impossible to do throttling on inbound packets, as it's impossible to control the rate at which someone sends stuff to you..

Re:OpenBSD

[TheNetAvenger]

OS-X can do this too. Nice to see all these lame pee cee os's finally come up with support.


Unless I missed a really big memo, Mac OSX and OSX server can only do Port Filtering via the Firewall. (Which is something that is also built into products like WindowsXP and basic Linux installations.)

I see nowhere that MacOSX or any Apple provided product for OSX offers Application Layer Filtering.

Even MS has offered ISA for Windows Servers with Application Layer 7 filtering for several years. They also have had QOS which is application controlled application filtering in Windows 2000 and XP.

If it is in OSX or provided by Apple in an add-on product, please let me know about it.

Re:OpenBSD

[Triumph The Insult C]

that's why you do

block out all

and be done with it. =)

picky: it's authpf =)

Re:OpenBSD

[shaitand]

It's not impossible to do throttling on inbound packets, I do it with my current configuration at home. Outbound is easy because you only have to queue the packets and send them out at the rate you want, inbound requires dropping packets... it really only works with tcp/ip though, basically tcp/ip determines your connection speed by flinging packets at you as fast as it can and seeing if they all are recieved, if not, it slows down until it's finally able to negotiate an acceptable speed, this is how that OC3 connected webserver is able to figure out to send your 56k modem data at 56k. So basically you have the packets dropped until the speed is where you want it.

This linux patch is different in those ways from ALTQ... because that's it's entire purpose? You can already do all the things altq does with iptables as it already stands. The entire purpose of this patch is that it allows you to shape traffic based on application rather than based on port. The inbound/outbound thing already works under iptables (like I said, I'm doing it myself).

Re:OpenBSD

[Anonymous Coward]

He just didn't want to admit that BSD is dying.

Re:OpenBSD

[Anonymous Coward]

It's not impossible to do throttling on inbound packets

Somehow, I don't think that's quite bandwidth throttling, as you can only do it on TCP, and you can't do it for the entire session - the beginning of the stream will still come full blast.

Not to mention that it actually uses more bandwidth, as the other side has to re-transmit packets that actually made it to you.

Re:OpenBSD

[shaitand]

it's also the same process that occurs every time someone with a slower connection accesses a fast server, my connections are just artificially slower instead of slower due to hardware limitations.

There is a small burst of faster connection but it's very small. It's not bandwidth throttling for the masses per say, because as you say (and I said to begin with) it only works for tcp... it works for things like... oh, a small tcp/ip based network called the internet.

It also does use more bandwidth true, there is some additional overhead incurred but it's not anything noticeable in practice, it works well for me since web page loads recieve a short burst at first and feel a touch snappier, whereas any connection that lasts longer than a second is slowed down to the throttled speed.

It definately IS bandwidth throttling by definition. There is overhead incurred. What computer process can you say incurs NO OVERHEAD, there are some that can rank "almost" no overhead, but none that literally have no overhead. It is still a hack using the protocols built in control mechanisms to do something it's not meant for (ok technically that's exactly what that control is for but not quite like this), but it's also the only way I know of to limit inbound internet bandwidth... any other ideas? I'm all ears... I only do this because it's the best I can come up with.

Re:OpenBSD

[Anonymous Coward]

Wrong.

TCP expects ACKs for packets it sends. It waits for the return ACK before sending the next set of packets. Throttling involves delaying the incomming traffic and then delaying the return ACK. Inbound throttling can simply be a FIFO buffer with a X millisecond delay. Exceed the FIFO and then you drop packets. If you drop packets, you just delay the ACKs or increase the FIFO size.

Re:OpenBSD

[zdzichu]

Packet shaping in accordance to packet content is in Linux for few years already. Look at STRING match in netfilter (iptables), then MARK packets and establish queues with CBQ or HTB or something other.

The only innovation in this L7 shaping is expansion from simple string match (accurate for Kazaa, btw) to full regexp match.

Re:OpenBSD

[kcurrie]

Speaking of altq-- I'm having issues where if I set port 22 traffic to go into a SSH queue, interactive ssh traffic is fine, but scp's and rsync over ssh hangs after a few K of trafic. Anybody seen this before?

Re:OpenBSD

[LarsG]

but it's also the only way I know of to limit inbound internet bandwidth... any other ideas?

Won't delaying ACKs have the same effect?

I think that products like Packeteer does a lot of stuff to "help" TCP rate control - playing with window sizes, delaying ACKs, sending ECN.

Priorities

[Rosco P. Coltrane]

you can set up your linux-router/linux-switch to prioritize mail over the web over kazaa or gnutella

I vote for more kazaa than mail. Unless someone sends me movies by mail.

Re:Priorities

[rusty0101]

The idea would be that time sensitive e-mail, perhaps a contract that has to be approved by the boss, would not have to sit around waiting to get into the corprate mail system while a couple of interns are downloading the latest full length movies.

It very well may be the contract that allows the interns to continue being an intern through the summer.

For home users, this may not be as important, unless you are doing a SoHo business, and would like the same kind of feature while your kids are getting their movie fix.

To each their own. I do agree somewhat with you in that well over 90% of my e-mail these days is spam, which has significantly less importance to me than my web browsing. At the same time I would like the web browser to have priority over any downloads I am doing.

-Rusty

Re:Priorities

[Anonymous Coward]

its really not for mail instead of kazaa on a home link. While web over kazaa might make sense for some home users, the reason i really want to run something like this is for interactive low latency programs. When i'm downloading something big it's annoying to use ssh in fits and starts. To say nothing about things like online gaming. I'm not sure if this solution could do a good enough job to let me play quake while otherwise filling my pipe but if it could that sure would be a godsend.

Re:Priorities

[Anonymous Coward]

Things you get more of should generally be at a lower priority, since the lower-volume stuff will finish first.

(posted anonymously because I'm ashamed to be explaining the flaws in a joke)

Re:Priorities

[arkanes]

Since email is, by design, an unreliable an insecure system, I'm not sure that prioritizing it is usefull for anything except a spamhaus (and a spamhaus probably wouldn't want the overhead of shaping and would just have a dedicated connection for the email anyway).

Another Shot in the head for Closed Source

[SkArcher]

Title says it all really. This will be of great help to those who implement/support Linux on a commercial level.

DOS potential?

[yozzle]

If an attacker knows that you prioritize a certain service, wouldn't he cause a greater disruption with his DOS with this?

Another thing: couldn't the ??AA get ISPs to use this feature, not to kill P2P sharing, but to reduce its priority (perhaps as a compromise from not being able to kill P2P outright)?

Of course, there are many benefits to this as well, I'm just pointing out possiblities.

Re:DOS potential?

[Anonymous Coward]

"Another thing: couldn't the ??AA get ISPs to use this feature, not to kill P2P sharing, but to reduce its priority (perhaps as a compromise from not being able to kill P2P outright)?"

This type of technology already exists and it easily afforable by any half-decent ISP $15-30 thousand maybe a little more. The fact that it currently possible and ISP's are NOT doing it right now answers your question for me.

As far as a DOS potentional goes this would actually help more than hurt. If someone is DOS'ing a particular service you can deprioritize the traffic and greatly reduce the impact.

Re:DOS potential?

[Anonymous Coward]

DOS? I havn't used that since 6.22 and then only with QEMM instlling everything into UMB's and the HMA. Not a TSR below 640k on my boxen.

Re:DOS potential?

[I)_MaLaClYpSe_(I]

As far as a DOS potentional goes this would actually help more than hurt. If someone is DOS'ing a particular service you can deprioritize the traffic and greatly reduce the impact.

Um, no.

The impact of a DOS attack is not reduced by deprioritizing the traffic, as the negative impact is caused by too much traffic. You can limit the amount of traffic from the linux box but not to it. And if your ISP-connection is not capable of carrying all the traffic (DOS + regular) a certain percentage of the traffic is discarded, depending on the ratio of DOS-to-regular traffic.

This is also true, if the bottleneck is not your ISP-connection but, say, your webserver which can only handle a certain amount of connections: You can limit the HTTP traffic to x% but if there are e.g. hundred times more DOS-related connections than regular traffic, almos all regular traffic gets discarded.

Disclaimer: But I could be wrong.

Re:DOS potential?

[Art Tatum]

Yep. I remember when Code Red hit. I couldn't get a decent ping in games for months. And of course, I certainly wasn't running IIS or PWS or whatever other POS service it was that was targetted. Packets dropped on the floor, but still taking up precious gaming bandwidth...

Damn you, sir!

[Anonymous Coward]

It is obvious to anyone that you could not possibly have developed such an advanced feature for the Linux kernel on your own or with the help of the community. This feature has obviously been lifted verbatim form the proprietary Unix code owned by SCO. I expect you to pay our standard SCOSource licensing fee of $150US per processor running this code, IMMEDIATELY. Failure to pay for this license within the hour is a violation of SCO's Intellectual Property rights and WE WILL SUE YOUR ASS OFF!!!!!!!!!!!!!

Darl "Sue em" McBride

Re:Damn you, sir!

[Anonymous Coward]

you're forgetting : 1000 companies must pay or we'll SHOOT THIS CUTE PUPPY.
How much? Well ..... (pinky at mouth) ONE BILLION DOLLAR

How does it work?

[goombah99]

How does a router know what the intended purpose/application a packet is destined for? Does not only the receiving computer actually know what applications have bound what ports?

Re:How does it work?

[demaria]

The same way Antivirus software knows which files are viral. It uses signatures to figure out what the traffic really is. No matter what port it runs on, you can always tell FTP traffic because of the format of the protocol, types of commands, and so forth. Part of the reason people buy commercial packet shapers is for these signatures. You can't do effective traffic shaping at just layer 4, you need to look at layer 7.

Re:How does it work?

[WasterDave]

It's really, horribly complicated. Basically the router has to build as little of the TCP stack as possible in order to look at the actual, data contents of the packets to decide what application is being tunnelled.

Dave

Re:How does it work?

[evilviper]

Obviously, it reads the data in the packets and recognizes the protocol. If it looks like HTTP traffic, it will give it the priority of HTTP. If it looks like SMTP traffic, it will get the priority of SMTP...

It doesn't need to know which port it is bound to, it just rocognizes the protocols in the packets.

Re:How does it work?

[zentigger]

Actually they code causes your hdd heads to modulate at such an exact frequency that the electomagnetic resonance opens up a worm-hole in the space-time continuum.

This portal is used to summon thousand of magic gnomes that sit in the spaces between time on your ethernet interface where they use their prescient abilities to determine who is trying to download pr0n so they know exactly when to reach out and "snatch" your packets. Depending on your configuration each gnome will hold the packets in stasis for a predetrmined amount of time, thus limiting your bandwidth.
duh!

Re:How does it work?

[Anonymous Coward]

Sorry to be dense but I still dont understand.

If I send a plane jane udp or tcp packet why is the protocol neccessarily evident? to be specific, suppose my packet just consisted of an encryted wad of digits with just enough UDP information to deliver it to a certain socket on a certain port at a certain IP address. How would the router know it was a gnutella packet or anything else.

I realize that perhaps mail and http packets have enough header info for a person to perhaps figure out what they are. But would it not be easy to disguise gnutella packets as say mail packets with a bogus header but sent to something other than port 110?

Re:How does it work?

[demaria]

Ignore encrypted for a moment. You can disguise stuff inside mail or http traffic. But if you look inside, you may find patterns. Say your HTTP encapsulated gnutella always contain the text string "gnutella-http" in the first 20 bytes. Boom, that's your signature right there. Signatures, of course, are reactionary not proactive. Say someone comes out with the encapsulated gnutella protocols. Your traffic shaping vendor (be it Packeteer, Allot, or the open source guys) does an analysis on this new protocol, discovers some form of a pattern, and makes a new signature. Then you update your traffic shaper's software.

Now encrypted is a different story. It's harder to inspect, as you can't actually look at the traffic data and it's mostly random looking. The most you can do there is try to see message length, frequency of messages, or responses to try and get a pattern.

Re:How does it work?

[djtack]

Yes, demaria (above) explains this pretty well. Certainly it's not hard to trick the filter (you could tunnel everything through SSH on port 22, and nobody would be the wiser), but that isn't necessarily the point. It's still useful if you can (mostly) trust your users not to cause mischief.

To better illustrate how this might work, consider this packet:

17:26:26.288988 66.35.250.110.http > azrael.47969: . 1:1461(1460) ack 446 win 6432 (DF)
0x0000 4500 05dc 67fd 4000 3106 07a6 4223 fa6e E...g.@.1...B#.n
0x0010 80ff 16e8 0050 bb61 0000 16ef 7765 bbbe .....P.a....we..
0x0020 5010 1920 e122 0000 4854 5450 2f31 2e31 P...."..HTTP/1.1
0x0030 2032 3030 204f 4b0d 0a44 6174 653a 2046 .200.OK..Date:.F
0x0040 7269 2c20 3330 204d 6179 2032 3030 3320 ri,.30.May.2003.
0x0050 3232 3a32 363a 3235 2047 4d54 0d0a 5365 22:26:25.GMT..Se
0x0060 7276 6572 3a20 4170 6163 6865 2f32 2e30 rver:.Apache/2.0
0x0070 2e34 3620 2855 6e69 7829 206d 6f64 5f73 .46.(Unix).mod_s
0x0080 736c 2f32 2e30 2e34 3620 4f70 656e 5353 sl/2.0.46.OpenSS
0x0090 4c2f 302e 392e 3663 0d0a 4361 6368 652d L/0.9.6c..Cache-
0x00a0 436f 6e74 726f 6c3a 206d 6178 2d61 6765 Control:.max-age

This is clearly web traffic, even if we ignore that fact that it's on port 80, you can see evidence of http in the data itself.

17:34:06.098988 mgc.ssh > azrael.46148: . 447953:449401(1448) ack 1296 win 9648 <nop,nop,timestamp 339772381 279677933> (DF) [tos 0x10]
0x0000 4510 05dc 088d 4000 4006 fd93 80ff 1605 E.....@.@.......
0x0010 80ff 16e8 0016 b444 7ee3 8e22 7d94 24ff .......D~.."}.$.
0x0020 8010 25b0 ff13 0000 0101 080a 1440 83dd ..%..........@..
0x0030 10ab 8bed 7fdd cb10 3f79 eb7e ffce 1950 ........?y.~...P
0x0040 a295 3003 bc21 4ffe 0e6b 231a 6ce7 748c ..0..!O..k#.l.t.
0x0050 e9aa 4d74 ea34 16ff a456 5795 2176 b4b4

Now this SSH packet could be carrying anything... it's hard to tell. Still, certain applications might have patterns, as suggested.

Re:How does it work?

[haqim]

can you please name the tool you use to "catch" the packets like in the above two examples.

is it ethereal in action ?

thanks in advance,

Re:How does it work?

[Anonymous Coward]

By the look of it, It's tcpdump using -E -s 176 as parameters.

Re:How does it work?

[Anonymous Coward]

err, -E = -X. Use something like (as root).. tcpdump -n -i -X -s 512 'port 80 or port 22'

Re:How does it work?

[Anonymous Coward]

Argh, slashdot ate my tags.. tcpdump -n -i -X -s 512 'port 80 or port 22'

Re:How does it work?

[joejoejoejoe]

Does that sig mean your are a sperm doner?

haha

Wohoo!

[Kirby-meister]

Yes! Hopefully my college's sysadmin will be nice enough to make Kazaa so slow that people will stop installing that spyware-infested, OS-breaking POS software, so that I (being a dorm's paid computer janitor) won't have to fix their computer later on :P

Now, if something could be done about stopping those fine young college girls inadvertantly running attacks on their campus's servers? :P

(Now that I think about it, I don't mind the girls needing help so much as the dumb college guys spilling beer on their laptop's keyboard...)

Re:Wohoo!

[SuperBanana]

Now, if something could be done about stopping those fine young college girls inadvertantly running attacks on their campus's servers?

Three words.

On
Site
Service.

Re:Wohoo!

[Anonymous Coward]

One Word.

Roophies.

Any documentation on this?

[azaze1]

Is there any documentation on this feature or any other QoS/traffic shaping for linux?

I'm really interested in finding a way to limit bandwidth based on IP or MAC address or both. I understand dummynet for freebsd is great at this but I don't think it will work with my shuttle box due to the SiS 740/961 north/south bridge combo. I could barely get it working in linux.

Anybody have a good howto or something for linux traffic shaping?

-Robert

Re:Any documentation on this?

[kableh]

Well the linked site links to the original Linux traffic shaping web page, located here: http://lartc.org/. That would be a good start =).

Re:Any documentation on this?

[mossmann]

Uh, did you try following the link?

"First, Please read the Linux Advanced Routing and Traffic Control HOWTO at http://lartc.org/"

New type of linux distro? (again)

[Lord Kholdan]

Why isn't anyone trying to make a home-server linux distro? "just put the cd in and wait, in half a hour you will have a printer-sharing, file-sharing server that will greatly enhance your internet experience! Now you and your family can download, surf and game without any problems in the bandwidth!" If Linux is going to break into home of joe average that might very well be the way. As a black box that does wonders for you. No learning, no configuring, just advantages.

Re:New type of linux distro? (again)

[Anonymous Coward]

Why isn't anyone trying to make a home-server linux distro? "just put the cd in and wait, in half a hour you will have a printer-sharing, file-sharing server that will greatly enhance your internet experience! Now you and your family can download, surf and game without any problems in the bandwidth!" If Linux is going to break into home of joe average that might very well be the way. As a black box that does wonders for you. No learning, no configuring, just advantages.

Well, get started! You have the code, the idea, and the opportunity. It's not like you have to wait for someone else to do it.

Re:New type of linux distro? (again)

[bryanzera]

>No Learning, no configuring, just advantages.

The people smart enough to think this way have already purchased macs.

Re:New type of linux distro? (again)

[Lord Kholdan]

Except... by the time I'd know Linux well enough to code it we'll all be running sentient OSes anyway. And if your point was about how everyone should code new stuff of Linux... Why isn't a good idea worth something? Or is there only one true(tm) way people should interact with Linux? By coding?

Re:New type of linux distro? (again)

[bogie]

Ever heard of Esmith? http://www.e-smith.org/
Mandrake and Red Hat will work fine as well.
Or I guess you could buy a Netwinder www.netwinder.net which really is plug and play.

"If Linux is going to break into home of joe average that might very well be the way."

Well realistically that's really not likely to happen. Joe average doesn't go around setting up servers. Of course no offense, but I'm not really sure what your initial point was ;) Are you saying the average home user needs Application Layer Packet Shaping or that there are no easy to setup linux "server distros"? I guess maybe you meant both, but considering most homes aren't even running the easy to use linux servers out there now the availability of ALPS probably won't change that.

For businesses it might spur more linux adoption though.

Re:New type of linux distro? (again)

[Lord Kholdan]

I'm talking about targetting homes with more then one computer. Parents complain because downloads prevents them from reading their email and surfing, downloads must be cut to get low ping, there are no file & printer servers... I'm pretty sure many people would fine those handy, even if they don't know what packet is.

Re:New type of linux distro? (again)

[Elwood P Dowd]

I was shocked to see all the "DSL/Wireless Routers with Printer Sharing" at CompUSA. If they're getting sold, then sure, there's a market for this shazz.

Re:New type of linux distro? (again)

[harryk]

Have you even looked at the LEAF distributions. While being very small, they are extremely effective at doing exactly what you are talking about. While it might not be as pretty as some would like, its still quite good. File sharing, printer sharing, etc, all based off a floppy boot,including routing, QoS, ipsec, etc.. check it out at http://leaf.sourceforge.net Linux Embedded Application Firewall enjoy

Re:New type of linux distro? (again)

[Lord Kholdan]

You all are missing my point... while there are dozens of server distributions to aimed at Linux admins there are not any Linux server distros aimed at people who ...it is geared towards people who have no Unix knowledge and do not intend to get any.

I'm talking about "my 10 year old brother can install and admin it" easy.

Almost anyone would get advantage out of a personal server.

And almost no-one can set up one themselves. (Let's be realistic, people who could set up a Linux server are less then 10% of the users... maybe even 5% or less).

Demand exists for Linux in that area, it could be used.

Re:New type of linux distro? (again)

[shaitand]

Your mac configures advanced routing for you? I wasn't aware of that...

Re:New type of linux distro? (again)

[King of the World]

try ipcop

Re:New type of linux distro? (again)

[qvatch]

ClarkConnect Its a Home server/gateway, works great.

Re:New type of linux distro? (again)

[bja]

They call this distro Mandrake. You can download it from www.linuxiso.org and similiar sites. Learn more at www.mandrakelinux.com.

Re:New type of linux distro? (again)

[CBravo]

http://connectux.nl/products does it, but you have to pay for it (and some knowledge of Dutch helps).

Packets at Layer 7?

[Cytlid]

For those of us practicing for our CCNA exams... packets are at layer 3, its known as data at layer 7.

Re:Packets at Layer 7?

[u01000101]

For us practicing for our MCSE... packets are at prayer 3, data comes only at prayer 7.

Re:Packets at Layer 7?

[Anonymous Coward]

Well, hopefully you fail - because this is about filtering packets ("layer 3") based on the contents of the data at "layer 7" (which is bogus, because IP and its associated higher-level protocols don't follow the seven layer model to begin with). Surely you should understand this, if you're trying for a CCNA.

Good try, though. You almost convinced us you were smart, until you said something stupid.

Re:Packets at Layer 7?

[Jennifer Ever]

Wow, you have to practice?

Re:Packets at Layer 7?

[afidel]

actually it inspects the packets at layer 3 and determines the layer 7 protocol being used so the desciption is correct =) /just a dumb MCSE

Re:Packets at Layer 7?

[autocracy]

Have you read the Cisco curriculum for CCNA? I've been looking at their ever-changing crud for 2 years now, and it's like this: there's the right answer, and then there's Cisco's right answer. You don't even want to think about what their current curriculum calls "normal mode" electrical connections. They entirely dropped the word fault from that. Guess how fun that is to explain to anybody...

Re:Packets at Layer 7?

[Mattsson]

My guess is that they are shaping the packets at layer 3 but doing it based on where it comes from / is headed to at layer 7.
So it still is packetshaping. =) (Haven't read the code though, so I might very well be wrong there.)
But maybe it should be labeled "packetshaping at layer 3 based on layer 7 data" instead. =/ Hmm...

Re:Packets at Layer 7?

[D_Gr8_BoB]

For those of us practicing for our Taco Bell exams...

• Beans
• Cheese
• Sour Cream
• Guacamole
• Tomatoes
• Lettuce
• Rice

Re:Packets at Layer 7?

[BitHive]

I don't get it. (Seriously, what's the gag?)

Re:Packets at Layer 7?

[Anonymous Coward]

CLUE: MCSE = Must Call Someone Experienced

Re:Packets at Layer 7?

[Art Tatum]

I can't believe people are still going on about ISO anyway. I thought that was DOA years ago...

Re:Packets at Layer 7?

[Cytlid]

I'll take criticism from an "Anonymous Coward" as a compliment. Should I listen to you, and take your advice I will fail, because appearing at the test as "Anonymous Coward" would be akin to not showing up at all.

Also, I'm not trying for it. I'm getting it. It's a mandate of my employment. And seeing as I've been deploying cisco routers for a few years straight now, it should be cake.

I'll come back when I have it, and ask how that anonymously criticizing thing is working for you.

Re:Real nice, destroy another market.

[alienhazard]

the difference between bill and linus in this case is that bill FORCES it on you, linus does not. and my guess it that this feature WONT be bundled, per se (not a defaultly enabled feature), but just another option u can choose to use when you make menuconfig, like isdn support or telephony support.

Dont Worry!

[Anonymous Coward]

If your ISP starts using stuff like this start using a encrypted protocal to transport and tunnel your normal software.

Your ISP can tell you have a SSL tunnel but it's really hard to check what the packets are for.

There are ways around it. Currently the problem is with large amounts of ignorant or clueless users just destroying networks with stupid Kazaa searches for porn. Now they'll have to upgrade or try harder.

Remember it's like attrition (like spam) each side just one upping the other side

Re:Dont Worry!

[SharpFang]

Your ISP may tell SSL transfers are minority, waste bandwidth, are uncontrollable (and whatever your ISP marketing drones can think of) and downgrade any SSL transfers till you switch back to plaintext.

Re:Dont Worry!

[Anonymous Coward]

Nothing stops you from switching to something else.
Look at the spam problem. It's really hard to filter. If you're given internet access there is always a way to get around filters. Nothing stops you from hiding stuff inside of HTTP that looks like a compressed file but is actually just encrypted.

Re:Dont Worry!

[SharpFang]

Yes, theoretically a HTTP-like (or even HTTP-based) P2P network app could be easily made and could easily beat the filter. But how many people would switch to the new network? Only a part of the affected by the filter, and not many will be affected. And even if you do - the filters by then would probably include downgrading big file transfers (something like HTB only on per-file basis, not per-host/per-connection) and would be able to detect "by special fingerprints" and distinguish a real webserver from a p2p host.

Re:Dont Worry!

[aminorex]

Your ISP may suck and you may wish to get another ISP.

Re:Dont Worry!

[SharpFang]

Theoretically - yes. But what if this is one of the the cheapest available, (means affordable for me), and I know it's the best available in this price class anyway? Anything better than them is 4 times more expensive and beyond my financial reach. (note I don't live in US...)

Shape Spoofer, read on

[appleLaserWriter]

This packet shaping software must be watching for embedded packet headers within the stream.

Suppose you have a Kazaa packet that is tunneling through HTTP. The shaper notes the HTTP header and passes the data according to HTTP rules until the embedded Kazaa packet is found. Now the shaper switches to Kazaa mode and shaping changes accordingly.

Now, if you want to defeat the shaper, tar and compress your kazaa files, then uuencode them and embed them inside html files. To the packet shaper, it looks like you are transfering some very large web pages. Alternately, drop your uuencoded text into mail messages, instant messages, etc.

Re:Shape Spoofer, read on

[Mal Reynolds]

I've often thought with very little work, the makers of applications like Kazaa could totally spoof packet shapers.
One would think with many colleges foisting these regulations on their student networks, some enterprising student programmers would have already developed work arounds.

Re:Shape Spoofer, read on

[SharpFang]

Errr, how? Copy&Paste the packet contents? Write a wrapper? And what about unwrapper? How many kazaa users worldwide will receive your kazaa packet if you sent it through ICQ and uuencoded?

Of course you may set up a tunnel between your home box and some remote host of some friend, outside the shaped network. But then the admin will notice excessive transfers over that tunnel between the two hosts and downgrade your transfers using old-fashioned source&dest IP match.

Re:Shape Spoofer, read on

[appleLaserWriter]

For the HTML scenario, you need two scripts, encode and decode. Encode simply uuencodes the target file, then sandwiches it between a html header and tail, renaming the file from hitmeonemoretime.mp3 to hitmeonemoretime.mp3.html. The decode script just strips out the html header and then runs uudecode on the naked middle.

You then drop the file in your kazaa directory. When the packet shaper sees the kazaa header it drops your packet down to kazaa quality. But then it sees the html header and presumably your packet is upgraded again.

In the ICQ case, you add an ICQ header to the file and an ICQ extension. Kazaa protocols are still used to transfer the file, but the packet shaper gets tricked into thinking it is transmitting a nice long ICQ packet.

When the sysadmin throttles your ICQ traffic down, simply send them a kind note explaining that your important business conversations seem to be slowing down to a crawl and could they please look in to the situation.

Essentially this scheme is just social engineering. We are trying to convince the packet shaper that our data is other than what really is by changing appearances.

Re:Shape Spoofer, read on

[Anonymous Coward]

uh yeah,
wtf?

Re:Shape Spoofer, read on

[SharpFang]

Sounds clever, unless the shaper recognises "single file" transfer unit (may do at this level) and considers only the outermost headers...
Plus the losers at the other end may be pissed off that Y0UR MP3Z AR BR0K3N

Re:Shape Spoofer, read on

[appleLaserWriter]

Single file transfer unit analysis would be a good feature for a packet shaper like this to have. I can't tell from the minimal web page if this project is a cool idea with minimal implementation or just minimal documentation.

The cool thing about the html scenario is that you can embed decode instructions in the html header, something like "go to http://warez4u.cx/uns7uffme/ for decoding toolz". The embedded file meme would need to be inserted in to the global consciousness.

Re:Shape Spoofer, read on

[Sabalon]

The first p2p that SSL encodes everything will also break this. Then it'll all look like noise.

Wondershaper

[Otik2]

Does anyone else use Wondershaper? It works very well for my cable modem and is extremely easy to set up and use. Any comments on how it compares to this one?

Re:Wondershaper

[51c4r1u5]

Yep I do :) But it operates at level 3... See the other posts for a discussion of level 3 vs. level 7 prioritizing (e.g. switch port numbers and mess up your shaper)

Re:Wondershaper

[Anonymous Coward]

wondershaper is great -- it can prioritize and group traffic by port number or host ip. It's not the same as layer 7 which can group by application though.

but really, the end result is the same. Also, wondershaper runs out of the box on redhat and gentoo linux from what I've seen.

it's just the traffic shaper modules are relatively unknown outside linux networking circles, I guess.

Re:Wondershaper

[CvD]

I like wondershaper... well, I used to, when it worked. It ran on my RH7.3 server perfectly, but now that I've switched to Debian stable, it no longer works. I'm using 2.4.18-586tsc, and whenever I run the script, all traffic through my ppp0 (pptp adsl connection) is blocked. Very annoying...

Anybody else with Debian stable with kernel version 2.4.18 who has a working wondershaper? Any tips???

Cheers,

Costyn.

Re:Wondershaper

[rob00si]

Hi,

The problem is the bottom of the wondershaper script - the 'downlink' section. Just comment out the last two 'tc' commands and the script will work fine.

If you want to get the 'downlink' section working, you need to recompile the kernel and add the section about network policing (if i remember correctly). It's not enabled w/ the default debian kernel.

The thing is, you really don't need to police incomming packets - i'd be more concerned w/ outgoing... which works fine

-Rob

Re:Wondershaper

[CvD]

Okay, cool. Thanks for the tip. But I'm kinda interested in shaping the downlink too. I mean, this is useful for when you are downloading a large file through FTP or some P2P program or BitTorrent, and you want to be able to continue browsing or whatever.

Hmmm... recompiling kernels sucks... oh well.

Cheers,

Costyn.

Finally...

[oaf357]

Application layer is how packet shaping should be done. Of course there are ways around everything but this is truly the best way to go about shaping.

Good work.

Correct me if I'm wrong, but CBQ anyone?

[Kris2k]

I've been doing traffic shaping based on port policies for months using the CBQ.init Script.

What's the advantage of using Layer-7 shaping, when CBQ does it quite efficiently?

Re:Correct me if I'm wrong, but CBQ anyone?

[SharpFang]

That's based on service, port number notwithstanding. Set up FTP on 25 and Kazaa on 80 and you still get FTP treated as FTP and Kazaa blocked completely ;)

Re:Correct me if I'm wrong, but CBQ anyone?

[Openadvocate]

Well it takes more cpu power. :)
If one chooses to use packet switching for load balancing fx. your webserver or anything else. It's fastest to keeep it as low level as possible, for example on the port level. Create a virtual IP and redirect all port 80 request on to several webservers. But maybe you want different webservers serving diffent things, fx. one serving up HTML another graphics and a third doing non-static pages, like search pages. You can do this by having a sub domain for each type like www.mysite.com graphics.mysite.com and search.mysite.com. But you can also do switching in the application layer and redirect all pages based on the url, a cookie or something else so that all *.gif|*.jpg goes one place, and all /cgi-bin/* goes another place.

Another place you might want to do layer 7 is if you need some sort of auth. to the webserver/application. If it has not been programmet to scale, loadbalancing might not work since you would only be auth. to one server. This can be solved in different ways, one solution is to still do port switching but making the loadbalancing "sticky" so that once user is loadbalanced to one server, he stays there and the allocation only runs out after a certain time of inactivity. Of course this could give some timeout problems so you could choose to do the loadbalancing on layer 7 based on the url or cookie and you would get a better intregration between the loadbalancer and your application. Then there are the cases of handling p2p application which does not confine their use to one port and are often these days user configurable. Then you would not know which port they are running on and need to look at the application layer and the actual traffic in order to determine what is going on, this is much more complicated as you can guess since it's not just about redirecting ports anymore. But dealing with the traffic based on the actual data, gives you more control but also more so maintain
Ok, I could go on for hours here so I better stop. :) But it's hard to say why just this solution would be better without looking at a specific problem you need to solve.
But as I said, try to get down to a low level as possible.

Re:Correct me if I'm wrong, but CBQ anyone?

[Anonymous Coward]

actually it can be based on ip block, which makes alot more sense to me in most circumstances to me. So, my neighbor's share ( 192.168.0.[1-2] ) about 128k down/56k up. That leaves the rest of the connection to me ( 192.168.1.0).

But this makes more sense on the coperate level.

Re:Correct me if I'm wrong, but CBQ anyone?

[Kris2k]

Indeed, you're right, however, for this to actually become interesting, they are going to have some sort of plug-in model where you can attach protocol signatures, to build your shaper as your network evolves.

Considering the fact that there are new apps being written every day, it's going to be difficult to have a layer-7 shaper that will be "up to date" with current applications and protocol uses.

Anyways, I still believe in the militant/BOFH sysadmin approach: NO INTERNET FOR YOU!

;)

Arms race ++

[Jeffrey Baker]

This only works until the protocols become smarter. An encrypted IPIP (or SSH, or IPSec, et. al) stream carrying kazaa traffice looks the same to a packet inspection system as an encrypted IPIP tunnel carrying data from your rotodynamics sensors. There will come a point when bandwidth usage will be dealt with at the social level because all technical solutions have been obsoleted by encryption and tunnelling.

Re:Arms race ++

[DigiShaman]

Yup. In fact, I wouldn't be surprised if Timewarner cable (RoadRunner) offers a tiered service. You pay extra to get priority over someone that just casually surfs the web. Are you a gamer and want ungodly ping rates? Pay up!

Re:Arms race ++

[rherbert]

You could also lower the speed of the connection for certain IPs that have been transferring an excessive amount of data over the last hour. Then it'd just hit the people who are using the most bandwidth regardless of whether they're tunnelling.

Re:Arms race ++

[way2trivial]

or- equate encrypted traffic with your lowest priority (P2P) service..

Re:Arms race ++

[Jeffrey Baker]

Oops, you just pissed off the rotodynamics research group.

Re:Arms race ++

[zaad]

This only works until the protocols become smarter.

This post is definitely right on and deserve to be modded up.

Basically, the l7-filter project is a pattern identifier based on packet payload (data) and not simply the headers. What this allows you to do is to generate signatures of protocols you wish to match.

This works right now because most firewalls and shapers do not look at the packet payload for shaping, and the applications AREN'T trying to foil that. But pattern based packet payload analysis becomes common enough, you can bet that certain protocols will start to masquerade as others to try and get through filters.

Just take Spamassain or other Bayesian based spam filters for example. Spammers are already modifying the contents of emails and inserting extraneous words to evade matches.

This is not to say that layer 7 filtering isn't worth doing. It just means that like the previous poster said, it'll be an arms race until it becomes too tough to distinguish legitimate traffic from masqueraded traffic that it won't be worth doing anymore.

Re:Arms race ++

[CBravo]

And in this protocol, you only get bandwidth if the receiving end says it is ok to get it? Sending extra layer 7 information for the company router (with different encryption) does not sound impossible.

correct me if i'm wrong

[pridkett]

Thankfully, once your packets get routed onto the backbone, you shouldn't have to worry about this. Why? Because your data is packetized, and the internet is best effort. That means that your packets may travel over several sources to get to the destination. Thus, it would be possible to fragment your packets locally to a very high degree so that a router in the backbone would never be able to tell what protocol is in use because the packets would be sent via various hosts. So, the MPAA can't go an install this in the backbone of the net to stop your l33t divx pirating.

On a local network, well that's another story. There will always be ways around stuff like this though. It wouldn't be hard to get another link (cellphone?) and send just enough packets over that to make stuff confusing.

Re:correct me if i'm wrong

[SharpFang]

Yep. Fragment your packets so much the router won't be able to recognise them. The admin will thank you, you've just downgraded your own performance yourself so much that no traffic shapers are needed. (Note: More packets=More overhead=Less data in one frame, plus what about incoming packets? How do you tell the remote host to fragment them?)

Re:correct me if i'm wrong

[vadim_t]

Heh, wrong. The admin will hate you for that.

Let's do a calculation: 1GB transferred with 128 byte packets gives 8388608 packets. With 56 bytes of TCP/IP data per packet that makes 448MB of overhead. Yeah, the download will be going slower, but a lot of bandwidth will be lost on TCP/IP.

The whole idea is useless, anyway. Many tools like Snort can already reassemble fragments to avoid being foiled by tricks like this.

Oh, and you can tell the remote host to send smaller packets by changing the MTU.

Re:correct me if i'm wrong

[Anonymous Coward]

Um, so Joe User has sacrificed data for overhead, but he's still sucking up the same amount of bandwidth. Why would Joe Admin be so grateful?

Re:correct me if i'm wrong

[Anonymous Coward]

OK, I will correct you. The internet tends to have very high route stability - so the basis for your argument is bogus. Routes change on the order of minutes at best, and typically not for days (unless there is last-mile load balancing going on), so you will not be switching routes. Also, highly fragmented data is suspect to begin with and should be placed in the "never forward" queue.

Re:correct me if i'm wrong

[buss_error]

Thus, it would be possible to fragment your packets locally to a very high degree so that a router in the backbone would never be able to tell what protocol is in use because the packets would be sent via various hosts.

Highly fragmented traffic or small MTU's get an instant squint on many networks.

A friend was telling me that someone where she works got fired for using internet radio at work. They had been told twice (and written up) not to do that anymore because of bandwidth issues.

I call it employment darwinism. Trying to avoid being discovered in AUP issues like this is a good way to piss off people that can rock your world. It gains you very little for a lot of risk. So, by all means, go ahead. Someone else behind you needs a job...

Re:correct me if i'm wrong

[SharpFang]

Oh, Joe User used to send 20 50K packets per second. Now he's sending 200 100b packets because firewall's HTB or other simple QoS won't let any more through. From 1M/s the bandwidth usage will drop to 20K/s. (why would anyone install the new high-level shaping filters, while leaving good old ones out?)

SCO Code

[attobyte]

There is no SCO code in that patch is there? :)

Sorry I had to do it for one of these posts.

Mike

Re:SCO Code

[attobyte]

Its funny that I posted this when only 10 comments was listed but I am redundent. I think i would subscribe if I had a way to post comments to my moderators. So I can tell them the are full of shit.

Mike

Re:SCO Code

[attobyte]

This moderator SUCKS please metamod him to shit

Thank You.

Re:SCO Code

[Anonymous Coward]

/me isnt that moderator, but...

It's redundant in the context that "SCO Code" karma whoring is getting pretty cliched...
In soviet russia the iraqi info minister has a beowulf cluster running proprietary SCO code

Score 5 Informative????

[Anonymous Coward]

I love you moderators. You mod up an incorrect post because you too are clueless. Here's a tip; when you don't really know the subject, don't moderate the post.

This dweeb is el wrongo and has a LOT to learn before he gets his shiny CCNA. I'll offer him this tidbit; your post would get you kicked out of the CCIE exam.

TTFN

Damn - nearly got excited

[BigBadBri]

until I read the howto and realised it's QOS and not layer-7 redirection.

Now that would be useful to have in the kernel.

I know you can do a certain amount with Apache, but to be able to slot a nice little Linux box in where an Alteon would normally sit would be a)cool and b)cheap.

Re:Damn - nearly got excited

[Scott Laird]

You mean like LVS?

Economics of Development these days

[appleLaserWriter]

I suspect there are uncountably many other p2p systems, but as you have already noted none of them have the adoption rate of Kazaa.

The problem of adding to Kazaa is that it is uncertain what you will get out of it. Not uncertain in the sense that I was uncertain that my MSFT options would ever have value (they didn't), but uncertain in that when you identify your effort you open yourself to legal sanction.

That coupled with the scarcity of programming work today may explain why we see innovative security tools and p2p software that is still focusing on 1998-99 technology.

Yippee!!

[archaic0]

Finally, I can make sure my pesky mail doesn't slow down my pop-up ads!

Not that you would...

[Bradee-oh!]

but if you were considering deploying this on any server of major importance, you may want to notice that they moved from 0.0.1 release to 1.0 release in 11 days. I for one, am now even more eager to fire up this patch and then break it. :)

Re:Not that you would...

[Bradee-oh!]

Scratch that. I'll rtfa a little more carefully. 0.0.1 to 0.1.0 in 11 days is a bit more confidence inspiring, I suppose. Still looking forward to breaking it :)

I think I'm gonna sue!

[L7_]

My name and thier protocol, it's kinda like Phoenix browser and database!!

Trickle

[Earlybird]

For those not ready to upgrade to Linux 2.5, and for those on other platforms, there is Trickle, a userland traffic shaper for Linux, *BSD and Solaris. It works on a per-process basis (or on groups of processes to limit aggregate traffic consumption), does not require root-level access nor kernel patches, and is, of course, open source.

Does SCO...

[shanestyle]

own the OSI model? =-).

Re:Does SCO...

[Anonymous Coward]

Latest news:
------------

SCO had formal source code documents
that proves without any doubt that SCOSource
owns the OSI 7-layer model and that the consortium that created the OSI 7-layer model
STEALED patented UNIX source code modeled ideas,
that could not be derived without violating
the SCOSource NDA.

this could be a help for me at home

[Archfeld]

My bro is an avid Kazaa/WinMX Pr0n colletor, and I'll come home and find 25 people downloading from him and his HUGE collection of trashy pr0n.
I'd like to be able to leave it running in a weighted environment without having to manually decide what share he should get or kill all the downloads :)

Re:this could be a help for me at home

[Anonymous Coward]

What's your brother's Kazaa username?

Re:this could be a help for me at home

[smeenz]

But Kazaa uses HTTP, so to the shaper, it would presumably look identical to your web browsing traffic.

Re:this could be a help for me at home

[Archfeld]

DirtyD, I think
somehow that is appropriate :)

Re:this could be a help for me at home

[X_Bones]

Jeff, is that you? Please don't tell Mom this is why our shared connection is so slow, OK?

Re:this could be a help for me at home

[JLester]

Not at Layer-7, that's what makes it ideal. The expensive shapers like Packeteers work the same way. It doesn't matter what port, it actually looks at the traffic itself at the application layer.

Jason

Re:this could be a help for me at home

[smeenz]

HTTP is a layer 7 protocol.. and both IE and Kazza make HTTP requests to a host to retrieve a file, so on that basis, at layer 7, Kazaa, IE, Mozilla and a bunch of other apps all look the same in that they're all HTTP clients.

Having said that, I you could check for X-Kazaa-* tags in the header and differentiate them that way.. except X- tags shouldn't be treated as a reliable identifier because they are, by definition, not a standard...

Or you could go by the fact that browsing on a port other than 80 or 443 probably isn't going to be "normal" web browsing traffic.

It really depends on how much effort they put into analysing the packet/datagram/data.

Re:this could be a help for me at home

[abdulla]

Are you sure its your brother that's the trashy porn collector? ;)

Re:this could be a help for me at home

[smeenz]

I just downloaded their protocol definitions and took a look - they differentiate kazaa and generic http by looking for the "user-agent: kazaa" line in the header.

so there you go.

Re:this could be a help for me at home

[Cyrus2001]

thats giving me an idea.;) make user-agent: Internet Explorer even more slower than it is right now.

Re:this could be a help for me at home

[JLester]

I haven't looked at this one, but I know Packeteers can still differentiate between regular web and Kazaa traffic. That's why so many colleges and universities have purchased them.

Jason

Re:this could be a help for me at home

[Krapangor]

You are not very clever.
In this case the most effective traffic shaper would be replacing all you brothers pictures by the infamous goatse image.
And as an additional benefit you'll place many people of the path of respectability and cleanliness.

(offtopic) Re:Good or bad?

[Anonymous Coward]

Yep, if you have c:\... They disabled access to "program files" and "winnt" directories. (including temp directory, so if you get some files created there and they get your profile over the disk quotas, you can't even delete them, and since you're not allowed to save anything, the logout session can't be saved, the only logout method is the reset button.)

Posting anonymously to save my freshly earned karma modifier from harm if this gets modded down as offtopic (which it is). - SF.

Re:(offtopic) Re:Good or bad?

[SmittyTheBold]

Dude, we all turn off the +1 modifier anyway, if you want to get to 2 on a post you have ot earn it ;)

Re:(offtopic) Re:Good or bad?

[Anonymous Coward]

Dude, we all turn off the +1 modifier anyway, if you want to get to 2 on a post you have ot earn it;)

I would think mosy people leave it on. I actualy have the -1 AC penelty off so I can browse at 1 and still see ACs but not users that have been modded down. And I post without my +1 karma bonus (only use it if I feel it is warented)

behind the times

[Anonymous Coward]

FreeBSD has had this for years. Why keep on reinventing the wheel? Fight NIH!

Re:behind the times

[shaitand]

show me where FreeBSD has this and I'll be happy to listen. Last I checked FreeBSD has the ability to shape based on port... that's what linux has had for a long time. This is filtering based on signitures matches in the data contained within the packets... I'd be more than happy to hear about what there is for BSD that does this.

Boobies!

[krumms]

Hmm ... ALPS for Linux ... sounds like it would go perfectly with my breast-shaped keyboard!

Whoa

[brsmith4]

It was just a few months ago that i needed a solution like this but had to bite the bullet for one of those $15,000 packetShaper routers. This is great and it sucks at the same time ;(

Re:Whoa

[demaria]

That packetshaper you got, assuming it's what I think it is, is a very advanced beautiful piece of machinery. This stuff is computationally expensive and complex, and has a good and frequently updated collection of signatures. It's money well spent.

Sure!

[dark-br]

More p0rn less SPAM :)

Re:Layer 7???

[Anonymous Coward]

Just because TCP/IP doesn't recognize the layers doesn't mean they aren't there...

The OSI model is a MODEL.

Re:Layer 7???

[JohnFluxx]

You filter the equivalent of the 7th layer.

duh

The uni I'm at handles bandwidth use "socially"

[smcv]

The computing service (who're responsible for the university and student networks) monitor general levels of traffic; if you've been using a lot of bandwidth for extended periods of time, they'll contact you, ask you what your excuse is, and tell you to slow down. The idea is that after a few warnings they'll disconnect your network socket, but most people take the hint.

Just looking at the stats rather than the protocol is also good for plausible deniability, since they don't particularly want to know the specifics of illegal file sharing and the like; they have been known to specifically stop a Direct Connect hub, but IIRC that was after another student had a private feud with the hub operator and decided to report them, after which the computing service had little choice.

They also occasionally scan random IPs for common server and trojan ports, then connect to some servers to see what banners etc. they produces, but this is more an anti-h4x0r thing than anything else; they don't even seem to mind students running low-traffic web servers on port 80, but they're likely to contact the student and verbally cluebat them if the server says it's IIS.

As long as you don't care about performance?

[frovingslosh]

As long as you don't care about performance.

Depends on the size of your incoming pipe. With my ~1.5meg DSL connection I expect my current hardware could keep up with it quite nicely; the hardware ain't where the bottleneck is. If I had some gigabyte fiber coming into the home it might be different, but for now hardware performance will not be a problem. What I can do with the new technologu is a different matter. Would still like to see more ideas than how to share porn.

Re:As long as you don't care about performance?

[shaitand]

It's not the size of the connection, that gigabyte connection you refer to is still pretty puny compared to what any pc today can handle... it's the traffic itself, it's the burden of resolving 50,000/requests a second. In an of itself, checking a routing table and sending a connection on it's merry way is a relatively efficient operation... but it can add up real quick.

Re:As long as you don't care about performance?

[Darby]

What I can do with the new technologu is a different matter. Would still like to see more ideas than how to share porn.

No thank you.
That alone will do fine.

I find your ideas intriguing and would like to subscribe to your newsletter.

I feel safe using this patch!

[Anonymous Coward]

+/* XXX Is it ok to do nothing here? This gets called each time a filter
+is added (not sure why). */

This ain't touching my kernel...

Re: Kazaa

[benjamindees]

Damn Kazaa users, how dare they saturate the network and degrade your Kazaa experience.

That's unpossible!

[benjamindees]

This 'Linus' Gnome' clearly violates the First Law of Thermodynaics.

SSL encrypted connections?

[SoulDrift]

I'm curious about this... how much luck would the traffic shaper have telling apart, for example, an SSL-encrypted IMAP session, HTTP session, or Jabber session. If they were going to arbitrary ports how would it tell them apart?

Does it need to perform its own man-in-the-middle attack to get at the transmitted data?

Packetlogic already does it!

[unix-oldtimer]

Guys, the XMMS team has been busy with exactly what these L7 guys are trying. Check out http://www.packetlogic.com No wonder XMMS is stuck at 1.2.7 :) It runs on Linux and blows the doors of anything Cisco, Allot, anybody else can do with Layer 7 protocol shaping/firewalling and better yet, you even get real-time surveillance.

Ssshh

[DreadSpoon]

Don't tell my boss; he might make me put this on the router so his EverQuest sessions don't start lagging when some secretary starts doing useful work online...

Re:Real nice, destroy another market.

[TheNetAvenger]

the difference between bill and linus in this case is that bill FORCES it on you, linus does not.

Not that there is an HTML browser in every windows manager for XWindows or anything. Oh wait, there pretty much is. Um...

And the difference between them and MS is? Oh wait, I know, MS is actually an HTML rendering engine for developers, not just an inaccessible browser that is built into the XWindow desktop managers.

IE was a HTML rendering technology; the IE browser was just the pretty face in the early days. It was designed so developers could have their application display an HTML page just like they can tell Windows to draw a Button or show a Picture or put text on the screen.

It was an extension that was added to Windows just like when the ability to display a picture by just calling a few APIs was added to Windows.

Microsoft wanted Windows to be able to offer HTML display support natively, the browser was not something they set out to conquer the world with. At the time, they were more focused on MSN and its 'Explorer' based online browsing service.

People complain about IE being shoved down their throats, but I don't see them complain about the BITMAP rendering engine in Windows being shoved down their throat because MSPAINT is included in windows.

If you don't like IE, delete the Icons for it. You can even delete the HTML rendering engine, but it will break any application (such as ones made by my company) that use the HTML rendering engine to display help and HTML content for our users.

Additionally, I also don't see people complaining about the Browsers being shoved down their throats that are included in almost every Windows Manger for Linux. I also don't see people complaining that IBM shoved a browser down people's throats with OS/2. (If anyone is old enough to remember, it was the first mainstream PC OS that shipped with a built in browser - Windows was not. And in OS/2, it was just a browser, not something developers could access or use in their applications.)

Give up the 'We hate Microsoft Goat', especially over something like IE and the few megs that is added to the core of windows that gives developers a mechanism for displaying HTML natively in Windows. This is no different than the BITMAP rendering core in Windows or any other piece of Windows that makes Windows what it is and the GUI of Windows what it is.

Why does everything from MS have to be bad and everything from Linus have to be good. Please stop this crap. This has become a religious war that is really stupid when you step back and look at it.

In fact it has gotten so bad here, that there are even flame wars and religions popping up on which Linux distribution is better or whether OSX, Solaris, AIX, or FreeBSD is better.

What happened to evaluating products for what they are instead of just hating everyone else?

Please don't reply with the 'MS Bully, Monopoly' stuff, Sun, IBM and Apple have all done very 'questionable' stuff with licensing and anti-competitive practices. -And yes I can provide examples.

Nope

[BigBadBri]

Wish it was, but I've looked at it and it'll only do portmapping (layer 4).

What I really want is to be able to redirect based on packet data, rather than port number.

If you can tell me how to do this on LVS, then I'd be much obliged - I can't see how to do it...

Re:Nope

[Scott Laird]

Ahh. I see. Combining the two (l7 and lvs) shouldn't be rocket science. Give it a few months.

Re:Nope

[bluehand]

Hi
yo've really got me curious
what are you triing to do ?! do you care to explain :)
you said you want to redirect based on packet data but i cant see where it would be usefull
please enlighten me :)

Re:Nope

[BigBadBri]

Scenario - a custom app, running over the Internet with a client talking to a bunch of hand-rolled servers listening on various ports on various boxes.

This is fine for a directly connected client, but can be a total PITA when running thro proxies and firewalls.

The hand-rolled servers all use their own protocols, easily identified by the start of each request.

Ideal solution would be to have the client connect to port 80 for all services, and hand off the sessions dynamically using layer 7 information.

The solution would also need to keep sessions open to each server while the client is logged in.

It's just a simple solution to a problem that was identified far too late in the development cycle...

Plus, I like the idea of layer-7 switching with persistent sessions just ofr the sake of it.

Re:Real nice, destroy another market.

[Anonymous Coward]

i think this is inappropriate

yeah, this is true

Code

[Daath]

It doesn't even see the code anymore, just - redhead - blonde...

and no...

[BigBadBri]

I'm not nearly a good enuff programmer to start screwing with kernels, before you ask... ;/

Re:Layer 7???

[marcilr]

Nice troll.

Re:Real nice, destroy another market.

[shaitand]

without a doubt, along with all the other advanced routing features of linux... none of them are default.

Protection against Slashdotting!

[Kwiik]

Vital services can still get in while your webserver remains overloaded.

Why are we advertising this? We're ruining the basis of our whole community!

Works like a charm

[flailking]

got a full t-1 going through a linux router box with QoS, I started out just scheduling based on port, but now packet size is the best way. The pipe could have very heavy traffic, but yet my shell is a smooth as silk, DNS querys are quick. I recommend it highly

your sig

[Planesdragon]

The American government is officially totalitarian
This is not a nightmare
It really is this bad

Please don't insult the suffering of all those who have actually lived under totalitarian rule.

So, if you happen to act like a terrorist the government will treat you one. They might even be blatantly racist and overzealous. But they're not totalitarian.

Dissent is still very much a part of America--and no one, yet, has been punished just for speaking out against the government. (Well, not citizens by the government. A few university professors and private citizens have lost their jobs, and a few immigrants have been forcefully emmigrated, but you get the point.)

(Not that Republican domination isn't that scary--[just what we need, tax cuts in wartime]--but it's not quite totalitarian. Might as well call Canada Communist.)

Good idea, perhaps in the wrong kernel subsystem?

[jjgm]

IMNVHO this would be better done in the netfilter (aka iptables) kernel subsystem than in the qdisc system. Not only can it then be used for more than just rate limiting (but also for firewalling, transparent proxying etc), but there is limited similar functionality already available in the patch tree (the STRING match) that could be extended to run regular expressions. Importantly, netfilter can communicate with the qdisc subsystem by tagging packates appropriately, enabling the rate-limiting and advanced queueing already discussed. I'm sure that the core work already done could be ported from one subsystem to to the other, although perhaps not overnight.

The fwsnort tool has proved the concept, I think, by translating rules from the Snort intrusion detection system into iptables rulesets.

For more complex protocols, such as FTP or IRC DCC, you'd also get Netfilter's connection-tracking support code for free.

I also think beginners will find the iptables tool easier to deal with that the tc tool, which has a steeper learning curve.

The equivalent Cisco technology, NBAR

[jjgm]

The Cisco equivalent of this is called Network-Based Application Recognition (NBAR). Rather than use regular expressions, Cisco ship PDLMs (Packet Description Language Modules) that can loaded and unloaded whilst IOS is running, much like you'd get by combining Netfilter's ip_conntrack_helper modules with the ideas these guys have.

(I still think they should be doing this inside Netfilter rather than qdisc)

NBAR can also be - and is - used to filter network worms at ISP borders, by matching the specially-crafted URLs used to compromise vulnerable systems. For example, here's the Cisco config to catch the Nimda worm.

Re:The equivalent Cisco technology, NBAR

[jjgm]

Maybe this is a better example. Cisco vs Code Red.

I guess you could fool the packet filtering

[kramer2718]

If you were going to design a new network application, you could add a wrapper around your packet to make it look like an http packet or some higher priority packet.

Re:Gimp?

[Anonymous Coward]

Companies won't give me a job because I'm drawing ass and tities in Gimp. So I throw eggs at their front doors.

Re:Real nice, destroy another market.

[arkanes]

IE started as just another application. Bundling it into the MSHTML components and ActiveX controls (which is rather more than a HTML renderer) didn't happen until much later.

Paint is a wrapper for metafiles, not bitmaps. So nyah.

Let me get this straight...

[Kjella]

You complain about the current bandwidth usage of your brothers pr0n collection, but when asked, you provide his KaZaA username on slashdot. That's like putting a gun to your head, pull the trigger and blame the bullet for harming you.

Kjella

Re:Let me get this straight...

[DickBreath]

You complain about the current bandwidth usage of your brothers pr0n collection, but when asked, you provide his KaZaA username on slashdot. That's like putting a gun to your head, pull the trigger and blame the bullet for harming you.

If more people download it, then more people are likely to share it from their own collections. So maybe it is a winning strategy. Besides he was only complaining about his brother's pr0n collection when he wanted to use the internet, not the rest of the time.

Re:Real nice, destroy another market.

[TheNetAvenger]

IE started as just another application. Bundling it into the MSHTML components and ActiveX controls (which is rather more than a HTML renderer) didn't happen until much later.
Paint is a wrapper for metafiles, not bitmaps. So nyah.

Ok, MSPaint is a bitmap editor, it does not handle metafile (WMF or EMF metafiles) so nyah yourself.

Secondly, the original code that MS licensed to create IE was intended to be used as an engine for HTML. This is in the documents that are available regarding their meetings with Netscape when they wanted to originally license the Netscape technology instead of Mosaic. Go look it up, it is court record now.

As for them making it an ActiveX (OLE) based engine as apposed to the OLDER mechanisms for components in the GUI, has nothing to do with the intent of what they were doing.

This also shows that you have little knowledge of ActiveX and why they choose to use this mechanism for implementing the engine as opposed to just creating a new API subset for IE.

Considering you can't discern a metafile from a bitmap shows that this debate is probably not worth continuing.

there are problems, and there are problems

[Archfeld]

This one is not an insurmountable problem, I AM the network admin, and the router is like 3ft from me, I was just thinking a dynamic weighting system, ala mainframe would be the IDEAL solution to the problem.

Fantastic

[Glyndwr]

This is utterly redundant, but I can't help but pat the team responsible for this on the back anyway: this is a fantastic addition to the capabilities of the Linux kernel. Well done guys!

Lose weight; eat more!

[Anonymous Coward]

Dearest crackhead moderators,

Why has this blatant, non-paid marketing drivel been modded as being "Informative"?

Please fix it.

Thanks.

[Next week: "Informative" magazine subscription offers, "Interesting" cheap long distance, and "Insightful" YOU'VE ALREADY WON proclaimations.]

Yet another "we intend to..." sourceforce project?

[pe1chl]

After seeing all the enthousiasm I read the referring page and their FAQ. It seems to be yet another 0.1 release of something that could be done. This is not something you could actually use and the n maybe refine a bit...

Also, from the FAQ it seems they are on a fundamentally wrong track:

A: Linux Quality of Service only looks at outgoing packets (once a
packet is in your computer why would you want to slow it down?)

First, there really is a point in slowing down incoming traffic, namely to throttle down the other side and avoid excessive queues to build up on a slow connection. It seems they did not get that.

But worse, it is a fundamental flaw to only look at outgoing traffic, and it also seems to be inferred that there is no state kept for a connection (they say it "classifies packets", not that it "classiefies connections"). This makes it useless.
E.g. when the incoming packet says "HELO domain" you may not want to slow that down but you may want to put that CONNECTION in the SMTP category. What can you do once the connection is transferring the SMTP data?

What you need for any packetshaping to be successfull is a state kept per connection that allows further shaping based on earlier matches.

Re:Good idea, perhaps in the wrong kernel subsyste

[lithium100]

I agree,
By putting this capability into netfilter you could also vastly improve on IP Accounting. Now not just accounting by service port but also by Application!

Luinux

[Anonymous Coward]

Try this: Luinux.
Still has some shotcomings but i'm working on them.

ssh forwarding

[Anonymous Coward]

of course I presume if I access something via an ssh tunnel or vpn (providing they have priority) this can't have any effect.

If you want more information ...

[staf]

The layer 7 filtering is only a small part of a shaping setup. First of all, you need a htb or cbq setup. This changes the behaviour of the packets you send. You can create multiple cbq/htb classes. Each class can be seen as a seperate traffic channel that sends the packets that are placed in this class. Each class can have it's own config like minimal and maximal bandwidth, delays and so on.

To place the packets in the classes, you need some filters. The 2 most used filters are fw (it can use iptables marks) and u32 (it can use any bit in a packet). So basically, you are limited to ip-addresses and ports.

But with l7 filtering, you can look in the packets and use the contents of the packet to filter the packets and place them in the class you want. To do so, you have to be able the recognise the traffic. And that's what the extra kernel patch will do.

The more people are looking for ways to recognise patterns, the better. So if you find a new way to recoginise kazaa traffic (use tcpdump or so to examine the packets), send it to the l7 patch author so he can update the protocol definition file.

Some handy URL's :
lartc.org : information about "linux advanced routing and traffic control"
docum.org : my site :) with more info about traffic shaping with cbq/htb.

Moderators on drugs?

[Anonymous Coward]

> It's not impossible to do throttling on inbound packets,

Why in the hell are comments like that given moderator points? It's yet another example of the garbage that makes /. less and less useful.

You can not directly control inbound traffic. Think about it. It can't be done. You can't control the other end of the connection. The other end should slow down if it doesn't receive ACK's, but again, it is impossible to throttle inbound packets. If you could, then most DOS attacks would not work.

Layer 5!

[LNN]

Judging by the patterns that comes along with the l7-filtering package, I'd say this is actually a layer-5 filtering process. All the patterns check only for the strings that are specific for establishing and maintaining the sessions, such as the POP3-server acknowledging that it is ready to recieve data, like the HTTP part where it looks for the request of a file.

What the patterns don't look at is the presentation layer, such as what kind of file it's trying to get, or in what encoding format it is transfered. It also doesn't care of the contents of the file requested, hence it is neither level 6 (presentation) nor level 7 (application), but simply layer 5 (session).

So, the patterns that come along with this are only layer 5. This however, doesn't mean you can't do layer-7 filtering. It seems easy to me to add words that just can't be searched for with google. just add something like:

sexgoogling
GET \/search\?q\=[a-zA-Z\%]*(sex|xxx|porn|pr0n)[a-zA-Z \%]*

to a .pat file.

--
Niklas
Quidquid latine dictum sit, altum sonatur.

Missing Something

[Cokelee]

I think everyone is missing something more important than blocking P2P networks. How about blocking SPYWARE clients (gator, cydoor, et. al.). I've blocked their ip address, but they simply use a different one. What about blocking THOSE packets because of their contents, because of their security risk. That seems like a much more worthwhile goal to me.

Re:Missing Something

[Anonymous Coward]

Sorry if this isn't exactly news to you, but if you like blocking SPYWARE clients, try Ad-Aware 6 or Spybot Search & Destroy.

Re:This will be nice - PCs faster than boxes?

[Glasswire]

Your astute observation that "...but it isn't going to beat custom hardware of the same generation..." (my italics) refutes your argument if you know what the state of dedicated HW vs off-the-shelf is right now.
In order to keep their BOM (bill of materials) parts cost down, vendors like Cisco put JUST ENOUGH processor, ram etc into those dedicated appliances to run highly optimized stacks like IOS effectively within the Quality of Service guaranteed by the specs.. Which often results in CPU/RAM combinations in the Cisco box that have NOWHERE near the raw throughput of a commodity modern cheapo consumer PC system.
So, your point about generations is right on - the reality is that a $500 PC might have many times the raw performance of that multi-thousand$ highly optimized network appliance device which is usually (for cost -and design continuity and other reasons) coasting on much older generations of hardware - because it doesn't NEED to be faster. And it often has expensive, specialized ASICs and network processors to make keeping pace with current generic CPUs even less necessary. By comparison, cheap PC can run a generic kernel and stack, lose a LOT to inefficiency and still potentially outperform the dedicated appliance. ( I have no doubt that if Cisco built all their appliances with 3GHz cpus with 1/2 gig of RAM, with all the other advantages they have there would be no comparison - but they generally don't)
So the question is, what's the cost benefit comparison between efficient/proprietary/expensive/dedicated and inefficient/generic-high-throughput/cheapPC systems, if the PC can perform well enough?
Cost effectiveness comes down to cases, but it is not, I suggest, a slam-dunk for the Cisco and other specialized boxes.