Slashdot Mirror
Ask Slashdot: Low Cost IP-based Traffic Shaping?
Deuteron
asks: "Hi! I work for an ISP and we're about to
deploy wireless net access and need a way to limit people
to the bandwidth they pay for. We're planning on starting
out with offering 128k, 256k, and 1M links. The wireless
hardware itself
(Breezecom if you're
interested) will handle the 1M part for us. The tricky
part is the lower speeds. I've done some extensive checking
and haven't found any IP or MAC address based shapers as
of yet. Can anyone point me to some free or extremely low
cost solutions? Any leads would be greatly appreciated!"
We are doing this exact thing to offer internet service to several companies off of out T-1 connection. Class based queueing works much better than the traffic shaper. To use it, get the cbqinit script off of Freshmeat, and turn on the experimental stuff under the 2.2.x kernels to enable class based queueing. After you do this, sizing the traffic is as easy as editing a text file to mandate bandwith on a per network or per class basis. It is much easier to configure and in my opinion shapes the traffic much better than the traffic shaper.
darkdave@uwyo.edu
[These links are long. If they get broken, go to www.cisco.com and search for "Committed Access Rate".]
Some of the more interesting versions of the Cisco IOS (the 11.1CA and CC tree I think, and v12 if you're feeling brave) will perform incoming and outgoing traffic shaping. The closest to what you'd like is probably Committed Access Rate.
It can be applied directly to an interface to limit all IP traffic, or you can define an access list so that it will limit all traffic that matches a particular protocol, QOS flag... or your customer's IP subnet.
This last option is useful to limit a customer's access to the internet at large while still giving them full speed access to, say, your local mail or FTP server. You perform the limit on your connection to the rest of the world, using a different rate limit for each customer.
The v12.0 documentation is linked above, or check this CCO search.
Dave
--
The Breezecom stuff is directional point-to-point, and not shared, so it's perfectly safe to sell the whole bandwidth of a connection.
See the traffic shaper pseudo device support in Linux-2.2.x. According to it's documentation it can shape from about 9600 to 256kb per pseudo-device. Documentation lives at:t xt
/usr/src/linux/Documentation/networking/shaper.
You might also want to look into the Linux Firewalling code if you're going to use it for traffic shaping as well. See:
filter and ipmasq.txt in the same directory.
Cheers!
J. Maynard Gelinas
I've got a collection of all the documentation I've been able to find on the 2.2.x network stack, including the QoS stuff.
It's all at my linux 2.2 site, check it out. Hope it helps.
Yes, you can limit downstream bandwidth. Routers have QoS, the linux kernel has several shapers to choose from, etc. But upstream might be more difficult. A malicious customer could simply type ping -s 1500 -f www.somewhere.com, and flood the entire wireless link he/she was on. There isn't an easy way to fix this. You can, however, confinscate their equipment and/or report them to the FCC for causing harmful inteference if they do decide to take down the link. I know that several cablemodems use snmp to inject QoS filters at the hardware level. Maybe there's similar offerings for other NICs. It could help during an emergency, and also to help limit upstream bandwidth.
--
Check out the IPChains HOWTO. You can implement TOS scheduling in your chain.
The previous poster suggests that you only throttle when bandwidth contention is an issue, suggesting that it will build goodwill.
I would suggest the opposite. You will certainly have a surplus of bandwidth when you roll out your service. If you open things up wide to everyone then you will probably have some very happy customers for a few months, and they will doubtless tell their friends. Soon you will have a growing customer base of people who are coming to expect more than they pay for. Then you have to start throttling down bandwidth. People are now getting less than they were getting before. Even if they are getting what they paid for, a lot of people are going to feel like they are getting shortchanged and they will start complaining vocally.
Maybe this isn't such a bad thing though. If you build a subscriber base quickly on word of mouth because you are giving away spare bandwith then you might be better off than if you build the subscriber base more slowly, or you have to advertise to build it quickly. It depends on how much the malcontents cost you once you have to start throttling connections, vs the costs of slower growth, or the costs of advertising. Unfortunately, the cost of the former is hard to predict.
As for dealing with the daily peaks of bandwidth utilization, again, I think people will tend to react better to consistant performance throughout the day or week, rather than wide fluctuations. On the other hand, if it is possible to allow maximal thruput on short (10-40k) spurts and throttle it down on longer downloas, then it becomes more difficult for people to quantify and less likely to engender ill will.
Well. its like any other area of life. "If other people are breaking the laws, then the laws must not mean much, so its OK if I do it too.." kind of thinking usually dosen't work well.
:-)
There's often reasons behind the laws, that put
them there in the first place. Oh, sorry about the "its the telcos, preventing competition" bit, these restrictions have been there (in one form or other) for much longer than those current issues.
Another thing, if you are going to operate illegally and possibly interfere with other services (the real reason the rules are there), I'd pick a piece of band that DIDN'T have Hams around to notice and probably seek you out over it, rather than someplace not so traveled.
Another another thing.. being an engineer and knowlegable in radio (and a Ham), there are problems with just upping your power. Narrower bandwith (like voice) = more sensitive recievers, and better range for same power = lower data rate in data mode. Higher bandwidth = faster data rates = less sensitive recievers = less range for same power/antenna situation. Its more of a challange for hi bandwith radio. Also, and worst, is the situation of multipath, and one station interfering with many others due to too much power. Digital signals need very clean signals, typically.. noise that you can hear and understand voice with can totally obliterate a digital connection. Having signal bounce back off mountains, tall buildings, airplanes, etc. can mess up a normally clear path, and that gets worse fast with more power.
There's much to consider in something like this, too much for a short mail. Basically, low power and lots of antenna gain (which equals directivity, i.e. dishes or beams) is the better direction to go. There is a website that has a paper on these issues, written by a Ham researcher in digital high speed Ham networks, if you are really interested. I belive its called the "Higher speed Packet" page, Packet radio being the commonly used digital mode of networking.
Search on Packet Radio, high speed, to find it..
Anyway, I don't mean to say you shouldn't try something, since I really don't know the ISM laws,
it may not be a problem to add antenna gain, and leave power the same, depends on how they wrote
it up. That would work better, be cheaper, and the more directional you get the less interference you cause to whoever is your neighbors. Antennas are pretty easy and cheap to make, once you know a little bit about whats what. Get a copy of the Amateur Radio Handbook at the local library, or buy one, for starters...
Didn't mean to write a book, but didn't want to see a place where common courtesy was also the better result way to go get by..
Hope it helped...
FreeBSD has a feature called dummynet. See the
following URL: http://www.iet.unipi.it/~luigi/ip_dummynet/
There is these little black vans with just 'FCC' on the side. I really didn't believe that they existed until I saw one with its 10 or more antenna on top.
Higher power is not a problem until you start interfering with somebody's TV set. They end up bitching to the FCC. Then the FCC sends the little black van to your area.
I never thought that they ever caught anybody. Until I saw some press clippings about some guy whose equipment was confiscated and he paid a hefty fine.
Well you say, I am not going to interfere with somebody's TV, the frequency is too high.
Then you end up interfering with somebody's cell site, or somebody's direct TV, in the future it will be TV all over again with HDTV micro signals. But I can guarantee before you interfere with any of the above, there is one thing you _will_ interfere with -
Aunt Myrtles old electric organ down the street. Every neighborhood including yours has one - an electric organ. They will pick up _any_ miss directed RF no matter the frequency. They are especially good with hi-power CB.
I personally don't use CB. I did have one once, and when I did turn it on, a couple of times. It seemed that there was this guy who would flip his multi-KW on at 8 oclock and slam my needle. Then he would go about making these weird slow throat noises with reverb mixed in.
It's those dumbshits that piss me off. I suppose I could have triangulated him quite easily with a couple of electric organs - or toasters for that matter. Then I could have stuck a needle in his coax and smoked his ass out - just never got around to it.
Just this last fourth some worker was killed when the fireworks spontaneously ignited that was being set up. It has been theorized that stray RF was to blame. My experience with the CB idiot certainly has me thinking. Frequency too high or just a little more power?? Just don't interfere with a plane's navigation system and send it down.
This sounds like an exploratory question for someone who hasn't yet come up with a business plan. Are you truly expecting to provide a huge amount of bandwidth to thousands of customers and then try to use a free linux program to enforce your ToS? I'd love to see you succeed with this, because the latest Linux kernels have some traffic shaping in them, and you could help out the coders with a real world test bed.
:-(
Why then aren't you taking advantage of Breezecom's built in Maximum Information Rate Class of Service? Do they charge too much for the management software? Have you even talked with their account reps? Their whole business is aimed at ISPs trying to do exactly this same thing. Breezecom modems emulate a serial connection, but their cheaper LAN products emulate an ethernet link. Their modems have a built in rate limiter, their LAN replacement is only aimed at office environments and not ISPs. It sounds like you have chosen the cheapest products, and are now trying to add something for nothing.
To properly implement a per user CoS, you must assign a static IP address to each end station, and possibly lock it down to a MAC address. Then you can set up a traffic shaper for each customer with little hassle. Easiest way to do is have different customers in different subnets, so all the 128k people are on one subnet, 512k on another. Rule writing is easiest that way. If you try to do CoS on systems dynamically grabbing an IP address (DHCP or equiv), you will spend all your time writing custom code to match addresses to customers to ToS to shaper rules and so on. Avoid it.
The best solution for packet shaping is Packeteer, who make a great box with a fairly good interface. The cost isn't that high compared with how much you will spend trying to implement the same thing with free software. Just buy one of their boxes and throw it in line with your ISP, then configure it a little and you can mostly forget it.
The next solution is Cisco, who have a bunch of different options built into their IOS for crude packet shaping. Presumably at some point you will have to buy a big Cisco router, probably when you get more than 50-100 customers. Since you are an ISP, what routers are you using now?
The cheap but limited solution is the latest linux kernel with IP Chains and Class Based Queueing. It should scale to handle a few subnets, each having its own CoS, but may not do 512k or higher. Crude, but should keep your bean counters happy until you have enough paying customers to afford something to cover a bigger user base.
Also check out NetBSD shaping, since I haven't yet.
No matter what you do, always enforce your bandwidth policy from the beginning, because you will lose all of your original customers later when you start to enfore the policy. Never give customers free bandwidth even if it is available, you are asking for a customer relations headache down the road if you do. Poor customer relations is the main reason small ISPs go out of business. This is the voice of experience learned the hard way
Remember, packet shaping is a one way process, if you want to limit the connection from the user back towards the internet, you have to install something at the customer end, either a small box or software on their machines. A nightmare you probably shouldn't touch.
Good luck, and tell us what solution you end up with and how it works. We geeks are a curious bunch.
the AC
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
http://lwn.net/1998/1119/shaper.html
Check out NistNet. It should do everything you are looking for.
http://osi.ncsl.nist.gov/itg/nistnet/
Wayne Walker Unix/Linux Advocate, SysAdmin, MUD addict in remission