Slashdot Mirror
Ask Slashdot: 802.11n Bake-Off Test Plans?
First time accepted submitter Richard_13 writes "I am seeking a bake-off test plan for an enterprise size deployment of 802.11n wireless. We are about to go to tender for a large scale deployment of 802.11n controllers and APs — and I need a bake-off (benchmarking) test plan that is focused on testing the *maximum number* of clients that an AP can handle before it falls over, in addition to the throughput for each client. We intend to test the latest products from the major vendors, Aruba, Cisco, HP, Xirrus, Ruckus, etc.; not consumer products like Linksys, D-Link or Netgear. Any bake-off test plans or useful links to multi-vendor wireless focused web sites would be greatly appreciated."
Just say "benchmarking" and you won't need to explain that "bake-off" means "benchmarking". Wait, what was the question?
Call a meeting of the competitors engineers. It's important you get them all in the room at one time with not too much advance warning of the topic.
Tell them what you think you want. Ask them as a group what you're missing. Then make them as a group come up with an eval plan and cook 'em off according to the plan they come up with.
If you need an independent judge, go to one of the labs that does independent third-party assurance and contract them to provide oversight.
Disclaimer: I've worked for one of those labs for the past 15 years.
Stand back and watch the fun......
Red
We have a box that can emulate up to 128 stations, including wpa, wpa2, etc. It can do
DHCP or static IPs and generate Ethernet, udp, tcp, http, and other higher level protocols,
including IPv6. Multiple systems can be clustered together for additional throughput and
radios. Each system can run on only one channel at once, but can talk to multiple APs
on that channel.
One of our systems can saturate any of the consumer grade APs we have, and some folks have
used it to stress very big systems (conference centers, etc).
Runs on Linux of course!
http://www.candelatech.com/ct520-128_product.php
Thanks,
Ben Greear
We run Cisco WLC and LWAPs as well. Do you have your LWAPs on different ports at the WLC side? That helps. Also different VLANs for different networks and SSIDs is important to keep the traffic down per.
Trolling is a art,
Cisco's WLC/LWAPs do load balancing among access points.
See here
Trolling is a art,
Maybe you could hold a bake sale and offer free wifi for all your customers. Just give them places to sit.
The (not so big) secret is that most WiFi AP rolls over with 8 or so clients. Only a few manufacturers themselves test their products beyond that, and those work all the way to over 100.
The company selling the test equipment you need is called http://veriwave.com./ You can buy the equipment from them and test all the vendors, or even better, just ask them.
They do of course know, since that is how they test their own test equipment. Problem is that they can/will not tell you because then 1. you would not need to buy their product, and 2. AP mfg would fix their products, and Veriwave would not have a market for their products.
Maybe just do some social hacking to get it out of them.
don't cut it off www.mgmbill.org
Virtually all "enterprise" offerings do things(typically by having all the APs reporting back to a central controller [$$$ incidentally] and tune themselves in various ways) that don't violate the wifi spec enough to be incompatible with (most) boring old wifi devices; but which are beyond the scope of the standard.
Load balancing, automatic power level adjustment to avoid excessive overlaps or voids, triangulation of clients and nearby access points, and various other stuff that can be quite handy; but may or may not require exciting license add-ons for.
I looked at equipment recently for wireless using the 'n' protocl - but noticed no mention of IPv\6.
No point of gertting new communications equipment, if it cannot be suicessfully usec with IPv6!
So make IPv6 part of the requirments.
Here are the results:
Vendors were tested with 30 then 60 wireless clients and 1 then 2 access points. So, 1 AP with 30 clients, 1 AP with 60 clients, 2 APs with 30 clients and 2 APs with 60 clients.
1: Cisco - Somewhat surprising. Great client density/bandwidth. Good load balancing between APs. Good management interface.
2: Trapeze (now Juniper) - Great client density/bandwidth (just a little slower (read less bandwidth to client, and just slightly less) than Cisco). Good load balancing between APs. Buy the extra Ringmaster management software.
3: Aruba - Significantly slower than Cisco and Trapeze. Good load balancing between APs. Good management interface.
4: Meru - Significantly slower than Cisco, Trapeze and Aruba. They did not have a network engineer available for the test to be present and we were unable to schedule another test before our purchase window closed.
We were going to test Xirrus but the rep we were working with left the company I believe in the middle of scheduling. We looked at Ruckus but were unable to schedule them.
These tests were performed in the spring of 2010. So products may have changed somewhat. You should be able to get demo hardware from any decent rep. We ultimately went with Trapeze after we put everything out to bid. Before that I was sure we were going to get Cisco gear but Cisco came in at twice the cost as Trapeze. We are deploying 128 APs without any issues. Client roaming and bandwidth are great (our primary requirements). All in all no complaints. Certainly liked the price point. Hope that helps!
In my somewhat limited(but rather painful) experience with attempting to use wifi as a serious connection, one of the issues that cropped up a lot was less with throughput, or with number of clients; but with client software behavior in the face of a glitch.
Dicking around at home and the wifi cuts out for a second? Reload the webpage and quit your whining.
Running your basic "enterprise" client configuration(documents directory is actually on a fileserver, authentication through AD, etc, etc.) and the wifi cuts out? Be prepared for frustratingly erratic appearances of apparently disappearing documents, authentication fails, not automatically reconnecting to the fileserver, Finder just twiddling its thumbs and thinking about infinity until that server either times out or comes back, etc, etc.
Even before any APs show up, you can start identifying the likely areas of sheer pain by using netem, switch jiggery-pokery, or just a $20 consumer AP and flicking your laptop's RF power switch: If your environment has client applications that don't play nicely if the network goes all to suck for a second or two from time to time, wifi deployment is going to be Fun.
Honestly, for most applications where wifi isn't a totally terrible idea(ie. heavily throughput dependent stuff), that would be the big focus of my testing(along with how useful the management tools and interfaces are). High throughput is far less valuable than stable connections.
Paraphrased from Pulp Fiction:
[brandishing gun]: "Say 'bake-off' again..Say 'bake-off' again! I dare ya..I double-dare ya, mothafucka! Say 'bake-off' one more god-damn time!!"
The only thing that matters is the bean counters and any ELA you have with existing vendors. Cisco might be good, or it might be crap, but if you have a pimp contract with them and good support, they're getting the contract. Live with it.
I want to delete my account but Slashdot doesn't allow it.
Wish I had some mod points for this one.
All the major vendors should be aware of what is going on at www.bufferbloat.net and have something in place to ensure that their products will reflect new updates soonest when things get fixed. This is an ongoing problem that crept up on the internet tech community and there is work in progress to deal with it but it will take time.
See (for example) Bufferbloat - Dark Buffers in the Internet, 1/20/2011
Been there, done that, paid for the T-shirt
and didn't get it
We run a thousand or so Aruba 125s at school here, covering all 600 or so acres of campus. Those are probably overkill for you (at about $750 a pop), but AFAIK even the lowest-end ones have the same essential features.
Basically, the network architecture puts the whole wireless network on a separate segment, all the way back to the aggregation points. They're gigabit wired into the building routers, but placed on a separate VLAN all the way back to one of the three aggregation points. Each AP is assigned to a controller, and will fail over to a second one if needed. The controllers pass the traffic to the rest of the network.
The controller architecture means you can do some pretty interesting things. Particularly, it means new APs are trivial to install - stick them into the controller's DB and plug it in to an Ethernet cable (it's PoE); it'll go and find the controller and pull down the config and any upgrades to the software. It also allows IP roaming between APs, even if they're in different netblocks. I can walk from one end of campus to the other (7 city blocks) while keeping the same IP and getting all my traffic, through about 150 different APs - much like a cell network. You can also do spectrum analysis through their management console - I once saw them find a broken microwave from all the interference they were seeing across the 10 APs in range a la Dark Knight.
The APs we have will band-steer clients over to the 5GHz spectrum if possible, which can support a huge number of clients, but you need the density for it to make a big difference. If you do, though, you can easily get 30 people per AP, with a few doing massive downloads/uploads and no hiccups. They don't recommend more than that, and in any case it's difficult to fit people densely enough that you wouldn't need a new one for signal purposes.
No, I don't work for them. I don't even work for the Networking department. I just really like the toys - though I suspect I might feel differently if I had to make the purchase! Quality isn't cheap...
I have developed a truly marvelous proof of this comment, which this signature is too narrow to contain.
Lmao..he said he needs to test a big deployment with non-consumer grade gear. Your approach outlines the least of his worries.
What is the building(s) architecture, power and existing cable plant like? (concrete and steel, stick; adequate/sub adequate/surplus power; CAT-3, CAT-5, CAT-5e, CAT-6, fiber, thin-net, coaxial) Access points need power and users need to be able to connect to something worth connecting to wirelessly
How is he doing authentication? (802.1x with cert?, challenge response?)
Is it against a central directory server? (what is the topology like to it and how well is it connected, as well as AD, NIS, LDAP?)
What is the acceptable bandwidth minimum per connection? (determines number of APs based on user environment, I.e., municipal-commercial open v. Education v. Corporate v. Government v. Medicine; beyond architectural and interference environments)
What's my budget and timeframe for deployment? (vendors will want to know this to not only help you meet your needs, but potentially offer access to unannounced products should the size, needs and timeframe line up with PR possibilities and $,$$$,$$$ you intend to spend)
As far as benchmarking (I refuse to use that other bs term the OP proposes), do the math for your needs, find the vendors that advertise specs that meet those needs and then call reps for 30-day demo units. Test the specs yourself, read reviews, talk to other big installations with the gear, then buy an initial number of APs and test a small deployment as a pilot before committing to a larger project with the vendor(s))
Lather, rinse, repeat. Your mileage may vary.
NEXT!