Slashdot Mirror
Ask Slashdot: 802.11n Bake-Off Test Plans?
First time accepted submitter Richard_13 writes "I am seeking a bake-off test plan for an enterprise size deployment of 802.11n wireless. We are about to go to tender for a large scale deployment of 802.11n controllers and APs — and I need a bake-off (benchmarking) test plan that is focused on testing the *maximum number* of clients that an AP can handle before it falls over, in addition to the throughput for each client. We intend to test the latest products from the major vendors, Aruba, Cisco, HP, Xirrus, Ruckus, etc.; not consumer products like Linksys, D-Link or Netgear. Any bake-off test plans or useful links to multi-vendor wireless focused web sites would be greatly appreciated."
one more time~
The Kruger Dunning explains most post on
Just say "benchmarking" and you won't need to explain that "bake-off" means "benchmarking". Wait, what was the question?
Call a meeting of the competitors engineers. It's important you get them all in the room at one time with not too much advance warning of the topic.
Tell them what you think you want. Ask them as a group what you're missing. Then make them as a group come up with an eval plan and cook 'em off according to the plan they come up with.
If you need an independent judge, go to one of the labs that does independent third-party assurance and contract them to provide oversight.
Disclaimer: I've worked for one of those labs for the past 15 years.
Stand back and watch the fun......
Red
We have a box that can emulate up to 128 stations, including wpa, wpa2, etc. It can do
DHCP or static IPs and generate Ethernet, udp, tcp, http, and other higher level protocols,
including IPv6. Multiple systems can be clustered together for additional throughput and
radios. Each system can run on only one channel at once, but can talk to multiple APs
on that channel.
One of our systems can saturate any of the consumer grade APs we have, and some folks have
used it to stress very big systems (conference centers, etc).
Runs on Linux of course!
http://www.candelatech.com/ct520-128_product.php
Thanks,
Ben Greear
Unless you're going to deploy like 100 APs or more i an skeptical that the vendors will work with you for such an effort.
Actually doing this correctly is going to be hard and expensive. Anyway, i'd read up on smalnetbuilder's methods and just run, say 10 or 20 concurrent client machines o a 3 or 4 AP set-up. make some of those clients mobile and walk around the space to see that hand-offs happen ok.
graph it all and look for major priods of drop-out etc. Again, though, unless you're doing a massive deployment or this is mission critical more than normal office lan this is not likely to be a cost effective exercise. I've previously had a good experience with Cisco APs
I will gladly accept all cookies created in the process.
We run Cisco WLC and LWAPs as well. Do you have your LWAPs on different ports at the WLC side? That helps. Also different VLANs for different networks and SSIDs is important to keep the traffic down per.
Trolling is a art,
Cisco's WLC/LWAPs do load balancing among access points.
See here
Trolling is a art,
If you're running a bake-off with access points, you're probably running just a bit too much power into the radios.
On-topic, I do like my redundant-controller, centrally-managed 160+ AP Aruba system.
Maybe you could hold a bake sale and offer free wifi for all your customers. Just give them places to sit.
Im not sure what you mean by no consumer stuff.but netgear has started offering small biz gear.including wifi setups supporting up to 150 APs. So not sure how big you thing big is, but they are one of the few mid size deployment shops. While my prior opinion of netgear was low they seem to be trying to break into the enterprise markets. Of course if you need mor APs than that, they are still too small.
Those who can, do.
The (not so big) secret is that most WiFi AP rolls over with 8 or so clients. Only a few manufacturers themselves test their products beyond that, and those work all the way to over 100.
The company selling the test equipment you need is called http://veriwave.com./ You can buy the equipment from them and test all the vendors, or even better, just ask them.
They do of course know, since that is how they test their own test equipment. Problem is that they can/will not tell you because then 1. you would not need to buy their product, and 2. AP mfg would fix their products, and Veriwave would not have a market for their products.
Maybe just do some social hacking to get it out of them.
don't cut it off www.mgmbill.org
Virtually all "enterprise" offerings do things(typically by having all the APs reporting back to a central controller [$$$ incidentally] and tune themselves in various ways) that don't violate the wifi spec enough to be incompatible with (most) boring old wifi devices; but which are beyond the scope of the standard.
Load balancing, automatic power level adjustment to avoid excessive overlaps or voids, triangulation of clients and nearby access points, and various other stuff that can be quite handy; but may or may not require exciting license add-ons for.
They may not be selling the cheap one anymore, which would be good, because it was vastly inadequate for any real N-style usage.
Guns don't kill people, "with glowing hearts" kills people.
Don't forget about Juniper's new wireless solutions, from their Trapeze acquisition.
I've heard a lot of good things about Aruba and Xirrus.
Having actually done Cisco wireless support and new deployments, I would highly recommend against Cisco. They call it a "Cisco caveat" for a reason. Sure that feature works... you know... under the right conditions which will never be met.
Man, if I could have one that does adaptive beam-shaping...
I looked at equipment recently for wireless using the 'n' protocl - but noticed no mention of IPv\6.
No point of gertting new communications equipment, if it cannot be suicessfully usec with IPv6!
So make IPv6 part of the requirments.
Here are the results:
Vendors were tested with 30 then 60 wireless clients and 1 then 2 access points. So, 1 AP with 30 clients, 1 AP with 60 clients, 2 APs with 30 clients and 2 APs with 60 clients.
1: Cisco - Somewhat surprising. Great client density/bandwidth. Good load balancing between APs. Good management interface.
2: Trapeze (now Juniper) - Great client density/bandwidth (just a little slower (read less bandwidth to client, and just slightly less) than Cisco). Good load balancing between APs. Buy the extra Ringmaster management software.
3: Aruba - Significantly slower than Cisco and Trapeze. Good load balancing between APs. Good management interface.
4: Meru - Significantly slower than Cisco, Trapeze and Aruba. They did not have a network engineer available for the test to be present and we were unable to schedule another test before our purchase window closed.
We were going to test Xirrus but the rep we were working with left the company I believe in the middle of scheduling. We looked at Ruckus but were unable to schedule them.
These tests were performed in the spring of 2010. So products may have changed somewhat. You should be able to get demo hardware from any decent rep. We ultimately went with Trapeze after we put everything out to bid. Before that I was sure we were going to get Cisco gear but Cisco came in at twice the cost as Trapeze. We are deploying 128 APs without any issues. Client roaming and bandwidth are great (our primary requirements). All in all no complaints. Certainly liked the price point. Hope that helps!
In my somewhat limited(but rather painful) experience with attempting to use wifi as a serious connection, one of the issues that cropped up a lot was less with throughput, or with number of clients; but with client software behavior in the face of a glitch.
Dicking around at home and the wifi cuts out for a second? Reload the webpage and quit your whining.
Running your basic "enterprise" client configuration(documents directory is actually on a fileserver, authentication through AD, etc, etc.) and the wifi cuts out? Be prepared for frustratingly erratic appearances of apparently disappearing documents, authentication fails, not automatically reconnecting to the fileserver, Finder just twiddling its thumbs and thinking about infinity until that server either times out or comes back, etc, etc.
Even before any APs show up, you can start identifying the likely areas of sheer pain by using netem, switch jiggery-pokery, or just a $20 consumer AP and flicking your laptop's RF power switch: If your environment has client applications that don't play nicely if the network goes all to suck for a second or two from time to time, wifi deployment is going to be Fun.
Honestly, for most applications where wifi isn't a totally terrible idea(ie. heavily throughput dependent stuff), that would be the big focus of my testing(along with how useful the management tools and interfaces are). High throughput is far less valuable than stable connections.
"The Pillsbury Bake-Off is a cooking contest, first run by Pillsbury Company from 1949 to 1976 as an annual contest. Since then, the contest has been held biennially."
https://secure.wikimedia.org/wikipedia/en/wiki/Pillsbury_Bake-Off
The only thing that matters is the bean counters and any ELA you have with existing vendors. Cisco might be good, or it might be crap, but if you have a pimp contract with them and good support, they're getting the contract. Live with it.
I want to delete my account but Slashdot doesn't allow it.
All the major vendors should be aware of what is going on at www.bufferbloat.net and have something in place to ensure that their products will reflect new updates soonest when things get fixed. This is an ongoing problem that crept up on the internet tech community and there is work in progress to deal with it but it will take time.
See (for example) Bufferbloat - Dark Buffers in the Internet, 1/20/2011
Been there, done that, paid for the T-shirt
and didn't get it
Plenty of companies out there make tools for testing this sort of thing. Spirent, Ixxia, and Agilent, to name a few all have layer 4-7 traffic generation appliance type products for stress testing.
http://www.spirent.com/Devices-and-Equipment/Base_station_testing.aspx
Netgear makes enterprise grade equipment -- we have Netgear equipment all through our manufacturing facility and IT center. having said that... We used Xirrus wifi hwardware and it works quite well.
What does enterprise size mean?
What class of construction is the
building? Straw, sticks, bricks... this wolf wants to know
Are regions RF isolated from each other
do you have multiple floors and multiple
buildings? Can RF pass between these
odds and ends.
First you need to wire the building and also decide if
your WiFi boxes will get power from the ethernet wire
links or from the wall.
Do you need secure access for all or is this install
wide open and plan to let VPN do the security.
i.e. guests will always want WiFi service.
Have you done any site research. If a neighbor already
has WiFi deployed and all the channels occupied you
may be the last fool in the pool. There are some
Android applications that I would use to see what is already
transmitting.
Windows and floors do you want to secure the inside
from outside listeners? And do you want to secure the
inside from external access.
Since all WiFi is tested to and operates to public standards
there is going to be little difference from vendor to vendor
at first glance. Placement and wired infrastructure will make
as big a difference as anything.
You are going to need wired links lots of them. Plan on
a robust wired infrastructure to start. The cell phone companies
often have less trouble on their last mile... than you might expect.
Will Cell phones have the ability to connect their WiFi links
to your network? If the company provides phones the answer
will be yes... Cell phones + laptops + iTouch three times the
load you might expect. Will you have to put Femto cells in
for executives?
You may have to screen areas with hardware cloth or some
RF limiting wall covering to keep areas from interfering with
each other....
Net nannie... do you have legal issue to audit and manage
both incoming and outgoing? Privacy issues that make shared
pass words a tangle.
Central management? Can one person walk around and
check them all? Backup...
Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities; Truth isn't. Mark Twain.
It is Beerfest 2. http://beerfestmovie.warnerbros.com/
You can call in all the experts and engineers to figure this out but never underestimate the importance of common sense and understanding of basics concepts involved. A couple of articles on Tom's explain it well. One story has some testing and benchmarking. May not be exactly what you need but may go a long way in ensuring good wi-fi. http://www.tomshardware.com/reviews/wi-fi-performance,2985.html http://www.tomshardware.com/picturestory/571-wi-fi-beamforming-networking.html
We run a thousand or so Aruba 125s at school here, covering all 600 or so acres of campus. Those are probably overkill for you (at about $750 a pop), but AFAIK even the lowest-end ones have the same essential features.
Basically, the network architecture puts the whole wireless network on a separate segment, all the way back to the aggregation points. They're gigabit wired into the building routers, but placed on a separate VLAN all the way back to one of the three aggregation points. Each AP is assigned to a controller, and will fail over to a second one if needed. The controllers pass the traffic to the rest of the network.
The controller architecture means you can do some pretty interesting things. Particularly, it means new APs are trivial to install - stick them into the controller's DB and plug it in to an Ethernet cable (it's PoE); it'll go and find the controller and pull down the config and any upgrades to the software. It also allows IP roaming between APs, even if they're in different netblocks. I can walk from one end of campus to the other (7 city blocks) while keeping the same IP and getting all my traffic, through about 150 different APs - much like a cell network. You can also do spectrum analysis through their management console - I once saw them find a broken microwave from all the interference they were seeing across the 10 APs in range a la Dark Knight.
The APs we have will band-steer clients over to the 5GHz spectrum if possible, which can support a huge number of clients, but you need the density for it to make a big difference. If you do, though, you can easily get 30 people per AP, with a few doing massive downloads/uploads and no hiccups. They don't recommend more than that, and in any case it's difficult to fit people densely enough that you wouldn't need a new one for signal purposes.
No, I don't work for them. I don't even work for the Networking department. I just really like the toys - though I suspect I might feel differently if I had to make the purchase! Quality isn't cheap...
I have developed a truly marvelous proof of this comment, which this signature is too narrow to contain.
I was going to reference the same article from Tom's - they basically have already done what OP is asking for, using hardware including Cisco, Aruba, Meraki, Ruckus, Apple, and HP. The second and third pages "Hardware And Methodology, Explained" might especially be of interest.
If you spoof a controller. One of the first lessons I've learned is to never use dynamically configured devices on a campus. There will always be a geek that will find a way to tell your equipment what the best route for traffic is.
I was promised a flying car. Where is my flying car?
The result was Cisco but this was 3 years ago. The Aruba gear is poorly made and it took about 70% more APs for the same coverage. We used 900 Cisco APs as opposed to over 1500 Aruba. That being said we also like the Ruckus Gear which also feels flimsy compared to the Cisco 1042 APs but the beamflex technology is second to none especially if your trying to carry a voice or video signal.
Be sure to add them to the list. Their virtual cell technology is pretty slick, and works well for busy environments. Just add more access points to a busy area or for an event, and the controller will take care of balancing the clients between the radios. No, I don't work for them, but I am a happy customer. We have over 250 access points installed across our campus.
Never ask for directions from a two-headed tourist! -Big Bird
For those fixated on the term bake-off, you obviously have never worked for a networking vendor or a major IT shop, or you would understand what the term means. And some of you demonstrated a level of intelligence that has me picturing you entering "snow white and the seven dwarfs" when prompted for an eight character password.
Ding ding winrar. The Toms article is very very good.
I can second Ubiquiti. It is really really great stuff for the price, and there are some 3rd parties providing alternate firmware options if you don't care for their software. they aren't crazy there with warranty practices - if you change the software it doesn't void the hardware warranty.
Get a web developer
See: http://www.tomshardware.com/reviews/wi-fi-performance,2985.html Great comparison.
If you know how to write firmware, you could probably use the MIMO access points to do some phased-array fun. Write the code to determine the approximate direction of the client (which, you should be able to do with multiple antennas for triangulation), and then increase the power using multiple antennas in said direction.
*goes to do some Googling*
It looks like there already is an access point that does this:
http://www.tomshardware.com/reviews/beamforming-wifi-ruckus,2390-3.html
The incredible thing is that chip-based beamforming, like MIMO, has been compatible with 802.11a/b/g for years. In fact, the technology is an optional part of the 802.11n standard. Despite its benefits, though, Cisco is the first to deliver on-chip beamforming to market. The enterprise-oriented AIR-LAP1142N access point is Cisco’s first and so far only product to feature beamforming, which it brands as ClientLink. It arrived in the first quarter of 2009, but the firmware that enables beamforming capability didn’t arrive until July. We tested with this firmware literally within days of its release.
Don't forget to check out Aerohive, another decent option.
While I understand that in the world of Enterprise IT it is standard to use a package from one of the large vendors complete with controller, I very much disagree with philosophy of setting things up that way. Please note that my network is not to the scale of yours, only about 75 APs on a small campus, but I've had great luck using small-business/high-end consumer grade equipment, clever setups on "Fat" APs, and some powerful controller software. For example, my current design just uses off-the shelf EnGenius stuff because it was cheap and at least did 2.4N piped through a communications-isloated vLAN and uses pfSense as a gateway so I can do my shaping and captive portal. I mean at $90 per AP and zero licensing costs, I'm willing to double-up on density to offset for slightly lower quality parts and we still make out like bandits in our budget. Throw in some basic scripts to check on AP status and you really have 95% of what Cisco or Aruba does, but at 5% of the cost. I understand that beyond a certain scale this just isn't feasible, probably around the 150+ mark. But "large-scale" means different things to different people and the author didn't provide a rough AP count. Besides, I had Cisco drop off some gear for me to test out their 1142s and a controller a year or so back. I was less than impressed compared to what my system was already doing. Only hitch being that I had to manually configure each AP, but its all about balancing practial usage against budgets in my world. And my Aruba rep was creepy. Final note - please bear in mind that while we have opted for this solution for wireless, the actual infrastructure is top notch. No copper used in between buildings, 1Gbps minimum fiber and 10G fiber from our Core router the server room. I think a lot of the time a high-cost wireless solution is designed as such to offset the downsides of a poorly designed or outdated network. Spend money on your foundation first, you know?
herve leger bandage dress herve leger skirts herve leger dress on sale herve leger dress wholesale