Stress and Volume Testing - Your Experiences?
Tarohann asks: "I work for a large software services company and I'm planning to write an article on Enterprise application Stress and Volume Testing (SVT) based on a couple of experiences I've had.
For the most part I feel that most Enterprise SVT initiatives performed by companies at the enterprise level are flawed as they are done on ideal/non-production environments. I was wondering if anybody here has had any good/bad experiences or pointers (websites, books) I can refer to on this subject. Full credits will be given. Thanks, Slashdot!"
Gather 20 of your closest friends and relatives, give them each a different OS cd, and a different used computer (some laptops, some desktops, different brands, etc.) and have them each install the OS to their liking. After that, let them swap and repeat. For better results, image those systems (with ghost, etc.), and install the images on other systems with different hardware, and do whatever it takes to get them going on the new system. Finally, take a couple of the systems, have someone who didn't work on it go over to it and remove the crap that the other person put on it, and install it to their taste.
After this, you will finally have an environment that reflects what really happens in most computer networks. Image again, and save one image per machine, and you should be all set. (Unless you want to swap some hardware around, like NICs, or modems)
I work for a company that writes systems management software for windows and *nix systems, shameless plug, and each of the QA machines have a different OS, and a mix of languages for color. We've got solaris boxen, macs, linux in many different flavors, and every x86 Microsoft OS released since September '95 (except Windows ME, 'cause no one really counts that as an OS ;) )
We've got used computers from many different vendors, servers, desktops, etc., and let me tell you that we encounter many more issues in captivity because of it...
If you can afford it, get help from Sun.
The Sun application server software team
has people with very deep backgrounds
and the right experience and equipment.
I was truly impressed working with them.
They do massive testing projects
and have all the necessary contacts
with other enterprise app companies.
Cheers, Joel
The best thing I can say to you is that if you want to 'sell' the test results to your management, etc., you are going to have to do both database/application server capacity and load testing and user-side 'performance' testing if you have a user-side to your application. For some reason, PHBs on up like to see the little application animating its brains out. I'd suggest Mercury Interactive's WinRunner and LoadRunner, but I was too lazy to RTFQ and beside that, I've been coding in WinRunner all day, and I now need beer to soothe my nerves.
I like how P&CP (performance and capacity planning) becomes S&VT when you are talking "Enterprise" applications. It's funny how far people will go to sell a new approach, or to pad their resume with more buzzwords that nobody understands.
Heute die Welt, morgen das Sonnensystem!
Last time I was involved in that kind of testing, we found out that our code worked as designed (yay!). Unfortunately, we also found out that the application server engine we'd written it for had some critical bugs buried very deeply, that had never shown up because no one had ever tried to do anything quite that stressful with it. :(
Oh well.