Slashdot Mirror
Testing Pre-Production Servers Accurately?
An anonymous reader asks: "Having been granted a 90-day demo enclosure of new blade servers from a major vendor the question I find myself asking is: On a limited budget, how does one simulate 1000+ attached clients and the activity of those clients? We're a K-12 school district and our current servers don't keep up with all the roaming-profile abuse from our Windows workstations. Are there tools or tricks available to simulate load on Netware/Linux servers? The user groups around here usually answer this question with 'Get some workstations for a test lab!', there's got to be a less expensive option, right? Can we leverage our existing client populous to achieve our goal, without interrupting or changing the quality of service at the desktop, substantially?"
Just add a program to the login scripts that runs some tests against the server in the background.
Then when everyone logs in in the morning, it just overloads the thing...
And simulates the load spikes you see. I don't know a program or script that would do this, but I'm sure someone out there does.
JC
On Arrakis: early worm gets the bird. Magister mundi sum!
Nah, way to expensive. Check out OpenSTA.org or check here for a comprehensive list of Load Testing tools (both free and commercial): http://www.performancetester.com/mambo/component/o ption,com_weblinks/Itemid,4/catid,68/
So why not just point each user's "My Documents" folder to a network share? This will keep the profile much smaller. I guess I just took it for granted that this was a standard practice. I've done it for years.
This is very do-able, but there are a few gotchas. Your plight made be think about how I would do it if I were in your shoes...here's what I would do:
1. Come up with a (hopefully short) list of items that need to be observed under load. For example, authenticating users and allowing logins, serving up files).
2. For each testing item identified, find a repeatable, scriptable method of testing it. Perl will be your best friend here. For questions consult the great folks at perlmonks.com. You can find very nice modules that will allow you to create scripts that can authenticate to your test server, and report back a time. If you dump all your responses into a csv format, it will make the reporting of the data even easier.
3. Write a separate script to track system performance during the testing period. I couldn't actually tell from your question if the servers you are testing are windows or *nix , but whatever the os, you'll probably want a snapshot of overall CPU utilization and Run Queue size, swap use, disk access times, and memory consumption.
4. Be sure you are testing what you think you're testing! You're testing results will be invalid once you hit a limiting factor. So if you're network link gets overloaded by your test script, then your file transfer test is not a good one.
5. Design your testing scenarios carefully. It's probably best to have as short a test list as possible, and then make them as iterative as possible. For example, first test with having 1 users logon and transfer a file. Second test has 10 simultaneous users logon and transfer a file. Third test has 100 users involved, etc.
The key to all of this stuff, is you want tests that are repeatable, valid, and indicative of production uses. The nice thing about developing these test scripts is that you can use them to test hardware from several diferent vendors. If it seems daunting, just remember that it's better to spend a week of late nights developing the scripts - then to spend the next several months wondering if you're implementing hardware that will hold up to your production workload!
There is lots of test data out there. You could use the serverbench test suite from PC Magazine - it's free. It uses a bunch of client PCs to generate huge amounts of network file serving load (read/write/bigfile/small files).
Do you have enough network bandwidth? 1000 computers will easily saturate fast ethernet connections. You need gigabit ethernet or better.
And frankly, a bunch of blade servers isn't a good way to solve a file serving problem. You want either a bunch of NAS boxes (network attach storage) or regular file servers with big scsi raid arrays.