Measuring Usage of Distributed Resources?

An Anonymous Coward writes "I work in a distributed development environment with some 1000 engineers running Solaris on different hardware platforms. We are initiating a project to move to a centralized environment. One of the biggest problems in this project is to identify the amount of resources (i.e.: RAM, disk space, etc) that the centralized environment should provide to the users. For example, each developer is currently building his code in a different desktop on the network. How could we effectively monitor how much resources are been consumed from each desktop during each build? If we find a way to capture this information, considering that the results were gathered from different hardware platforms, how could we normalize the data to get meaningful metrics that would help us define a solution?"

Comments

WTF cares?

[SuiteSisterMary]

You don't want to know anything about your current set up. It's got nothing to do with what you want to be doing. What you want to do is figure out the work load that this new box is going to be doing, multiply those requirements by 1.5, and buy a box that, half loaded with equipment, can do that. It's called 'needs analysis.'

Test a smaller case?

[cperciva]

Given that software never works properly the first time, you're going to have to get the bugs worked out with a small group of developers before moving everyone over to this new system. Why not use that to your advantage? Measure the utilization of a box which has ten developers, multiply that by 500 (in order to give yourself a good safety margin), and that's what you need to buy before you move all 1000 people across.

You're trying to measure the wrong thing.

[hamjudo]

Only look at the heaviest users on the biggest, fastest machines. Once developers have easy access to faster systems, they will use them much more than the small slow systems. The only developers whose use usage won't go up much, are the ones who already have machines that fast.

See what resources the folks on your fastest systems use, and assume everyone will use that much.

Expect it to ramp up to that usage over the coarse of a few months. Expect the usage to keep going up forever.

Bigger issues with centralized development

[tshoppa]

I understand your desire to quantify the hardware needs for moving to a centralized system. But IMHO there are many bigger questions that have to be answered first:

• Policy - Exactly what is being centralized and what isn't? Are browsers going to be run on the central machine and displayed on X-terms? Or are they going to run on the satellite machines? You have to answer (or guess the answer) to this question for each and every application you may have. And the answers will not only determine the server CPU/storage architecture, it will also be vitally important to the network infrastructure.
• Development - What sort of development environment will there be? Will all software - from the littlest dinkiest shell script up to the giant mega-app be forced under CVS? Will you allow "checkout" to remote nodes, or only on the central node? etc.

All that said, as for remote CPU utilitization the ruptime command is a start.