What Do You Look For in a Big Iron Review?

ValourX writes "We're starting to write more reviews of enterprise-class hardware and software and although we've done pretty well with our reviews, the high-end products are a lot trickier when it comes to testing and evaluation. Obviously it is not possible to build an enterprise-grade 'your neck is on the line' production environment just for writing reviews, but maybe we can do something smaller, just for testing purposes. What do you as an IT professional want to read in a review for a server OS or a high-speed switch, or a big iron server or proprietary workstation? What tests should we run? What results and feature comparisons are going to be most meaningful to you?"

Comments

True costs

[ChiaBen]

I have had problems in the past looking at various hardware and comparing the true costs of it, especially support. With the third party support companies out there (we use Terix, amongst others), there are so many options, and with yearly support contracts in excess of $100,000, for our relatively small company, mis-calculating these in a recommendation can be a very big deal.

Just my $.02... oh, also just plain reviews of support companies on different hardware would be good also.

Scaling claims & Installation complexity

[akad0nric0]

I've worked with too many companies whose products *do not* scale the way they claim, or whose products will techincally scale, but are at that point virtually useless. Use bogus data, who cares, but test the data volume, throughput, storage, archival, etc. to the limits and make sure the product is still useful. This is the single biggest problem I've had with enterprise installations, and the problem as an architect is that it's difficult to test on a very tight timeline for product evaluation. I've had egg on my face more than once because I had to take the vendor's word for it.

Second, install the application yourself. Don't let the vendor do it for you. And when you install it, install it as an enterprise would. That is, if it's an n-tier application, or has multiple components, don't take the "default" installation and put all of the components on one system. Of course this will work. Try distributing the components over multiple systems like an enterprise would. Often this is where the complexity comes in and products falter.

One company I worked for purchased some software from Tivoli. After 6 months, and a team of engineers onsite from the vendor, they still couldn't get the components to talk for more than a day without problems (after weeks of installation), and still couldn't get useful data out of the database due to its size, so we took our $500mil back and bought something else. Having an evaluation that would've tested this would've saved us a bundle.