Slashdot Mirror
Benchmark Software For Windows 7 Rollout?
tdisalvo writes "We are doing a Windows 7 rollout and I will have to compare major PC vendors. I am looking for vendor-neutral tests that will give me the data I need to present an educated opinion to my CIO. Clear, pretty charts are nice since it is for C level execs, and we need to make it understandable for nontechnical as well as technical people. More specifically, I am looking for something that will clearly show how the same processor performs (better or worse) with a particular build, motherboard, RAM, power supply, etc. My plan is to get very similar machines from major vendors and see which one's build has the highest independent benchmarks. Something with which I could test multiple computers and report on the differences in score would be ideal."
As usual, free is an advantage.
The only thing I can say is a lot of benchmarking software that offers charts and nice graphs tend to be skewed. Not all of them however. A lot of hardware companies design the parts to get somewhat abnormally high results on benchmarks, thus inflating the numbers, and providing inaccurate results. Your best bet is unfortunately more time consuming. You should have multiple software testing the machine, and then make your own chart. This is much more accurate. Try rendering a 1080p video file and record the amount of time it takes. Things like that.
It doesn't run on Windows but Phoronix Test Suite would give you a good baseline for the hardware.
Is this really necessary for a Windows 7 rollout with corporate desktops? Most machines are already overpowered for the average user using Office and what not.
I'd think the cost per machine for good 3-4 year warranties would be more important. At least, it has been in my experience.
I could see doing something like this just for developer machines, but general roll-out? I dunno. Seems like you'd just compare pricing and go with the one that makes the most budget sense.
You haven't said what you actually do with these computers. The relevant benchmarks should look like your actual workflow, otherwise you are just drag racing.
I haven't benchmarked in a while, but this new game came out recently. iD Software's Quake3. All of my hip friends use it to test their machines.
My Slot-A AMD Athlon rocks out like 75 frames a second! Try it out!
Phoronix Test Suite ( http://www.phoronix-test-suite.com/ ) supports Win7 now. It also allows comparison against OSX and Linux ( http://www.phoronix.com/scan.php?page=article&item=linux_windows_part3&num=1 ).
It's Free, it's Open Source and has a bucketload of tests already. You can combine result sets and you can even get the results uploaded for comparison at http://global.phoronix-test-suite.com/
Creating your own tests is nice and easy too.
(Full disclosure - I am one of the project members).
Questions:
A. Do have the authority to make the decision?
B. Are you tasked with giving him your "expert opinion" on the matter?
C. Are you tasked to actually educate him enough about a technical decision that he has no technical skills to currently evaluate an answer?
Answers:
A. Evaluate on the specs you know are important on the job, give him a specific brand, and say "trust me, buy these"
B. Evaluate on the specs you know are important on the job, give him a specific brand, and say "trust me, buy these"
C. You're boned.
This is pointless. Really. All the machines will test within a few percent of each other. It's not like a Dell is significantly faster than an HP (especially if the software image is the same).
If the machines have different CPU/Chipsets/Video Cards, that's a different story, but a PC -is- really just the sum of its parts.
Tell the C-level execs that the best value would be to skip the benchmark and go right to the bidding, let the vendors undercut each other for an extra month.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Why should C-level execs care about what model processor is used in their computers? Office users aren't looking for the absolute greatest performance, they're looking for reliability, manageability, and cost. I can guarantee that no typical* medium or large size business will make a decision on which vendor to use for office computers based on the performance benchmarks. Frankly, who gives a shit about the motherboard in a typical office user's computer. It doesn't matter, certainly not to upper management. Choose something that has a reasonable cost, a solid long term support contract, and is easy to manage in your existing environment. If anything, the support contract, expandability (adding dual monitors later, or adding more memory for heavy data analysts or future software upgrades), and the existing vendor relationships are far more important than performance benchmarks. *Assuming they're not using them to render lots of graphics or do other very specific, specialized tasks.
Every time you post an article on Slashdot, I kill a server. Think of the servers!
Performance? Really? Personally I'd want stability, reliability, and top notch support. Your average computer user loses far more productivity from downtime due to cheap hardware dying, unstable drivers, etc than to their machine starting (insert app of your choice) .2 seconds slower. I want to be able to order an exact replacement 2 years down the road if a machine dies. I want replacement parts available for the forseeable lifetime of the machines on which I standardize.
Like they will know you did?
Here's what you do.
You go to techbargains.com slickdeals.net techdealdigger.com techdeals.net etc.
CTRL+F DELL
Note specs and prices.
Do this for a week.
Then next week, jump on the first deal that meets or beats the best deal from last week.
Then order up a bunch of machines.
If the number you're ordering is an issue, just call Dell, ask for the supervisor, and then get X machines at the quoted price after agreeing to upgrade them all to the 3-year, NBD warranty.
Corporate will love the price.
Whoever manages the machines (you?) will love the NBD warranty for when a PSU fails, or a fan starts getting noisy. (When, not if.)
You won't have had to do any real work.
Everyone wins.
Not only were the machines similar they were virtually identical. Unless you were looking at the case badge, or the PCI vendor strings, you would have been hard-pressed to tell which was which. Same intel silicon, very similar HDDs and optical drives(not that that really mattered, neither party was willing to quote anything other than a capacity, so the brands we got in the test boxes were assurances of nothing). The RAM was within spitting distance of one another and(again), the vendors would assure us of nothing other than "X capacity, verified compatible) so it wasn't as though the specifics of the test samples told us much.
We ended up going with Dell, just because they were cheaper, their driver download pages are modestly less unpleasant, and their "ImageDirect" tool is actually pretty handy.
Unless you have particular reason to believe otherwise, exhaustive benchmarking will be a waste of your, and the exec's time. The only exception that I can think of would be if you were advocating for something unusual but potentially interesting(ie. Most corporate desktops are brutally I/O bound, straining under the load of A/V, constant patches and updates, and so forth. SSDs would make them fly, comparatively. Particularly if your company actually has a lot of expensive people running around, a "number of minutes from cold boot to productivity" benchmark could be eye-opening.)
This is specialized for architects but we went from a five minute "save to central" time cost with Windows XP 64Bit to two seconds on Windows 7. Our cost to deploy ROI was achieved in one week's work. "Save to Central" is an Autodesk feature for writing to a SQL Server database on a server. Talk about low hanging fruit. Management surely understands that when people are no longer standing around yapping, more money is being made; not to mention happier workers as well! Your results may vary. Of course with Windows, any new deployment is faster on the machine than a stale DLL hell one.
Here's why this guy is being asked that... Suppose Machine A is "5% faster" than Machine B at the same price point for a common task. Let's say that task is something everyone does often and is easy to measure: booting up. So, if Machine A takes 60 seconds to boot, Machine B takes (0.95*60)=57 seconds--3 seconds faster.
So, here's how the C-level execs think... Say you have 1000 employees, each saving 3 seconds/day in bootup time. 1000 employees * 3 seconds/day = 3000 man-seconds/day. 3000 man-seconds/day * (approx) 225 work days/year = 675,000 man-seconds/year = 187.5 man-hours/year saved! Just think of how much more productive we are due to that 5%!
Of course, that assumes that all your employees are robots and use every second of time productively. To add, by the time the OP gets all the machines, runs the benchmarks, and creates the pretty PowerPoint slides for the C-level execs, this little experiment probably cost the company a lot more than 187.5 hours... (Although you could probably shoehorn a 3-4-year NPV calculation showing a savings for this project...)
Windows 3.1x calc: 3.11 - 3.10 = 0.00
There's no business/office productivity software that requires Vista or Windows 7. In fact, I'm not aware of any software of any kind that REQUIRES Windows 7.
You can run everything on XP.
Now ask yourself: "Why are we spending -any- money on upgrades?"
Two paths from this point.
1) Slap yourself, rebuild your corporate image with a nice current minimal build and give users the option to rebuild their machines with said image dynamically, at boot time. This will produce vastly greater productivity than any attempt to upgrade.
OR
2) Continue on your current path to justify your continued employment and claim Windows 7 is necessary and the upgrade is "a must have" to remain competitive.
In no test, on the same hardware, will you see any performance increases, by any time based measurement when comparing Windows XP SP3 vs. Windows 7. Windows 7 will always be slower. Boot time, shutdown time, application launch time, or install time. All slower. And you don't have to take my word for it, break out your stopwatch, you can see it for yourself.
The "may see an advantage" you mentioned is exactly what I was asking for detail on. My empirical testing showed that 64-bit isn't a magic bullet, unless your apps have outgrown the memory limitations of the 32-bit world. There are a few natural disadvantages of 64-bit. For example, pointers are bigger, causing more cache misses. I've met way to many people that think that it is a foregone conclusion that 64-bit is faster than 32-bit all the time. I actually had someone respond "How can that be?" to a benchmark result showing 32-bit XP 15% faster than 64-bit XP for one specific workload. For those who may fall into this category, there are many reasons, from immature 64-bit drivers to hand-tweaked 32-bit app code whose 64-bit equivalent hasn't had the same number of years of care and tweaking.
I hope you were merely tasked with finding benchmarks and that you're just a tech. If your CIO tasked you with picking the next platform and you decided to perform technical benchmarks, then you really missed the boat.
1. First, you need to be analyzing support you'll get. Don't get too hung up on it, but you need something better than a 90-day warranty. There are diminishing returns though, at some point it's not worth getting a 3 year or 4 year contract.
2. Next, you need a vendor that will help you with license management. Being able to audit your licenses for Office or Symantec or whatever quickly will help you. If you don't have volume licensing, now is the time your vendor should be helping you with it.
3. Usability matters a lot, but what matters almost as much is how cool your laptops are. When your marketing director, you know, the one that always wears cool clothes and would have to have his iPhone pried out of his cold dead hands, goes to a conference you better make sure he has the coolest laptop of any of the other marketing geeks. A lot of companies overlook this, but I guarantee you he doesn't want to be carrying around a Latitude E6510 clunker.
4. There's a nice price point right now around $1000 for decent corporate laptops and you'll get about 3 years out of them.
5. You need to be negotiating with your sales rep hard if you're making a purchase like this. Your rep isn't going to be able to make huge discounts on laptops like they can on server equipment or some software licenses, so see if you can get some killer pricing on servers while you're shopping for a big laptop package.
Skip the benchmarks, it's not worth your time. Anything you teach your boss about Core duo, i5, etc will be useless knowledge for him in six months when Intel introduces some new spec.
----- obSig
Performance benchmarks for typical desktop office machines are pointless. What is FAR more important is: driver stability/support and vendor support in the case of hardware malfunction. So long as your desktops have > 1gb ram they will be fine for 7, for normal office use.
We're currently a dell shop (sigh), my baseline cut-off for Dell laptops is Latitude D510 with 1GB ram for Win7 pro x86. Desktop machines - anything with similar spec to that is fine for x86.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.