Conducting a Unix Desktop Usability Study?

cyclop asks: "I am a close friend of a Ph.D. student on human interface usability. She's now working to tailor a KDE-vs-Gnome usability study (a pretty hot topic these days), and I have been called to help, as a long time GNU/Linux desktop user. What kind of advice -- both technical and theoretical -- would you give us on conducting a deep and objective study on the Unix desktop, that can be useful for the developers and the OSS community?" "She has installed GNU/Linux and used both KDE and Gnome to get to know them, while I provided her a number of links on background information and previous usability studies on both DE, and advised her to subscribe to relevant mailing lists of both projects. However, I feel that it's not enough and that there are a lot of potential pitfalls and misconceptions that wait for us, me being a geek and she being a Linux newbie. Moreover, she found that most of the previous studies on the web were quite sloppy, in comparison with common usability research standards."

Comments

for meaningful results...

[smash]

I'd ensure first up that the study runs for "long enough".

3-6 months perhaps?

"Usability" imho, in the usual meaning of the term, is a load of wank.

Who cares if the first time someone uses the environment that it takes a little orientation to get used to? In the real world, if a couple of weeks of pain makes you much more productive after that, it's a net benefit imho - the remainder of your time using the environment outweighs the significance of the learning time.

I'm not saying that initial learning is not important, but I think that these studies need to show both sides of the equation...

smash.

Re:My thoughts

[Anonymous Coward]

It might be better to test it on childern.

most people have used windows at one time and expect things to have that type of layout.

children who havent gotten used to what windows is like might find it a bit easier/harder to move around in.

you could have 2 groups of children

set one group of children to use gnome for the first week/month/year and kde for the second week/month/year whatever
and set the other group to use kde for the first week/month/year and gnome for the second week/month/year

and compare there reviews of how easy it is to move around.

however it might be better to test it out on teenagers are they will be able to take more infomation in.

You missed one last

[DrYak]

#4. I have no social life : These have seen so much different systems that they understand well the general concept and can adapt themself very quickly to whatever system you throw at them. As open to new solutions as #1, but very quickly get as agile to whatever system as #3 is.
(In fact that was my case when I started learning both KDE and Windows 95 in parallel)

The best system ever should be as easy as possible for #1 out of the box, but need to be very easily configurable to whatever complex system #3 and #4 need.

If the system is newbie-friendly but can't evolve you'll end with Clippy and this kind of stuff that gets in the user's way with pointless tips (tips that would have helped a total beginner, but sorry now I know exactly what I want). Attracts #1 users, but repels #3 and #4.

If the system is configurable to extreeme you end up with emacs or vi : the most powerfull tool around you can't ever dream of in your wildest dreams, but you can't do anything without unless you've spent the first year learning it the harsh way. #3 only are interested, #4 must ponder if they want to re-learn everything once again (albeit they do it faster), #1 will prefer to commit suicide.

#2 are pointless, they won't accept anything that isn't their original system, they're the one that will never switch to MacOSX or Linux because it's not Windows+Office, and they'll cry each time MicroSoft revamps the interface and everything is moved around (Windows 3.11 -> Windows 9x -> Windows 2k -> Windows XP -> Windows Vista and same for the Offices). Just wait until the next "GUI is completly changed one more time" period, and they'll be as good as #1 users (or #4 if it's not their first change around).