Slashdot Mirror
Novell's Releases Linux Usability Testing Videos
sp3298622 writes "Novell is releasing primary desktop research, including over 200 videos and analysis of usability tests, at betterdesktop.openSUSE.org. Vice president of collaboration and desktop engineering for Novell, Nat Friedman: As a programmer, it's sometimes difficult to know how ordinary people with no technical experience are reacting to your software. Linux people tend to know other Linux people. In these usability tests, we selected test subjects who were experienced with Windows, but who had never heard of Linux, and asked them to perform basic tasks using the Linux desktop."
That headline is just embarassing.
http://www.useit.com/alertbox/20000319.html
OK Here is (I think) the complete list from the article:
I found it interesting that eight out of twelve succesfully completed the "Find out if the computer is online" task. I also wonder if these users could complete all these tasks in Widows.
Insert Generic Sig Here:
The required population size depends on what you're trying to test and how carefully you select your population. If you're trying to test the failure rate of moderately experienced Windows users performing tasks on a Linux system, and you can accurately identify and select moderately experienced Windows users with no prior Linux experience, then you only need a tiny population.
Testing the failure rate is important: 100% of 11 users succeeding at a task can give you at most ~ 90% confidence that all similar users will succeed. 1 of 11 users failing is a far stronger result, telling you that you can expect at least 9% of all users to fail.
Various usability experts suggest that as few as 5 or 10 individuals are required for usability testing, and the remaining usability issues are discovered and resolved via the bug reporting and maintenance processes.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
89% sounds like a very good success ratio for the date and time test. However, RTFA and you'll see that only eleven people participated, most of them female.
Eleven people is a pretty good sized group for a usability test. This sort of testing is pretty expensive and time consuming, it's not like a survey or something. From a group that size, you can get a pretty good idea of how the average person will try to accomplish a task and some problems they may encounter. I've worked on projects where usability tests included only three people to test the interface to a product costing tens or hundreds of thousands of dollars. I'm going to have to disagree that these results are meaningless. The important thing is how did the user who failed try to do the task. What stopped them? What problems did other users have?
So only one of them had problems? Sounds good.
You shouldn't let the small numbers put you off. Respected usability professionals say you only need five people for meaningful results.
Bogtha Bogtha Bogtha
Actually, it is FAR more likely to be this easy under Linux. All the tools are gratis. None of it is shareware/crippleware. You can always depend on at least tar/unzip being available if not the KDE or Gnome equivalents of winzip. Contrast this with Windows where the tools are not at all standardized or standard.
I mean really, pullleeeze. Installing winzip or somesuch is something that I nearly always have to do when getting my hands onto ANY WinDOS machine for the first time. (mine or someone elsese)
The average mother-in-law is not going to have any clue that they need winzip or something like it. This has actually always been a problem where Linux has been inferior.
grep may be arcane. Ditto for gzip.
However, I can depend on both being available regardless of what year it is or what version or vendor of Unix is under discussion.
Nevermind the whole quaintness of your example to begin with...
A Pirate and a Puritan look the same on a balance sheet.
I believe your answer lies here: http://www.betterdesktop.org/welcome/reports/repor t-date-time.html
# Issues encountered:
1. The date and time configuration tool is not easily discoverable from the menus, and is not listed in Personal Settings.
2. Users assumed the root password request meant they had to log in as root.
3. Users wanted the click behavior of the clock applet to be similar to Windows.
# Recommendations:
1. Fix time and date settings to not require root access.
2. Add time and date settings to the Personal Settings window.
3. Make the clock applet behavior be more similar to that of the Windows taskbar clock.
Too bad the tests have such small sample sizes.
It would be good to get some big numbers on these issues, although I acknowledge that it would be very expensive.
Hmmm witty sig or funny sig? Maybe elitest techy sig!
Comment removed based on user account deletion
Putting a CD into a Windows machine pops up a box, one of the options of which "Copy the music to your computer". Your boss must be really dumb.
Claiming it is non-trivial on Windows is fallacious though, it's certainly no more difficult. Windows Media Player is included with Windows (except in some versions in the EU!) and is capable of ripping CDs.
Hopefully this is a general trend for the industry. Maybe everyone will start using these techniques that have been available since the early 70's. Maybe instead of designing from the seat of the their pants they will start testing their interfaces just like we do with all software.
This isn't hard. Usability labs like this aren't necessary. I only have to sit any member of my immediate family down, my parents friends, co-workers, etc. to get an idea. Maybe more serious testing needs specialised workers but by no means do we need these specialised facilities.
If you want a cool way to benchmark in terms of speed, acuracy, and rate of habituation try GOMS. No testers needed. (For those who know about GOMS, please clean that article up. I haven't had time.)
Is there anything better than clicking through Microsoft ads on Slashdot?
If Tyranny and Oppression come to this land,
it will be in the guise of fighting a foreign enemy. -James Madison
Frankly, I think you just insulted my mother. She knows how to do "apt-get install quake3" or "emerge quake3". She also knows how to insert the disk when it asks for it. And you know what? Later, when the game tells her to patch it, while she may not remember it, I can certainly give her a link to "apt-get update && apt-get upgrade" or "emerge sync && emerge -uDN world" -- which updates not only Quake3, but the entire OS and (almost) all installed programs.
Cryptic? Maybe. But so would typing in "wget quake3.com/.../q3.exe" or whatever. But you're using Firefox, so the cryptic symbols are hidden. Guess what? Linux can do that too. In fact, although there are several good portage and apt frontends, I can write a frontend myself to the basic stuff. Takes about two minutes.
Maybe I should submit it to Slashdot?
Don't thank God, thank a doctor!
yeah unfortunately Yahoo and other IM services go to some lengths to make sure that third party messengers have great difficulty with compatibility. Which means that IM Clients frequently need to be updated. A gtk upgrade could be painful over Dialup I suppose though.
MP3 encoding is often not shipped with Linux Distrobutions - The reason being Intellectual property - I think it is (or was) a 40,000 license fee to ship systems that can encode MP3. From memory - though It might of changed - all that K3B needs to encode MP3s is a copy of lame installed on the system.
Mepis is an excellent starting point for Debian Unstable (what I am using)... Theres is still some setup required (upgrade to KDE 3.4 - set up device automounting - I am pretty sure this won't be necessary in the next version). But other than that everything pretty much works out of the box (for me at least).