Novell's Releases Linux Usability Testing Videos

sp3298622 writes "Novell is releasing primary desktop research, including over 200 videos and analysis of usability tests, at betterdesktop.openSUSE.org. Vice president of collaboration and desktop engineering for Novell, Nat Friedman: As a programmer, it's sometimes difficult to know how ordinary people with no technical experience are reacting to your software. Linux people tend to know other Linux people. In these usability tests, we selected test subjects who were experienced with Windows, but who had never heard of Linux, and asked them to perform basic tasks using the Linux desktop."

Comments

So much for this

[Raelus]

89% sounds like a very good success ratio for the date and time test. However, RTFA and you'll see that only eleven people participated, most of them female.

If you don't have a diverse testing population, you aren't going to produce meaningful results. The idea is fine and all, but the results are mostly useless.

Finally!

[Animats]

This has been needed for so long. This is how it's done.

Developers, you don't get to check in code until you've watched the video of users struggling with your program. OK?

Fortitude

[minginqunt]

It's a hard thing is to admit that free software has a usability problem. The natural temptation is to sit and watch these videos whilst screaming "You idiots! You don't click "Send and Receive" if you want to send an email! What's wrong with you?!?!"

It is difficult, but it's vitally important. These people aren't stupid losers- they are fluent in another operating system, where they can achieve whatever it is they want.

The problems on show here are ours, not theirs.

Martin

Re:Hm.

[I confirm I'm not a]

Might this only result in the Linux desktop becoming more like Windows?

Maybe... but...
There was a brief comment in an article in, I think, last month's Linux Format (UK magazine) (I'm at work, so can't get at the article, sorry). Usability testing had been done on Evolution, and it was observed that one volunteer repeatedly used the "send/receive email" when they wanted to create a new email. The testers realised that the traditional "send/receive" button was not particularly intuitive. To my mind, that's the kind of useful information we might well get from this kind of testing - not assistance in turning Linux into Windows 2.

I mention this only because I believe there's still hope ;-)

Re:Mod parent +25.

[TuringTest]

Of course the "easiest" (and therefore the "best") user interface will be the one that is as close to 100% identical to the only one they've used before.

That's false. The best interface is one that reflects the user expectations. The Windows interface doesn't reflect user expectations in many ways, so it's possible to create a better interface than one which is just identical.

That should be the aim for the Linux Desktop design, not just to attract former Windows users but to best serve previous Linux users as well.

the driver hurdle

[mcraig]

Personally I think the hardest task for an average user to perform on linux at the moment is driver installation, and lets face it for most people getting all their hardware working is the first step towards adopting a new system. I recently tried installing drivers for my ATi Radeon 9800 Pro a pretty mainstream card from a well known manufacturer, needless to say it's not straight forward by any stretch of the imagination. You simply can't expect joe public to ever learn how to compile his kernel and even messing around with kernel modules is probably asking too much.

Linux is certainly making progress synaptic does a great job of alleviating dependency hell and almost entirely masking it from the end user. I'd like to see the linux community not necessarily looking to emulate the functionality in Windows or Mac OS X but instead looking for what would be the most elegant solution. Perhaps something like an online database of drivers that manufacturers could update, which could be automatically 'pushed' onto your computer overnight and silently rebooted (with your permission in a preferences box) so that you don't even have to worry about having the latest drivers it all becomes automatic would be neat, in the event it failed to reboot it could roll back to the previous driver and notify you in the morning of its attempt.

You could allow users to rate drivers and add the ratings to the database, this way you could specify you only want to automatically update to new drivers that are rated 3/5 or higher for example. This could be like linux's answer to Windows update only better.

Not with the testing they're doing.

[khasim]

That's false. The best interface is one that reflects the user expectations. The Windows interface doesn't reflect user expectations in many ways, so it's possible to create a better interface than one which is just identical.

Not with the testing that they're doing.

You are correct, in theory. You are incorrect in this specific instance because their testing procedure will not yield the information necessary to find a "better" interface.

That is because they are only testing prior Windows users.

Those Windows users have been trained to seek certain items in certain places.

Even if you added a button that said "Complete this test with one click", the users would NOT find it unless they could not FIRST find the Windows button/menu that they were trained to look for of if that button was in that location.

That should be the aim for the Linux Desktop design, not just to attract former Windows users but to best serve previous Linux users as well.

Again, I agree with that, but that will not be achievable through these tests.

Microsoft Word used to have an option to use the WordPerfect keystrokes. This was because the people with the most experience found it very difficult to maintain their productivity while learning a new system. Even if that system was "better" for other people. Back then, the most experienced and productive people had spent years learning WordPerfect for DOS.

Novell has learned nothing in these past years. To make it easy to migrate users, you make it an option to have an interface that is 100% identical to what they are familiar with.

Real "usability testing" requires more people with more experience levels on different systems, including people with little or no computer experience at all.

If you REALLY want to make the system easy to use, you have MULTIPLE options:

# 1. Basic level. Almost no menus and lots of "I want to" included in the icon's name ("I want to send an email to someone" or "I want to look at web sites").

# 2. Emulation level. 100% Win2K look-alike.

# 3. Whatever other interface you design.

The key is to build the interface to the user and what the user expects/knows.

It's No Less True

[aarmenaa]

I should start by saying that I use Windows on my primary desktop. For a while, I really tried to switch to Linux - Unbuntu, Mandrake, Slackware, and Fedora have all been on my box at one time or another. For the most part, getting the OS installed isn't rocket science. What is difficult is working around all the crap once you get a Linux distro installed.

Getting the desktop to look like anything except blurry ass requires an hour of reading about how to install your video drivers. Why? Because after installing your package using the really nice script, it still doesn't work. So you google again and figure out you need to edit that ghastly xorg.conf file. And then Google to figure out why the resolution is stuck. And then Googling again to figure out why the refresh is stuck at 54 Hz and giving you a massive headache. Dual monitors? TV out? You may as well just go cry yourself to sleep unless you're an uber-leet nerd, because that stuff takes hours to set up. That shit is a matter of one click in Windows; my mother can do it.

Then there's networking. Support for your wireless adapter may or may not even exist. If it does, it's probably in one of the generic Prism2 drivers or something like that. Great, but it doesn't help me a whole damn lot - mine says Netgear on the front. Back to Google again. It's also intresting to note that Linux's DHCP client and the server in my Linksys didn't get along real well, even on a wired connection. There's no way someone who doesn't know how that crap works would be able to troubleshoot that.

Of course, there's always multimedia playback, right? The install I liked best so far, Unbuntu, couldn't play anything out of the box. I know it should have been able to, but for whatever reason my install was futzed no matter how many times I reinstalled it. I never could figure out how to make it play videos. There were several settings for decoding and such (as well as about 10 different players to choose from), but nothing seemed to change no matter how I tinkered with those settings. Oh, and Unbuntu comes with several options for audio input and output including ALSA and ESD. WTF is the difference? I've heard of ALSA before so I'll use that one. Oh wait, that one doesn't work, but the ESD one does. Well, as long as I hear sound I don't really care. At this point, I don't even want to Google it.

This is why there aren't more Linux desktops: there are severe usability issues. I find it easier to get a webserver complete with PHP and MySQL up and running on Linux than a desktop. Why? Because I don't need video drivers, audio, or wireless networking. I also don't change my server hardware every month or two. Linux makes a great server, for sure. But as great a server as it is, it's a shitty desktop. And you'll please excuse my anger, I just got finished configuring my Linux install and promptly broke it...again.

Here's what desktop distros should be working on:

• -When it says it's installed, it'd better work (video drivers)

• -Drop the funny names. Yes it's superficial and shouldn't matter, but it does.

• -Make the defaults work. If the driver's there, the comptuer should play sound. And it should always be able to play video out of the box. And at least try to support the mouse wheel. I use mine a lot, and they come on every modern mouse. Why do I need to Google to figure out how to make it work in Linux?

• -Where possible, make it one click. Things like multiple displays shouldn't be so hard. Things like resolution and color depth should be changeable in an applet, not a config file.

• If a luser asks how to do something and your response is to call them a retard and tell to open up a terminal, your software's fucked. Lusers don't ask hard questions, which means that what they're asking for is a fundamental basic. And you just buried it under a shitload of command line.