Slashdot Mirror
Novell's Releases Linux Usability Testing Videos
sp3298622 writes "Novell is releasing primary desktop research, including over 200 videos and analysis of usability tests, at betterdesktop.openSUSE.org. Vice president of collaboration and desktop engineering for Novell, Nat Friedman: As a programmer, it's sometimes difficult to know how ordinary people with no technical experience are reacting to your software. Linux people tend to know other Linux people. In these usability tests, we selected test subjects who were experienced with Windows, but who had never heard of Linux, and asked them to perform basic tasks using the Linux desktop."
You know that ruddy "Linux vs. Windows Usability: The Quake Installation Test" troll is lurking around here somewhere.
That headline is just embarassing.
Might this only result in the Linux desktop becoming more like Windows?
89% sounds like a very good success ratio for the date and time test. However, RTFA and you'll see that only eleven people participated, most of them female.
If you don't have a diverse testing population, you aren't going to produce meaningful results. The idea is fine and all, but the results are mostly useless.
"It is the stillest words which bring the storm. Thoughts that come with doves' footsteps guide the world."
Developers, you don't get to check in code until you've watched the video of users struggling with your program. OK?
While I've seen things like this before, I'm liking Novell's approaches to Open Source more an more these days. With the excellent SUSE 10 (still may replace Ubuntu on my main workstation) and projects like Beagle and Hula, they're set to really make a splash if they take this useability idea seriously. They seem to be gelling more on the desktop than anyone else of late, 3 years ago who'd have expected Novell to be doing this? Awesome.
fak3r.com
200 videos + slashdot link = massive conflagration.
I don't get it.
It's a hard thing is to admit that free software has a usability problem. The natural temptation is to sit and watch these videos whilst screaming "You idiots! You don't click "Send and Receive" if you want to send an email! What's wrong with you?!?!"
It is difficult, but it's vitally important. These people aren't stupid losers- they are fluent in another operating system, where they can achieve whatever it is they want.
The problems on show here are ours, not theirs.
Martin
but another entirely to start working on a solution. The barrier to desktop Linux lies in simplicity, and without conducting a study or showing you a video, I can explain it easily.
Go to the web on a Linux PC (provided you've got a browser pre-installed), and download a tarball of say, Firefox. You are a Windows user but you're 'elite', so you use Firefox, and since it's available for Linux, you want to have the same browser.
You have downloaded the tarball, presumably to your desktop. You double click on the file, and it gets opened by Archive Manager. And from here, you can bet that 99% of the Windows folks that would like an alternative to their PCs will not make the adaptation to Linux.
It has to be EASY. Apple set the benchmark for this -- and if imitation is the greatest form of flattery, then do it. Who cares about inflating Apple's ego? If Linux makes a breakthru on the desktop because it's as easy to use as an Apple, or even as easy as Windows, how does that hurt anybody? The true geek can rely on the the commandline only distros, or drop to terminal to get their tasks done using regular expressions and grep or whatever they want, while the 'idiots' (and I would venture to say, that I'm one of them) can use the nice GUI that's simple to follow and easy to use.
Then folks, when developers see that they can cross develop applications that work in Linux (with little overhead), and that people will be able to easily use and access them -- THEY WILL. The open source community just needs to see that fact and start making solutions happen. With the extremely fast and accurate nature of Open Source, the feats that have come from it are amazing. It's more amazing, that the basis of Open Source -- Linux -- remains fundamentally unchanged to accomodate the eager Windows users (read: ME) to switch fully to Linux. Until the snobbery stops and changes start, Linux on the desktop is going nowhere fast. And that's upsetting for a Windows user tired of his OS, and not wanting to get tied into another corporate entity (Apple).
The price is always right if someone else is paying.
Missing Tests:
;)
1) Ooops! Find your kernels source, kill X, and install the drivers for your video card. Oh, and updated XF86Config. Or Xorg.conf. Whichever one you happen to have.
2) Damnit, another kernel panic. Find what obscure change caused it to happen this time!
3) Ah, so now you have a wireless card? Try to get it working! You might need to use ndiswrapper. If you get another kernel panic, go back to #2.
4) Ah, can't get above 800x600 resolution, eh? Yeah... find your monitors horizontal and verticle refresh rates. Google it, and you might get lucky.
5) Figure out how to resolve RPM dependancies. Shit, that package needs Python 2.4.2, huh? Ah well, 2.3.9 is installed. Guess you're out of luck.
All joking aside, this was a pretty intresting study.
Hmmmmm .... take people who are experienced with performing a certain function on Windows ....
Then put them in front of a different system (like say a Mac) and see if they have any problems performing that same function.
Of course the "easiest" (and therefore the "best") user interface will be the one that is as close to 100% identical to the only one they've used before.
That's great for Novell because they're trying to get a slice of the Windows market.
But this does not provide ANY information that any person could not get just by spending 10 minutes on a Windows machine and copying down menu locations and order and wording.
OK Here is (I think) the complete list from the article:
I found it interesting that eight out of twelve succesfully completed the "Find out if the computer is online" task. I also wonder if these users could complete all these tasks in Widows.
Insert Generic Sig Here:
Next week's article: Microsoft released a video of people who never used Linux, being asked to use a partly configured LFS system. The users were dropped at a command prompt and asked to simply fire up the windowing system, open the word processor, listen to music, etc. Every single user failed. MS points out that this is 100% accurate and solid evidence that Linux's TCO is 400% higher than Windows.
I have users who were quite skilled with Win2K who are lost with WinXP (until I show them how to make it look like Win2K).
So, which interface should Linux emulate then? Win2K or WinXP? Or Mac? Or something else?It is difficult and it is important
This approach will give you completely different answers depending upon whether the group you select is familiar with:
a. Win2K
or
b. WinXP
or
c. MacsYep. And so the "best" interface for Linux would be
Novell could have saved all that time and money and just spent 10 minutes with a Windows machine, copying down menu locations, order and wording.
There is NO "usability testing" being performed here. No one will learn whether a specific Windows implementation of a menu is less optimal than a different one.
All that will be "learned" is whether those users can find the Linux equivalent and that will always be easiest for them when the Linux menues are 100% identical to the Windows menues that those users are familiar with.
And we all know that programmers have no frickin idea how to satisfy a woman.
like shooting karma-fish in a slash-barrel. :)
My beliefs do not require that you agree with them.
As a programmer, it's sometimes difficult to know how ordinary people with no technical experience are reacting to your software.
How about "always completly fscking impossible" for more precision.
Not that there's nothing to gain from training and experience in usability design. Far from it: it will let you skip many obvious problems, and help you resolve others that users find for you in better and more efficient ways. But until your interface is tested on "real people" in at least a couple of iterations, there is no way in hell you can call it "good", "finished", or anything of the sort. If you don't agree, you've probably never done any real usability trials. There are always surprises. Often really big ones.
Your fine tuned detail somewhere may work just as plannned, but it will easily be swamped by problems stemming from inadvertent signals the interface is sending which never occurred to you, or from assumptions you never questioned or even spotted, which utlim ately make people (rightfully!) misunderstand the whole metaphor and do the wrong thing.
There are good news though: If you are willing to really really accept that the user is right (the way people percieve your product is in fact the way they perceive it, and you won't be around to explain to them that their thinking is wrong), and have set aside reasonable time to correct the problems you will find, - usability trials are fun!
Seriously. Fun, enlightening, and humbling (but in feelgood way), and they will broaden your horizons by illustrating just how differently from your daily assumptions it is not just possible but common to think. Do them. You'll like it.
Just resist the urge to explain the problem to the subject (except to be able to move on to test other things). Write down the problem in stead. The trial is for your instruction, not theirs.
sudo ergo sum
But it ALREADY is in a semblance of English... I mean it looks like English, I can understand every word...
I just can't parse it.
"Novell's Releases Linux Usability Testing Videos"
Let's take it a word at a time.
Novell's Ah, it's got an apostrophe s on the end, so it's either possesive or contractive. As Novell is an entity I'm assuming that we're talking possessive here. Something belonging to Novell. Good start...
Releases. Well, this can't be a verb as we're expecting the noun that is possessed by Novell, so while it might be nice to think that "Novell Releases" is the start of the sentence, instead we're looking at somethings (it's plural) that Novell owns. So Novell's Releases. Some items owned by Novell that have been released. Excellent, now what about these mythic Releases...
Linux... This isn't so good. Linux is a noun, and not a verb... Three nouns in a row? It's probably not unheard of, but in this case I'm expecting a verb. I want to know what Novell's Releases do... Well, let's soldier on and see if the verb appears later... Perhaps Yoda wrote this.
Usability... Nope...
Testing... Hmmm, perhaps test is being used as a verb and the entire portion in front is being used as a compound noun as favoured by Germans...
Videos. Yes, that's it....
The "Releases-Linux-Usability" (whatever that is) owned by Novell is testing Videos!!! Are they testing VCRs? Video Codecs? Movies? Perhaps if I read the article it would tell me.
Or perhaps they REALLY meant "Novell Releases Linux Usability Testing Videos" NAAAAH!
Z.
-- Under/Overrated is meta-moderation, and therefore is Redundant.
So, I wanted to install Quake. I use Linux a lot. I'm no expert, but I'm pretty comfortable with it. I've heard Windows was great for people who want to play games with no hassles. I knew microsoft made it, so I pointed konqeror at microsoft.org, and away I went. Ooops. MS is a company, not an organisation. Microsoft.com! Okay, so I tried to search the website for an .iso, so I could install Windows. Nothing! I realised that it was Commercial Software. I should have known this upfront, but I'm no windows expert. So, for the privelige of *playing games,* I went down to a local computer shop and invested over a hundred dollars in a copy of Windows. (I guess there are a lot of hard core gamers who wouldn't have a problem with putting down hundreds of dollars just to play games, but it isn't something I normally do.)
Installation was pretty smooth. I had to download nvidia binary drivers to get fully accelerated OpenGL, just like Linux. Windows is a supported platform for the drivers. I had to reboot the whole OS after installing them, because Windows won't let you easily drop back to a command line mode and just restart the GUI. No worries - I didn't have a server running on the machine, and it only takes a bit longer to reboot than to just restart a GUI.
Caution - Windows only comes with a special limited feature browser that doesn't support tabs, or anything. It is apparently only provided so you can download the latest version of a real browser after you install Windows. Windows doesn't come with a lot of useful stuff that you expect from a Linux distro...
So, I start reading docs to find out how you install apps on this new OS. I was having a pretty good time. Then, I learned that there is no equivalent of apt-get. If there is free software you want to download and install, you have to do it manually. So, I used the funny miniature "IE" browser to get the Quake source online.
Ooops, bad idea. Windows doesn't come with a compiler. You can download a free version, but the full featured "Visual Studio," costs a lot of money. I didn't feel like investing the effort to understand the differences. I decided to just get binaries. Again, there is no tool to automatically download and install an app, so I had to manually google for windows binaries. Thankfully, Quake is a very popular game, so it was very fast and easy to find, but still, it is an extra layer of inconvenience.
After a flurry of clicking "next" and eventually "finish," I finally had the game installed. Hooray. I tried to run it and I got a "BSOD." (Crash error screen) Of course, I already pointed out that Windows comes with no development tools, so it wasn't like I could try again with the debugger to see what happened. I had no way to see exactly what the issue was. What's worse, I couldn't get back to the system. This *game* had caused the equivalent of a kernel panic. It wasn't just the app that had crashed, but the whole system! this, from a system that is supposedly really great for games! It lets a game kill it!
Okay, so I rebooted into Linux. I already knew of a website with binaries for Quake, so I went there in Konq (Which came installed by default! I didn't have to go and download it!), downloaded a package...
dpkg -i Q
That was all there was to it. This "Windows is great for games" garbage is just horrible propaganda.
Now, if only I could get sound to work in Linux...
Personally I think the hardest task for an average user to perform on linux at the moment is driver installation, and lets face it for most people getting all their hardware working is the first step towards adopting a new system. I recently tried installing drivers for my ATi Radeon 9800 Pro a pretty mainstream card from a well known manufacturer, needless to say it's not straight forward by any stretch of the imagination. You simply can't expect joe public to ever learn how to compile his kernel and even messing around with kernel modules is probably asking too much.
Linux is certainly making progress synaptic does a great job of alleviating dependency hell and almost entirely masking it from the end user. I'd like to see the linux community not necessarily looking to emulate the functionality in Windows or Mac OS X but instead looking for what would be the most elegant solution. Perhaps something like an online database of drivers that manufacturers could update, which could be automatically 'pushed' onto your computer overnight and silently rebooted (with your permission in a preferences box) so that you don't even have to worry about having the latest drivers it all becomes automatic would be neat, in the event it failed to reboot it could roll back to the previous driver and notify you in the morning of its attempt.
You could allow users to rate drivers and add the ratings to the database, this way you could specify you only want to automatically update to new drivers that are rated 3/5 or higher for example. This could be like linux's answer to Windows update only better.
You are correct, in theory. You are incorrect in this specific instance because their testing procedure will not yield the information necessary to find a "better" interface.
That is because they are only testing prior Windows users.
Those Windows users have been trained to seek certain items in certain places.
Even if you added a button that said "Complete this test with one click", the users would NOT find it unless they could not FIRST find the Windows button/menu that they were trained to look for of if that button was in that location.Again, I agree with that, but that will not be achievable through these tests.
Microsoft Word used to have an option to use the WordPerfect keystrokes. This was because the people with the most experience found it very difficult to maintain their productivity while learning a new system. Even if that system was "better" for other people. Back then, the most experienced and productive people had spent years learning WordPerfect for DOS.
Novell has learned nothing in these past years. To make it easy to migrate users, you make it an option to have an interface that is 100% identical to what they are familiar with.
Real "usability testing" requires more people with more experience levels on different systems, including people with little or no computer experience at all.
If you REALLY want to make the system easy to use, you have MULTIPLE options:
# 1. Basic level. Almost no menus and lots of "I want to" included in the icon's name ("I want to send an email to someone" or "I want to look at web sites").
# 2. Emulation level. 100% Win2K look-alike.
# 3. Whatever other interface you design.
The key is to build the interface to the user and what the user expects/knows.
Sounds interesting! But I can't find any data regarding that comparison. Sure, there are tests about logging in, but no data about comparing KDM and GDM.
3 Gnome-apps, 2 neutral apps. Where's KDE-apps? Looking at the data-section, I see this:
Again: Where's KDE? Going thropugh the test data I see that every single test was with Gnome. Where's KDE? So instead of being called "Better Desktop", maybe this should be called "Better Gnome" instead? then again, what can we expect from having a Gnome-guy running the show? So much for equal handling of the desktops....
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
Getting the desktop to look like anything except blurry ass requires an hour of reading about how to install your video drivers. Why? Because after installing your package using the really nice script, it still doesn't work. So you google again and figure out you need to edit that ghastly xorg.conf file. And then Google to figure out why the resolution is stuck. And then Googling again to figure out why the refresh is stuck at 54 Hz and giving you a massive headache. Dual monitors? TV out? You may as well just go cry yourself to sleep unless you're an uber-leet nerd, because that stuff takes hours to set up. That shit is a matter of one click in Windows; my mother can do it.
Then there's networking. Support for your wireless adapter may or may not even exist. If it does, it's probably in one of the generic Prism2 drivers or something like that. Great, but it doesn't help me a whole damn lot - mine says Netgear on the front. Back to Google again. It's also intresting to note that Linux's DHCP client and the server in my Linksys didn't get along real well, even on a wired connection. There's no way someone who doesn't know how that crap works would be able to troubleshoot that.
Of course, there's always multimedia playback, right? The install I liked best so far, Unbuntu, couldn't play anything out of the box. I know it should have been able to, but for whatever reason my install was futzed no matter how many times I reinstalled it. I never could figure out how to make it play videos. There were several settings for decoding and such (as well as about 10 different players to choose from), but nothing seemed to change no matter how I tinkered with those settings. Oh, and Unbuntu comes with several options for audio input and output including ALSA and ESD. WTF is the difference? I've heard of ALSA before so I'll use that one. Oh wait, that one doesn't work, but the ESD one does. Well, as long as I hear sound I don't really care. At this point, I don't even want to Google it.
This is why there aren't more Linux desktops: there are severe usability issues. I find it easier to get a webserver complete with PHP and MySQL up and running on Linux than a desktop. Why? Because I don't need video drivers, audio, or wireless networking. I also don't change my server hardware every month or two. Linux makes a great server, for sure. But as great a server as it is, it's a shitty desktop. And you'll please excuse my anger, I just got finished configuring my Linux install and promptly broke it...again.
Here's what desktop distros should be working on:
"I do a grep for shit, bollocks, and tits before checking in code. I'm professional..." -RECURSIVE_META_JOKE, reddit.com
-1, Troll.
Another one bites the dust