Slashdot Mirror
Usability Testing Hardy Heron With a Girlfriend
toomin writes "Reviews of the latest Ubuntu version, 8.04 Hardy Heron, are everywhere, but most of them are undertaken by geeks familiar with Linux. This guy sits his girlfriend down at a brand-new Ubuntu installation and asks her to perform some basic tasks. Some of them are surprisingly easy, others frustrate and annoy. There are lots of little usability tweaks he stumbles upon just by seeing the desktop experience from the point of view of the mainstream user."
That's how user testing should be done. It is really much too difficult for someone familiar with the program or OS to see what is not obvious or confusing to a novice user. The people that program the UI don't always think like a user - they usually think like a programmer, and that doesn't always work.
This entire review is focused on that Ubuntu should work and act just like Windows
... real geeks need things to be unique to their "domain", so they can lord over the rest of the plebs with their advanced technical knowhow.
Yes, because that would be FAR too easy
</sarcasm>
2) Watch youtube. Unsuccessful. No Flash.
11) MSN. Unsuccessful
Yeah well, I have a girlfriend too, and all she ever does on a computer is watch music videos on YouTube, write e-mails and chat on MSN. Maybe you'd like to weight your rating based on how important something is to the person tested (by asking them). Downloading a torrent and changing your mouse speed will probably rate to 0 while MSN will probably rate to "Why the hell would I need a computer if not for MSN?".
You just got troll'd!
I wish I'd taken that train of thought a little further before clicking submit.
... it's not necessarily "better" because it's "different" ... in fact I think the opposite.
IF Ubuntu (or release of your choice), WAS more like Windows, just think how much higher the adoption rate would be for it. Imagine how EASY it would be if you could show people with only a Windows background, "look, you do the same things and get the same result - only this one is free, doesn't come cluttered with DRM, isn't susceptible to malware etc etc".
It could be the BIG selling point, if only you'd give in to the pressure. But comments like this just perpetuate the "elitist" attitude of Open Source
If only you'd see the wood for the trees, you could have your open source "revolution" we've been waiting for for 25 years.
well they have to when transitioning from XP to Vista...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
And most people would rather stay with XP instead of moving to Vista. Just like they'd rather stay with XP than moving to Ubuntu.
Yeah, have had that problem with YouTube on ubuntu as well, and it only happens if YouTube is the first site you go to, they fubar their refer to adobe.
If she had of gone to, say, ANYWHERE else first it would have been fine and just popped up with a bar saying "i can haz plugin?", say yes and bam, its all good.
Long and short, it should have flash already on it.
Oh and she was lucky it was the 32bit version, installing and using skype is pretty hard when there ain't a 64bit binary available for the platform (last time I looked anyway).
...
>she should have read the release notes
Ahaha, good one. I thought the very point of this exercise was that users do not behave like developers expect or would like them to. Reading release notes is certainly among the things they rarely ever do, and so this hints at Ubuntu doing something wrong more than anything else.
- Frozen Bubble is available on Linux, as well as a lot of good games
- Less viruses than XP, and so no need for an anti-virus
- Firefox and "MSN" chat are available on Linux
- Free IT support when I'm available at home
I know that it may seem redundant but computer noobs switch for very strange reasons and we must listen to their needs if we want Linux to "succeed on the desktop."Wow, what a novel, totally-unheard-off idea! Usability tests! And actually, you know, doing them, instead of just talking about them. And with, you know, actual users. Wow! Quick, get a patent about that!
Frankly, this is why I gave up about Linux-on-the-desktop back in, I'm not sure, 2000 or so. When the Gnome User Interface mailing list was full of people with great and groundbreaking ideas, most of them blatant violations of everything that HCI had long dumped as bullshit, others completely untested, the rest copies of windos ("because that's what people know and expect"). Most importantly, there wasn't a single expert in the field on the list or - to my knowledge - in the entire Gnome project. Yes, including me, having read a bunch of books on the subject doesn't make me an expert, it just allowed me to spot the I-have-no-idea-but-I'll-pretend-I-do guys more easily.
Linux has suffered tremendeously due to this disregard of the normal, non-geek user. You know, the kind of person whose VCR flashes "12:00" because the UI on VCRs is total shit and only geeks really bother with it because we are the only ones who consider bugs and technical problems to be a challenge instead of, say, bugs and technical problems.
Assorted stuff I do sometimes: Lemuria.org
The article several times suggests that the solution to some of these problems is, essentially, user education: having balloons that signal "new item installed" or wizards open the first time you launch a program, telling you how the program works.
The problem is that this approach often doesn't work. For one thing, it annoys the piss out of experience users. For another thing, new users tend to ignore most of that information... mainly because they are being overwhelmed by new information and can't possibly assimilate it all.
Take, for instance, the problem that was encountered when changing screen resolution. The tester changed the resolution easily, but then she clicked the "Keep settings" immediately, which locked her into graphic settings that were hard to change back. Part of the problem, I suppose is that the system allowed the user to make a ridiculous change. But part of the problem is also, perhaps, that the user is very used to clicking "OK" on any dialog that gets in the way: there are too many new things to read and learn, and the easiest way to get things done (in the mind of a new user) is to dismiss those annoying boxes as quickly as possible. Would a second popup, that described in detail why this low resolution was a bad idea (and how to undo it when desired), have changed anything? Doubtful. Most users would just click "OK" without reading it.
All this to say that I'm by no means convinced that adding more balloons, wizards, and dialog boxes will magically make it easier for users to figure out what's going on. I don't know what the solution is: usability is a tough problem. There is a place for helpful information (balloons, tool-tips, etc.), reminders, and wizards. But too much of this becomes decidedly counter-productive.
Using a list of common tasks to test a piece of software is simple, brilliant, and done all too infrequently. I can't count the number of times I've been using an application and can't work out how to do something that hundreds of other people must also want to do, and yet the developer has not made this task obvious.
Open source tools often suffer from this because they are so proud of their features. I remember trying to burn a data CD under Linux 3 or 4 years ago and the tool I was using did not make it at all obvious how I drag files into the workspace and then burn them. It did, however, make the SCSI options for me CD drive immediately available.
Web pages often suffer because they are style over substance. Yes, it looks very pretty, but how do I buy stuff? Well done for having text on the front page that tells me how "obsessed by quality" you are, but where is the contact address so I can ask you questions about your product?
Software libraries suffer when they are more interested in recruiting developers than serving users. I remember trying to use a Perl tool that required a specific module. The web page for the module had a great deal of links about writing applications using that module but nowhere on the whole page did it tell me how to download and install the module so it could be used with an existing application.
When designing any piece of software think: what will this be mostly used for? Does it make it easy to do that?
I think Windows achieved 12 out of 12 at least 5 or 6 years ago!
I'd love to see the same test done a default current (let alone 5 or 6 years old!) windows install.
Advanced image manipulation? Torrents? Finding stuff on a non-windows-format partition? I doubt it.
There are shills on slashdot. Apparently, I'm one of them.
Of Windows was easier.
The test was how well can a windows user do common tasks on Ubuntu. I sure hope they are better on Windows. Everything is easier when you know how to do it already.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
And how well do you think your girlfriend would go on a vanilla windows install with no flash installed, MSN account not setup, etc?
Quite fine, there is the Windows Messenger that comes by default in Windows Xp.
And about Flash? the first time you go to a flash-based site, it tells you you must install Flash and asks you if you want to install it, after there, just clicking YES YES NEXT NEXT ACCEPT YES will take you to a fully working Flash IE.
If the article's author had setup flash / pidgin / explained the difference between GIMP & Open Office draw, his girlfriend would have had few problems.
Yeah, but that is the main idea with this Usuability testing/QA issues. You do not want having to explain to everyone about such things. By the time you are telling them "GIMP... which stants for GNU Image Manipulation Program.. GNU is for GNU is not Unix" they lost all interest and just tell you to please resintall Windows.
Ubuntu is an African word meaning 'I can't configure Debian'
The problem is indeed persistent. It's pretty easily summarized:
The price of ignorance is subject to inflation.
-1 Uncomfortable Truth
And what do you mean this joke has been done before?!
The number of comments in this thread from people who are dismissive of usability tests is indicative of the gap between Linux fans and everyone else.
Usability is more than checking to see if a user evenrtually figured out how to get something deon. E.g., it's disastrous if they can't figure out how to open a file. But, the fact that they managed to open the file is not necessarily an indicator of a good design.
Usability is something people pay cash for. Just ask Apple and the Photoshop folks.
-- Slashdot: When Public Access TV Says "No"
Believe it or not, not everyone buys their laptop with built-in wifi with the idea of someday installing linux on it. Lets not even get into that many, many people don't even have a clue which chipset is used in their device...
Machine9dotNet
Yeah, and it's a huge part. It's the 800 pound gorilla part.
Testing for useability needs to come in much, much earlier in development, and it needs to involve a much wider cross-section of human beings. And as it's being done, development of adequate documentation and help needs to go hand and hand with it.
It's so easy to disparage girlfriends, the middle-aged, and the elderly--in short, anyone whose job or study is not technical--that I think it's becoming ingrained in the cultures responsible for developing the various operating-system distributions and open-source software packages. This is going to cause them to suffer over the long haul. It's what makes them such a tough sell to people in business.
There's an immense population of middle-aged people, for example, still in the work force. And interestingly enough, they've actually now all got 20 or 25 years' experience as end-users of computer systems. They're not stupid. They all have jobs that they need to get done. They're not interested in being part of user communities and forums. They're not interested in the ideals of free and open-source software. They're not interested in sticking it to Microsoft. They're not interested in that warm feeling of accomplishment that until recently accompanied getting your printers hooked up to OpenOffice--after wasting hours of productive time doing it. They're interested in using their computers as tools to accomplish their current day's work.
Issues of usability and documentation aren't much fun. They're probably the least glamorous and most boring functions of developing the software. That's why they get such short shrift in open-source development. Nobody really wants to take them on, so we're treated to excrescenses like having people guess how to get out to a command line to install their audio player or their scanner or their printer.
Large-scale developers of proprietary software know precisely where their bread is buttered, and they attend to all this as a matter of course.
"Girlfriend" articles seem to appear quite regularly every few months, so at least somebody is thinking about this even at a ridiculous level. A lot more people need to be thinking about it at a much more serious level.
"Here's what's happening. You're starting to drive like your Dad..." - Red Green
You have to shop around.
And if you buy a Windows machine, what version of Vista does it have? Now play a DVD on a vanilla version of Vista Basic. Contact someone on AIM on Vista. Edit photos on Vista basic. Run torrents.
What, they require installation?
Ah, we're back to Ubuntu being easier.
And how difficult is it to install Ubuntu anyway? Pretty damn easy.
Fud fuddity fud.
"Say what you want about the command line and editing the config file, and how users shouldn't have to do it, but at least the option is available. Any idea if the same can be done under windows?"
Yes. You can boot up into safe mode and get a fully functional GUI at VGA resolution so that you can launch whatever gui / wizard non-command line xorg.conf editing program you need to get your desktop back.
Granted, only tech-savvy people would know how to boot up into safe mode. But I still give this one to Windows. Ubuntu could include a "safe mode / recovery" option in Grub that boots up into X with VGA resolution and a **USABLE** GUI and there's no reason they couldn't also keep a command-line recovery mode for people who prefer / need it.
Another crtique that I have for Ubuntu is that they give weird (at least to non-tech-savvy people) names to their boot options in Grub with the kernel version etc. To a geek that information is very important but to a new user it's confusing. A new user who installeda dual boot would expect to see something like "Linux ~ Windows XP ~ Linux (Recovery Mode)" or something. I remember seeing this same critique on Brainstorm so hopefully they'll do something about that.
What's up with the mindless psuedo-feminism?
Yes, a guy is going to be interested in treating a girl as a sex
object. More likely than not, the girl will get upset if you don't.
They will complain that you don't appreciate them or some such.
Girls aren't just chaste nuns. Guys aren't just rampaging huns.
A Pirate and a Puritan look the same on a balance sheet.
I'm sorry, but I'm sick of things like this - his girlfriend HAS to be some untutored user who has no clue about computers, tee-hee. As a female computer programmer, should people assume my husband is computer illiterate? No? Then why assume his girlfriend is?
Isn't it enough to say that the installation was tested with a novice user instead of putting stupid assumptions and implications right in the freaking headline?
And of course, what kind of replies do I expect to my post around here? People marveling that a woman is posting on slashdot that will be modded up as funny. Given the nature of my post, I also expect some responses telling me to calm down or calling me a feminazi. There, I've taken care of those responses, you can stick to ones that actually address what I've said.
The big stumbling block I think most UI developers hit is the fallacy that elements of the interface need to mimic something "popular" or "familiar" (typically a Windows UI, given their market-share).
Doing that amounts to a short-cut, where you're settling for pleasing people who "already struggled to learn a previously designed interface" - at the expense of winning over a crowd that never mastered that one in the first place.
I'd say usability testing like what was done in this article is the ONLY way to succeed, if you're going to proceed with a true goal of "ease of use". Anyone savvy enough to help build an OS is FAR too removed from comprehending all the points of confusion the "average user" might run across.
(And again, this is why a developer typically thinks so "inside the box" when it comes to concepts like "open file" dialogs, "save" and "export" options on menus, or assumptions about which plug-ins and "extras" a typical user will need.)
Personally, I think "file extensions" are a pretty evil concept, the way most operating systems today handle them. I'm not saying they're "pointless". Rather, they're too technical in nature for the casual user. I think Apple was on the right track when they decided, years ago, to let the OS decide what application a file belonged to via hidden "resource fork" data, instead of the 3 character extension on the end of the file defining it. The problem is, Apple tried to do this when it went "against the grain" of what every other OS was doing, and it created too many headaches for moving data between platforms.
In addition, it used to create a lot of problems where an app would "marry itself" to a certain file-type, and that wasn't always the most desirable behavior. I think they need to brand "metadata" onto files so apps can check there to know what a file is, but provisions are made to allow *several* apps to become preferred options for working with a given filetype. (EG. When a user double-clicks a document, a dialog opens asking if they'd like to open this with "MS Word, CoolJoe Writer, or Starview? Or click here to modify your selection of preferred programs used to work with this type of document.")
The problem with programs being "helpful" is exemplified in one of the main features of the Firefox 3 beta that Hardy comes with. That URL bar, where starting to type a URL you've been to before brings up a list of matches? In the new FIrefox, the developers figured it would be better if the page titles were matched too, not just the URL. And while it's doing that, they decided to display the page titles larger and below the URLs, rather than to the right of as before. And it doesn't just match from the beginning, but from anywhere in the page title. So you type the letter "t," and whereas before you would have seen the several URLs beginning with "t" that you've recently visited, now you see every page where the letter "t" occurred in the page title or URL. Then you type "h," and instead of (in my case) being down to "theoildrum.com" and "theonion.com" there's a long list of every page title or URL with the letters "th" in them. Adding "e" doesn't narrow it down much.
Oh, and while there used to be a way to revert Firefox to its old, much more useful, behavior, that option was removed two months ago because the developers are so in love with their "helpful, friendly" method of showing every page from your history with "the" in the page title.
Okay, here we have the best-funded project in open source, free software. We have Google as the main funder - who know something about simple interfaces and usability. And we have Ubuntu choosing to default its users to this new Firefox, despite the Mozilla project's upfront statement that it's not considered ready for regular use yet. (And it's not in other ways too: Elements swim around the page much more than they do in Firefox 2, meaning that while it may be faster on the back end, on the front end there are longer periods where you just can't look closely at it without quesiness.)
So here are developers trying to make their packages ever-so-helpful. And they fail, for the precise reason that Microsoft so often fails. The best tools, for people who will use them every day (or even weekly) are well honed, not cluttered with cruft.
It's also why the girlfriend test is of limited utility. What matters isn't how it looks to a brand-new user, but how it looks to that user after a month of use. That's the point where they say, "This looked promising at first, but damn do I miss the better fit-to-hand of the tools in my old OS." As in any new relationship, the most critical test isn't the first date, but where you are after a month together. Optimizing for the first date often directly undermines the long-term viability of the relationship.
"with their freedom lost all virtue lose" - Milton
If you're talking to a novice you're not going to go into tons of detail about acronyms and such, you're going to say "GIMP does stuff that photoshop does"
I think your argument is silly.
I've performed usability tests in the past and regardless of if you're a coder or a usability expert, they almost always reveal usability issues that seem unintuitive or which people want to "explain away" because they can't wrap their heads around what is happening. Usability testing is, however, a very underused scientific process in computing today.
That said, over the years a lot of general guidelines have emerged that can help designers avoid some of the really common mistakes that happen over and over and over again. From my own use of Ubuntu, it is clear there has not even been consistent use of those guidelines to get the low hanging fruit before going to the expense of a real usability test. There is much work to be done.
Personally, I think "file extensions" are a pretty evil concept, the way most operating systems today handle them.They are, indeed, a common usability problem, especially when one goes into hidden extensions and multiple extensions. Expecting users to know what three letter combination mean "program that does stuff" and which ones mean "data you can read or see or hear" is absurd, especially when they can only sometimes see them and what they see is not always the "real" extension. It is certainly an area where OS's could be significantly improved.
You do realize that people who actually have girlfriends have plenty of time to spend with them, and doing one thing doesn't preclude another?
In any real-world relationship, even one with a good amount of sex, most of the time will be spent doing other things.
Ubuntu certainly has come a long way in terms of usability for the average person. But it's still not nice enough for the average person.
I built a computer for my fiancee and put Ubuntu 7.04 (and later 7.10) on it. She thinks it's just OK. She recently quit her job to go back to school. She wants a laptop, and she insisted that it have Windows XP. I didn't even suggest that she let me put Linux on it. Her current Ubuntu workstation is used for casual tasks that aren't that important: MySpace, YouTube, email, listening to music, etc. Since these tasks aren't of great importance, she's willing to wait for me to fix things that break (and they do break). But I can't expect that of her when she's got a paper to turn in or an online assignment to complete.
I know there's a huge debate (maybe even a holy war) about whether open source desktops should innovate or just copy Windows. Fortunately there's enough choice that different projects more or less have both avenues covered. But---and it pains me to say this---emulating Windows (pre-Vista Windows anyway) is probably the way to go if converts are really wanted. Even though doing things in Windows isn't always consistent or elegant or even easy, it's familiar. Computer geeks like myself like differences: they are fun to explore and play with and foster ideas. But the rest of the world just wants to get their work done. They don't care what's going on behind the scenes, and they don't have the patience to mess with "restricted drivers", the command line, "Synaptic", or different document formats just to get something done.
Right now sound isn't working on her Ubuntu machine. All I did was run the update. Looks like a buggy ALSA driver got installed (do a search for "ALSA lib pcm_dmix.c:864:(snd_pcm_dmix_open) unable to open slave"). But for the life of me, I can't find a fix, other than to upgrade to 8.04.
When I initially set up her computer, there was some version mis-match or something like that with the packaged flash player. The solution was to download a specific version of the .deb file and do a manual dpkg (or apt-get) install. Relatively easy for me, but my why should I expect my fiancee to have to go through all that? Even in the article, the author said it was YouTube's fault for redirecting a browser that doesn't have Flash installed. It doesn't matter---in Windows, when you go to a Flash site without having Flash installed, you get the same behavior: "Click here to install flash", and you're done.
I installed Skype on her computer; it mostly works, except that the emoticons aren't animated like they are on the Windows client. Not a big deal, but it's just one more thing that makes Linux feel like a downgrade to her. This is one example where the Linux world lacks the polish of the Windows world. Polish may not mean anything to the geekier among us, but it does make the typical person feel like they've take a step backwards.
She recently got an iPod Shuffle... this sort of works. I forget the name of the application that auto-loads when the iPod is plugged in, but about half the time it crashes. No error message or anything, the whole Window just disappears. Sometimes we'll start sync'ing songs to the iPod, minimize that application, go do something else for a while, and... where did that program go? Looks like we have to start the process all over again.
There's been a number of other issues, off and on, that require my intervention (i.e. track down the bug or quirk on the web, then install/uninstall/upgrade/downgrade a package, and possibly hack some things on the commandline). But the biggest killer for her is still OpenOffice.org. That program also crashes randomly. (Ironically, one of the crashes was when I was having her make a list of things she liked/didn't like about Ubuntu, and what she needed for a laptop. I can't think of a more simple/straightforward task: start OOWriter, create a bulleted list and save.)
But the MS Office to Ope