Slashdot Mirror
Portable Usability Labs As User Research Tools
Pete Gordon writes "Do Portable Usability and User Research Labs make sense in the software development life-cycle? This interview (my bias--it's with me, and I have a tool in beta now) covers some of the issues and questions on KDE's news site.
I don't have the right answers necessarily, just looking for others input and opinions.
Also, here are other links about the subject over the past few months.
Info World and
Harry's comparison."
Open Source projects, more than other types of projects, have serious financial constraints. Is the cost/benefit ratio of performing these labs worth it? Seeing as how Open Source projects typically form the backbone of systems and rarely form the front (user-facing) end, is it worth it to spend time and money on projects that will only be used by developers and hackers?
At least he admitted that he was posting an interview with himself. And it helps that he tossed out a couple of other links to lend some credibility to the discussion.
After the recent slew of self-plugging stories and the article about 'marketing' which didn't call it astroturfing but described it, this is something of an improvement.
Any effort to get usability information is worth it, whether it's a full usability lab, or just sitting behind someone who is trying to use your software with a pad of paper.
The only people who don't think that usability is worth measuring are the people you wouldn't want working on UI to begin with.
RomSteady - I came, I saw, I tested. GamerTag: RomSteady / http://www.romsteady.net
As Engineers and coders etc etc, we tend to take alot of things .
.
.
.
.
.
.
.
as granted, and already understood ie. intuitive for our mindset
But for the common man sometimes it does not jive
If you think that alot of ppl still have flashing 12:00 on their VCR's
its gonna take some serious simplicity to get past their (fear?) of
the technical or just grasp of it
I think monitoring computer usage amongst beginners and maybe even
intermediates could show were ppl are frustrating themselves, and
perhaps tools that could be provided to ease the road more
travelled , ie. the electronic office/school/home
Ergonomics is for human physical comfort, this might provide
one for mental comfort of sorts
Best example I can offer is tech manuals that leave out a step
that is obvious to the person that wrote it or coded the app,
but leaves the first time user sitting there thinking its obvious
something was left out, but not sure how to proceed
Computer literacy is still pretty weak IMHO, and on the level of
Linux and nomenclature of OS subcomponents even more so
The more we understand the users, the better we can adapt the
interface
Ex-MislTech
google "32 trillion offshore needs IRS attention"
Great. Here's where I can get into the corporate BS, thanks.
At least he's honest.
Elizabeth Neal has recently written on this subject, and the title says it all:
Why You Don't Need a Usability Lab
Promoting the mindset for usability and user-centered design inside the KDE project is a very good thing, though.
Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
The most famous comes from anthropology. Watching and Observing Chimps in the wild like Jane Goodall.
Aha. Up to now I thought that Margaret Mead deserved the honour.
CC.
TaijiQuan (Huang, 5 loosenings)
Sheesh!
What're you? Some kinda shrink?
Man, you do realize that you're sounding like Sigmund Freud, right? Stop spooking me out.
why do you ask a question instead of making a statement?
Usability testing is absolutely essential to producing good software -- I've seen too many applications that left the developers hands and went right to market and were utter crap, because the developers put together what *they* thought people wanted, rather than actually verifying at any point that they were on the right track. (And then, most developers have the nerve to get pissed at the user for suggesting they make changes. Go figger.)
Usability testing also mitigates most of the round-and-round arguments developers will always have between themselves over some feature or another. Instead of butting egos, ask the users.
Portable usability test environments are not all that hard to come by. Here, we use a couple of Windows Laptops with TechSmith's Camtasia to record users sessions. We can take the laptop to them, present them with whatever we're testing, record the sessions, bring them back, play back the sessions, make our notes and changes, and go about our business. It works rather well for us, and it's much more affordable than building a dedicated facility. Much more convenient for the users, too.
Ed R.Zahurak
You know, oblivion keeps looking better every day.
That's interesting. How does that make you feel? Does it evoke memories of your childhood? Potty training with your mother, perhaps?
Why did you evade my question about the qualities that you admire? Are you hiding something?
Besides the cost factor, other advantages of portable setups over formal labs include ease of getting test subjects and public relations. Getting test subjects is easier if you go to them vs. having them take time out of their day to come to you. The public relations part has really worked well for us. Suddenly the users, their managers, *and* the people making the buying decisions see that we care enough about them and their business to take the time to go to meet them.
Big, Fat Shameless Plug: I regularly cover usability issues on my blog. I cover everything and anything related to usability, customer experience, human factors, ergonomics, human-computer interaction (HCI), user experience, interface design, and so on. I don't have any bias (e.g., open source, Microsoft), and I don't have any religion. I just report news on usability, offer comments, and write articles about usability. If you are interestd in the topic, check out my blog.
WebWord Usability Blog
Thanks for letting me throw this plug at you...
How to Download YouTube Videos
You can actually build a portable usability testing lab pretty cheaply. I've started to do so out of parts I already had laying around. The core of it is a pair of PowerBook laptops (one G3, one G4), each of which has an S-Video port, coupled with a Canopus ADVC-100 firewire video capture box. This lets me record a user's screen in its entirety into iMovie.
Couple that with an iPod with a voice recorder, an iSight camera to watch the user directly, and a key logger, and you've got a pretty decent usability testing lab. Since the mac has an X server, a VNC client, an RDC client for Windows Terminal Server, and Virtual PC, I can actually test software for pretty much any platform with stuff I cobbled together myself that fits into a suitcase.
Another component I've been playing with a little bit is "vnc2swf", which purports to attach to any VNC server and record everything that happens in the form of a flash file.
Either the author interviewed himself, or he has a formatting problem. If you are listening, there are no breaks or boldface to demark question and answer sections. Disconcerting to read.
Interesting read though. Here's an idea, how about taking advantage of reusability in open source by producing a software toolkit to enhance user feedback before and afer development?
Talkback is neat, I am thinking now of a really simple-looking "push-to-talk" button that (if your computer is set up correctly of course) would let people actually speak about their gripes so developers can get an in-your-face gritty idea of how others find it to use. In a few days of using OpenOffice which I like a lot heavily, I took the time to write up two pages of dense bugs and user issues. But it takes a while to type, and even way more if you want to input into a bug tracking system. I think many users would find that too hard/daunting. Assumes you have a mic working, which is unfortunately a big assumption with computers these days, but I think it could be really useful. Comments?
Creating truly usable software is a difficult task, and it makes sense that we'd want to apply the Power of Science! to the problem. So, we get Usability Testing.
Generally, the usability tests I've heard of work like this: you get a bunch of people and stick them in a lab (portable or not), and watch everything they do with the program for a while, as they complete a checklist of tasks. It seems to be prefered to get users who have no previous experience with the program, to prevent "bias".
Well, that's great, but it doesn't really address usability. It addresses short-term pickupability. Now, that's really important, and it _should_ be addressed -- but if it's the *only* thing you're concerned with, you'll miss other important issues relating to long term software use.
There's a unix-geek saying: "Unix *is* user-friendly -- it's just picky about its friends". Like all such jokes, there's a kernel of truth here. There's a very steep learning curve to the command line tools that are at the heart of the Unix environment, but once you've gotten up it, they *enable* you as a user to more easily do complicated tasks that would be very tedious in a GUI.
I don't mean that this is about the CLI vs. GUI thing, though -- that's just an example. I'd certainly be frustrated if my web browser were exclusively designed based on the reports of people who use it for a few hours to complete basic tasks. I'm concerned about the line of thinking that removes features which save huge amounts of time every day simply because they might confuse new users.
I won't name any names, but I will cough subtly in the direction of the GNOME project and at metacity.
Please, usablilty people, don't just think of first impressions. Think of the long-term relationship. Both have to be good.
It's Human-Computer Interaction (HCI). Every year there is a large ACM conference on this called CHI. There are also hundreds of HCI researchers all around the world at some of the top institutions working on problems like this.
Georgia Tech and Carnegie Mellon have two of the bigger masters programs available. Each program pumps out between one to two dozen people a year who should be well equipped to perform usability testing, among other things.
And you don't need a whole lab. You don't need to videotape often, and you don't need to buy some special software/hardware (you can, and they help, but you can get a lot of mileage from much less). Jakob Nielson and his cohort Don Norman have published a few good books that should be accessible to the uninitiated. Often times, some scribbles on paper are a better choice than prototyping the interface (scribbles usually give you higher levels of feedback, as opposed to "The font is ugly.").
There really are much better sources than articles like this one where people are just discovering HCI methods (not to rag on the article). Do a little google searching (you now have the right keywords: usability, hci), read some books (amazon is bound to have something up your alley), and maybe even ask some people in the field. There's a lot of really cheap, really quick things you can do to help yourself out (lookup Nielson's Discount usability, or you can hire an HCI person onto your team, we're very worth the cost).
BTW: There are many more excellent sources than Nielson, he's just the easiest to cite for applied HCI in a short period of time.
I agree with Elizabeth Neal that it's not necessary to go for a high-buck portable usability lab. I've done quite a bit of usability testing in the past 10 years or so. The most valuable? Paper prototyping.
It's cheap, it's quick, and the subjects aren't intimidated. Multiple design concepts can be tested quickly and changed on the fly and subjects are quick to cut loose with useful criticism.
By the time a design is mocked up for testing in a usability lab, it seems set in stone to the subjects. Their comments become fiddly, focused on minutiae ("I would use Ok rather than OK on that button") while whopping big issues of UI flow go ignored (e.g., an actual "tested" UI: a 40-page wizard (!) without a single back button).
What if necessary features aren't in the prototype?
Testing is a poor tool for doing feature selection and coming up with the concept, functional spec, and interaction design for a product.
Better than testing is doing up front field research to really understand user needs; internal business interviews to understand business goals; and then looking at features that will meet needs (instead of going on a feature frenzy first, as many many open source projects do)
However, on most OSS projects, if you don't code, you're a second-class citizen. There are regular threads every year on user experience lists about "why the OSS community should listen to us" that are filled with anecdotes of rejection by dev teams when a designer or usability person has tried to get involved.
I don't have any particular answers either, other than that I'm sure there are good OSS developers who would like UX design talent on the team - but there's not a real venue for getting them to work together, and there's not a culture of involving noncoders in most OSS projects.
Open Usability is trying to bridge the gap, but still has a long long way to go. (from getting profile in the OSS community and UX community to getting rid of the focus on 'usability' professionals and a focus on testing / evaluation)
Woah, I know a lot of people have 12:00 flashing on their VCRs. How many have it flashing because they can't set it after trying, how many because they could but don't feel it is worth learning how, and how many have given up because the power keeps going out. Most people have a nice clock in their living room set to the correct time. Unless they are trying to program the VCR to record while they are at work, they do not need a clock on the VCR. Most people just toss movie in and hit play, something that doesn't need the correct time.
In fact I would argue that it is a bug in VCRs that they don't automatically turn the clock display off when you press a button (other than one to set the time). That flash is annoying, and it will come back again the next time someone decides to try car VS power pole challenge.
test post maybe they finally unbanned my IP jesus 3 months..