Portable Usability Labs As User Research Tools

Pete Gordon writes "Do Portable Usability and User Research Labs make sense in the software development life-cycle? This interview (my bias--it's with me, and I have a tool in beta now) covers some of the issues and questions on KDE's news site. I don't have the right answers necessarily, just looking for others input and opinions. Also, here are other links about the subject over the past few months. Info World and Harry's comparison."

Comments

Usability

[RomSteady]

Any effort to get usability information is worth it, whether it's a full usability lab, or just sitting behind someone who is trying to use your software with a pad of paper.

The only people who don't think that usability is worth measuring are the people you wouldn't want working on UI to begin with.

Ppl and PC's

[Ex-MislTech]

As Engineers and coders etc etc, we tend to take alot of things
as granted, and already understood ie. intuitive for our mindset .

But for the common man sometimes it does not jive .

If you think that alot of ppl still have flashing 12:00 on their VCR's
its gonna take some serious simplicity to get past their (fear?) of
the technical or just grasp of it .

I think monitoring computer usage amongst beginners and maybe even
intermediates could show were ppl are frustrating themselves, and
perhaps tools that could be provided to ease the road more
travelled , ie. the electronic office/school/home .

Ergonomics is for human physical comfort, this might provide
one for mental comfort of sorts .

Best example I can offer is tech manuals that leave out a step
that is obvious to the person that wrote it or coded the app,
but leaves the first time user sitting there thinking its obvious
something was left out, but not sure how to proceed .

Computer literacy is still pretty weak IMHO, and on the level of
Linux and nomenclature of OS subcomponents even more so .

The more we understand the users, the better we can adapt the
interface .

Ex-MislTech

Re:Cost?

[metlin]

Short term, yes. Long term, no.

In the long term, it would be worth it. Hate it as you will, the precise reason Windows does so well in the market is its user interfaces.

User interfaces play a very very vital role in user behaviour, and usage.

I do not understand the argument that developers should not have good UIs. Why not? Would you not use a Visual IDE for your development if it had more features that you would use? Or would you rather that we all stick to CLI?

In fact, I really *like* Microsoft's Visual Studio .NET's IDE -- it's really quite well done, and very well designed with the developer in mind. And guess what? It increases my productivity by a significant amount when I code.

I'll just say this -- if Linux has to make it big, user interfaces _are_ a big deal.

There is a HCI maxim that says that the best designs are those that you do not notice -- that is what we should be striving for, Opensource or not. You never know who would be using your Opensource application for what.

And a good UI design is only going to help it.

Re:A good thing, but not indispensable

[platos_beard]

I don't think the most important factor determining usuability is addressed in either article. User interface design should be done by user interface designers, not programmers. They need a completely different skill set. While programmers need to understand the working of a computer and be able to extract essential information from documentation, UI designers need to understand the people and processes in the domain they're designing for and be able to extract essential information from people.

Put simply, programmers need computer skills while designers need people skills. Sometimes they overlap, but no more than random variation dictates (and possibly somewhat less). And even if they do, its a different mindset while doing one job vs. the other.

And both jobs are hard. A good UI designer has to get beyond the specific suggestions from users to see what the underlying need. UI designers have to find a way to get users to envision a system that doesn't exist yet and figure out how it could work best. Prototypes are essential. Skills to run meetings are essential.

The toughest part is dealing with criticisms of proposed designs. Sometimes the criticisms are because the new design isn't understood well enough, but other times the criticisms reveal a design flaw. Distinguishing between the two, and correcting misunderstandings of the proposed design without stifling further criticism (which you need) is a delicate art.

Re:Cost?

[sgtrock]

I think I understand the point that your CIO was making, although I don't know who or what JWZ is. I've heard similar comments in the past from people. I have found that the best response is to turn it around. Ask how much time and money is spent dealing with problems with an existing solution. :)

Seriously, OSS is not a panacea. There are some OSS projects which are mind-bogglingly successful precisely because they provide a solution that is more cost efficient overall. This includes usability for the pieces that matter. The classic LAMP implementation is one such solution that springs to mind. OpenOffice.org is another that meets the needs for many (not all!) organizations. Lyx is also one that I'm personally fond of for document creation.

However, this is not necessarily true of all OSS projects. Even OOo won't fit the bill for some organizations. The best advice that I can give is to stay calm and objective. When another project comes up where OSS might fit the bill, ask yourself if your research shows a shallow learning curve in addition to equivalent or more functionality. If so, suggest that a bake-off be done between your proposed OSS solution and the closed source candidate(s). Make sure that the bake-off targets usability and functionality.

If the OSS project passes, great! You've got a foot in the door. If it fails, don't be disappointed. Don't beat a dead horse and whine about how OSS Teh Way. Instead, use the opportunity to find out what the end users and other techies thought was missing. Use that information as part of your search criteria when you go looking for the next possible candidate.

It may take you a couple of tries before you get an OSS project in, but as long as you stay calm and professional in your relationships with the other people involved, it won't cost you any skin. Keep your CIO in the loop as you do this exercise. He'll appreciate it.

Short-term testing alone is harmful

[mattdm]

Creating truly usable software is a difficult task, and it makes sense that we'd want to apply the Power of Science! to the problem. So, we get Usability Testing.

Generally, the usability tests I've heard of work like this: you get a bunch of people and stick them in a lab (portable or not), and watch everything they do with the program for a while, as they complete a checklist of tasks. It seems to be prefered to get users who have no previous experience with the program, to prevent "bias".

Well, that's great, but it doesn't really address usability. It addresses short-term pickupability. Now, that's really important, and it _should_ be addressed -- but if it's the *only* thing you're concerned with, you'll miss other important issues relating to long term software use.

There's a unix-geek saying: "Unix *is* user-friendly -- it's just picky about its friends". Like all such jokes, there's a kernel of truth here. There's a very steep learning curve to the command line tools that are at the heart of the Unix environment, but once you've gotten up it, they *enable* you as a user to more easily do complicated tasks that would be very tedious in a GUI.

I don't mean that this is about the CLI vs. GUI thing, though -- that's just an example. I'd certainly be frustrated if my web browser were exclusively designed based on the reports of people who use it for a few hours to complete basic tasks. I'm concerned about the line of thinking that removes features which save huge amounts of time every day simply because they might confuse new users.

I won't name any names, but I will cough subtly in the direction of the GNOME project and at metacity.

Please, usablilty people, don't just think of first impressions. Think of the long-term relationship. Both have to be good.

Why many user experience peeps don't do OSS

[count0]

It's a nice thought - open source user experience design (user research, interaction design, functionality, UI, not just visual design).

However, on most OSS projects, if you don't code, you're a second-class citizen. There are regular threads every year on user experience lists about "why the OSS community should listen to us" that are filled with anecdotes of rejection by dev teams when a designer or usability person has tried to get involved.

I don't have any particular answers either, other than that I'm sure there are good OSS developers who would like UX design talent on the team - but there's not a real venue for getting them to work together, and there's not a culture of involving noncoders in most OSS projects.

Open Usability is trying to bridge the gap, but still has a long long way to go. (from getting profile in the OSS community and UX community to getting rid of the focus on 'usability' professionals and a focus on testing / evaluation)

Re:Why many user experience peeps don't do OSS

[TuringTest]

I would fight this battle with the "Pixels Are Code"(tm) mantra. The idea after this slogan is to remark that HCI experts create a precise, exact output which is a high level description of the final application; something akin to the intermediate bytecode produced by a compiler, just with a very descriptive language. I.e. UI experts produce one step of the design process.

You should persuade OSS developers that the work of a usability analist requires technical skills and produces careful designed specifications, then they will recognize your hacking ability as a part of the programming process.

The problem I've seen with OSS developers is that they are not familiar with the knowledge and glossary of the HCI field, so they tend to consider advice from a UI expert as mere opinions. In order to break this trend we'll need to better marketing the usability techniques, in a language that they could understand and relate to the "code-sharing" experience of Open Source. Thus the usefulness of the "P.A.C." mantra.

Re:You can't test your way to a great solution

[Pete Gordon]

I totally agree with this. I really want to communicate the information gathered from field research back to everyone involved in the product--especially the developers. That is the real goal of this lab!

Screen capture or not, doesn't matter. You could just use the video (multiple cameras) and audio to capture the business process of an individual or a focus group setting.

I really want to see a User-Centered Design approach.

Barbara Nelson at Pragmatic Marketing (a marketing IT shop that is focused on the Product Marketing Manager) said it this way...

"Listen to customers everywhere - online, onsite, at user group meetings, customer advisory boards, technical support, usability labs, point of sale, focus groups (in person and online), and through email. Anywhere they might be. Their natural habitat is by far the most fertile, but take advantage of other places they might congregate."
http://www.productmarketing.com/magazine/1/4/09bn. htm

Thanks!
Pete Gordon