The Future & History of the User Interface

An anonymous reader writes "The Mac Observer is taking a look at UI development with lots of video links to some of the latest developments in user interfaces. It also has links to some of the most interesting historical footage of UI developments, here's one of the 1968 NLS demo. From the article: 'Sadly, a great many people in the computer field have a pathetic sense (or rather ignorance) of history. They are pompous and narcissistic enough to ignore the great contributions of past geniuses... It might be time to add a mandatory "History of Computers" class to the computer science curriculum so as to give new practitioners this much needed sense of history.'"

Comments

Overlapping windows

[Peaker]

Heh, the issue of User Interfaces always makes me laugh at the incompetence of seemingly the entire world when it comes to User Interfaces (or the whole computing world in general).

Some obvious trivial faults:

1. The whole overlapping window model is bankrupt. You want to minimize the amount of information, especially redundant information, that the user has to input, and output as much information in an accessible way. The overlapping window model does the opposite: it requires that you tile your windows manually (or through tedious, inaccessible menus) rather than specifying which windows you want to see. If you don't do that (and due to the required effort, most don't) then you don't see all of the information you want even though most of the screen is wasted space!.
   For reference, just look at your screen now, and watch how much of it is covered by empty "gray areas". When you open a new window, does it hide gray areas, or real information?
   This is even more absurd when there are just a couple of windows, hiding each other, when the entire screen is free space! The computer expects YOU to work for HIM and move these windows from hiding each other.
   This phonemenon is also felt in list boxes, where you are expected to adjust the column widths manually to not be too short/too long, even when there is an optimal adjustment readily available. You again have to work for the computer, and ask for a ctrl+plus to set it up. Most people don't even know about ctrl+plus in column-listboxes.
   Some programs make it even worse, and don't let you resize their windows when the entire screen is free, and you have to scroll through their data in a little window.
   
2. Internationalization and shortcut keys.
   What's so fascinating about this example - is how common it is across platforms, programs, operating systems.
   The feature is called "shortcut keys" and yet everyone is implementing it as "shortcut symbols".
   This is terrible - when you switch between languages, all shortcut keys break!
   
3. Multitude of widgets, with overlapping functionalities. This is just silly and confusing to beginners. We need less widgets, not more.
   
4. "Jumpyness". Today's GUI's all "jump". What I mean by that is that they don't smoothly switch from one state to the next, but rather do that with a single screen refresh. The human mind doesn't read that very well. For example, scrolling down "jumps" down a pageful instead of scrolling down a pageful in a smooth motion.
   The fact that fixing this would require modifications of all existing GUI programs is a certificate of poor architecture of GUI software.
   

There are many more trivial issues to fix. Until they fix these, I find it very funny to talk about future directions for the User Interface. We haven't even gotten the basics right yet!

Re:Overlapping windows

[CaptainCarrot]

I disagree.

1. Overlapping windows are used to make more information available to the user than can be displayed on the available screen real estate. The RL metaphor is a collection of papers on a desk. You can't see every paper all at once, but you bring to the top of the pile those which you need. You do this for your own benefit, based on the needs of the moment, not for that of the desk -- or the computer. The whole point is that the space isn't tiled. I don't like working that way personally, and I suspect the reason we've moved away from that model is because most people don't. Remember the early Windows versions?

   You asked how much of the screen was empty space and therefore wasted? Very little of it, most likely. Very little of mine is as I type. Space with no content in it is not necessarily wasted. In fact, it most likely isn't. Space is crucial to how our brains orgainize what we see. If every square inch of space on the screen was being used, we'd see it as a jumbled mess. The best and most eye-pleasing data presentation use of designs very carefully balance empty space against that occupied by content. Take, for example, your original post against my reply. See how I create spaces between my paragraphs with properly structured P tags? See how much more readable that is?

   I agree that some programs are badly designed and make poor use of the model. That doesn't mean the model itself is broken.

   Yes, it would be nice for those very particular about their screen arrangements if they could save state between sessions and recover it immediately when they start back up again. This is an implementation issue 00 remembering, of course, that most people prefer not to tile.

2. Yes they're shortcut symbols really, but people have a hard time remembering arbitrary symbols. That's why we employ mnemonics, which naturally relate to the language of the interface. For example, it's easy to remember the shortcut to open a file in most word processors (ctrl-O) because "O" is the first letter in the word "open". It's not reasonable to expect such mnemonics, input through an alphanumeric keyboard, to work any other way -- unless you can think of a better one where alphanumeric input is both easy to remember and language-independent. Good luck.
3. This is not an inherent fault in the model, but is a failure across an industry to standardize. In my own GUI design work in Motif, this is why I use the default behavior of the default widget set as much as possible. The users most often know exactly what to expect then.
4. I remember when some word processors and the like included a "smooth scrolling" option. No one used it. It turned out that most people wanted the screen to scroll quickly instead.

The Future is easy to predict here.

[Anonymous Coward]

In the long term, we'll be communicating with computers the same way we communicate with our pets, kids, and coworkers - with a combination of body language, voice, gestures, etc.

In the short term, we'll see Longhorn slowly and sloppily copy whatever Apple's doing; and we'll see KDE and Gnome both copying the bad parts of what the Gnome and KDE are doing respectively; and we'll see all real computer users using emacs/vi/pine/xterm/screen like they always did.

Nobody's paying attention

[quokkapox]

At least not to common consumer devices. I cannot even count the number of remote controls, microwaves, cellphones, dishwashers, ATMs, and other devices which are seem to be designed completely without thought for the human who will need to use them.

Remote controls - ever heard of making the buttons distinguishable by FEEL, so I don't have to look down to tell whether I'm going to change the volume or accidentally change the channel or stop recording?

Microwaves - make the buttons we use all the time bigger and obvious. I can't use my microwave oven in near dark because the stupid thing's start button is indistinguishable from the power level button. That's just dumb. I don't need two different buttons that say "Fresh vegetable" and "Frozen vegetable" which I never use; and I have to babysit the popcorn anyway, so I don't need a "popcorn" button hardcoded for some random time limit. A microwave should have a keypad for entering time and bigger buttons labeled +1minute, +10seconds, ON, and OFF. That's all 99% of people use anyway.

The people who design interfaces should be made to use them for long enough so that they work out at least the most obvious design flaws.

I keep putting off buying a new cellphone because I know I will have to learn a new interface even to set the freaking alarm clock and it will probably take six menu choices to do it.

Re:Nobody's paying attention

[evilviper]

I can't use my microwave oven in near dark because the stupid thing's start button is indistinguishable from the power level button.

Better question: WHY THE HELL ARE MICROWAVES DIGITAL? What part of "close the door and turn the dial" was so hard for people to understand, and how did typing in digits help? Microwaves aren't phones.

Was it the extra precision? People need to be sure they are microwaving their sandwich for exactly 2 minutes and 45 seconds, and ABSOLUTELY NOT 2 minutes and 46 seconds?

Are there a lot of people out there with only one finger, who find it faster and easier to type in 1-0-0-0-Start rather than turning the dial a quarter turn to "10m"?

What in the world makes people believe replacing analog with digital is the answer to absolutely everything?

apply it to a calculator...

[3seas]

take these fancy UIs and use them to control a calculator and then decide if it right for the job.

"Right for the Job" is the key phrase.

There are three primary UIs:

the command line (CLI)

the Graphical User Interface (GUI)

and the side door port used to tie functionality together. known by many different names, but in essence an Inter Process Communication Port (IPC)

Together they are like the primary colors of light or paint, take away one and you greatly limit what the user can do for themselves,

But if they are standardized with the recognition of abstraction physics (in essence what a computer impliments) then the user would be able to create specifically what they need for the job they do via understanding and applying abstraction physics. The analogy would be mathmatics and the hindu-arabic decimal system in comparison to the more limited roman numeral system.

There are all sorts of user interfaces that can be created but they all are made up of some combination of the primary three, perhaps lower down on the abstraction ladder but none the less there.

The reason why this is unavoidable is simple due to the nature of programming.

Programming is the act of automating some complexity, typically made up of earlier created automations (machine language - 0's and 1's is first level abstraction - all above it is an automation). The purpose of automating some complexity is tocreate an easier to use and reuse interface for that complexity. And we all build upon what those before us have created. Its a human unique characteristic that make its our natural right and duty to apply.

What the failure of so called computer science is guilty of is distraction by the money carrot, starting with IBM and wartime code cracking paid for by government/tax payers.

This distraction has avoided genuine computer science, or abstraction physics as it would be far more accurate in description.

Abstraction physics to the creation and manipulation of abstractions as mathmatics is a creation and manipulation of numbers, as physics and chemistry is a creation and manipulation of elements existing in physical reality.

With the primary three colors of paint you can paint anything you want, but you cannot call a painting "the painting" any more than you can call a mathmatical result mathmatics. Nor can you call some interface built upon the primary UIs the silver bullet of UI's.

All this will become much more clear, common and even second nature once we all get past the foolish fraudlent idea that software is patentable.

A roman numeral accountant, in defending his vested interest in math with roman numerals, promoted that only a fool would think nothing could have value (re: the zero place holder in the hindu arabic decimal system.)

Re:Multi-touch

[Eideewt]

Agreement here too.

My biggest gripe with today's computer interfaces is that attempting to funnel everything you might want to do through a mouse plus (if you're lucky) a keyboard forces you (as an interface designer) to make a difficult decision: either waste huge amounts of screen real-estate on functions you need to include, or hide them away.

What we need are interface devices that aren't so bandwidth limited. When we want to make the computer perform an action, all we are generally able to do is locate it on the screen and say "Do." On systems with multi-button mice the situation is somewhat better. Most Firefox users are familiar with "left click to follow a link, middle click to open it in another tab, and right click to get an [ick] context menu" idiom. Scroll wheels are another instance of a bandwidth-increasing addition to the system. Rather than clicking an arrow to scroll, we are now able to spin a wheel while pointing in the general area of the thing we wedant to scroll.

Some systems put the physical controls available to even better use. The Sam text editor, its successor Acme, and basically all of the Plan 9 operating utilize the mouse buttons to perform distinct and consistent actions. In Plan 9, button 1 selects, and the other two buttons, when used in conjunction with it, perform other useful actions. The exciting feature of this setup is that it moves the selection of possible actions out of the computer, where navigation is inefficient, and puts it literally under your fingertips. Rather than selecting an object on the screen then selecting an action, or vice versa, one can simply point at it and say "do this." The ability to convey specific actions in one fell swoop is what makes command line junkies (myself included) swing the way they do. What could be more exciting than marrying that power with a GUIs flexible expression?

An even more extreme example is the five button keyboard (for the left hand) + three button mouse featured in the Doug Engelbart video linked from the summary. I'm not sure how his system used them, but this setup allows for eight functions using the most obvious mapping, many more than modern interfaces. Not only that, but with chording, it's possible to increase the number of possible actions to a dizzying 255, which is probably way too many to actually make use of*: Engelbart's system uses typed commands as well as clicks rather than attempting to assign a meaning to each combination of button presses. One good way to cope with the number of possibilities is to assign a general funtion to each button, and to combine those functions to perform actions. For example, if the left mouse button selects text, the middle button pastes it, and the right mouse button cuts it, one can copy by selecting with LMB, then pressing RMB, MMB in succession. Other button actions might be "system" to trigger global system functions, "window" to do window management and "inspect" to look more closely at an item. What might happen when you press these together? Exposé, anyone? But without reaching for the keyboard.

Anyone interested in user interfaces should take a look at the Sketchpad computer program for starters, which was simply amazing, and at "Alan Kay: Graphical User Interfaces" on Google Video. GUIs have a rich history that is not evident in modern interfaces.

* This is a good thing! Even when a the system allocates a set of global button presses and applications implement their set of commands, there will still by plenty left over for the rest of us to allocate as we see fit. This is the one point where I disagree with Bunions: I love that multimedia keyboards aren't standardized, because it means that programs don't depend on the buttons they provide, which in turn gives me 32 keys (on the keyboard I have) that I can bind to any action I want, without losing any functionality.