Slashdot Mirror
Fold 'n' Drop Window Interaction
Mints writes "Following up on recent "Desktop Innovation" stories that have left some disappointed, I thought Pierre Dragicevic's exploration of Fold 'n' Drop warranted mention. Described as "a new interaction technique for seamlessly dragging and dropping between overlapping windows", Fold 'n' Drop allows the user to interact with layered or overlapping windows in a very intuitive manner. Refreshingly, Mr. Dragicevic provides both a sample implementation, in Java, and video demos. Mr. Dragicevic is a researcher in Human-Computer Interaction at Intuilab, Toulouse."
In my experience, few things can improve on keyboard shortcuts for navigating between windows depending on the amount of windows open. Reaching for the mouse just adds more time.
So THAT'S what the backs of windows look like.
Kind of neat. My only comments thus far is that if you "discard" a window (fold it all the way over so that it dissappears off the screen) there's no easy way to get it back without dropping the object your dragging first. Similarly, it's too easy to folder over too many windows, by accident.
-dave
http://millionnumbers.com/ - own the number of your dreams
Huh? I was expecting an article on laptops.
"Kittens give Morbo gas!"
If you ask me, it'd be much easier to use Ctrl+C and then navigate where you want to go and use Ctrl+V. It's difficult to hold down the mouse button while violently jerking the mouse back and forth to get to the right window.
Don't get me wrong, it looks really neat, but it's not terribly useful. Sounds like the kind of thing that would fit GREAT in Longhorn.
http://www.mirrordot.org/stories/bf5e7607538684c45 385314562c610b0/index.html
On Mac OS X, we can do this with Exposé. Start a drag, move the mouse to a hot corner, drag over the formerly-obscured window...
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Tried the Java demo. It's a neat idea. It takes a minute or to to get used to it, but then it starts to feel as natural as clearing off your desk with the back of your hand when you and the secretary need someplace to ... well, put something down.
Cantankerous old coot since 1957.
Thank you. I will be announcing my API Real Soon Now.
I think Apple's existing implementation of Expose is quite powerful. Not everyone realizes that drag-and-drop works with it, and more unfortunately Apple does not default to using a "screen corner" to activate Expose (yet this, too, is possible).
I have it set up so that I can literally "yank" the mouse in the general direction of the lower-right corner to show all windows, perhaps after picking up a file with the mouse. This then allows me to drag the file to any window. Further, I can use spacebar (like in spring-loaded folders in the Finder) to immediately choose a window instead of pausing for a second to have it selected automatically.
This action is so natural and powerful, I use it all the time. And though I use Linux at work and it is fantastic in many ways, I sorely miss features like Expose in Mac OS X.
"Microsoft killed my company, I hold a personal grudge. I don't use Microsoft products and neither should you."-JWZ
On windows, we can do this with the start bar.
Start a drag, move the mouse down to the title of the window on the start bar, drag over the formerly-obscured window.
It's always nice to see new ways to interact, but I can't recall a single time this would have been useful in the past week. My memory can't recall much more than that, but the folding corners would certainly annoy me more often than it would actually be useful.
I found the demo to be a little clunky, but he did have to implement it himself.
Once I figured out how it was worked, I found myself wondering how useful it would be to be able to just fold back the corners of a window when I wasn't dragging a file.
The general idea of peeling back the corner of a window seems like it might be actually useful at times. Sometimes the rigidly rectangular window can get in the way.
Of course, I'm sure it would eat CPU like all graphical candy, but cycles are cheap I guess.
Lost at C:>. Found at C.
Combine this with some kind of hand gesture sensors (yes a mouse is one but I mean more complex) a la Minority Report and you would have a very intuitive "virtual paper" interface. This looks like it will be very handy (no pun intended) for use with a mouse but I think using more complicated hand gestures (in the future, when possible) will really make this paper-like manipulation of windows even more intuitive and useful. Funny how the further we move away from paperwork, the closer we get to being just like it.
Seriously, if you still have both hands, you don't need this. At least not under Windows. While dragging, you can still press Alt+Tab or Win+D (Desktop), so you should be able to get everywhere you want to.
I don't suffer from insanity, I enjoy every minute of it.
Unsurprisingly, Microsoft's way is the worst:
:Grumbles: Friggin advanced users think they can just add dimensions wherever they damn well please
[Fuck Beta]
o0t!
... I find a sharp blow to the Solar Plexus will also produce a satisfying fold and drop.
Don't go to a brothel if you want to buy broth
For navigating between already-open windows, when you have more than, say, 4 apps open, I've not seen any keyboard-only technique (or any other technique) that comes anywhere close to the speed of navgation via Expose on OSX (which uses mouse-only or mouse+keystroke) (Expose = mouseclick or keystroke reveals all open windows and lets you choose between them quickly) if you know of a faster keyboard technique, please do tell.. and I'll use it. (but.. before osx i was w/ you.)
So when was the last time you used a keyboard shortcut to drag and drop something?
This isn't merely for switching between windows. If you can't RTFArticle, then RTFSummary at least.
Looking at the video, I'm struck again by how ill-fitting the "window" metaphor is. "Leafing through windows"?? Come on -- the model here is the page or the sheet.
Maybe it was never very appropriate. Windows and icons and menus... on desktops?? Oh my!
Windows's taskbar already lets me drag and drop amongst arbitrary windows. You just drag the stuff to the task's button on the taskbar, wait till it goes foreground, then you drop it wherever you want.
I usually have more than 10 windows open, I don't want to waste time peeling through them one by one, especially when I know exactly which window it is (I just recently clicked its task button after all).
Once I have a taskbar, I don't often have to remember which windows are "below" or "above" each other. I just need to remember which task button represents the window to get to it.
Which comes to a related point - KDE orders the tasks on the taskbar top to bottom, left to right. This means that if you remove a task, the ALL of the tasks to the right of it will change their vertical positions. This is bad UI IMO. However the person in charge prefers it the way it is[1].
Windows does it left to right first then top to bottom. This means that only leftmost and rightmost tasks change positions if you remove one, so it's not as much of a mess trying to remember where a window is.
[1] Nope he doesn't go check with the "people in charge of Usability", because there aren't any. Which probably explains why Linux still has a mediocre GUI in terms of usability.
jesus, you zealots can't wait to shit on anything that doesn't come from the mecca in cupertino. I've read more bitter posts from apple fanboys in this story that I can credit. if it's on a computer and it didn't come from stevie's ass it's just no good, right? fuckin jerks.
for the java demo:
http://kafene.org/foldndrop.jar
That should give you an idea about the functionality. That's all I snagged before it got slashdotted.
Exposé actually shows the windows, not small previews. It organizes and resizes them on the screen temporarily so you can see all the windows and their content.
No existe.
I honestly love exposé, sometimes you just lose track of a window and can't find it any more and I don't want to have to put everything in the dock and then open it up one by one. I'm still wondering what Leopard will have in store. I honestly can't wait.
- Qua
a few minutes ago...
Ctrl X/C
Ctrl (Shift) Tab
Ctrl V
For me, it's actually faster (but I'm also ambidextrous, so I am equally efficient with my left hand), plus you have more control over exactly what you want in windows. When you click and drag something, in windows it will move it if it's on the same partition, or copy it if it is across partitions. Using the control keys, you don't have to second guess what partition the folder you are dragging it to is on. KDE (IIRC) will ask by default when you click and drag, if you want to copy or paste (via a pop up menu). In windows, this is possible, too, but you must click and drag with the right button. Plus, in windows, if you have a bunch of software installed, you will have to scroll past a million items in the context menu before getting to copy/cut/paste options.
You are confusing copy/cut and paste with drag and drop, same as the other person who replied. Just because you can cut and paste some things with drag and drop, doesn't mean drag and drop is the same thing as cut and paste. What about dragging something onto a printer or application icon, for example?
The Ion window manager for X is quite fast for keyboard only navagation. It tiles and tabs window. I've never used Expose so I can't say for certain if Ion is faster, but I usually have around 10 apps open and I have no problem getting to the one I want quickly with the keyboard.
Win a signed Stephen Carpenter ESP Guitar from the Deftones: http://def-tag.com/?r=0008781
Step two: select a file in window "A" and drag over window "B" (which is overlapped and beneath window "A")
Step three: Wait half a second for window "B" become the modal window and release.
What happens if the target window is completely obscured by the front window? If there is no overlapping edge for you to move towards and wait for focus to be given to the underlying windows?
From what I've seen, Mac OS really is the best with regard to user interaction tricks. It's the smoothest and best interface around. However, this new technique seems to have some advantages in terms of smoothness and it is intuitive. Clicking on a keyboard button may accomplish the same thing in the current Mac OS, but then again in Windows you can drag down to the taskbar and wait for that window to gain focus. It's just not as elegant as what's being proposed here. I, for one, think this sounds cool! You can push away the front window(s) and see what was previously obscured.
(Then again, I have not used Mac OS X that much and maybe what they already have is better than what is being proposed here... but still I think it's a neat idea worthy of consideration for any GUI.)
This looks cool and was rather fun to play with in the Java demo, but just how useful is it really?
How many people actually move/copy files in this manner anyway? I suppose Jo(e) Average may but surely anyone who uses a computer at all regularly would copy and paste - I've even seen people copy/cut and paste using menus more than I have seen them drag and drop between open windows.
Neat trick, but... next!
In stories like this you can see how much the Slashdot userbase has changed over the years.
Here's a new UI concept, that is very promising and hasn't been implemented anywhere yet. A true opportunity for Linux to score a "first" in UI design -- this could be the next generation of window shading/rollup, the possibilities are endless.
And the comments are "in Mac OSX you do such-and-such instead", "in Windows you do such-and-such instead". Things like "this problem is solved" -- as if there was One True Solution in UI design! -- and "before doing your research you should stop at the Apple store" -- as if PhD research didn't do related work assessment! --, enumerations of Windows key sequences, and so on. And those are ranked "5, Insightful".
A few years ago the comments would range from the usual "GUI? Give me a CLI any day" to discussions on how to implement this on Linux and which wm would get it first, which would (d?)evolve to a healthy wm flamefest.
The Slashdot audience truly has changed. *sigh*
The filesystem is the package manager
This technology seems like the perfect killer app that would require Force Feedback. Imagine for a second... the more windows you leaf back, the heavier they become. You could blindly lift off a few windows...
jm2c
.: 2+2 = PI SQRT(1+N)
Win a signed Stephen Carpenter ESP Guitar from the Deftones: http://def-tag.com/?r=0008781
You're right - sort of. It's not that the mouse or the keyboard are burdening you - it's when an application forces you to use *both* of them in combination that your efficiency really starts to suffer.
Some programs are primary mouse-driven (eg. 3D modeler). Others rely primarily on keyboard input (eg. word processor). I would argue that an interface strives to be most efficient when it maximizes the functionality of its primary input device.
For a 3D modeler, this means an intuitive 3D interface for rotations, use of all mouse buttons, and carefully positioned clickable buttons and toolbars.
For a text-editor/word-processor, this means loads of keyboard shortcuts. emacs/vi come to mind (MS Word falls short in my mind - I *need* ctrl-a, ctrl-e, ctrl-k).
What become challenging for the developer (and, consequentially, the user) are applications that are sometimes mostly mouse-driven, sometimes mostly keyboard-driven. These apps necessarily force the user to switch between the mouse and keyboard frequently. Often, there is some gain to be had via keyboard shortcuts, but the interface does not always allow these to be most efficient.
A web browser, for example, or a file browser require both a mouse and keyboard (OK, so many of us get by just fine with the terminal, but let's ignore that). In the case of the web browser, many folks are drawn to opera and firefox specifically because of their extensive keyboard shortcuts and mouse gestures, allowing you to stick to whatever input device you were already using.
So, for example, in opera, i can gesture left if I'm using the mouse, or I can hit 'z' if I'm using the keyboard. It's *never* more efficient to reach for one or the other, but ultimately, I'll always need both (I didn't type in this post using cut and paste, and I certainly didn't click on the link to your post by tabbing through everything).
The *real* problem is that your window manager does not allow efficient navigation via the mouse. If, for example, you had mouse gestures, hot corners, or OS X expose functionality bound to extraneous buttons on your 5 button mouse, you would reach less often for the keyboard when you were already using the mouse.
You're right - reaching for the mouse just adds more time, but so does reaching for the keyboard. A window manager is best when it forces you to do neither.
Not that I post on slashdot or anything.
You want to move text to a different application? Select it, and then drag it through Exposé to any window.
If you drag it to the desktop, it creates a file called whatever.textClipping. The file can be moved around or stored. It's a normal file. If you drag the file to an application, the full text of the file is pasted into your window. If you double-click the file, you can read it just like a normal file. I have three of these files on my desktop right now.
Want to add an image to your document? Drag it to the window. Want to add an attachment to an email? Drag it to the window. It's much simpler.
Albuquerque PC
So I take it that you're always delighted when Clippy offers to format your document as a letter?
For that matter, are you delighted when your mother-in-law suggests that it would be quicker to make a left at the next intersection?
AI will be good enough to lay out my windows how I want them to work optimally about as soon as AI can actually do the work for me. Neural nets? You must be kidding, right? Predicting what a human is trying to do is hard enough for other humans, let alone computers.
That's not to say I don't agree that the metaphor is pretty ridiculous. Hey, it looks really cool. Everyone else in the office was looking over my shoulder. But would I actually want it in my window manager? Not likely.
This doesn't seem too intuitive. Can you imagine trying to explain this concept to your mother? Look at the Discarding Windows part of the video, where does the window go? Bring it back! It's like the drag-lock on the trackpads, inexperienced people hate it because it gets in their way, interrupting their work.
Does look pretty cool though.
People will start making origami windows
If Practice Makes Perfect, And No One is Perfect, Why Practice?
Well, at least half of the comments I've read have given 'Fold n Drop' that kind of review: Who would use this if they can just use Control-C And Control-V?
Uh... Excuse me... but have you ever told an end user that you can copy and paste a file? I have... and they're often dumbfounded.
End users, and by that I mean the 'those who need all the help they can get' loser end users, will eat this up. Why? Because the first time they try to drag a file from one window to another, they'll see the window fold... and either freak out, or understand what it's for. No instructions. With 30 seconds of experimentation, they'll understand. It's intuitive. Remember when 'intuitive' was the goal of every GUI? Yeah, me too...
So, we have the windows users: "Drag to the taskbar, bring up the window you need!" Our 'loser' user doesn't know this one.
How about the Mac User: "Expose does this already! And Springloaded Folders too!" Yeah, it does... Springloaded folders are in the right direction, and so is Expose. But this is better and eaiser than Expose, and on par to Springloaded Folders.
Then you have your traditional Unix user: "Focus Follows Mouse! DUH!" Or even better yet: "Mouse?! Whatever happened to useing the keybaord? If you have to use a GUI, use Ratpoison or Ion!" These guys give me the giggles... First off, I'll say yes... keyboards are much more efficient. Give anybody an on screen keyboard to type an essay, and they'll agree. But focus following the mouse is more anoying than useful to most all of our 'loser' enduers. As for the mouseless window managers... You forget that the target audience doesn't like the keyboard for anything save typing. If more than one button needs to be pushed at the same time, the user will tend not to understand. The only major exception would be Ctrl-Alt-Del, which has Windows users on Macs very frustrated from time to time.
Then you have these guys: "What a waste of my CPU and GPU! Can't we get rid of the eyecandy?!" Quit your whining. Today, our 'loser' end user can purchase a system from Dell with 2.5 thousand times the processing power of the computer that the Apollo astronauts used to get to the moon, And our user will be doing things much less important. As for the GPU, I don't belive the Java script is 3D accellerated, and it seemed to run just fine... IN JAVA mind you... This in C++ or C would run much faster still. If a few wasted cycles make the system easier to use and understand, I'd call those cycles well used indeed.
So quit your moaning. Give this guy the credit which is due: It's a very facinating idea, and I hope to see it implemented in all the OS's as an option. From watching the Video, then trying the Java... I'm hoping to get it an option on all my computers, be them Windows, Mac or Linux.
-Pathway
http://www.apple.com/macosx/theater/expose.html link to what everyone seems to be rabbiting on about.
I'm all for the improvement and innovation of interface design. Making a design intuitive increases productivity, even among power users who may do a particular action only once among a few thousand, and may not have it as ingrained as expected. That said, there are some major drawbacks to this design that may serve to make it less productive and even less intuitive.
Counter-Intuitive Metaphors
Metaphorical abstractions for computer objects only work if they have a clear representation of being similar to object they represent. While some windows (text boxes, for example) have a clear similarity to being a leaf of paper, many others do not; directory windows seem fairly unrelated to 2d objects: they contain multiple objects inside of them, likening them more to being a box or drawer, some 3d abstraction. Thus it is not only counter-intuitive to "fold-over" an object which has depth, but also brutally forcing a metaphor onto an object of which could suggest a completely alien mental abstraction from the one a user original envisioned. For this reason, almost all interface references to real-world objects are either extremely obvious or very broad in definition. The "focus" metaphor works, for instance, because you can bring any object (one with depth or no depth) and put it on top of another object, thus bringing it into "focus" or plain-sight; it is an extremely simple and all-encompassing concept.
Temporality and Spacial Complexity
The second problem with this method is its inherent temporality. Most GUI operation requires no timing, and in the rare cases that timing is required (ie: double-clicking, hovering over spring loaded folder), the operation is extremely simple and requires no precision. The one exception is double-clicking, and you can witness its result by watching any surface user fail to open a folder because they can't keep the mouse still while clicking the left mouse button. The folding operation illustrated here, on the other hand, is an extremely complex operation that takes some very precise timing. Even I, an experienced computer user (as we all are), had to practice it many times to double-back on my mouse movement fast enough to correctly "fold-over" a window. Since windows move and change in organization, the operation is slightly different each time it is performed. I can already tell that even if it the operation becomes somewhat natural, I'll always continue to miss on occasion because of it's complexity. And if I'm having trouble with it, I can't imagine what it would be like for my parents!
UPDATE: I had my mother test it out to see if a surface user could cope with it, and after struggling with it for a few minutes, finaly gave up.
Accidents and Set-backs
The third problem I for-see is that folding can easily occur unintentionally and is difficult to undo. Spring loaded folders and "snap-to" focusing work well because their actions inherently require a very specific action: going over a folder and waiting for about a half a second for the window to pop up. Since the cursor is going to be generally moving while dragging objects, a half-second wait over a folder or partly obscured window is abnormal and requires intentionality. Even then, it is as easily (if not more easily) reversed as it done by simply moving off the newly focused window. With folding, on the other hand, it's easy to see how any quick movement during a drag could activate the effect, and when the process of folding is started, it takes an even more complex spacial action to set it back, that being the looping around and back onto the fold from the other side.
Just a few thoughts on intuitive interface design, using this as an example of what works and what doesn't.
Multiplayer Gaming (defined): Sitting around, discussing single-player games with my friends, at the bar.
The first post still wants to use his keyboard.
../ Of course the latter is faster if that's all you're doing.
Personally, I wish the computer cognoscenti *would* give more emphasis to truly graphical computing.
The fact that the keyboard is more efficient for interacting with the majority of computer operations that people do really just goes to show that our culture hasn't advanced from thinking in pipelineable data chunks to true objects.
For much software, config files, switches, and option params still dominate over graphical dialogs, and even those that do exist in polished software are still just checkbox and radio equivalents of config settings, not real objects in the sense of "chopsticks interacting with noodles" (associating entities with containers).
Even most GUIs are simply visual equivalents of the same verb-noun operations that CLIs have always used, eg, graphically foo.txt dragging up a level is the same as mv foo.txt
I think the future is somewhere in the way non-linear video editing suites and graphical art programs work, but more consistent.
Hopefully now that OSes are moving to 3rd gen windowing architectures that allow much more complex visual depictions (OS X a few years ago, Longhorn next year, Linux real soon), more experiments like this will be tried, and new interactions will emerge.
Although this post has made no sense, here's to truly graphical computing!
Yes, ten Word icons followed by "C:\Docu..." is a REALLY great way to drag stuff from window to window within an application (or multiple apps)!
The problem with the Task Bar is that it relies upon the window title, which is overloaded with the application and document name typically, and provides no clue as to the actual content of that window. Expose provides a visual thumbnail approach for finding the "right" window, and allows for "tooltip"-like document names to pop up if you hover the mouse.
Well, once you get used to them, they are neat stuff, though true they are odd at first. I'm a heavy mouse gesture user, I can't comfortably use a webbrowser without gestures. A friend commented on me: "You know, it looks funny. You don't point at anything and things change, you click in completely random places and move the cursor around in some chaotic manner, then things happen, or not, I know you're using mouse gestures but just watching you webbrowsing freaks me out."
Anagram("United States of America") == "Dine out, taste a Mac, fries"
If you install the windows XP powertoy, which improves the alt-tab switching you get a preview of each window before selecting it!
.
Being user friendly is great for things you're only likely to do once, but if its something you'll do houndreds of times a day, even saving 10% of the time adds up easily, Which is why I use vim. "zc" to close a fold may not make much sense to someone that doesnt already know it, but once you map that in your mind you save a lot of time.
Pain lasts, kid. Its how you know you're alive. Sometimes I think this growing up thing is just pain management-TheMaxx
"You cannot drop an item onto a button on the taskbar. However, if you drag the item over a button without releasing the mouse button, the window will open after a moment, allowing you to drop the item inside the window."
Took me about 5 minutes between sitting in front of Windows (95, mind you) for the first time and getting this message. If it happens so often that you feel the need to design a message box around this user behaviour, well gee maybe users really do want to be able to drop items onto taskbar buttons and you should just let them? Morons.
(Yes, I'm aware that you said hovering until it becomes the active window. It's still a stupid interface design.)
Switch back to Slashdot's D1 system.
It DOES save you time. That demo absolutely rocks. I've been in situations where I've had a ton of windows overlapping each other and I want to move an object to something that's on the bottom layer. So, now we have a choice between clicking a bunch of times to minimize the first few windows, resizing another one and then finally moving the object into the correct position dropping it, then restoring the other windows back to their original position. OR click on the object and fold a few windows back (which is totally cool by the way) and dropping it. Much better! THIS should be a standard feature in feature window managers.
My understanding is that the objective is to move a file from one folder to another, not to drag and drop. That is a means to do it. Cut and paste is another. Why drag to a printer (I've never even heard of this one) when I can hit ctl-p? Lets not get so obsessed with drag and drop that we lose sight of why its there, the objective. Besides, its just a metaphor, you're really dragging anything.
Really new users find drag an drop about as intuitive (or less) as mv dir1/file1.txt dir2
Which to use depends on the person using the computer. Options are good.
Personally, I'll take keyboard use, its an order of magnitude faster, *for me*. I can do a keyboard shortcut in less time than it takes to remove my hands from thekeyboard to grab the mouse, let alone move and manipulate the mouse. If I'm just lounging out, keyboard out of reach, surfing the web, then I'll use the mouse because its more convienent.
----- If communism is a system where the government owns business, what do you call a system where business owns govern
Your mother (and mine, by the way) isn't alone. There's a zillion people out there who can't figure this out. I know a Harvard professor who double-clicks on everything. I know a high-tech Marketing professional who spastically double-clicks on stuff, then double-clicks again when the app is slow, just in case it missed her first double-click.
It's hard to explain. "Double-click invokes programs." Invokes? What the hell does that mean to the average person?
Paradoxically, single-click on a link causes a new web page to be displayed. That sure looks like something got "invoked", doesn't it?
How about all those apps that do something unexpected when you select an entry out of a combo box? Isn't the metaphor supposed to be SELECT, followed by PUSH A BUTTON?
So it's not all that easy to explain when you're supposed to single-click and when you're supposed to double-click. And, the fact that Windows misses some of the double-clicks you do because of the stupid-ass way double-click is implemented at the message level doesn't help, either.
Oh, and let's not forget the fact that Windows drops internal messages occasionally anyway (about 1 in 100,000 the last time I measured it, in case anyone cares), so remember that the next time you decide to create your own message class and "trust" Windows to deliver your messages.
It's funny how you can always tell the OS a person uses by their mouse-prefs.
Linux: the mouse is only good for click, drag, and select/copy. Users believe the mouse is a useless add-on. On Linux, I agree.
Windows: good for getting those right-click menus. Also the only way to do things that don't have obvious keyboard shortcuts - preference dialogs, toolbar buttons, etc.
Mac: Drag and drop everywhere. Bind the middle button to Expose. Eventually you just keep your hands in the Quake position: left hand on the kb, right on the mouse. You know, a GUI.
Care about electronic freedom? Consider donating to the EFF!
I think the original idea was that you could drop the file onto different bits of the window, but I'd still rather just drop the file onto the taskbar and have it do the most logical activity.
How many people can read hex if only you and dead people can read hex?