Slashdot Mirror
Gosling: If I Designed a Window System Today...
An anonymous reader writes "In his blog entry for the 10th August, James Gosling (finally) publishes a short paper he wrote in 2002 entitled 'Window System Design: If I had to do it over again in 2002'. His design is to make the window system do the absolute minimum and move all the work into the client."
So. Who's with me to create tihs sourceforge project? Dead serious folks, not a troll. BUt who has the gumption to get it started and make it run VERY fast, then after a while see how the X.org people would think of merging or using it? Eh eh?
let me know, use my gpg key to encrypt messages (it's the wave of the future!).
--zoloto
I can't count how many times I hear on /. someone saying that X is too bulky, etc, etc. And here's Gosling saying (2 years ago) that X is headed in the direction of slim and lightweight.
Am I misreading what he's saying?
Key to financial independence: Spend less than you earn. Save and invest the difference. Do it for a long time.
First, it says he wrote this article 2 years ago -- talk about a slow news day for /.
Second, isn't this more or less what X11 is doing today anyways? Yes, the xserver is getting some new extensions added to it to handle things like compositing or polygonal shapes efficiently, but a lot of the cool new features we're seeing in applications come from userspace libraries on the client side of the X11 protocol. Cool little hacks like shadows or translucent windows are just the result of rendering a window off screen and using some compositing rules on the server for generating the final image.
Cairo looks like it will make a nice DPS-like library to use for graphics (it really looks like Postscript minus any programming language features which presumably get replaced by your own choice of language) and that hits Gosling's "postscript-like" display feature.
Is this just an attempt by him to say "told you so" or something?
On the other hand, windowing systems progress slowly and that's why linux forces are pushing through client side. They mature very fast, and can do many tricks but not all (see transparency).
Still, I don't see the harm of a well designed, capable windowing system, which is at least easily extendable.
But someone mentioned on Slashdot actually implemented what sounded to me like a pretty good idea. They used the structure of XML to transfer data between client and server.
It makes sense, if you think about how graphical elements are grouped and have properties...
tasks(723) drafts(105) languages(484) examples(29106)
20 years ago it might have made sense to make this very modular since nobody knew how things would end up looking. Today, let's face it, windowing is "done." All the various libraries over X look and work very similarly, just different enough to clash. Windowing is mature, I say it's time for more integration.
Modularity should be at the level of source code, not runtime components.
As Gosling mentions, X is moving in this direction today. In a year or two, when the newest X changes are stable, the average GTK+ or Qt app will talk to the server via OpenGL. On most DRI-like setups, the route from GL to GPU looks like:
OpenGL -> userspace command buffer -> graphics memory (DMA via Direct Rendering Manager).
Text layout, fonts, etc, are all done server-side, and the only thing the "server" sees are pixmaps and GL commands.
A deep unwavering belief is a sure sign you're missing something...
Maybe its because Gosling is coming from X11 land and its sucky drag n drop/clipboard implementations but this is seriously a big deal is a Windowed operating system. In a Windowed Operating System, it should be easy to move data from one application to another---even though they are made by different companies. And not just text either---things like pictures as well. Going beyond this, dynamic linking and embedding is a handy feature as well.
Does this mean that Rasterman and the Enlightenment crew are going to have to re-impliment Enlightenment DR17? Maybe next year will be the year of E... but no matter, doing the job correctly is more important than rushing a release. Still I wish they could get sponsored or something to speed up their developement. How long has it been? *sigh* and no, I'm not even a [great] programmer, that's why I have to make such a silly comment to begin with...
Seriously, all we are talking about is modularizing the windowing system. If the WS is as simple as possible, people are going to rely on libraries and windowing toolkits to get their work done. I guess that's already happened with GTK, etc.
Computers are useless. They can only give you answers.
-- Pablo Picasso
For my fellow Amigaites out there:
*sniff* That brings back memories. Sadly, my Amiga RKMs now support my monitor, but oh... this is so familiar. :-)
For the rest: the Amiga had a graphics library layer that talked directly to the hardware. On top of that was built the "Layers" library which does what Gosling is talking about. It just handled clipping lists and "stacking" without any other details. On top of this layer was built the GUI.
Also, the Amiga used a single message port to communicate with the application. You could have more msg ports, but rarely needed it. You waited politely for a message, fetched it, then acted upon it as you will. All your GUI events queued up nicely in the message port.
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
Gmail's interface is an interesting design, hiding away functionality until you need a specific feature.
I always wondered if a generic window system could be designed in the same modular way, with smart application modules anticipating your behaviors and adding more and more specific, functional components into the UI as you perform a particular task.
In order to get good performance out of such a simple window system, applications need to be reasonably intelligent. One thing I think this entails is getting rid of immediate-mode APIs as the standard way to draw, and make retained-mode APIs the standard way to draw. To refresh your memory:
- An immediate mode API is something like GL or Cairo. The app sends drawing commands, and the engine executes them immediately. If something moves and needs to be redrawn, the app musdo all the work of redrawing the scene.
- A retained-mode API is something like EVAS. Instead of submitting drawing commands, specifies what the scene looks like in a scene graph. The canvas library does all the dirty work of redrawing scenes efficiently when things change.
The plight of X (which has very fast drawing, but often has brain-dead application redraw behavior) shows that no matter how fast your graphics API is, many application programmers (who usually aren't graphics programmers), will still make it look slow by writing apps that redraw the whole scene on even the smallest change. A good canvas API like EVAS fits very well with how most apps work. Canvas APIs are slower when scenes change quickly, but for most apps, most UI elements stay static. Where canvas APIs excel is in allowing simply-coded apps to demostrate good redraw behavior, because all drawing optimization can be done in the canvas.
Of course, for scenes which are animated and quickly-changing, apps should be able to access the underlying immediate-mode API, but this hsould be the exeception rather than the rule.
A deep unwavering belief is a sure sign you're missing something...
Curiously, the Mac OS X window system implements almost the exact design Jim Gosling describes in his paper.
All drawing work is done on the client side, and the window server has nothing to do with fonts, cut/paste support or much other higher level work. The window server simply assembles the drawing buffers to the displays (via hardware or software) and routes events, using hints of the foreground application and the visible window area to manage the task.
A consistent look and feel is derived by providing a consistent set of high level toolkits, residing on a set of lower level drawing frameworks.
Shared libraries make sure the needed code is readily available and resident in memory. Font are cached and vended as shared memory resources using Mach's virtual memory semantics. Drawing buffers also leverage Mach VM semantics.
uhm, I think that's what he said......
but generally yes you have to be a dick when you are a UI designer. you have to say "No, you don't want that" almost all the time. it leads to a better design. This is why KDE and Gnome usability will never be as good as the mac or even windows. Unless they become complete assholes and remove features by the pound, tell users who come up with stupid feature ideas to go take a hike, and block any non-KDE or gnome app from even being installed.
"You don't want flexibility."
Here's what happened with X11 as I see it. Fundamentally, it was a network protocol spec and client/server model. Then they built Xlib to implement the network protocol. Then, they ginned up the Athena widget set, sort of a quickie prototype on how one might actually start to build a UI on X. Having done that, they called it a day, leaving it for others to implement the look and feel, and basic functionality like cut & paste. As a result, for years most developers just used the (crappy) Athena widgets as-is, while some others started off in several directions making something worth using (e.g. Motif). Finally a decade or two later we have some decent Windowing toolkits built on X, and a look-and-feel morass.
X was overly focused on the juicy technical aspects of the day (like networking) and stopped short of providing an application-ready windowing system.
Instead, focus on delivering 1) a rock-solid, high quality API and 2) a great-looking, high performance implementation for the common case - an app running locally on a PC.
In other words, pick good API (e.g. GTK) and implement it over a small, relatively primitive rendering library to access hardware (e.g. OpenGL).
If people want to come along later and re-implement the API to insert a network transport layer, fine. They can write a shared object to do that, and slip it in place of the local version. Its backend might be VNC, X, whatever.
If they want to re-implement it to look different, or have different functionality, fine. But there probably won't be a lot of motivation to do this (except maybe to default to a different skin, or make this year's buttons round instead of square, so people feel better about paying for an OS upgrade). And if you replace the default shared GUI library with something else, *all* apps will link against it and hence look the same. (Unless you want to get fancy for some reason and run them with different link paths or something).
The problem is that there isn't enough RAM to preload all the applications. My PC during the day will run (and this is a typical work day) Word, Excel, Outlook, Visio, Project, Firefox, Internet Explorer, and an assortment of programs that don't concern me like virus scanners.
If all of these applications tried to preload themselves on startup then your swap would grind itself into dust and boottime would be in excess of 30 minutes.
It's false reasoning to say that Word takes only 2 seconds. It takes 2 seconds plus whatever time it added to the boot sequence. And if the first application you run isn't Word then there is a good chance that the preloaded Word will be swapped to disk anyway, making the next instance of Word take significantly longer than 2 seconds.
Take note that Mozilla also uses the preload trick. My work machine has consumed all 256MB of RAM and 450MB of swap after a fresh reboot and a login. That's 450MB of intensive swap activity that slowed down my boot sequence. If I just want to check my appointments in Outlook then why am I forced to wait for Word and Mozilla to fight over the swap? It's ludicrous.
Subtle point here. The hardware the apps have access to may not be the screen, but an off-screen surface which the graphics acceleration subsystem (such as OpenGL) can draw into. The window system takes care of getting the bits drawn in the off-screen surface onto the displays.
These surfaces can live in VRAM, or DMA addressable main memory. Lots of tricks can be done here by having the app draw at what is essentially the front end of the display processing pipeline.
Consider for example the GL buffer-as-texture path. Apps draw into a buffer, which when flushed is treated by the window system as a texture to be applied to the app window. The whole GL pipeline can be applied, scaling or warping the texture, altering the geometry the surface is to be applied to, mixing the texture with other texture sources, and so on.
If I designed a window system today, it would have themeable standard widgets, and the protocol (function calls for local, some sort of RPC for remote) would only have to specify the widgets to be used, as opposed to all the drawing operations, which is what X11 does.
...) to be communicated between server and client. Rather, clients would be able to indicate which events they wish to receive for each widget (basically like onclick, onmouseover and friends in HTML).
Also, it wouldn't require each and every event (mouse move, click,
All this is simultaneously going to do away with the many competing and incompatible GUI toolkits for X and the non-themeability of Windows and Aqua, and make network transperency work without huge bandwidth requirements and sluggish responsiveness.
It's worth pointing out that this window system exists in the form of PicoGUI. Sadly, the site is currently down.
By the way, what is it about OpenGL that makes it so suitable for acceleration, yet it's horribly slow when implemented in software?
Please correct me if I got my facts wrong.
Not fair to blame Gosling for Swing. He mostly participated in actual Java development back when Java was still called Oak, and was still viewed mainly as an embedded systems language. Swing didn't appear until after Sun began its big push to sell Java as a general-purpose platform. By then, Gosling had been promoted to "Chief Scientist", a job that seems to consist mostly of giving speeches and writing papers.
This like saying that once cars could go faster than it was safe to, no more innovation was needed.
What would happen if such a windowing system appeared would be this: the GTK+ folks, the QT folks, and some Xlib folks would port their libraries to the new system, add in a few missing things, and we'd have the same thing we have now, but faster, and easier to maintain.
It would also move importants bits out of the server, like the paste buffers and so on, into plain user space, where they could more easily be standardized. Free of the legacy swamp of X, clean designs could spring forth, and innovation could happen.
For instance, I'd love for there to be an easy to use clipboard stack, that could hold as many clips as there was diskspace, and an interface to help maintain it. Click the clip you want, second button it into place. This would make things like document editing easier, and make using the clipboard less of an annoyance.
All the technology in the world won't hide your lack of vision, talent, or understanding.
From the Article:
<i>Once you accept that fact and admit that
it’s actually the right way to go, the design falls out, simply by
stripping away legacy stuff that isn’t needed any more.</i>
This is actually the hardest thing to do. Todays Computer systems ar still mostly based on the concepts of 30 and more years ago. So many things that got hacked in into Unix and/or Windows in the last decades could be unified in the way it is accessed. Plan9 is actually a nice step in this direction.
Just because I can imagine doing a hippopotamus, doesn't mean I'd like to do it.
The fact is that's there's pretty much nothing you can do in software that you can't optimize in hardware to run faster, sometimes orders of magnitude faster. But the fact is that there are far more general problems than there are specialized problems. 3D rendering and by the way, encyrption and compression, are just some of the specialized problems common enough to warrant the creation of dedicated hardware. Even then the tendency is for one or two generations of moore's law to cause generalized processors to surpass even the best hardware. 3D is still a relatively immature technology, so this hasn't happened for it yet, but give it another 10 years and you might find 3D accelerator cards are as rare as dedicated hardware encryption devices today.
Jherico
What can the average user can do to ensure his security? "Nothing, you're screwed"
His new design duplicates the big mistake of X... putting policy in the application... and makes it worse. Why is it that people use Mac OS or Microsoft Windows? Because they have a consistent GUI, however it's implemented, that isn't subject to the whims of each applications programmer.
And it's unnecessary... most applications need a fairly limited set of graphic primitives, and where composition of those primitives is needed scripts in the window system can virtually always do the job: the limiting factors in a GUI are rendering, which would still be handled in native code, and the human. Yes, some applications need tightly coupled high performance control over their display, but this is still and for the forseeable future an exception. Even art software really doesn't need the kind of GPU-intensive performance he's shooting for. The applications that need to do their own direct rendering of complex scenes, rather than just a fast way to pump bitmaps to the display, are pretty rare and can be dealt with as they are now with a shortcut through the window system. With OpenGL you can even have multiple applications of that kind running concurrently without interfering with each other.
So the special case he's optimising for is already well handled, we don't need to build the window system around it. And in the general case it wastes the performance of the graphics card by keeping the application way off in the processor intimately involved with the mechanics of moving images around. As GPUs get more power and memory it will be more and more practical to move more of the window system into the GPU, and it will be more and more desirable to handle rendering in a common layer that's close to the display (in the GPU, where possible) the way Mac OS X already handles compositing.
Quartz Extreme is pretty crude. It shouldn't be necessary to do rendering in the processor and compositing in the GPU (the normal case, because it doesn't copy rendered windows back from the GPU to the CPU and maintains the master of each Quartz window in main memory at all times), with all the extra memory traffic that creates... but it shows the way forward. A truly 3d GUI where windows and more complex application objects are managed in 3d space the way a window system handles them in 2d space should be possible and efficient.
But consider what happens when you move a window into the 3d background... the GUI moves it away from you and tilt s it at an angle so you can keep it in view "off to one side". You can't keep going back to the application over and over again to re-render its part of the screen as your viewpoint changes. Instead you let the GPU map it onto a surface, and navigaton of the environment is smooth and more or less invisible to the application. Perhaps one might send the app a signal that say "suspend updating" when it's too far away or out of your viewpoint, but that's an optimization.
No, this is exactly the wrong time to go back to the X model of a dumb server and smart applications.
Hmmm...I recognize you from the old comp.sys.next.* usenet hierarchy. Didn't you disappear after the acquisition to go work on creating Quartz? If so, it must be fun to be a few steps ahead of Gosling. Oh, and thank you for the working implementation that I'm using right now.
The "cue the foo posts in 3, 2, 1..." posts will commence with no subsequent foo posts in 3, 2, 1...
Have you tried with prelink? I'm still 50/50 whether the effect is placebo or not. (partly because how the system used to work isn't anymore in my short term memory due to having to spend a few days in fixing what prelink -a did after my /usr got full..)
I think you are going to be proven wrong... my feeling is that there is going to be some key changes in the future with 3D acceleration. I'm expecting window systems to radically change in the future when 3D features of graphics cards are used. Let's face it: we have 3D cards, which are not used for anything. Doesn't it seem plausible that the windowing system will start using the unused capabilities of modern video cards? Doesn't it also seem logical to have everything in 3D rather than 2D (which is old school)? For instance, having everything as vectors (as under 3D) would significantly improve desktop quality as it pertains to resizing windows, changing icons, chaging colours, etc. Arguably, the display may also improve significantly (things like pixellation and blurriness on some monitors/laptops won't be as bad).
So to sum up, I think there is a revolutionary elephant knocking on the doors of window systems. That elephant is none other than 3D features of modern graphics cards...
Sivaram Velauthapillai
Seeking the meaning of life... @slashdot of all places
The plan 9 way of doing the windowing system is very interesting. The way /dev/bitblt is handled is probably the nicest way to implement a windowing system...
An intro to 8 1/2
In short, it ain't gonna happen so start trying to persuade existing widget developers to harmonise, and application developers to hurry up and move to the new harmonised versions. *That* is the only reason there is a consistent look and feel on Windows and MAC OS. As an example, The GIMP was ported over to Windows, and it looks like a GTK application even though Windows has a standard widget set.
- James Gosling wrote the NeWS Network Extensible Windowing System.
- Don Hopkins was one of the absolute wizardly programmers using NeWS.
- JCR was a NeXT Weenie, and is now an Apple Weenie. (I forget if you worked directly for NeXT or only were a heavy user...)
- The Unix-Hater's Handbook was written in the late 80s, so the comments mostly relate to X11R2 or thereabouts, and when Don was referring to a 486, that was a fast computer. I thought the book was misnamed - it was largely the sendmail- vi- and emacs-hater's handbook, and many of the things that were evil about Sendmail, Vi, and Emacs then are still evil today.
So when Don was ranting about architectural choices that were wrong with X, he was comparing it to more interesting architectures like NeWS or various experimental things that have been left in the past, or to Windows which was relatively lightweight (though badly buggy) back then (the "lightweight" part has since been fixed.) And the things that he complained about mostly haven't changed a lot, except that Motif was really bloated back then, while these days Gnome looks bloated (but does a lot more) and Motif looks comparatively lightweight.The one major difference is that today, the main uses for X are to run a browser as well as Xterm. To some extent, this balances the really big advantage of X (or NeWS) over Windows, which is being able to access resources on lots of machines at once.
What Don *didn't* rant about, because he was ranting about X, was that NeWS had its own horribly broken tradeoffs as well. The way it implemented really good WYSIWYG rendering and let applications decide which pieces of work should run on the client vs. the server was to pass executable chunks of Postscript code around for the interpreter on the NeWS Window Server to run. When everything worked, it was excellent, a joy to work with, and received the kind of raves that NeXtStEp enthusiasts give NeXt'S environment, but it had an almost total lack of security between applications, and if anything broke your screen tended to explode into an angry fruit salad. There were few people capable of doing real debugging in this environment, though Don, who was one of them. wrote an amazing debugger for it.
Java took many of the good ideas from NeWS about letting you safely hand chunks of code between cooperating interpreted processes, but did so with an actual security model built in from the beginning. There are people who don't like it (especially ObjectiveC fans), but one of its worst problems, slowness, has been fixed in part by JIT compilers and in part by Moore's Law. Unfortunately, some of Java's competitors, like ActiveX and JavaScript, didn't have a solid security model built in to them, so while they also let you divide labor between clients and servers, they're a security nightmare that could, and should, have been avoided.
Disclaimer: JCR and Don are friends of mine. I've met Gosling occasionally, but it's been a few years.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks