Frontiers: A New Xlib Compatible Window System

alucard writes "The JourneyOS people have published this overview of their upcoming window system. It looks like it is OpenGL based and uses XML as the communications protocol. The biggest news is that it is supposed to have Xlib compatibility, but uses HyperQueues instead of Unix domain sockets. Could this get rid of the speed problems of XFree86 while still retaining Xlib compatibility? I think this is something everyone wants, but projects to create alternative GUIs such as Fresco and PicoGUI have given up any hope of compatibility with X11 or Xlib. Can we expect another alternative out there soon?"

Comments

That depends...

[Motherfucking Shit]

Could this get rid of the speed problems of XFree86 while still retaining Xlib compatibility?

If you're talking about my Pentium 75 firewall with 40 megs of RAM, I'd say that the answer is probably "no." Then again, I think I saw a pig flying on the way home from work today ;)

Re:That depends...

[Anonymous Coward]

Regardless of what computer you run it on, you had better count on the flying pigs being there before a window system based on OpenGL and XML beats X11. After all, despite all the slagging it gets on slashdot, X11 does have a reasonably sane binary wire protocol and XFree86 does utilize most of the 2D capabilities of your video card.

Of course XFree86 depends on the card manufacturers giving them documentation for the hardware, but if the manufacturer doesn't see any advantage in helping Linux use their cards to their fullest, don't expect for a minute that they would help some unknown group like JourneyOS.

bah.

[Anonymous Coward]

This will be SLOWER than X, for god's sake, at least locally. Unix domain sockets are zero-copy on Linux! XML requires full-blown parsing, X messages are fixed binary format.

Re:bah.

[Moraelin]

Bingo. It never ceases to amaze me through how many loops people will go, just to stay "fashionable". One of them being the retarded use of XML on an _internal_ data pipeline.

Don't get me wrong. XML is great for its original purpose: a standard format for exchanging data with other programs. Programs not made by you, nor under your control.

XML was not designed to be fast, nor to take up little memory. It was supposed to just be easy to parse, in a standard way.

It's supposed to be used in situations like, "ok, we have this program, and have to import and use the data from 20 files from 20 other companies." (E.g., a travel agency wanting to be able to use data from completely unrelated companies, like hotels or airlines.) At which point, you don't care as much how fast it is to parse, you just consider yourself lucky to have only _one_ standard data format to parse. Even if it takes all night to get that data converted, you're still happy to have it in your computers at all.

Open Office using XML to save the files makes perfect sense, for example. It's _external_ data, and it's supposed to be readable by other program. That's exactly the intended use for XML. Again, the keyword is: external data.

But using XML _internally_? Gimme a break. If that's not a syndrom of being a SFV (Stupid Fashion Victim), I don't know what is.

I thought it was bad enough to see programs where perfectly good relational data isn't just printed to the screen as such. No siree, bub. They first convert it to XML, then run through an XSLT transformation to get the same data back. _Then_ they print it.

So I was thinking, "gee, surely nothing can be a more retarded waste of memory, CPU power and development funds." Well, now I'm proven wrong. There _are_ more retarded uses for it. This here X replacement has to take that crown.

Hrmm

[acehole]

I can't see how creating more X alternatives or derivatives are going to help linux become more mainstream on the desktop?

Perhaps someone could explain that to me...

Re:Hrmm

[digitalunity]

The idea here isn't to create an X alternative, but an X replacement. Something modern, fast, light. I know everyone is going to chime in with 'XFree86 is good enough. And it isn't bloated or slow'. It is an excellent piece of software, but it isn't as fast as it could be. There are commercial X11 servers for Linux that really are as fast as advertised, I've seen it in person. Another of the 'problems' with XFree86, and even commercial X11 servers is that they don't take advantage of modern GPU's. Granted, OpenGL isn't an ideal method of rendering *everything*, but for some things, it would offload much of the graphics subsystem from the CPU. There are cases where OpenGL would be much slower, but with a little smart programming, this can be avoided. OpenGL also has the added benefit of adding some eye-candy features like transparency without taxing the CPU. Anti-aliased vector fonts are easy with OpenGL.

The only problem I see here, besides the usage of XML(it's flexible and easy to program, but too large as a metadata medium), is that many have tried this before and failed. Only time will tell.

I'll give it a week before declaring it 'YetAnotherDOA sourceforge project'.

When hell freezes over.

[Anonymous Coward]

Replacement for X. Not bloody likely.

No networking capability and difficult to port since it is tied into a x86-only features.

Anyways, remember X only uses networking IF IT HAS TO. No TCP crap if your on the same computer as the X client, it all goes thru sockets, which are plenty fast.

I wouldn't mind a replacement to X, but this isn't it.

XML

[sql*kitten]

XML is a good choice for storing data when you might want to extend the attributes of a record/properties of an object without breaking existing applications. That's inherent to the design of XML - so long as you query your documents in a structured way - say with a DOM parser and XPath - then "extra" tags don't make a blind bit of difference, except for taking up some memory and some processor time.

But it's an extremely bad choice if you want to move lots of the same type of data around, when you know the format of the data in advance, and you do if what you're sending is drawing primitives, GUI widgets from a standard toolkit, etc etc. That is because of all the redundant metadata. If you are sending (for example) a CSV file the metadata comes once, in the header, and all the rest is pure data. An XML file stores the structure with the data - if you did this in a CSV file, it would mean repeating the header every other line, doubling the size if your file (or stream) for no advantage. It's worse with XML where you might have <somelongtagname>1</somelongtagname> - that is, your metada being far larger than your data. All compression does is add both processor and memory overhead at both ends, assuming the network isn't already saturated.

I know that XML is "trendy" but it's the wrong tool here - a binary protocol should be used.

Re:XML

[hummassa]

a binary protocol should be used.
A binary protocol is being used: it's WBXML over HyperQueue. RTFA. :-)

could it be...

[SuperBanana]

Could this get rid of the speed problems of XFree86 while still retaining Xlib compatibility?

Could it be that the poster hasn't read more than 1 page into the comments on the last dozen times this X-is-slow BS has come up? I thought we all agreed on this one, but apparently not.

This is very frustrating to see, because this makes for system #3, and things were already bad enough with 5 billion different window managers. Choice is great and all, but there's a reason some things are standardized, and "xlib compatibility" is not the same as "X"; I guarantee this new system will break all sorts of things in new and exciting ways.

Perfect example of how open-source has failed us; EVERYBODY's gotta invent their own wheel instead of helping to make the existing wheel(s) better.

Didn't Apple teach us anything?

[daBass]

"This allows for one of the goals of JourneyOS, which is eventual Win32 compatibility"
Oh no. Not just X, but also Win32 compatible.

I doubt we'll ever see this project finish. When is anybody going to just start from scratch, like Apple did with OS X? Build it and they will come.

Re:Didn't Apple teach us anything?

[sql*kitten]

When is anybody going to just start from scratch, like Apple did with OS X?/i

Umm, Apple didn't start from scratch. They bought an existing platform - OpenStep - and customized it. OpenStep and NeXTStep before it were both mature products in the MCCA space (mission critical custom app) for example financial applications, telecomms, etc.

As an old NeXTStep user, in many ways OSX was a step backwards... still it is good to see that Apple are able to make a more of a mainstream success than NeXT did. NeXTStep was a great OS that should have owned the business desktop in the early 90s, if Jobs hadn't screwed up marketing it so badly. Businesses were banging on his door to buy it back then and he said no, we only sell to education, but he priced his machines at $10,000!

Oh. my. god.

[quigonn]

and uses XML as the communications protocol.

Thanks, no, I never want to try this one. XML as communication protocol is a nice generalization on the paper, but in practice, it sucks. GUIs should be optimized for speed, and thus, the protocols should be specialized, too.

Re:Oh. my. god.

[vidarh]

The X protocol IS extensible. Exactly what do you think the shared memory extension uses? Or Xv? Or DRI? Or XRender? The X protocol was designed specifically with extension in mind. Now, I can think of quite a few thing in the X protocol that sucks, but lack of extensibility is not one of them.

The "slowdown" of optimizations for X is not a result of the protocol, but of the complexity of the task. Look at all the alternative GUI's available for Linux, and take a look at the number of drivers they support and the feature set - most of them are highly specialized because the amount of work that needs to go into a general purpose GUI system to make it useful is simply staggering, and few people have the skills to do it well.

Extending the X protocol is the easy, almost trivial, bit.

total vaporware

[rmm4pi8]

nobody seems to have read the project page yet, or you'd note that the 'journeyOS people' are exactly one, the 'founder', and that so far the product is a conceptual description of an OS and another of a GUI. yep, that's it, no programmers, no code.

not that i think noble adventures dont start small, but if the tons of smart people working on wine cant keep full compatibility, and the xfree86 team is having trouble getting substantial performance improvements....well, i wish this guy luck.

sure sometimes we need a fresh start, but it seems as though journeyOS has perhaps a little more ambition than is healthy (posix and win32 compatibility, in full, and with good performance), and little in the way of resources. perhaps he could ask hans reiser about just how hard it is to design a good FS?

seems to have bitten off more than he could chew, and as another poster has already noted, may have made some bad strategic choices at the same time. why xml for screen drawing when metadata is so often repeated? why design the GUI for the libOS that doesnt have any programs supported, rather than those that do (ie, POSIX, win32)?

perhaps all in all 'dream big' would be a better motto for them...

X using sockets..

[Chris_Jefferson]

OK, repeat after me... X is NOT SLOW BECAUSE IT USES SOCKETS!!! Please understand this! When you are on a local machine it uses optimised sockets, which are the fastest way of streaming information between programs, and were designed into linux for that specific purpose. Also.. you say "Speed up".. and they are going to use XML?? Sending information via a bloated ASCII (or even unicode)-based protocol? Yep, that will be fast won't it.. espically if they parse all the XML they recieve to make sure it is correct.. *sigh*

Re:X using sockets..

[countach]

I agree. I'm far from convinced that sockets have anything to do with X being slow. In any case, doesn't X have a shared memory option?

The point is, let's see some benchmarks PROVING that sockets slow things up before going off at some tangent to replace them.

ARGGH! X isn't where the slowdown is!

[MarcQuadra]

Alright, people need to listen closely. XFree86 is NOT SLOW, the toolkits you lay on top of it are what bogs it down.

I get full-window dragging and resizing in WindowMaker on my old 366MHz laptop, and it has an ATI Mach64 video chipset. Windows crawls when I do similar operations on the same hardware.

I seriously suggest that if you think X is slow you check out a more lightweight window manager and apps. GNOME and KDE have a LOT of overhead because they run on top of an extra layer of abstraction (GTK+ and QT, respectively). WindowMaker is written in C to interface with X more directly and it shows.

You can still use your GNOME/KDE apps under WindowMaker, I'm using konqueror as my file manager right now. Try it.

As for this project, it sounds cool to me. I think X is plenty fast as it is, but that doesn't mean I think someone should take what we've learned over the last decade and apply it to a more modern 'ground-up' solution. It would be nice if there was an X replacement that had QT and/or GTK+ tied in more closely, or if we had a quartz-extreme-like OpenGL windowing system and font renderer with postscript-esque qualities (i.e. run my desktop at high resolution, but zoom everything appropriately for 'real-world' DPI).

Re:ARGGH! X isn't where the slowdown is!

[MarcQuadra]

That still doesn't make X slow. If you insist on KDE and GNOME you should be complaining about KDE, GNOME, Qt, and GTK+. XFree86 and Windows have similar graphics throughput, the Windows UI is a LOT lighter than KDE or GNOME, and it runs partially in kernel mode.

If you have issues with the performance of your linux desktop you should be looking at the applications, desktop environments, and toolkits, because XFree86 is definitely the wrong tree to bark up.

One thing you learn with computers is that you can speed things up only as much as the smallest bottleneck will allow. X is not at all the bottleneck for graphics performance on Linux machines.

Re:ARGGH! X isn't where the slowdown is!

[Arker]

What is 'desktop'? A mac metaphor, ripped off by windows, which has no place and is not needed in X.

Quit trying to make your unix box act like a toy and start using it the way it was designed to work, and you'll quickly find out it's a lot better that way.

Re:ARGGH! X isn't where the slowdown is!

[irc.goatse.cx troll]

Why is it doubtful that the kernel comes into play?
You have no idea how much of a difference it makes (I'm not saying this to be derogatory, just expressing a point).
A well configured kernel provides so much more usability, some examples of things to look into:
- Anticapritory I/O Schedualer. -- Some disk read operations actually take multiple reads which get schedualed in a weird way when theres currently something being written to disk. The reads get schedualed in a way that the writes are between reads, causing the original read operation to take a lot longer due to the added latency. This patch will anticipate such situations, and cause each read to pause for a few msecs so that the next read can be schedualed instantly.
- Interactive patches (preempt, ll, et al) -- Guess if an application is interactive based on how long it sleeps between reads. An app that constantly reads is probably not interactive, so it should have less priority. You know what they say, the slowest part of a program is the user, and that is kind of how this works. When a program sleeps for input between read its marked as interactive. This way bash responds with quickness, but gcc can wait for your interactive procceses.
- IP QoS -- Not really X related, but really makes a huge performance diference. With QoS enabled, You can set it up so that small packets (500), then limit your outstream to a little less than your max so that SYN|ACKS can still get out. The result is that you can max your upload/download to your hearts content, but your latency NEVER takes a hit. I can even scp to my server while sshed in (both using the same sshd, which is why packet size comes in) and my ssh session still remains responsive.
There are more, But these are just a few examples. I'm debating writing a paper on properly setting up a modern Linux system for maximum usability, as theres a huge difference and a lot of it is in the kernel.
Sorry for any bad formatting, ironicly enough I typed this in lynx due to a few bad apps(python + wxwindows is the kiss of death, both soulseek and bittorrent tend to trash my usability). Still need to address that, heh.

try Quake3 over X- THEN tell me X sucks.

[SlightOverdose]

Yeah sure. in THEORY, the server-client overhead of X slows it down. I have, however, yet to notice it. Hell, I get a Higher framerate with Quake3 in Linux than in win32.

To take the quake3 thing a bit further- one day I was setting up a quake3 server on a remote machine. However instead of running q3ded (the server), I ran quake3.x86 (the client) by mistake.

Imagine my horror when the screen goes blank and I realise what I have done, and SURELY, this would fsck both boxes. Theres no way quake3 can X over a 100mbit network link (with the overhead of SSH thrown in). Or can it?

A few seconds later up pops the menu. It ran fine. As a quick experiment, I loaded q3dm17. It worked, and I was getting a good 15fps- quite playable.

Dont believe me? try it for yourself.

I think that little demo alone is enough of a demonstration. X my have it's flaws- namely bloatedness, but it CERTAINLY doesn't seem slow to me.

Re:try Quake3 over X- THEN tell me X sucks.

[Ari Rahikkala]

X is not concerned with 3D stuffs except for the initialisation of the GL context.

As soon as the app has its own GL context, it directly talks to the hardware. X has nothing to do with this business except in the forwarding of events (mouse, keyboard). Only the driver, the app and the 3D card play a role in this case.

The parent was running Quake3 remotely over X. This means that the client talks to the server in GLX (OpenGL encoded in X11 packets). What you're describing is DRI, which can be used only locally.

Now, Quake3 can be expected to have a rather advanced renderer that pushes as little stuff as possible to the graphics card, so it's not such a big surprise that it can do with one hundred megabits. The bandwidth of AGP 1x is 266 megabytes per second, which does put the network one magnitude lower, but then again if you can get 10fps over the network where you can get 100fps locally, the values are in the ballpark.

I've sort-of tried this with Half-life, but the client side ran Wine between HL itself and the network (which might cause a lot of overhead outside - or then not, I can't tell) and the server side had an old ISA network card, so I got at best something like one frame per second in the hazard course. 15 fps on much better hardware is, IMO, conceivable and credible.

Hold on cowboy!

[palad1]

Before everybody and their mothers starts flaming them because they use XML as their protocol, I would like to point out that they are using WBXML to send their messages.
WBXML is a packed encoding for XML, thus saving space and all those that can be highly compressed. Just have a look at the specs, it's not because it has WAP tagged to it that it's a load of crap.
And then, to those saying that the parser will kill the perfs, I'd like to point out that this protocol doesn't rely on string parsing but _byte_ parsing..

WBXML

[DrXym]

WBXML compresses XML down to bytecodes. It's used in WAP and other places. It works off the DTD, assigning bytes to represent the various tags and attributes.

But even if it were plain old text XML, it still poses some real advantages, not least of which you could transport it over SSL, web proxies and other barriers that would stop an X-session cold.

Besides which the transport is probably not as important as the data you are send. If the schema had sophisticated drawing primitives ala SVG you might find it is considerably faster than X which might be forced to move bitmap data around for similar operations.

Re:WBXML

[sql*kitten]

WBXML compresses XML down to bytecodes. It's used in WAP and other places. It works off the DTD, assigning bytes to represent the various tags and attributes

That might help, but you're still shipping a lot of redundant metadata around.

But even if it were plain old text XML, it still poses some real advantages, not least of which you could transport it over SSL, web proxies and other barriers that would stop an X-session cold.

You can quite happily tunnel anything you want over HTTP, even X11.

If the schema had sophisticated drawing primitives ala SVG you might find it is considerably faster than X which might be forced to move bitmap data around for similar operations.

Not if you want to remain fully compatible with existing XLib code.

Re:WBXML

[luisdom]

Do you ever read sources?

That might help, but you're still shipping a lot of redundant metadata around.
From the WBXML specification abstract: ... Meta-information, including the document type definition and conditional sections, is removed when the document is converted to the binary format.

Not if you want to remain fully compatible with existing XLib code.
What? They provide a library that is xlib compatible and then extend it with more drawing primitives. What it makes xlib-incompatible is applications that use that specific primitives with plain X. xlib applications are not affected.

It seems you dismissed the whole thing just after reading the post and you are just justifying that without checking other's arguments...

How about speeding up the GUI, like KDE?

[mrb000gus]

My biggest speed problem on X at the moment is the startup times etc of applications. KDE takes 20 seconds to start up and each application takes a couple of seconds to load. Could this be sped up by a faster X server? Or is the problem there more likely Qt, KDE libs, etc?

X is fast enough

[leoboiko]

I used to run ratpoison. As others have already pointed, X is actually very fast, the problem are the toolkits. It's nice to have alternatives, but IMHO toolkit optimization is much more urgent.

Today I'm using Gnome. It's beautiful and useful, but the feature that hooked me was the superb i18n support (now I can finally do with X11 apps what I always did with Emacs: switch the input method! No more restarting apps to write multilingual Japanese/Portuguese texts!). But it is not fast, and that's in a Duron 800, 128Mb RAM. Gnome can be great for the third world, where we still have lots of Pentium 100s hanging around. Gnome and KDE are both excellent desktops, but they need speedups.

The main problem with X is (still) video card support and configuration (ever played with Trident TGUIs 9680? I have an LTSP net full of them). There's a lot of work to do, but we have come very far in this aspect. I doubt a "modern window system" wannabe could easily offer similar support.

Why I am tired of this "let's replace X" s&%@.

[DrWhizBang]

"X is slow"
"X is bloated"
"X is old"
"X needs to be rewritten"

Blah, blah blah, blah, blah. Blah. I'm going to be honest here, workstation performance is abysmal on any of the recent flavours of Unix, expecially gnome/KDE, and expecially XF86. There are basically two reasons for this:
1) XFree86 sucks
2) and XF86 sucks
(I am not being sarcastic - please hear me out...) Xfree86 sucks because it does not have good drivers. Many functions run unaccelerated on cards that have all kinds of cool acceleration features. It seems some of these features have been written, but not added to the tree ( or so I have been told.)
XFree86 sucks because it has not been proactive in implementing the features/extensions required by the newer toolkits like gtk+/qt. Are you aware that gtk pushes everything as a bitmap through the X protocol for each expose event? XRender is probably the only instance where the toolkits waited for an X extension to be developed before added in a feature that required it - generally the toolkit authors, rather than wait for an X extension or piece of functionality, they will implement it in the toolkit so that the client just pushes the pixels dow to the X server.

Do not look to discard X. X is in fact the one thing that we have that Windows and Mac do not have. It gives us years of backwards compatiblity, and an extensible, network transparent architecture. Instead, we should put our hopes in Xouvert and similar projects that are looking to give us a world-class X server, and the pieces that the toolkit authors need to optimize their toolkits for X.

Re:Why I am tired of this "let's replace X" s&

[benjamindees]

Many functions run unaccelerated on cards that have all kinds of cool acceleration features... Are you aware that gtk pushes everything as a bitmap through the X protocol for each expose event?

Thank You. No one knows that; or seems to care. I'm currently playing around with a P200 with an S3 Trio64 that gives better (single pixel) Xperf results with the VESA driver than with the (accelerated?) S3 driver. KDE seems to run faster with the unaccelerated driver, which is completely unexpected.

XFree has little to no information on problems like this because they discourage discussion of performance to avoid playing favorites among video card manufacturers. There needs to be someplace that Joe User can go to get realistic advice on graphics performance in Linux, something besides the usual "buy an ATI/nVidia" spiel.

I think it's a good thing that these type of articles get posted once a week because someone needs to start talking about this; it is a serious problem that most noobs think graphics performance on Linux sucks and everyone just tells them that it doesn't or that they should use something other than KDE/Gnome. It seems that graphics and, by extension, gaming are the only areas left to hinder Linux from widespread desktop adoption by the average Windows user.