The State of X.Org

An anonymous reader writes "Phoronix has up an article looking at the release of X Server 1.4.1. This maintenance release for X.Org, which the open-source operating systems depend upon for living in a graphically rich world, comes more than 200 days late and it doesn't even clear the BugZilla release blocker bug. A further indication of problems is that the next major release of X.Org was scheduled to be released in February... then May... and now it's missing with no sign of when a release will occur. There are still more than three dozen outstanding bugs. Also, the forthcoming release (X.Org 7.4) will ship with a slimmer set of features than what was initially planned."

Comments

Anything else out there?

[CastrTroy]

Is there anything else out there? Why is there such a lack of interest in X.org, when so many other projects depend on it. Most of the big projects have been moving quite quickly, making a lot of headway in the past couple of years. What's holding x.org back?

Re:Anything else out there?

[larry bagina]

I agree. I recently discovered the xfree86 project. It seems like a good alternative to x.org.

Re:Anything else out there?

[Kjella]

Uh, a strange thing to say but your posting history seems normal so...

In February 2004, with version 4.4.0, The XFree86 Project adopted a license change that the Free Software Foundation considered GPL incompatible. Most Linux distributions found the potential legal issues unacceptable and made plans to move to a fork from before the license change. At first there were multiple forks, but the X.Org fork soon became the dominant one. Most XFree86 developers who were already annoyed at other issues in the project also moved to X.Org. In short, x.org was xfree86 but that project is practicly dead. Pretty much everyone worth mentioning have migrated from xfree86 to x.org and while x.org may be moving slow, xfree86 has almost stopped dead. Going back there would do little to nothing to bolster X development. Tbe question is rather why there's so little work overall (or so it claims, I don't have enough knowledge to say) since the competition is basicly non-existant.

Re:Anything else out there?

[Yetihehe]

Tbe question is rather why there's so little work overall (or so it claims, I don't have enough knowledge to say) since the competition is basicly non-existant.

Wow, you have just answered your question in the same sentence.

Re:Anything else out there?

[CastrTroy]

First, I believe your parent post was a joke. Second of all, the reason most open source projects go stagnant seems to be for a couple reasons. First, the code is an ugly mess, and nobody wants to work on it. Second, internal conflict about where the project is going, so nothing gets done, or every new feature or bug fix becomes a big argument, about how it should be done. Thirdly, very boring project subject matter. The last one seems to be the big killer. There could be a nice open source Outlook/Exchange replacement. It wouldn't be inherently hard to build, but it seems that nobody is interested in doing it. Most of the stuff that gets a lot of attention is the stuff that developers are interested in building.

Re:Anything else out there?

[fireboy1919]

I have a theory.

We're using X as our windowing system because it's what we have, and we need it. But I don't think anybody (or not many people) really *believe* in it.

That is to say, I doubt anybody takes a look at it and says "this is it! This is the way we should do Windowing!" And so the followup, "...and if it this thing worked, then it'd be more awesome."

What people actually say when they start looking at it looks more like this.
"Okay! X.org is a good project! I think maybe I'll contribute my time to it! Hold on...what is this? Why does it have all these features that nobody cares about? Why the nonstandard build system? What's with all the crazy legacy code? This thing is way too complex for me to spend my time on, and what I learn won't transfer to any other work. I'll pick something else."

Re:Anything else out there?

[CastrTroy]

Open source does not work like big business. It doesn't stagnate because there's no competition. It stagnates because people don't want to work on it. There isn't much competition for the Linux Kernel either. That doesn't slow down it's development.

Re:Anything else out there?

[Enleth]

It's a HUGE, and I mean, REALLY HUGE codebase that deals with hell lots of different concepts (graphics, input, networking, ...) that requires a good deal of understanding to modify it properly. Yet, there are less than 30 (!) active developers. That's probably one of the most under-manned Open Source project out there, proportionally to its size. And it's, again, because it's complicated as hell and nothing can be done about this - it's just the way a windowing server has to be.

I've tried to help the project two years ago, I did dome work on input hotplugging and while not much of the code I wrote finally made it to the upstream (Daniel Stone, the man behind the input subsystem, finally decided for a different solution than what I was thinking about - maybe that was a good decision, I'm not the one to judge), I could experience myself how difficult developing X is. Besides skills and experience, you need to be able to keep track of such a big structure mentally, all the time. Not every programmer can do that, even skilled and experienced.

And, no, you can't always abstract everything out and make a nice, clean structure for the code to adhere to. Maybe the X code could be a bit easier to modify, but just a bit. Trying to force that, you would end up with an Xserver counterpart of GNU Hurd, if you know what I mean...

Re:Anything else out there?

[pebcak]

There isn't much competition for the Linux Kernel either. You mean like FreeBSD, OpenBSD, NetBSD, Darwin and OpenSolaris?

Re:Anything else out there?

[Kjella]

Wow, you have just answered your question in the same sentence. The Linux kernel for example, is completely without competing forks that I know of, yet seems to be making good strides. xfree86 was going slow, and even after the x.org revival things again appear to be going slow. With a lot of people focusing on the desktop performance of Linux, why isn't this project interesting? A lot of other projects seem to be progressing nicely even without the competition breathing down their necks. I wonder if part of the reason is that the code is X11 licensed, which is fairly close to BSD so you don't have even LGPL protection of your code. If Stallman is looking for a non-(L)GPL project to replace, maybe he should drop the Hurd and start working on a GPL'd X instead...

Re:Anything else out there?

[larry bagina]

Smelly hippies?

Re:Anything else out there?

[Enleth]

Geez, people bash on the network transaprency all the time, while it's actually the least of the problems. And it's completely irrelevant when an application connects locally because it happens over a shared-memory IPC (which unix sockets actually is, despite having "sockets" in the name).

I'd say all the old, device-dependent xfree86 code is to blame for most of the needless complexity and while it is being rewritten, it's a slow process that requires more developers than are involved with the project. Working with the new X.org code, while still demanding, wasn't really bad, just required thinking and getting "the bigger picture" well.

Actually, the new code is perfectly capable of dropping network transparency, integration of needless extensions and so on *when it's appropriate*, just take a look at Kdrive. But still too many important things remain in the xfree86 part.

Re:Anything else out there?

[Anonymous Coward]

or new developers to tackle bite-sized portions of the stack without being overwhelmed. Anyone care to comment on whether this was done?

Yes, it was - hence rapid development things like mpx, xrandr, xrender, composite xinput 2.0 and so forth. Have people really forgotten so fast that a couple of years ago linux /didn't do/ all those gee-whiz window explodey effects?

Really, the /. story is inflammatory flamebait. Loads of pretty cool new stuff has appeared recently in X.org. Actually, IMO that's more likely why the release schedule slipped - stuff like MPX is very cool but represents very major changes.

Emacs' release schedule recently slipped too - but it was because they're merging ECB and window groups into Emacs 23, not because emacs devel has stopped!

Re:Anything else out there?

[kripkenstein]

In addition to those factors, I'd say the problem is also a lack of interest.

On the one hand we have things like GNOME and KDE, Firefox, Blender, etc. etc. - software that the user knows by name and interacts with directly. People happily join such projects and contribute code to them.

On the other hand you have software that the typical user might not not even know exists, like the Linux kernel. However, for geeks the kernel is perhaps the pinnacle of programming, and furthermore by lucky coincidence (or unhappy, if you are GNU) the name of the kernel has become synonymous with the entire OS, making it high-profile just like the more obvious software projects mentioned in the previous paragraph.

Whereas the X server is somewhere in the middle. It isn't well known, even geeks might not know exactly what it does (i.e., where the separation is between X, the window manager, and so forth), and for some reason it lacks the 'coolness' factor of the Linux kernel.

All of this is unjustified, and a shame. Perhaps more stories on Slashdot like this one will raise awareness? Maybe we should also motivate people, by e.g. telling them that hacking X is even harder than hacking the kernel ;)

Re:Anything else out there?

[fastest fascist]

Seems a bit derogatory to call the BSD, OpenSolaris and Darwin users "smelly hippies". Not all of them are like that. Not all of them are hippies?

Re:Anything else out there?

[kosibar]

Okay, so here's an idea...

We should revive XFree86. To start, we should generate a list of features for the next release. We'll spread some rumors about what we're doing, let the world see how hard we're working on it.

This should get some attention from /. and other sites to get people involved but we'll freeze the code and not allow any new developers/submissions on the project. Frustrated, they'll go over to X.Org to try to work for the competition.

Now for phase II. About this time next year we announce a release date, delay it a few times, then release it about two years from now. Make it a big deal. Major release. Get everybody talking about it.

For the release we'll drop all of the major new features on the list. We'll fix a bug or two, something major like a spelling error in a log report. Of course, we'll add a few new bugs. We could drop support for some hardware. For new features we could change a few things in the conf file. Instead of "Section" you now have to use "Block". We could totally change the format of the ModeLine to something totally crazy (crazier?)

If this follows the corporate model we have today it should drive major innovation and more frequent releases from X.Org, though our XFree86 project would unfortunately take away most of X.Org's market share.

Open source projects would probably earn the respect of more businesses and government agencies if it would just follow these common sense models from the corporate world.

Re:Anything else out there?

[peragrin]

that was the point of the fork though. Xfree86 developers moved slowly before X.org was formed, and wouldn't introduce changes like 3D accelerated desktop, period.

Developers where complaining about xfree86 for years before the fork, When the license changed it was just enough to push the fork. X.org began a long boring process of breaking X into smaller modules which will accelerate overall development. The problem is that process is still on going, and will take a few more years before any major upgrades can take place.

Think about the Mozilla project. They spent years cleaning out the core codebase and upgrading the core gecko engine from Netscape before they even had a decent beta. X.org is doing the same to something far larger, and uglier.

Re:Anything else out there?

[0xABADC0DA]

I would like to contribute to the X, but mostly what stops me is that the code is written for the 80s. Lots of internals are using macros and bit values and optimization hacks, and directly 'speaking' X11 protocol. The code is disorganized, with files in weird locations and with two-letter names. I'm not blaming the developers, because in the 80s this is what had to be done. And they are making huge progress making it modular and organizing the code. But it's still not an attractive codebase to dig into.

Then, once you have decided to work on it and have fully absorbed X11 protocol into your being, you basically need a vmware license in order to develop. It's almost as hard to try out the changes you made as it is for kernel developers... slightly easier, especially for debugging, but you still need to either shut down everything you are doing to run a new build or have multiple development systems.

So basically it is a really step hill to overcome just to start developing X. Perhaps steeper since the kernel at least has excellent, 'simple', modular code to work with.

Re:Anything else out there?

[jeiler]

Agreed. X needs to fork--have half the team maintain the current implementation, the other half do a start-from-scratch coding marathon.

Re:Anything else out there?

[nguy]

A lot of Mac's success *IS* because its gui framework.

The GUI framework on the Mac is Cocoa. The equivalent of Cocoa is Gnome (or KDE). The underlying display server, the equivalent of X11, is Quartz.

it appears to be managed in frame buffer

But it isn't. OS X has the same client/server display architecture as a Gnome desktop.

with custom rom that makes sure you never see bios info -- just pretty pictures.

What you see on OS X is that the boot loader quickly throws up a gray screen to keep you from seeing the boot loader text; the text itself is still there. If you like, you can boot OS X completely in text mode, just like a Linux system.

the removal of large swaths of abstraction make it load and "talk" faster.

The OS X display server has at least as many layers of abstraction as X11. It is not intrinsically faster than X11 (if anything, it's slower). Mostly what you perceive as speed on OS X is massive amounts of backing store.

the use of pdf rendering

OS X doesn't really use PDF rendering.

and enforcing policy rather than just providing tools means that things like cut and paste work from app to app, every app.

I own several Macs. The notion that "cut and paste work from app to app, every app" is laughable, and Apple couldn't enforce that if they tried.

Furthermore, if anything, policy is determined by the GUI framework, not the display server.

That is the sort of thing that X fails on for the casual or home user.

Whatever problems you think the Linux desktop may have, they have nothing to do with X11; consistency and policy is determined by the desktop environment, not the display server.

Re:Anything else out there?

[ThePhilips]

My personal bet is that X is overly complicated.

E.g. it takes 20-30 minutes to start doing something with Linux kernel. Entry bar is set low - many people like to participate. Needless to mention that to compile (properly configured) Linux kernel (with subset of drivers and features you really need) only few minutes. There are piles and piles of documentation and forums where you can find anything.

E.g. KDE + Qt. To compile KDE - you might need days. Or just grab precompiled binary packages. But after that you can in 5 minutes create something useful and interesting. Documentation is near perfect and complete. Also reading source code is quite easy, since most of the code is human readable.

But X is different beast. Even compiling it is challenge on itself. There is literally no documentation on its innards. There is no "Hello World" for X. There are bunch of example modules which you need to spend hours after hours to only understand where they plug into the all X mess.

I'd say main X problem is its strive to be cool and sit on all chairs. I'd say they need to scale down the project and split it into smaller independent pieces. Forget large releases (installing apt-get would help! kidding). The smaller sub-projects would have more chances attracting people, since (at least theoretically) then entry barrier would be lower.

Re:Anything else out there?

[0xABADC0DA]

Personally, I'm still unconvinced that X is a particularly "good idea." 15 years later, and the promises of simplicity and compatibility are still unrealized, as every single implementation of the protocol has suffered from numerous problems. Perhaps it would be best to start from scratch, and revise X11 to be a more realistic/practical specification. The main problem I think is that X is written in C. Originally the X server did graphics itself, scan converting lines and such, so it had to be in C (and there weren't many real alternatives then). Now all it really does is manage a lot of information -- and C is a really really bad language for managing lots of information. Even a simple desktop has over a thousand "windows" that all have position, change listeners, and other properties. Then there are all the bitmaps, pens, backing stores, repaint regions, etc. Events, queues, messages.

The X server should be mostly scrapped and rewritten in Java. Java is a language that is suited for managing information like that, while still being high-performance (enough). The server could be rewritten in C++, but C++ is messy and is a complicated and archaic language at this point anyway.

Take a look at for instance weirdx which basically one person did. It handles most of the core functions of X and plenty fast (of course it is incomplete since it is one person's hobby). Or see Sun's Project Looking Glass, an opengl X server written in Java -- that was also written in one guy's spare time. With more development on these they could be real competitors to X.org while being more approachable, and I'll bet faster than the C code.

Re:Anything else out there?

[Anonymous Coward]

My personal bet is that X is overly complicated. Overly complicated? Or just inherently complicated? A graphics driver (which is essentially what X is) has to interact with the kernel via an interface defined by the kernel, with hardware, and with the window manager. The only one of those that it can define is the relationship with the window manager.

By contrast, the linux kernel can define both its driver interface (how it interacts with hardware) and its API (how software interacts with it). The kernel pushes complexity into the driver space. X is stuck managing that complexity.

X also suffers from having a complicated function. Managing graphical relations is computationally intensive and difficult. That's why the GPU is separate from the CPU -- to allow for specialization of these complex functions.

A third problem is that X is more general purpose than many drivers. It would be easier if it only had to work with one monitor and graphics card, but the reality is that it is expected to work with all of them. This is also true of the kernel, but the kernel has greater ability to push the heavy lifting into the hardware's driver than X does.

Finally, I think that one of the biggest advantages that the kernel has over X is that it is adjacent to more things. X lies between the kernel, the graphics hardware, and the window manager. Generally, X uses the kernel rather than the other way around. Therefore, it's only people that are working on graphics hardware and the window manager who are interacting with X and therefore likely to find something that they want to change. By contrast, any program might make a kernel call.

Similarly, KDE/Qt and Gnome are accessed directly by many programs. Programs should be accessing X through the window manager.

All that said, you are probably correct about X needing to split itself into smaller, more elegantly abstracted pieces. However, even if X does that, it is still likely that it will be under documented with relatively less participation than other projects. While fundamental, it's simply not adjacent to enough other software to recruit easily. Without recruits, who is going to write the documentation and tutorials needed to gather more recruits?

Of course, we may simply be confusing words here. You may be using complicated to refer to what I would call inelegant or non-modular.

What's the problem?

[larry bagina]

It's Open Source -- unlike proprietary software, we're not at the mercy of a company to dictate the release schedule or fix bugs if they get around to it. If bugs aren't fixed, it's because we failed to fix them.

Typical of Microsoft

[elrous0]

Oh, wait.

Re:Finally, developers' ignorance and childish

[Anonymous Coward]

Do they still sing "network transparency out of the box" mantra every time someone suggests changing architecture ? That's the complaint you're going to go with? Seriously? Something that degrades gracefully into the ideal solution (shared mem and unix sockets) for a local-only graphics server?

There's a LOT wrong with X.org right now, even mentioned in TFS. I personally wish they would put a lot more work into the transition to evdev and HAL, so we can get rid of xorg.conf and finally make strides to being as user friendly as "the other" OSes.

But network transparency? You're fighting the wrong battles here.

Re:Haven't really noticed any reduced quality ..

[RobBebop]

I agree wholeheartedly. The current release of X is suitable and works well for me.

The "upgrade every year" mentality is the wrong one to have. They missed their date? Okay, that's fine. As long as they don't buckle under the "release schedule" mentality compromise quality. I may be naive, but I don't know any reason they would want to push/rush their next release.

Paid developers?

[siride]

Aside from Keith Packard and Dave Airlie, and maybe one or two others, how many paid full-time X developers are there? Watching the mailing list, it seems like there's a couple of volunteers, some people who submit the occasional patch but otherwise work on Qt, or GNOME or whatever, and then the core listed above. I think there just isn't enough manpower right now and the distros, instead of fixing that problem, just maintain their own patchsets and do a "good enough" job to make X work smoothly for their releases, and leave it at that. Clearly, nobody wants to make X a priority and it shows. The Wiki is almost never up to date, it's nigh on impossible to build a working X system from git, even with the couple of half-arsed build scripts available from the mailing list (for my part, I have never been able to get it to build completely, and not for lack of trying or ability). The mailing list is full of academic arguments over color specs and other pointless things, or people asking for help. There used to be a lot of discussion on how to improve X and also, how to get things done. That no longer happens. What the distros and Linux companies need to do is get more people working directly on X and get serious about making X a serious project. It's not just some option piece of software that nobody has to care about. It's only one of the most important aspects of desktop Linux. And it just makes no sense to me that no distros are really spearheading X development. If they don't take the time to make this an issue, X will continue to atrophy, further limiting Linux's potential in the market.

Re:Paid developers?

[axxium.us]

What the distros and Linux companies need to do is get more people working directly on X and get serious about making X a serious project. Hi. I am the X11 maintainer for Zenwalk Linux. While I can't fix it all myself I have been updating the wiki and fixing documentation with the available time that I have to commit to it. I agree and think that if each GNU/Linux distribution had at least one developer helping in what we he/she can it would make a significant improvement.

Duh

[Gazzonyx]

Mostly the time we spend posting on Slashdot.

Re:Finally, developers' ignorance and childish

[Anonymous Coward]

Internally, X11 running in local mode works the same way as Apple's window server - using shared memory and local sockets. Hell, even Windows Vista works this way (except using Windows IPC mechanisms instead of Unix ones).

Everyone who suggests changing the architecture of X by removing network transparency is arguing from a position of ignorance. There isn't a faster mechanism for doing a GUI server without either building the windows server into each app (allowing only one app at a time), or building the window server into the kernel (bad idea).

Re:Finally, developers' ignorance and childish

[kaiman]

But network transparency? You're fighting the wrong battles here. That is so true. Using Macs myself since a couple years. I have a recent MacBook Pro (mostly occupied by my wife) and an iBook G3 left for my stuff. While I can ssh into the MacBook Pro and do command line stuff fast, I so wish I could simply

    export DISPLAY=skarabrae:0.0

and get actual work done fast!

Network transparency is *the* feature of X.

Re:Phoronix will pay to fix X

[The Ultimate Fartkno]

Bribery? I completely fail to see your logic here.

BattleCat needs to have a bug fixed. He approaches coders who, for free and in their spare time, code.

"Hey there, coderman. I see that you do this sort of thing for free and for fun, but what would you say to doing that coding thing that you love to do, hitting this one bug that I really need fixed, and ending up with all the satisfaction that you normally get from your work and a shiny nickel on top of it?"

"ZOMGBRIBERYYOUCALLOUSBASTARD!"

Really? Is that what you call bribery? Where I come from, bribery entails a breach of ethics. All BattleCat wanted was to add a little icing to the job that people were already doing for free in an effort to have something fixed that was a priority for him. That's about as straight-up, ethical, and non-bribery a way to get things done as I can imagine.

Well, excuuuse me!

[Chemisor]

Well, excuuuse me! Blame the community. I would blame the code instead. I happen to be one of those few people who actually wanted to contribute something. Specifically, there was this bug where the server would crash after a VT switch, so I thought I'd take a look. Have you seen the X.Org tree? It's not just huge. It's unreadable. I honestly didn't even know where to start. Documentation was minimal. If you wanted to trace one of your Xlib calls, you wouldn't be able to. There are modules, but they don't seem to have any clear purpose. There are libraries that are wrappers around something which is a wrapper around something else. Try and find the real code! I dare you! Even just building the damn thing is a major ordeal. With the current XOrg tree from git, I can't do it at all. Yes, that's right: I can't even compile it, and that ought to be the simplest thing you can do with a project. You want to know why I'm not helping the XOrg project? Because it's a pile of steaming crap, that's why, and I have better things to do with my time than trying to build a windowed skyscraper out of it.

Maybe it's time to dump X

[archeopterix]

Here[pdf warning] is an interesting article from James Gosling and it pretty much explains why making the GUI system a server was necessary in the past but is not such a good idea anymore.

Perhaps X should be replaced, not improved.

How X got to where it is

[jonsmirl]

Three main trends got X to where it is.

1) Proprietary hardware. NVidia and ATI didn't release specs. That resulted in what little dev talent there was being used to do reverse engineering. ATI has gone a long ways towards fixing this.
2) Insistence on cross platform support. Cross platform support means no device drivers - everything in user space. There are all kinds of security issues with everything in user space. This also mean no integration with the underlying kernel. OOPS isn't visible, VT interaction, mode setting, no intergration with framebuffer, etc. Insistence on cross platform means that one OS can prevent progress from occurring on the others. There seems to be some movement on this issue.
3) Failure to endorse OpenGL-ES as the core driver system. The embedded world went OpenGL-ES and ignores X (N810 is an exception). There is money in the embedded world and not in the desktop. The money went to OpenGL-ES.

From a developer's point of view the architecture of X has not evolved in a way where a developer can work on one chunk of the code without having comprehensive knowledge of the entire system. Requiring that level of knowledge really reduces the number of potential developers. Finally there is a giant amount of NIH that goes on.