Slashdot Mirror
XFree & Rendering
Keith Packard from SuSE (and a member of the XFree core team) is doing A New Rendering System For X. You can check here to see what he has done so far. Looks great! Help is needed, so if you can give a hand - please do. Keith is going to show some more rendering stuff on LWE.
Eh? Whether you can cut and paste stuff is independent of how it's rendered on the screen; cut and paste is largely done by the client(s), with the only server involvement typically being the use of the selections such as the primary and clipboard selection.
...but they'll only use it if and when either the application, or the toolkit it uses, is changed to use the rendering extension, unless they somehow bury that stuff in Xlib itself, which I have the impression they're not doing (and I have the impression might not be doable).
However, this shouldn't be a big problem, since, for the most part, only the window manager needs to know. The X server could send an "physical screen size changed" event to the window manager and the window manager would rearrange the windows to fit onto the new screen size. If the X server disables edge scrolling, the effect is the same as on Windows; whether the off screen pixels are still stored or not is not user visible in any important way.
Windows actually has a lot more of those kinds of "legacy" assumptions built in; for example, many Windows applications cannot conceive of resizing or moving their main window while a modal dialog is up or while they are busy computing. I'll take the fixed screen size assumption over that any day...
Is there any indication that this is more than vaporware? It's easy to promise a consistent, efficient 2D graphics API with transparency support, anti-aliasing, and hardware acceleration, it's a lot harder to deliver on it.
(F^G*A^(G-1)+B^G*(1-A))^(1/G)
And it does seem that without hardware support this can be really slow. It might be best to do whatever the hardware does as fast as possible, but not define the exact results of alpha compositing so that such changes are doable.
It also appears that your "mask" is in fact a non-premultiplied alpha channel. Thus both are provided by the design.
I am still really confused by the design, though. Is a "picture" in effect a graphics context?
I also want to state why I feel it is very important that the extension do everything with a single graphics context. Many people have said "use GLX" or "use DPS". But I need the ability to write a library that does something like take a gc and set the foreground color to a special value. This is not possible when there are 3 (or 4) different types of gc.
An alternative is to add the missing stuff to GLX: add antialisaed fonts, clipping, and perhaps a few other things to OpenGL and it can be used for all the graphics, and only a single gc object needs to be worried about (though I would also like the ability to switch the gc between windows).
If that's what he's saying, then he's very confused.
On the X server there's no "text" to be selected on the clipboard no matter how you rendered it on the screen - there's just pixels. There is no guarantee that the text characters sent over the wire in a "draw text" request are somehow "remembered" by the X server, and, even if the X server happens to remember them, there's no X request to fetch those characters back (you can perhaps read the pixels back, but not the text).
As I said at least once, selection is done on the client, though, so whether the server was told to put pixels on the screen by a "draw text" request or was just handed the pixels by the client is irrelevant.
The client has to keep a copy of the text, which it can do regardless of whether it renders that text on the screen by sending the X server one or more "draw text" requests or by converting it to pixels internally and sending those pixels over the wire; that copy is what's selected in the clipboard (or, rather, what's supplied if somebody asks the client for the contents of the clipboard).
Well, first of all, I don't know whether there's a universal consensus that the X Rendering Extension constitutes "[putting] lipstick on a pig"; the fact that a person, or even that many people, happen to believe so doesn't ipso facto make it so.
Second of all, "help[ing] the Berlin Consortium" doesn't help all the zillions of X-based applications and toolkits and... that are our there now; to quote the Berlin Vs. X. FAQ:
Even if, when completed, Berlin is superior even to X with the Rendering Extension (which might well be the case), if the X Rendering Extension comes out significantly earlier than a version of Berlin sufficient to replace X, dumping the X Rendering Extension work and going off to work on Berlin may be a case of "the best is the enemy of the good".
Maybe Berlin would come out sooner if everybody quit working on X - but
I think that both the X Rendering Extension and Berlin are interesting projects; I'm unconvinced that it'd be a Good Thing if either were abandoned, as, if nothing else, they mean that people are thinking about More Than One Way Of Doing Things, which could lead to a group working on project A picking up Good Ideas from the group working on project B.
It is true that programs will not know the screen size correctly unless they pay attention to this event. But this is probably not going to break things too badly. Programs just use it to set maximum window size and to force menus/etc to be on-screen. In many cases the window manager will override them anyways.
Well, in the Berlin FAQ, they say:
so they at least realize that it will probably ultimately need to be X-compatible - there are just too many X applications out there for them to treat X as irrelevant.
It may not have X support now, but, in that FAQ, they quite explicitly say that it's not ready for prime time yet:
so the fact that it currently lacks X compatibility may not really mean that much.
Are you asserting here that anti-aliased text would somehow be stored differently in the clipboard than non-anti-aliased text? I have yet to see any indication that it would have to be; if you're cutting from a word processor, the character sizes and fonts might be available through the clipboard when using a format that includes it (although it'd probably also be made available as Boring ASCII Text, or Boring ISO 8859/n Text, or Boring JIS Text, or Boring UTF-8 Text, or...), but I wouldn't see an indication of whether the text happened to be rendered anti-aliased or not being made available via the clipboard.
(The fact that the text might have been rendered on the screen by sending raw pixels rather than sending text is irrelevant, given that, as I said, the client handles the cutting and pasting, not the server, and the client has the characters available.)
...which, of course, doesn't mean that they necessarily will use them, unless they're modified to use them, or the toolkit they use is modified to use them.
For most applications, X11 works OK. If it gets some support for anti-aliasing and compositing, people will be able to do a lot more with it than they can now. Keith's effort seems like a very rational starting point for that: add a small set of server functionalities that addresses the core problems people have.
If X11 additionally got paths, floating point coordinates, and other features in the server, would that significantly expand the application areas for X11? Would it allow existing applications to be speeded up in some important way? If people want those kinds of features, why aren't they using the existing DisplayPostscript extension to X11, already donated by IBM? And is it worth adding a complete 2D rendering API when 3D APIs already provide coordinate transformations, compositing, and all the other features, hardware accelerated, and available on pretty much all platforms, for those applications that really need them?
I can't claim to know the answers; my intuition is that the more minimalist approach Keith is taking is probably the better one for now. But whatever approach might be better in the end, unless significant additional resources materialize, I suspect that is all that realistically can happen. Any volunteers?
There is a slight fault in your reasoning.
The X server is a single-threaded application right now, and turning it multi-thread would be VERY hard work.
If it was in charge do do computation-intensive tasks, while performing them it would stop responding at all. I wouldn't like that much, would you?
From a "perspective" point of view: in the scenario you're presenting (application server plus x-terminals), the app. server is usually a "big iron", while the Xterms are tiny systems. Moving a lot of computational load unto them would worsen the usage experience a lot, IMHO.
http://www.microsoft.com/HWDEV/vid eo/GDInext.htm
--
Never underestimate the relief of true separation of Religion and State.
1) Whether or not the X Extension is trying to compete with Berlin, it is. Simply by their existance, they are competing.
2) What people don't seem to understand is that OSS isn't the magic bullet. Sure there is a near infinate number of developers, but the realities of the market are still present. There is only so much public interest, whether or not a product is Open Source. Public interest is ultimately what defines who wins and who loses. Great projects die because they don't have enough public interest. Say the X extension results in a rendering interface even half as good as Berlin. Now you've got a less superior product. Certianly a more bloated product (because it incurs the bloat of regular X, plus it's own protocol) and one that really is cobled together and "hackish." However, it has backwards comptability, and by that sole trait will win. What happens then? Probably Berlin dies. By stealing Berlin's turf, X leaves Berlin with no reason for existance.
3) Competition isn't the magic bullet either. Competition only works when to products are compatible. Take a look around the software world instead of thinking idealistically. There are a lot of graphics programs competing. Why? Because they are all comptabile with each other. Then, look at Office suites. Notice how much fewer products there are in this space. Why, less comptability. Take a look at operating systems? Notice the fact that there are no competing products. Why? No comptability. When there is no comptability, userbase is everything. Look all over the software world. Everywhere you can see products suceeding soley based on userbase. Wordperfect is a better word processor than Word? Who won? OS/2 is better than Windows. Who won? Why? There is always a winner and a loser, eventually. OSS slows the ending, but doesn't stop it. Ultimately one will win. By introducing competition to an interesting product, then the OSS community is hurting itself. Things would be different if both were competing on the same level, but they aren't. Soley because X has more comptability, it will win. No matter which is the superior product. By supporting one or the other, there are bigger implications than just one project getting more code time. After all, code time is infinate. Interest is not. By drawing interest away from Berlin, the X extension is almost guarenting its demise.
A deep unwavering belief is a sure sign you're missing something...
Placing the primitive rendering routines inside the X server means I can use hardware acceleration. Plus, sending higher-level commands means X will work over a network again. Ever run Enlightenment across the net?
The extension raises an interesting issue
This extension is starting to shift some of the load onto the client side:
Rendering geometric figures is accomplished by client-side tesselation into either triangles or trapezoids. Text is drawn by loading glyphs into the server and rendering sets of them.
In most cases, the client IS the same machine as the server, but in some cases, for example if I run an X-program on my shell account and the X-server locally, it means that the shell account starts to do the hard part of the rendering.
The shell acount might run X programs for 50 different users, but each X-server probably serves no more than a few screens, so I think it should rightfully be server-side.
X-Has-Sig: yes
It should be noted that Berlin uses a completely different approach to visuals than anything X ever had. It takes the approach that the application never need know or assume what the visual will look like. 'Widget's for lack of a better term are coded modularly into the server, creating a single unique interface for all applications to call upon. No more need for Qt/Gtk wars, just pure simplification. Hell, no more SQUARE windows like X is limited to. (anything none-square is a hack...)
I wish these guys would get more support and maybe one day it'd even be accelerated :)
Besides, image composition is for more than anti-aliasing; it forms the basis of an entirely new rendering model that replace 25 year old raster operations useful only on monochrome displays with operations that make sense in color.
Hi,
One thing I have always wondered, is if the X Consortium is still willing to change the specs of the X Window System to address current problems.
Most problems today are being addressed (we have seen this story, and recently the one of Jim Gettys who believes X is ideal for pocket devices, if only some of the "bloat" code that slipped into XFree over the years would be revisited), if only because of the popularity of Linux for graphics-intensive tasks.
But most of the solutions provided are extentions to X11. Even though it is nice that X11 is made so extensible, this still means that a standard X11 setup uses none of these extentions, practically bringing us back to the mainframe ages: even transparency (heavily relied upon by some window managers, I guess) seems to be such an extention.
Now in my humble opinion, the X Consortium could release a new spec, in which they call the most well-known and useful extentions more or less official, and in which they would probably provide some more solutions as well. This can be done without breaking radically with current X implementations, thanks to the extension system, while it would take away the "officially, this can't be done with X" complaints.
(Maybe X12 would be the wrong name, because it would be a mere upgrade to the X11 spec, with backwards compatibility, so something like X11R7 or X11R6.5?)
Would the X Consortium still consider changes in their specs, or have they merely become the maintaners of a historical document? I am just asking because during the young times I have been with UNIX, it has always been X11R6.4 (or maybe X11R6.3 too?). So I actually don't know how active the Consortium is.
It's... It's...
"We can confirm that Debian does *not* ship the version with the trojan horse. Our version predates it." [CA-2002-28]
now text in X can look as pretty as in windows.
Windows does not alpha blend text. Mac does not alpha blend text.
Antialiasing without alpha blending is substantially worse than with alpha blending.
Windows has great fonts. Great fonts are bought, and hardly ever given freely (a few notable exceptions occur in the case of TeX).
Alpha blending and antialiasing will help X font looks quite a bit. But good fonts help a lot too. And Microsoft has bought a lot of good fonts.
Why not set X up to use TeX fonts? That should be possible; everything's open that needs to be open...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Ok, I'm dumb...I understand what *brightness* is, but what exactly is "gamma"? I've seen these terms used independently, so I assume they are not exactly the same thing?
It's 10 PM. Do you know if you're un-American?
Well, anti-aliased text is actually much more readable in general than standard text. On low res displays, it makes it possible to read anything at all (TV is a good example.), and on higher res displays it makes a huge difference to the apparent quality of the text. Using anti-alising and sub-pixel glyph placement you can also eliminate the odd 'jumpiness' you get when the width of the font glyphs interferes with the pixel spacing of the display. You can see this most with rotated text: With non-antialised text, the baseline of the text appears to shift up and down, with anti-aliased text it's a straight line And transparency and alpha compositing make decent colour vector graphics worth doing :)
Now, you can do all this at the moment if you want to by rendering to a pixmap on the client side, copying the pixmap to the Xserver and exposing it. Unsurprisingly this can be *really* slow. It also means that if you're rendering anti-alised text then you can't cut+paste it! Not Good. Putting all these primitives in the Xserver will result in *every* app being able to use these new features efficiently. I can't wait...
nosig
So? Doesn't everybody use the fonts from some old windows cd for their X server anyway?
Windows fonts are TrueType fonts. True Type font technology, while largely being a subset of metafont technology, is patented by Apple, and it is possible (although it currently seems unlikely) that Apple will call those patents into action.
You can, however, convert True Type fonts into very large Type1 fonts with the ttf2pt1 package. Then they can be used by ghostscript (provided you place them in a font directory, use type1inst and mkfontdir, and edit your gs Fontmap). They can also be used by X if you rebuild the X font loader (it chokes on large fonts without a rebuild - instructions on the ttf2pt1 web site). Of course this still doesn't get you antialiasing, but it does get you some really nice fonts. And that is a large part of the problem with fonts under X. And we know that everyone loves building X and has several hundred megs just waiting for the X source code.
X also now comes with Freetype font rasterizer, so native TrueType fonts can be used. There is really no sign of Apple pulling the plug on it yet, but they can do this whenever they like. Don't forget Apple and Windows dominate TrueType font usage, and presumably Microsoft is paying a lot of money to use the patented technology.
Now that this is posted to Slashdot, I thought people might be interested to know how this extension relates to Libart, which, after all, shares many of the same goals of providing high performance 2D rendering.
Libart and the Render extension are both fairly low-level mechanisms for 2D rendering. Libart nails down the data structures and buffer layouts (24-bit packed RGB for most work), which has desirable consequences for both performance and simplicity. On the other hand, Render is really optimized for the case where there is hardware support for the primitives.
In many ways, Render is even lower level than Libart. For example, Libart handles vector paths, stroke outline, miters, line dashing, and so on. Render only handles low-level primitives such as triangles and trapezoids. To render more complex shapes, you have to tesselate them into triangles client-side before sending them to the screen.
Neither of these projects currently provides an ideal high-level API for application developers. While it will be possible to write directly to the Render layer, it's probably too much of an impedance mismatch for most applications. Similarly, the current Libart API doesn't provide any way to use the hardware acceleration that Render provides.
Thus, I'm working on a slightly higher level API for Libart, one with basically the same semantics as the existing API, but with the actual representations of data structures abstracted. Thus, a Libart+ "shape" can be a bezier path, a vector path, an SVP (sorted vector path), or a pile of triangles optimized for the Render extension. Similarly, a "picture" can be a packed RGB or RGBA buffer as now, or may be a handle to a pixel buffer actually living on the video card, accessible through the Render extension.
I want this to be a beautiful API for applications to write to. The main advantage is that it can automatically negotiate which functions should be performed server-side and which client-side. For example, no video cards today can actually deal with bezier paths directly. If this ever changes, then Keith can add the relevant stuff to the Render extension, I can add the code to Libart+ to recognize it, and apps will automatically win.
Further, this architecture provides for richer functionality than Render provides, without loss of performance in the common case. Right now, I'm adding the PDF 1.4 blend modes (Multiply, Overlay, Hard Light, Soft Light, etc) to Libart. These blending modes just get implemented in software.
Lastly, I have to say that I think Keith is making exactly the right decisions to keep Render low-level, lean, and simple. I have great confidence that he will ship something soon that provides useful access to hardware acceleration. It is refreshing to see well-engineered stuff coming out of the X world again, after many years of bloated, nearly useless crap from the X consortium and The Open Group. Gambai!
LILO boot: linux init=/usr/bin/emacs
Representing the range of brightness of a screen as 8 bits of intensity is actually very efficient and almost exactly matches human visual capability, if you use an exponential version where brightness(N+1)/brightness(N) is a constant for all 8-bit values N. This means brighness is a function pow(N,1/G) where G is a constant called "gamma". If you normalize N to be 0-1 then G is about 2.2.
Quite by accident this brightness function is exactly what cheap CRT's that are on PC's do when fed the linear conversion from D/A converters. So the hardware is actually perfect.
If the screen has a gamma value of G the correct math for adding images A and B so the result is the sum of brightnesses is pow(pow(A,G)+pow(B,G),1/G). The proper math for multiplying A by a mask M is pow(pow(A,G)*M,1/G).
Fortunately the data is 8 bits and thus both the above functions can be provided by 256x256 lookup tables. You can also use 256-entry tables for the two pow functions. Assumming G==2 may allow a lot of other math tricks as well, and is plenty accurate enough for vision.
The biggest problem with their design is that they specified premultiplied. This prevents the above math from being applied correctly, also gamma introduces questions about exactly what value you premultiply by.
I would recommend that they reformat their stuff to describe everything in an unpremultiplied area, and that all math is in intensities, not numeric values. The gamma is a constant built into the server and frozen at 2.2 or 2.0.
There can be a "premultiplied" flag in a picture. This should be used to fix "bad premultiply" that is produced by most (all?) 3D renderers, where the colors are in a gamma space but have been linearily multiplied by the alpha. To unpremultiply these, replace R with A?(pow(R,G)/A,1/G):R and same for G and B, leave A alone, and continue with the composite.
I disagree with your desire to keep the pixel values and the raster ops visible.
Yup, I was wrong on that one; raster ops are replaced with compositing operators on color values.
It may be important to support "premultiplied" alpha compositing
Reading Jim Blinn's work along with Porter & Duff convinced me that premultiplied alpha was the "one true representation". You can still use non-premultiplied alphas if you like; the extension can be twisted to make that work, it's just that premultiplied alpha is the "obvious" way to do things.
I think you *should* use IEEE 32-bit numbers and put the "transformation" into the server.
Maybe someday, but I'm sticking with client-side tesselation for now. The number of 2D geometric objects drawn on the screen is miniscule today; maybe when we're drawing thousands it will make sense to move some of that across the wire. The neat part is that we can do that transparent to the application by tesselating in the library for servers which don't support the new stuff.
Having the transformation in the server is necessary for supporting font rasterizers and to take advantage of hardware acceleration for transforming images.
Font rasterization is now the client's problem. The extension does support affine transformations of images using fixed point coordinates; it's hard to imagine what floating point would add to that.
I do think paths should be in the server, because otherwise people will write wrapper libraries that will be slower than doing it in the server.
Given that hardware requires tesselation to triangles, I believe there isn't a significant performance advantage to doing that stage in the server. The key here is getting the huge hardware acceleration speed increase and not worrying to much the small speedups from shifting work from one general purpose CPU to another.
I do not want the Windoze-style 12-parameter call to sepecify a font,
By moving the rasterization to the client side, we can experiment with lots of different font naming schemes to see what we like; there is no longer a performance penalty for using non-"native" fonts.
I just wrote my first FreeType application today; that API seems much nicer than either Windows or XLFD.
Again, I STRONGLY disagree with the conventional wisdom that the only fast way to do images is to have the program know the details of the hardware.
With a color-based rendering model, it's now obvious how to transform color data from one format to another. The extension transparently converts image data.
Make a new type of "gc" that includes the damn window!
It's called a Picture.