Slashdot Mirror
The Full Story on GStreamer
JigSaw writes "Gnome's Christian Schaller has written an intro/status document on GStreamer, the next generation multimedia development framework for Unix. Christian explains what it is, why it is important, its use in both the desktop and server side, its use on embedded Linux, Gnome and even KDE. He also discusses its current competition and the plans for the future."
There was one snippet of news buried in the last page that I think is pretty big:
:)
"Another interesting development is that we currently got a team of about 7 french students who are going to make a GStreamer-based non-linear video editor as the final year project."
7 students running a final year project suggests it ought to be good, so does that mean we might finally have some really high quality video editing software other than Cinelerra? If so, that's brilliant!
I like the fact that GStreamer support is now filtering even into non-GNOME apps like Juk (in KDE). Good stuff
The "pipeline" he describes is somewhat similar to what you can do with VST plugins in Windows. E.G., you could hook up a microphone, then attach some distortion filters and eventually terminate the pipeline at some output device. All in all, this is a great article in my opinion. For the technically inclined, there are much more in-depth docs here, including all the gory API details.
Buy some computer games!
One of my terminal windows looks this:
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail
KDE has a runaway process killer. Why doesnt gnome?
I wonder what will happen when MPlayerG2 comes out from an incubator. Will the two projects simply compete, or will they work out some way to integrate/support each other?
In fact, there is little out there to compete with GStreamer, at least on Linux. The nearest equivalent would be DirectShow on Windows which has nowhere as nice an architecture.
You're probably thinking that GStreamer duplicates Xine and MPlayer (though mplayer isn't really a library). To a certain extent it does - they all allow you to play back files, however GStreamer allows you to do a whole lot more.
Having said that, at the moment XineLib is more robust than GStreamer is, but the competition actually is spurring them forwards.
Although the main integration isn't planned until 4.0, the upcoming 3.2 will support gstreamer in JuK, the new music player for KDE. It will replace the slow and buggy noatun. Ive tried it, and its really quite good. Its one of the reasons why KDE 3.2 will rock.
GStreamer does not depend on GTK. The only dependence is on glib. I've yet to see anyone make any rational argument against a glib dependency in KDE. glib is just an extension to the C library, and no more a GNOME technology than libxml or libpng.
It's hard to be religious when certain people are never incinerated by bolts of lightning.
glib, gobject, gdk and gtk ARE a common package ... in most if not all distros
/usr/lib/libglib-2.0.so.0 /lib/tls/libc.so.6 (0x002f1000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x004c8000)
/usr/lib/libgmcop.so.1.0.0 /usr/lib/libmcop.so.1 (0x009e9000) /usr/lib/libgobject-2.0.so.0 (0x0015b000) /usr/lib/libgmodule-2.0.so.0 (0x00111000) /lib/libdl.so.2 (0x00425000) /usr/lib/libgthread-2.0.so.0 (0x00c68000) /usr/lib/libglib-2.0.so.0 (0x00224000) /usr/lib/libstdc++.so.5 (0x0082d000) /lib/tls/libm.so.6 (0x002f2000) /lib/tls/libc.so.6 (0x0044a000) /lib/libgcc_s.so.1 (0x00fb0000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00c2d000) /lib/tls/libpthread.so.0 (0x00ef0000)
Which? I've never used one. At least Debian, SuSE and Red Hat/Fedora package them separately, and those are the major dists.
GTK+ is not built on glib and gobject : they belong to GTK+.
You're just wrong there. Many projects use glib and its gobject, and have no runtime dependency on gtk+. I, myself, have developed projects in C using glib on systems that do not have gtk+ installed.
Observe that glib is not linked to gtk+ on a Fedora system:
[gordon@wanderlust:~]$ ldd
libc.so.6 =>
Further, observe that KDE's "arts" includes libgmcop which uses glib and not gtk+:
[gordon@wanderlust:~]$ ldd
libmcop.so.1 =>
libgobject-2.0.so.0 =>
libgmodule-2.0.so.0 =>
libdl.so.2 =>
libgthread-2.0.so.0 =>
libglib-2.0.so.0 =>
libstdc++.so.5 =>
libm.so.6 =>
libc.so.6 =>
libgcc_s.so.1 =>
libpthread.so.0 =>
I would like to see some kind of cross-platform framework of this kind. DirectShow is Windows only. QuickTime runs on Windows and Mac, but not anything else. Are we going to see GStreamer on other platforms? The transform filters should be relatively easy to port, as should the file readers. The only parts that should require more than a quick recompile (I would imagine) will be source and sync filters that interface directly with hardware (cameras, microphones, speakers, displays).
Oh, and one more question: DirectShow exposes an interface to graphics hardware allowing filter developers to take advantage of hardware IDCT and MC features relatively easily. Does GStreamer have an equivalent?
I am TheRaven on Soylent News
There are two ways to read that - a) It's impossible to abstract the details in any useful way or b) all APIs to date have made bad choices in how to abstract things.
Looking over the SGI article, I seem to get the sense that it isn't really possible to have an abstract API that works on wide varieties of hardware, and you need to communicate with hardware vendors about a large number of issues. Of course, we all know how well vendors like to pay attention to open source developers, so let's not worry about being able to do that for a while. Gstreamer would have to be a force to be reckoned with before they pay any attention at all.
I don't really understand some of these objections, but I suppose it's because I'm not a graphica guru. For example, the square vs. non square pixel issue - can't a library be defined that knows what various devices do about that? AFAIK, all one can do with video shot for one pixel length when displaying on another length is add black lines to the edges to make it size correctly, or do some kind of averaging to add or subtract pixels from a dimension. I suppose if combining video from different sources the latter would have to be attempted. Even so, I can't see that this is necessarily a bad thing to have in the API - if someone has a different method for scaling between systems, just impliment it as an option for the resizing part of the API. I know if I were developing a video editor/manipulator I sure wouldn't want to deal with that myself if I could help it. Assuming there are standard ways for dealing with these issues, and maybe even default ones used most of the time, why should all the various video editors out there have to worry about it? Solve it once, define it as a standard option for the autoadapt API, and move on.
Am I missing something? I don't know much about Gstreamer, but I don't see why they can't do things in such a way that allows detailed specification of behavior if the developer is after something specific, and use the standard or accepted best solution to common video compatibility problems if not told otherwise. Maybe Gstreamer could even become a part in standardizing some of the hardware insanity SGI had to deal with, if it is successful and powerful enough. It's open source, but you never know. Ogg Vorbis is starting to get hardware support, and I would never have guessed that either.
"I object to doing things that computers can do." -- Olin Shivers, lispers.org
I am JACK's primary author. I hope I can explain some of the basics to you.
1. What the hell is a signal graph (re: your response above)? Of what I've read about JACK, that's the first time I've seen that expression? Or by "signal graph" do you simply mean "a graphical environment for stringing together a sequence of signal processing modules into an overall application"?
When audio programmers talk about a signal graph, they are using the term to refer to a rather abstract conceptualization of what is happening in software (sometimes in hardware). The model is of a series of "nodes" each of which processes a signal in some way. Each node is connected to one or more other nodes, for input and/or output. You can build a very simple graph, such as some kind of node that reads from a disk file and sends output to another node that delivers it to an audio interface. Or you can build incredibly complex graphs in which the signal is routed all over the place, possibly even including through feedback loops.
JACK is merely one of many systems that use the model of a signal graph internally; GStreamer is another.
2. You say that JACK is for communications between different processes. My understanding was that JACK was for communication between different sources/sinks of audio signal. Those could be processes, but they could also be hardware devices. For instance, when I start jackd prior to running rosegarden4, I tell it to use the ALSA driver for output. In fact, I thought that it could really be anything that could provide or accept an audio signal (even files, network URLs, etc.), since some sort of "virtual device" could be specified for them. Is that not correct? And if it is correct, how is that different from Gstreamer then?
Gstreamer is really a toolbox to be used by a SINGLE program to construct processing pathways (aka "signal graphs"). It offers no facilities (other than connections to JACK) that allow MULTIPLE processes to route data among themselves.
As to what a JACK client does with the data it receives - that is entirely up to the client. We have some clients that stream to an icecast server, other people are working on UDP and RTP-based networking, others write data to disk etc. But JACK knows nothing about this, its entirely internally to each JACK client.
3. What do you mean by "with Gstreamer the whole graph is in-process"? Are you saying that you use the graphical signal path editor to create an application out of modules, but when you're done it links (in the post-compilation sense) the modules together into a single executable which has the capability described by the network? Because otherwise -- if the modules do their work independently and pass data between each other -- that sounds like processes talking to processes, just like with JACK. What am I missing?
As I mentioned above, Gstreamer is used by a SINGLE application to build processing pathways. It is of no use whatsoever in building multiprocess pathways, other than its connection to JACK.
4. My understanding of the whole point of JACK is that it's for low-latency audio work. But it sits between processes, or between devices and processes, or whatever; how can that be lower-latency than if JACK wasn't there at all. For example, rosegarden4 uses JACK to pass data to the ALSA driver for my soundcard. How can that be lower-latency than if rosegarden4 just talked to the ALSA driver directly?
For a situation involving only one process (such as rosegarden), its certainly possible for direct access to provide marginally lower latencies than with JACK. But when I say "marginal", I really mean it. On a modern CPU, and with the right kernel, you can basically JACK as low as your audio interface can handle. The reason that JACK's design matters for latency is 2-fold. First of all, it imposes the correct model of int