The Full Story on GStreamer

One big bit of news by Telex4 · 2004-01-13 11:23 · Score: 4, Insightful

There was one snippet of news buried in the last page that I think is pretty big:

"Another interesting development is that we currently got a team of about 7 french students who are going to make a GStreamer-based non-linear video editor as the final year project."

7 students running a final year project suggests it ought to be good, so does that mean we might finally have some really high quality video editing software other than Cinelerra? If so, that's brilliant!

I like the fact that GStreamer support is now filtering even into non-GNOME apps like Juk (in KDE). Good stuff :)

Re:One big bit of news by Goonie · 2004-01-13 12:00 · Score: 5, Informative

I wouldn't get too excited about final-year student projects.

They are usually evaluated according to their adherence to software development methodologies, rather than the actual quality of the end product. To that end, students spend more time making the paperwork good rather than the code good. Whilst this is a necessary part of building really big projects, it's not an optimal method of building small projects by inexperienced part-timers who often have only a very partial understanding of the problem domain.

--

Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
Re:One big bit of news by Hatta · 2004-01-13 16:07 · Score: 4, Funny

"Another interesting development is that we currently got a team of about 7 french students who are going to make a GStreamer-based non-linear video editor as the final year project."

Sounds good, but don't you think it would be better if they were comp sci students?

--
Give me Classic Slashdot or give me death!

GStreamer is a very useful framework by Anonymous Coward · 2004-01-13 11:24 · Score: 2, Interesting

That was a good reading. I hope Rhythmbox and Totem will be included on Gnome 2.8 at least when GStreamer 0.8 is out and more mature.

The BeOS had the Media Kit and it was great, allowed for cool stuff easily done on apps. Check Cortex for example: http://www.bebits.com/search?search=cortex and its surrounded plugins.

Hmm... by My+Secondary+Account · 2004-01-13 11:25 · Score: 5, Informative

The "pipeline" he describes is somewhat similar to what you can do with VST plugins in Windows. E.G., you could hook up a microphone, then attach some distortion filters and eventually terminate the pipeline at some output device. All in all, this is a great article in my opinion. For the technically inclined, there are much more in-depth docs here, including all the gory API details.

--
Buy some computer games!

Re:Hmm... by CoolVibe · 2004-01-13 11:44 · Score: 4, Interesting

Ever saw artsbuilder? Arts (despised and misunderstood by many) also is more than meets the eye.
You can plug in modules, and synthesise any sound you like though plugins and modules, not unlike the pipeline editor in gstreamer.
Re:Hmm... by Spy+Hunter · 2004-01-13 14:21 · Score: 4, Insightful
aRts is despised for several reasons:
1. It has problems with some linux sound drivers causing it to skip or play brief loud screeching sounds randomly, even when no sounds are playing
2. High CPU usage by other programs causes sound to skip
3. It keeps the sound device open, meaning non-aRts programs display cryptic error messages or freeze
4. It uses a non-trivial percentage of the CPU even when idle
5. Video support was tacked on and sucks
6. It adds a large, very noticable delay to all sounds
aRts is a synthesizer. KDE shouldn't have adopted it as a general-purpose sound system. Some of its problems have been partially addressed in newer versions, but it is still a bad choice. The way it SHOULD work is the kernel should provide unlimited virtual sound channels and let an unlimited number of programs open the sound device at once. It should use hardware mixing if possible and software mixing if it must, but it should keep latencies absolutely as low as possible. Then apps can choose whatever sound framework they want: aRts, gstreamer, jack, or plain old alsa, and everything would Just Work.
--
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
Re:Hmm... by egreB · 2004-01-14 00:45 · Score: 2, Interesting

Since I don't know a whole lot about kernels, that sounded like a good idea to me. But isn't OSS and ALSA at least partly kernel-driven? Sound drivers are in the kernel, and I would think /dev/mixer is a kernel-thingy. Is the mixing done in userland? Is this supported by hardware mixers? Are there any arguments against doing it in kernel? Could it be done in a nasty hack? How is other OSes doing this (like BeOS, Good'ol-MacOS, Darwin, Windows or HURD (if they at all think about sound))? Would this be different in microkernels? Why hasn't anybody done this yet?

To stay remotely on subject, I your points about aRts is quite my experiences, especially on some hardware (like my laptop). I just turn it off.
Re:Hmm... by iantri · 2004-01-14 00:58 · Score: 2, Informative

FreeBSD does this -- in fact, that's where his terminology comes from, I believe.
http://www.freebsd.org/doc/en_US.ISO8859-1/books/h andbook/sound-setup.html
Re:Hmm... by Spy+Hunter · 2004-01-14 12:44 · Score: 2, Interesting

Right now the only mixing done in the ALSA/OSS drivers is done by the sound card hardware itself. All cards can mix together at least one channel of sound output from programs plus MIDI, CD Audio, Mic input, and sometimes other things. Some cards can mix together two or three channels of sound output from programs. But many can't, so the result is that only one program can output sound at a time (which is mixed together with the MIDI/CD/MIC channels on the card, not by the kernel). It is part of the kernel's job to virtualize hardware for access by multiple programs, so IMHO the kernel should provide an infinite number of virtual chanels and use software mixing to mix them together. There is an ALSA plugin that allows software mixing in the kernel, but last time I tried it it didn't work, and I don't think there's a way to add more channels on the fly when requested by programs. If ALSA provided only this one benefit over OSS, people would switch to it in droves. I can't understand why it hasn't been added yet.

--
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}

I'd settle for . . by Leroy_Brown242 · 2004-01-13 11:25 · Score: 2, Interesting

Video playback I could resize on the fly!
Call me lazy, but I hate putting in all those switches for mplayer.

--
Pretty Pictures!

Re:I'd settle for . . by dinivin · 2004-01-13 12:01 · Score: 3, Informative

That's strange. I've never had a problem grabbing the corner of an mplayer video window and resizing it as needed/desired.

Dinivin
Re:I'd settle for . . by superjaded · 2004-01-13 12:32 · Score: 3, Informative

Try using the 'xv' video driver ($ mplayer -vo xv). mplayer, by default, uses the 'x11' video driver and won't scale on the fly.

the 'xv' driver will, however.

KDE usage huh? by kidgenius · 2004-01-13 11:26 · Score: 2, Funny

I'm surprised that KDE users would use something that started w/ a "G" instead of a "K"....and vice versa ;-)

Server or Client audio? by vanyel · 2004-01-13 11:28 · Score: 2, Interesting

The article wasn't clear if Gstream addresses this problem, but one of the things I've been looking for is X-server based audio. I have a variety of types of systems and try to run or two desktops. Since Windows and Mac won't remote natively, they're the ones I'm currently stuck with, and my unix systems, being capable of it, are off in another room somewhere and I get to them using a local X-server or ssh. But that means no Unix multimedia, because no audio.

Breaking news by Anonymous Coward · 2004-01-13 11:29 · Score: 2, Interesting

Multimedia on Linux, Film At 11.

And here I believed the rabid zealots that told me in no uncertain terms that Linux was a viable multimedia platform... 3 years ago. 3 years ago Linux wouldn't detect most soundcards.

OT really, but you guys should think more before blathering it up in the trenches. Coming back with a zany "we have that, fucker" and pointing people to a page for a project maintained by a kid in Romania barely out of alpha that's been abandoned for 2 years as an alternative to a mature, stable commercial application is not my idea of "we have that". The computer is not just a browser, office suite and MP3 player.

Re:Breaking news by gnulxusr · 2004-01-13 18:44 · Score: 2, Informative

The computer is not just a browser, office suite and MP3 player.
No, it's definitely not just that. It's also a SoftSynthMIDI sequencer with audio capabilities, a DV video capture system with editing and effects facilities, a multitrack HDR and probably more. No, I see no multimedia this side of the mountain.

100% CPU Usage by Anonymous Coward · 2004-01-13 11:31 · Score: 4, Interesting

My experience with GStreamer on RedHat v9 with Gnome has got me annoyed. Nautilus uses GStreamer to make the thumbnails and when browsing directories I'll often see GKrellM screaming about 100% CPU usage. A quick 'top' almost always reveals the culprit as gst-thumbnail.

One of my terminal windows looks this:
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail
killall gst-thumbnail

KDE has a runaway process killer. Why doesnt gnome?

Re:GStreamer, Gnome, and KDE : WHY KDE IS WRONG by forevermore · 2004-01-13 11:35 · Score: 2, Insightful

By using Gnome you are re-affirming your American ideals and supporting the open doctrine of truth, liberty, and justice for all.

Yeah, even if they don't ask for it, don't want it, and fight you to prevent it from happening. How "free" is so-called freedom that's shoved down your throat?

Seriously - gnome, kde, whatever.. As long as they all interoperate (which could use some work, but is improving), choice is a good thing.

--
Do you really need reason for beer? Wingman Brewers

Explain it to me quickly, by plastik55 · 2004-01-13 11:37 · Score: 2, Insightful

But where are the places where GStreamer innovates over the DirectShow APIs? The basic concept seems to be the same. DirectShow even has a filter graph editor which GStreamer's stream editor is eerily reminiscent of.

--

I have a positive modifier on Troll. When I mod someone Troll their karma should go UP!

Re:Explain it to me quickly, by TheRaven64 · 2004-01-13 13:06 · Score: 4, Informative

I was wondering the exact same thing. I used DirectShow a couple of years ago, and it was a joy to work with. The example code was very clear (for example, it ships with a null-transform filter, which takes any input and then outputs it unchanged. This could easily be specialised to create a new filter). At the time, I looked at GStreamer as a possible alternative, but found that it was nowhere near mature enough for real use.
I would like to see some kind of cross-platform framework of this kind. DirectShow is Windows only. QuickTime runs on Windows and Mac, but not anything else. Are we going to see GStreamer on other platforms? The transform filters should be relatively easy to port, as should the file readers. The only parts that should require more than a quick recompile (I would imagine) will be source and sync filters that interface directly with hardware (cameras, microphones, speakers, displays).
Oh, and one more question: DirectShow exposes an interface to graphics hardware allowing filter developers to take advantage of hardware IDCT and MC features relatively easily. Does GStreamer have an equivalent?

--
I am TheRaven on Soylent News

Nice. by Bluesman · 2004-01-13 11:38 · Score: 3, Interesting

This really has come a long way from when I checked it out a while back.

It's a fantastic idea, although it's been around for a while. But being able to apply different filters to an audio stream is really cool. It's unix pipes for audio.

What would be great is if gnome standardized a bunch of filters like this for everything. Imagine being able to apply a tar and then a gzip filter in this manner. Or perhaps a .doc decode filter and a grep, then to a .csv. All file conversion could be handled by the environment, rather than individual programs, which is messy and inconsistent.

Gstreamer is a big step in the right direction. Way to go guys.

--
If moderation could change anything, it would be illegal.

GSteamer and MPLayer by JarekC · 2004-01-13 11:40 · Score: 4, Interesting

Recently I read a short but interesting discussion of GStreamer in context of MPlayer, triggered by an announcement of a bonobo component wrapping MPlayer.

I wonder what will happen when MPlayerG2 comes out from an incubator. Will the two projects simply compete, or will they work out some way to integrate/support each other?

Re:GSteamer and MPLayer by IamTheRealMike · 2004-01-13 11:50 · Score: 5, Informative

I wonder what will happen when MPlayerG2 comes out from an incubator. Will the two projects simply compete, or will they work out some way to integrate/support each other?
To be honest, the two don't really compete... mplayer is purely playback focussed. It has no pretensions as a multimedia framework, or anything of the sort. GStreamer is all about being a powerful multimedia framework.
It's easy to forget how much code sharing goes on between these projects. They are all liberally licensed, all import each others code all the time and swap codec implementations etc. This isn't like standard capitalist competition where people constantly reinvent the wheel in order to stay ahead - if mplayer has a codec the GStreamer guys want, licensing issues nonwithstanding they'll go and take it.

Who cares what a framework is called? by brunes69 · 2004-01-13 11:42 · Score: 2, Insightful

You think mom and pop, or even corperate customers for that matter, know or care what DCOM, DDC, or OLE stand for or mean? Guess that means that Windows will never be taken seriously...

Frameworks are only used by developers, they can call them whatever the heck they want.

Re:Wow, there's nothing more useful than . . . by IamTheRealMike · 2004-01-13 11:44 · Score: 4, Informative

Did you actually RTFA? I assume not.

In fact, there is little out there to compete with GStreamer, at least on Linux. The nearest equivalent would be DirectShow on Windows which has nowhere as nice an architecture.

You're probably thinking that GStreamer duplicates Xine and MPlayer (though mplayer isn't really a library). To a certain extent it does - they all allow you to play back files, however GStreamer allows you to do a whole lot more.

Having said that, at the moment XineLib is more robust than GStreamer is, but the competition actually is spurring them forwards.

GStreamer summarized by Jeffrey+Baker · 2004-01-13 11:45 · Score: 3, Interesting

I think gstreamer is a cute hack, but it is also *exactly* what Chris Pirazzi warned against in his "Video I/O on Linux: Lessons Learned from SGI".

One can build fancy mechanisms which have network transparency, compression/decompression, format conversion, graph-based dataflow management, etc. on top of a well-designed video I/O API, and such mechanisms might be useful for some applications. But SGI's big mistake--one which hampered development of useful audio/video applications for years--was to try to build and offer those fancier mechanisms to developers instead of offering a simple API that worked on multiple video devices.

Substitute audio for video when necessary.

Re:GStreamer summarized by Vann_v2 · 2004-01-13 11:58 · Score: 4, Informative

From the article:

GStreamer provides you with an easy to use API that lets you focus on your actual application instead of worrying about what kinds of things happen at the lower levels.
Re:GStreamer summarized by Jeffrey+Baker · 2004-01-13 12:12 · Score: 2, Insightful

Obviously you didn't read the SGI article. His point is that such encapsulating APIs make the details inaccessible, thereby frustrating attempts to make any decent video applications.
Re:GStreamer summarized by starseeker · 2004-01-13 16:02 · Score: 4, Insightful

There are two ways to read that - a) It's impossible to abstract the details in any useful way or b) all APIs to date have made bad choices in how to abstract things.

Looking over the SGI article, I seem to get the sense that it isn't really possible to have an abstract API that works on wide varieties of hardware, and you need to communicate with hardware vendors about a large number of issues. Of course, we all know how well vendors like to pay attention to open source developers, so let's not worry about being able to do that for a while. Gstreamer would have to be a force to be reckoned with before they pay any attention at all.

I don't really understand some of these objections, but I suppose it's because I'm not a graphica guru. For example, the square vs. non square pixel issue - can't a library be defined that knows what various devices do about that? AFAIK, all one can do with video shot for one pixel length when displaying on another length is add black lines to the edges to make it size correctly, or do some kind of averaging to add or subtract pixels from a dimension. I suppose if combining video from different sources the latter would have to be attempted. Even so, I can't see that this is necessarily a bad thing to have in the API - if someone has a different method for scaling between systems, just impliment it as an option for the resizing part of the API. I know if I were developing a video editor/manipulator I sure wouldn't want to deal with that myself if I could help it. Assuming there are standard ways for dealing with these issues, and maybe even default ones used most of the time, why should all the various video editors out there have to worry about it? Solve it once, define it as a standard option for the autoadapt API, and move on.

Am I missing something? I don't know much about Gstreamer, but I don't see why they can't do things in such a way that allows detailed specification of behavior if the developer is after something specific, and use the standard or accepted best solution to common video compatibility problems if not told otherwise. Maybe Gstreamer could even become a part in standardizing some of the hardware insanity SGI had to deal with, if it is successful and powerful enough. It's open source, but you never know. Ogg Vorbis is starting to get hardware support, and I would never have guessed that either.

--
"I object to doing things that computers can do." -- Olin Shivers, lispers.org

KDE 3.2 will come with partial support for it. by Anonymous Coward · 2004-01-13 11:47 · Score: 4, Informative

Although the main integration isn't planned until 4.0, the upcoming 3.2 will support gstreamer in JuK, the new music player for KDE. It will replace the slow and buggy noatun. Ive tried it, and its really quite good. Its one of the reasons why KDE 3.2 will rock.

Re:KDE 3.2 will come with partial support for it. by Makarakalax · 2004-01-13 12:28 · Score: 3, Informative

JuK doesn't replace noatun in 3.2, noatun will still be there. JuK is a media Library, Noatun a media player that plays audio and video. They are barely similar appplications.

GStreamer is looking more likely to be adopted by KDE. Arts is a little unmaintained and not well liked. GStreamer is good, has few new dependencies (arts depends on glib too as it happens), and supported by freedesktop.org. All things going for it. But frankly I don't know who will decide this sort of thing.

Re:Dependencies ... by IamTheRealMike · 2004-01-13 11:57 · Score: 3, Informative

Could this prog be separated between UI dependant and toolkitn agnistic (yes, gtk+ is a freakin' X widget toolkit .. anbd I hope it will saty so)

You don't know what you're talking about. GLib and GObject don't even depend on X, let alone a widget toolkit. GTK is built upon these libraries, but to say GStreamer depends on a widget toolkit is flat out wrong.

Re:Dependencies ... by CoolVibe · 2004-01-13 12:02 · Score: 3, Informative

Glib is not a widget toolkit. Sure, it has ties to gnome, but it's independant. Just like libxml2 (which is desktop agnostic) is 'tied' to gnome, but KDE uses that too.

Gstreamer needs glib to run. So what? :)

Re:Dependencies ... by damiam · 2004-01-13 12:23 · Score: 4, Informative

GStreamer does not depend on GTK. The only dependence is on glib. I've yet to see anyone make any rational argument against a glib dependency in KDE. glib is just an extension to the C library, and no more a GNOME technology than libxml or libpng.

--
It's hard to be religious when certain people are never incinerated by bolts of lightning.

Cinelerra is mediocre by Anonymous Coward · 2004-01-13 12:38 · Score: 2, Informative

Cinelerra is honestly pathetic compared to ANY non-linear videditor available on Windows or Mac.

Even that piece of shit known as Adobe Premiere -- which Cinelerra trys and fails to immitate -- is lightyears ahead. If you want to get into the billions of lightyears ahead, then compare to FinalCutPro or Vegas Video (which was recently bought from SoundForge by Sony Pictures).

Linux has a loooooooooooooooong way to come in this department, and that's no troll.

Re:Dependencies ... by MSG · 2004-01-13 12:58 · Score: 4, Informative

glib, gobject, gdk and gtk ARE a common package ... in most if not all distros

Which? I've never used one. At least Debian, SuSE and Red Hat/Fedora package them separately, and those are the major dists.

GTK+ is not built on glib and gobject : they belong to GTK+.

You're just wrong there. Many projects use glib and its gobject, and have no runtime dependency on gtk+. I, myself, have developed projects in C using glib on systems that do not have gtk+ installed.

Observe that glib is not linked to gtk+ on a Fedora system:
[gordon@wanderlust:~]$ ldd /usr/lib/libglib-2.0.so.0
libc.so.6 => /lib/tls/libc.so.6 (0x002f1000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x004c8000)

Further, observe that KDE's "arts" includes libgmcop which uses glib and not gtk+:
[gordon@wanderlust:~]$ ldd /usr/lib/libgmcop.so.1.0.0
libmcop.so.1 => /usr/lib/libmcop.so.1 (0x009e9000)
libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x0015b000)
libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0x00111000)
libdl.so.2 => /lib/libdl.so.2 (0x00425000)
libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0x00c68000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00224000)
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x0082d000)
libm.so.6 => /lib/tls/libm.so.6 (0x002f2000)
libc.so.6 => /lib/tls/libc.so.6 (0x0044a000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00fb0000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00c2d000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00ef0000)

Re:JACK, Gstreamer, ALSA's mixer, etc. Very confus by paulbd · 2004-01-13 16:03 · Score: 4, Informative

I am JACK's primary author. I hope I can explain some of the basics to you.

1. What the hell is a signal graph (re: your response above)? Of what I've read about JACK, that's the first time I've seen that expression? Or by "signal graph" do you simply mean "a graphical environment for stringing together a sequence of signal processing modules into an overall application"?

When audio programmers talk about a signal graph, they are using the term to refer to a rather abstract conceptualization of what is happening in software (sometimes in hardware). The model is of a series of "nodes" each of which processes a signal in some way. Each node is connected to one or more other nodes, for input and/or output. You can build a very simple graph, such as some kind of node that reads from a disk file and sends output to another node that delivers it to an audio interface. Or you can build incredibly complex graphs in which the signal is routed all over the place, possibly even including through feedback loops.

JACK is merely one of many systems that use the model of a signal graph internally; GStreamer is another.

2. You say that JACK is for communications between different processes. My understanding was that JACK was for communication between different sources/sinks of audio signal. Those could be processes, but they could also be hardware devices. For instance, when I start jackd prior to running rosegarden4, I tell it to use the ALSA driver for output. In fact, I thought that it could really be anything that could provide or accept an audio signal (even files, network URLs, etc.), since some sort of "virtual device" could be specified for them. Is that not correct? And if it is correct, how is that different from Gstreamer then?

Gstreamer is really a toolbox to be used by a SINGLE program to construct processing pathways (aka "signal graphs"). It offers no facilities (other than connections to JACK) that allow MULTIPLE processes to route data among themselves.

As to what a JACK client does with the data it receives - that is entirely up to the client. We have some clients that stream to an icecast server, other people are working on UDP and RTP-based networking, others write data to disk etc. But JACK knows nothing about this, its entirely internally to each JACK client.

3. What do you mean by "with Gstreamer the whole graph is in-process"? Are you saying that you use the graphical signal path editor to create an application out of modules, but when you're done it links (in the post-compilation sense) the modules together into a single executable which has the capability described by the network? Because otherwise -- if the modules do their work independently and pass data between each other -- that sounds like processes talking to processes, just like with JACK. What am I missing?

As I mentioned above, Gstreamer is used by a SINGLE application to build processing pathways. It is of no use whatsoever in building multiprocess pathways, other than its connection to JACK.

4. My understanding of the whole point of JACK is that it's for low-latency audio work. But it sits between processes, or between devices and processes, or whatever; how can that be lower-latency than if JACK wasn't there at all. For example, rosegarden4 uses JACK to pass data to the ALSA driver for my soundcard. How can that be lower-latency than if rosegarden4 just talked to the ALSA driver directly?

For a situation involving only one process (such as rosegarden), its certainly possible for direct access to provide marginally lower latencies than with JACK. But when I say "marginal", I really mean it. On a modern CPU, and with the right kernel, you can basically JACK as low as your audio interface can handle. The reason that JACK's design matters for latency is 2-fold. First of all, it imposes the correct model of int

pipe dream? by Doc+Ruby · 2004-01-13 18:49 · Score: 2, Interesting

"GStreamer is that of a pipeline system which your media streams through"

Linux programs are filters in pipelines with data streaming through them. GStreamer is a special case for media. "Programming" GStreamer is executed through a pipeline viewer, a flowchart for GStreamer components. How about a general purpose flowchart programing tool for Linux?

Perl, for example, is internally compiled into a graph of primitives. How about a program that parses Perl into graphs, enforces Perl graph grammar in a GUI, and reconstitutes Perl code for saving? The three tier form has Perl code for data, Perl graphs in the "business", and flowcharts as presentation. Is there such a thing? For Python? Ruby (hint ;)?

--

--
make install -not war

39 of 201 comments (clear)