Notes On The Future of Video on Linux

Dina's Dream points out two interesting articles currently running on LinuxPower, and linked from Gnotices (GNOME news site) as well. "The first article is a really good summary of the current state of affairs of video under Linux and the direction we should take. Questions are bounced back between a few very knowledgeable people, including GStreamer developers, SGI people and Alan Cox. The second article is a set of lessons learned by Chris Pirazzi while working at SGI. Chris was involved in a lot of Video API programming at Silicon Graphics, and raises a few very good points based on his experience. All people even remotely working on video drivers or software should read these points and take them to heart."

Comments

Re:video focus

[vektor_sigma]

Paul: You mention that it makes little sense to sync audio to video and then state that this is what most applications do.

On the contrary, syncing video output to the audio is much easier, since audio resampling is a big pain. It's much easier to just watch the soundcard buffer and decide approximately when to show the frame as you imply. You only run into big difficulties when you want to sync audio to the video (or more accurately, sync video blits and audio output to the vertical refresh rate of your output device).

There are times when this is a very important. You imply that video framerates are between 20-30fps, which is quite small. For playback of video sources we need to handle framerates of 50fps and 59.94fps. At such a high framerate, the monitor refresh must track the video input in order to achieve smooth video. See the link to Dave Marsh's faq on judder I mention in the article.

In these cases, you need to set the monitor to the correct refresh rate and then watch the error between the refresh and the sound card clock, and resample the audio when necessary: playing with the refresh rate on the fly would cause a monitor resync and disturb playback!

If video cards let applications drive the refresh of the monitor (software genlock), we could run it based on the audio clock and get the advantages you describe. However given the current state of the hardware it's better to do these small resamplings. That said, actually doing this in linux is still infeasible without some updates to the APIs and drivers, which we are working on.

-Billy Biggs