Slashdot Mirror
Ask Slashdot: Synchronizing Sound With Video, Using Open Source?
An anonymous reader writes: I have a decent video camera, but it lacks a terminal for using an external mic. However, I have a comparatively good audio recorder. What I'd like to do is "automagically" synchronize sound recorded on the audio recorder with video taken on the video camera, using Free / Open Source software on Linux, so I can dump in the files from each, hit "Go," and in the end I get my video, synched with the separately recorded audio, in some sane file format. This seems simple, but maybe it isn't: the 800-pound gorilla in the room is PluralEyes, which evidently lots of people pay $200 for --and which doesn't have a Linux version. Partly this is that I'm cheap, partly it's that I like open source software for being open source, and partly it's that I already use Linux as my usual desktop, and resent needing to switch OS to do what seems intuitively to be a simple task. (It seems like something VLC would do, considering its Swiss-Army-Knife approach, but after pulling down all the menus I could find, I don't think that's the case.) I don't see this feature in any of the Open Source video editing programs, so as a fallback question for anyone who's using LiVES, KDEnlive, or other free/Free option, do you have a useful workflow for synching up externally recorded sound? I'd be happy even to find a simple solution that's merely gratis rather than Free, as long as it runs on Ubuntu.
What you want is a simple Hollywood style clapboard. Use any two track + video editor and visually line up the audio spike to the video frame where the board is closed.
That is what the movie claps you see in documentaries are for. You produce a clap with your hand, resulting in a spike in your external mic and in the video cameras mic. That way, you know where to sync the two in your favorite video editing tool and you're fine. No need for special software or something similar.
Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.
So you either have to have something that can do an advanced auto-sync and make sure the sync gets corrected in multiple places, or you'll need to do it manually. Depending on how long the recording is that may not be too bad and you may not need to adjust it that many times, but it is really all you can do.
Now of course if you gear has some kind of clock input and output you can slave it together, but I'm guessing it doesn't.
Finally your request for Linux stuff makes it really hard. The Linux video editing scene is, well, really really bad. There are no good tools that I've come across. GRanted I haven't looked in awhile but last time I did all I found were things that were incomplete, or buggy, or not very useful (or all 3).
Something like Sony Vegas Movie Studio would do the trick and make it pretty easy to do what you needed manually, but it does cost money and isn't for Linux.
I haven't used it in a long time, but would Avidemux do the trick?
I think FFMPeg can mux audio and video from separate files. If not, search for muxing tools, and you'll have your answer. Of course, bear in mind that sample rates can drift. so the longer the video, the less likely this is to work without resampling. And you'll need to edit the start points to be precisely the same.
Your issue is very similar to what Twitch streamers go through with delay between audio and video. I'd suggest checking out OBS and there are quite a few how-to videos on YouTube to show you how to sync.
The video editor / compositor in Blender is quite nice. I'm sure with a little setup in Blender (and using a ClapperBoard at the start of every one of your videos) you could automate this process down to just having to drag an audio clip back and forth to match a spike in the audio with a frame of video.
If you REALLY have to do a LOT of videos and don't want to take the time to do it manually, then maybe you should think about dropping the $200 for PluralEyes. I have never seen or heard of any free / open source program that will do this automagically.
Actually I'm not even sure what visual / audio processing algorithms / methods PluralEyes would use to be reliable enough to do this every time - Unless they also require you to use a clapperboard...
Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.
~~~~~~~
"You are not remembered for doing what is expected of you." - Atul Chitnis
mencoder will do exactly what you need, muxing separate video and audio tracks into one container file of your choice, with tunable offsets for each track.
https://github.com/allisonnico...
I've done this with OpenShot. It's fairly easy and intuitive to use. Save often -- it crashed frequently for me, but when it worked, it worked well.
I use avidemux and audacity to add Rifftrax to movies. Basically copying out the existing audio, merging the rifftrack with the movie audio in to one track and then putting it back in as a separate audio track so the original audio is there too if you want to watch the movie without jokes (pretty rare for me actually, but it's nice to have options ;)).
Someone else already suggested OBS, I've used that too for recording gaming video. It can be pretty intensive and it takes some fiddling to make it workable but it's not bad.
If you want the software to "automagically" do its job, it is indeed not a simple task. You'd need to correlate what you hear with what you see, this is generally hard. But if you let the user to the "hard work" and associate e.g. the image of clapping hands with the clap sound, or other visual occurences with their audio counterpart, the software only has to calculate an interpolation, with the easiest setup being splines.
If you're doing this for surveillance, I use iSpy you can define a range of your camera that if it detects motion will start recording and you can choose how far in advance the buffer is and how long it waits before it stops. It can be triggered by audio too, I've found it to be very nice for watching maintenance at my apartment and people trying to break in to our cars.
They do charge for remote access features but you can "roll your own" for that if you need it (I didn't). It can use an external mic as a source or one on the camera either way.
I've done that a couple of times with kdenlive for some amateur skydiving videos. It probably wouldn't work so well for production quality stuff, but it works fine for what I'm doing. Most of my videos are less than 10 minutes, though.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
http://thinkinghard.com/blog/H... http://www.hecticgeek.com/2013... http://alien.slackbook.org/blo... There are lots more blogs, website etc that have solutions
There's a reason so many are "stuck" using Windows, no matter how onerous Microsoft gets (or OSX, no matter how walled garden Apple is) - some industry standard software simply doesn't run on other systems. Compared to the cost of someone to do the work for you, buying a dedicated Windows machine and the software to run it is less than a single edit session.
Or, as they say, use a clapperboard and do it yourself.
Is it just my observation, or are there way too many stupid people in the world?
I can't help with the automagic part, but...
I capture with this method too, except when working with pre-recorded studio performances. I highly recommend rolling the audio and video capture device for a few seconds before "action" to give you time to clap or tap an object in front of the camera, as close to the sound source (not necessarily the microphone) as is practical. you will then have a definitive reference point in your audio and video streams against which you can synchronize in your editor of choice. There are many FOSS editors that can help with this. The one I am currently using is OpenShot.
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
should be a pretty good hint that you didn't read it and just responded to the title.
He's screwed for what he wants: No cost Linux software that does auto-sync. There is none.
There's a reason so many are "stuck" using Windows, no matter how onerous Microsoft gets (or OSX, no matter how walled garden Apple is) - some industry standard software simply doesn't run on other systems. Compared to the cost of someone to do the work for you, buying a dedicated Windows machine and the software to run it is less than a single edit session.
Or, as they say, use a clapperboard and do it yourself.
Just one of the big reasons I use OSX. Even iMovie can make a surprisingly professional movie. A lot of times when under insane time pressure, I used it instead of FCP.
And I used a clapboard. Quick and efficient. And it works.
caveat: if I were to be making a long single cut video, I'd find a professional tool to keep everything in sync. But that never happened.
The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
Have you thought about modding the device to add a mic input? If you check the web, someone might have already done it, or the schematic might be available, or it might be obvious if you take it apart.
I'm no expert at video editing, but did you look at Cinelerra ? It exists for years (I mean, before VLC existed, before mplayer, even before Xine if I remember correctly... Who remembers aviplay ?) and apparently it is still (kind of) maintained.
For Debian it's available on Christian Marillat's multimedia repository.
reaper has added video editing capability recently. while windows/osx only, the devs have made efforts to make it run well under wine. should allow for syncing/editing your audio into your video. i believe it acts largely as a front end for ffmpeg for a purpose like this. there should be a lot of help at the reaper forum. the demo is fully functional and not time-limited. BabaG
If you haven't tried VirtualDub you need to go do so. That nobody here even speaks of it is mounting testament to the declining mental health of this community.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
Hit record x2, CLAP, sync in post. You really can't zoom in & line up two tracks like the big boys?
I have a little Sony handycam and made a video of a headless 8 string acoustic guitar I built. Using the Blender video sequence editor, the audio kept drifting. So I just split the audio into two minute chunks and sync'd each one. I figured I was doing something wrong, but it appears this is a widespread problem.
Different device clocks drift at different rates, mostly due to heat. Oven-controlled crystal oscillators are employed in more expensive devices to provide more reliable clocking.
Sources that are simply synchronized at the start will typically quickly drift and lose sync. Unfortuantely the drift is often not linear. So if you sync them at the start, and then stretch or shrink one source via re-sampling, they may sync at the start and end, but not in the middle.
A good solution for the OP's problem would compare the audio streams, identifying common reference points. The sources would then be adaptively corrected, matching those reference points. I've always thought that would be a very fun project, and useful to many people, but haven't been able to align my clocks to make it happen.
Oops! Posted this before logging in. Anyway, hope you find a good solution that fits your needs. To get Shenidam working, development packages and boost libraries must be installed before it complies successfully. Once it works, you can write a bash script and do things in batches and bring in other utilities like avconv / ffmpeg / sox, etc. as needed to for instance, convert a left-channel-only audio to true mono / stereo.
Precisely. I read this post, and heard the voice of Christine Baranski in my head..."So needy." If you're cheap, and really want to promote open source software, write it yourself. You say it's a simple task; just do it, and then share. If you're not willing to pay for the simplicity and ease of something like iMovie (free with every Mac since 1998 or so, so the barrier to entry is below $100), you need to pony up.