44.1 kHz vs 48 kHz Music Video Problem

[wrawlind]

Hi,

I'm trying to create a music video. We shot it on a Sony PD150 camera with the audio sampling rate set at the default 48 kHz. During the filming, the musicians lip-synched with a CD audio guide track. I thought post would just be a matter of synching the audio track in our footage with a .wav ripped from the guide track from the CD (I'm using Adobe Premiere Pro to do this) but this is proving to be pretty difficult. At first things match pretty well, but as the music continues to play, it becomes more and more out of sync with the video.

Is this is due to a difference between the two sampling rates (CD 44.1 vs DV Cam 48 kHz). I've converted the CD audio track into a 48kHz file with Cool Edit but it doesn't seem to make a difference.

Can anyone offer any suggestions?

Thanks in advance.
-R

[hech54]

Strange.
I just tried to duplicate what you did(44.1 to 48.000) and also got no difference. I thought it would work too.

[mpiper]

I am not sure what caused the problem, but here is a solution I have used for similar issues in the past. as long and the change is minor, no one will notice.

First, find a FRAME-ACCURATE marker on the video track at the start. Perhaps the start of the first word from the lead singer.

Then, find a second Frame-Accurate marker at the end of the video track.

Write down the EXACT time difference between the start and stop markers in minutes, seconds and frames.

Now, Find the exact same start and stop markers on the audio track.

Write down the exact time for it as well, also in minutes, seconds and frames.

Now, divide the time of video by the time of audio to see the difference. For a simple example: 5.25 minutes for video and 5.0 minutes for audio gives a difference of 1.05 or 5% longer video. (I normally convert everything into total frames and use 9440 / 8991 = 1.0499 for 5% change!)

With this difference, change the speed of the audio (or video) by that percentage so they are exactly the same length. Now you have a match!

With Premiere Pro, you can stretch and shrink time on audio and there is a checkmark to maintain pitch, so everything sounds perfect.

[mpiper]

Just wanted to check in and see if you got it fixed.

Any news

Mike

[ozymango]

Yeah, if you get this fixed please let us know, as I had the same problem myself and had to do the long "time the whole thing out, divide by 7, add 2, stretch the audio, remux, lather, rinse, repeat."

With me the situation was this: I recorded a small concert using two DV camcorders and a laptop for separate audio (patched into the soundboard), and I tried to edit it all together (using Premiere) and the video tracks and video sound synced perfectly. I was so happy. I could cut from one to another over the entire 90+ minutes of the video (four tapes total, 2 from each camcorder) and once I got the start points synchronized -- the hardest part, but not terrible -- everything stayed in sync right to the last frame.

BUT -- my ultimate goal was to also have a separate soundtrack containing the audio from the PC, very clean audio from the soundboard, but when I dropped it into my mix, it started and played in sync for a few minutes, and then verrrry sloooowly started to shift. Long story short, the audio recorded on the PC (WAV, 44khz) runs about 15 seconds longer than the DV audio, over the entire 90+ minutes. So it's not a big drift over time but way enough to get the lip sync off.

Or maybe it's 15 seconds shorter, I can't remember ... ... anyway, it's off by about 15 seconds. So I spent quite a bit of time doing the math and various stretching and shrinking to get the audio in sync, and I got it pretty close, but -- to my eye and ear -- it's off by just the tiniest bit towards the end and it drives me NUTS! Because the lips are just enough out of sync that my eye catches it and it bugs me. But I don't think I can stand adjusting the time by .0001 seconds anymore!

My audio recording software (Adobe Audition) does allow recording at 48khz and I kick myself now because maybe if I had initially recorded it at 48khz it would be in sync? I tried the 44-48 conversion and no difference, but maybe that's because it had to be 48khz to start? I dunno ... it doesn't really make any sense to me that the differences in sampling frequency should make a difference in running time, just in frequency response and sound quality (subtle as it is), but I could easily be missing something.

Anyway, next time I try something like this, I'll do the original audio recording at 48khz sampliing and see if that changes anything. Not that this would probably help you much as you're working with a CD to start with and so you're stuck with a 44khz source ... ...

Help, we need an audio engineer to give us some pointers!!! :P

[Cornucopia]

I would say that it was a fairly simple pulldown problem (i.e. The difference between recording at SampleRate = 44100Hz vs. 44056 vs. 44144, or similar 48kHz variances of 47952 and 48048), but it sounds like your working procedure doesn't include film sources.

It could be a problem with your editor application misinterpreting the rate (including drop/nondrop timing)--this has happened to mine in the past. But I don't think that's the case here.
The comment that bothers me most is the part about converting from 44.1 to 48kHz. Sounds like you did a standard SampleRateConvert ("ConvertSampleType") process. This wouldn't have been what you want, because that process is MEANT to maintain the same time duration and you specifically wanted to "fix" it by changing its duration.

What you might try to do is this:
Open in CoolEdit/Audition, and run "Adjust Sample Rate". This won't change anything about the file except the header, which will tell it to spit out the same samples at 48k instead of 44.1k. This will make your CD audio program faster and higher-pitched. However, the difference is on the order of ~8% and I didn't think that's the size difference you were talking about.

Note: If you actually tried the "Adjust Sample Rate" process above and it was STILL the same, something is buggy with the app. Try a different app.

There is still the so-far-unmentioned matter of clock reference...
When doing Music Videos, the Pros use a common sync clock and timecode reference, not only for the cameras, but also for music playback. They either playback via industrial CD deck that can be externally locked, or they play via digital tape (such as DAT, etc) that can also be locked, or use a 3rd camera (with digital lock) as audio playback source.
They also use Clappers/Slates all the time--sometimes both at head and tails of takes--VERY IMPORTANT. (Also helpful can be a syncable metronome for the talent).
Even if you can't get common clocks (superclock/wordclock/timecode genlock) for the cameras, you should still rely on Clapper/Slate. That would alleviate much of the problems, should they arise again in the future. But that's next time...

Going with seperate clocks--The internal clock crystal in most CD players DOES vary, but the extent of variation shouldn't be enough to bother you when doing lipsync. I'm talking on order of .001%
(I'm assuming you've already taken the time delay of playback unit audio vs. video into account since it sounds like the intercamera edit went well.)

Wish I could give you more help. Maybe something I said will resonate with what you tried and put you on the right path, otherwise, doing what mpiper said is the best option...

Good luck,

Scott

[wrawlind]

Hi, guys,

Thanks for all of your help on this. I'm on holiday and away from my equipment right now, but will try mpiper's suggestion as soon as I get back.

Cornucopia, thanks for your detailed post, but I'm not sure I'm entirely clear on everything you said. Can you elaborate on how my working procedure didn't include film sources? Also, can you say a little more about the drop/non-drop timing issues you faced in the past? We recorded our video in non-drop frame mode--will this make a difference?

Thanks again, everyone, and I'll let you know if I make any progress.

-R

[Cornucopia]

Cornucopia, thanks for your detailed post, but I'm not sure I'm entirely clear on everything you said. Can you elaborate on how my working procedure didn't include film sources? Also, can you say a little more about the drop/non-drop timing issues you faced in the past? We recorded our video in non-drop frame mode--will this make a difference?

Thanks again, everyone, and I'll let you know if I make any progress.

-R

You said you shot with PD150's. These record DV/DVCam with standard 59.94Interlaced NTSC frame rate--No 24p film source there. Audio would have no "framerate", but could be recorded as audio-for-video tracks, where the video is a particular frame rate. But you said you had CD source audio, so no "film rate" there.

You do have to remember that 2 minutes of CD audio = 2 REAL minutes, and 2 minutes (as counted) of DROP-FRAME timecoded video = 2 REAL minutes, but 2 minutes (as counted) of NON-DROPFRAME timecoded video DOES NOT EQUAL 2 REAL minutes.

If that difference amounts to your sync difference, then you need to have your video interpreted differently. You can do this via Virtualdub, using the framerate controls--try "Source rate adjustment = 30fps" or the matching option...
Still not sure this is really your problem though.

Good luck,

Scott