Synchronizing music clip trouble

[valecala]

Good morning all!
I would be very grateful to anyone who could give me any advice on this issue I am having.

I shot a music video in a studio. It is a group that does impro-jazz, so basically they have a track, but they improvise quite a lot on the solos.I was in the room with them to shoot the video (total shot, faces, expressions, hands on instruments, etc..), and we could hear just the drum and not the instruments (linked directly with the sound engineer who was in another room).I have 4 shots of total group and 5 others on close ups during solos. I managed to synchronize one of those 5 shots with the actual final track (because it is actually the take they decided to keep for the final track). As we hear just drum, I guess I managed to synchronize it because of the length (the other shots have different lengths as they improvise and as they modified the length each time they played, like the shortened the intro, for example). Now....THE question: how do I hell synchronize the others? I tried with Plural Eyes, but no way. So I thought that there are just 2 possible ways:1) One of the member of the group help me recognize the solos on the instruments with the final track on the other 4 videos2) If I manage to have all the recording of all the other tracks, then I can first synchronize all the videos with the respective track and then all of them with the final track.
The big problem is that they didn't play one song like the other, like for a pop song but the song actually changes each time.Now, do you see any possible solution that I cannot see (I hope that I was somehow clear in my explanation)?

Thank you in advance for you precious help!

PS. I am using Premiere Pro 5.5 (and Plural Eyes 2.0 plug-in)

Valentina

[JVRaines]

If you want to use closeups of the instruments being played, you will have to sync with the audio recorded during the shot. There's no way around that if you don't want it to look fake. Wider shots, shots of faces, etc. can use the audio of the final take. You have a tricky music editing job ahead of you! You'll definitely need the assistance of the band leader.

[Cornucopia]

You sound like you don't have a musician's aptitude for music (on the rhythm side: understanding tempo, bars/beats, phrasing, lag/lead, shuffle or swing), so that's going to make your job IMMENSELY harder. To do this correctly, you have to have the guidance of some musician/musicians, even if it isn't the band leader (though, if they're game, that would help since they were there at the shoot).

But you might still not be able to do this correctly...

Synchronization (in this context) is the matching of sound and video. Sometimes it is the matching of A-to-V, sometimes V-to-A. In normal circumstances (dialog, sound effects of movies, etc), this is more easily accomplished by adjusting the Audio to the Video, because audio has a finer timebase (~0.02msec vs. ~40msec), as well as being more robust (as it is often uncompressed vs. video almost always being compressed). And audio is easier to manipulate in general because of the much smaller footprint (and accompanying resource burden).
Unfortunately, you are not in a position to take advantage of this kind of synchronization. You are editing to a particular take. Thus, it's rhythmic/synchronization needs are already set for the audio, and your only recourse is to sync the video.

Video's timebase, being much coarser, doesn't allow you the leeway of adjustment necessary to track everything going on in the audio, so you will have to concentrate on the more obvious movements. And that means: obvious to you and your audience, not necessarily what seems obvious to (trained) musicians, who would be much more sensitive to discontinuities here - unless your audience is mainly musicians.

You can always sync up lead frames ("heads") - or some other single peak moment, and sometimes head & "tail" frames, but with using shots from differing takes, you will never be able to sync up a whole clip because people make music & movement much more randomly than machines (not repeatable). Had there been a strict metronomic click track running during shooting that all the musicians stuck to, it would be easier, but that would also go against the mantra of that style of music.

While it is POSSIBLE, using AE, twixtor, etc., to "time remap" your visual motion to match your audio, it will never look "normal". If you were shooting high framerate, maybe that would alleviate some of this battle, but not enough.

The LONGER you dwell on a particular scene, the easier it will be to pick out the lack of sync, so you will need shorter clips to mask the desync. And as JVRaines already said, wider shots makes the movement less obvious, so less prone to noticing the desync. "Beauty" or "B-roll" shots are also good here in that they are inherently "without sync" or tempo, and not expected to be synced, so can be used as band-aid patches. However, because they are what they are, you don't want to use them too much or you'll lose the "groove" of the synced items.

You may want to go back and shoot some special "ISOs" (isolated segments) as particular filler, using NOT a drummer but the actual recorded take as your rhythm track. The musician(s) can either fake it ("lip sync" or "air guitar") or not, depending upon their own ethics on this, though it's usually much easier to fake it, as that allows them to concentrate on the movement syncing rather that on the musical/tonal accuracy.

Good luck, you'll need it!

Scott

[newpball]

...I tried with Plural Eyes, but no way.

Then if seems you are going to have to spend a lot of time manually, no ways around it.

Those problems would not exist if brilliant engineers came up with something as simple as a TOD timecode on videos coupled to an at least quartz accurate synchronizable TOD clock on the video equipment.

But I know, that's too hard, it's only 2015 where we still need (digital) clapboards.

[Cornucopia]

@newpball, that has NOTHING to do with the problem the OP is encountering. Problems would still exist, because the OP is attempting to sync up elements/segments of takes B, C, D, etc. with desired audio take A, for example. TOD is always going to be different, unless you have a time machine up your sleeve.

Scott

[newpball]

@newpball, that has NOTHING to do with the problem the OP is encountering. Problems would still exist, because the OP is attempting to sync up elements/segments of takes B, C, D, etc. with desired audio take A, for example. TOD is always going to be different, unless you have a time machine up your sleeve.

Well, apart from the fact that it will look unnatural that's just wrong in my humble opinion!

I can understand one gets away with it on some 'lip synching Justin Bieber clip' or to 'touch up' some short moment but with live jazz?

[JVRaines]

I tend to agree — if you're making a jazz music video, you're not going to get away with out-of-sync solos. Might work in a 10-second segment for the TV news, but not in a video for the fans.

[Cornucopia]

Well, there's no point even calling this a "live jazz" video - that ship has already sailed. That would have been a multicamera shoot of a one-take "concert-style" performance (either live-switched/mixed, or genlocked with master timecode sync for later editing).

And the use of the drummer for the click track of multiple takes leads to the inaccuracy that is all too evident now.

But you are fooling yourselves: the use of multiple takes (with appropriate recorded click track replay and non-TOD genlocked cams) and the use of lip syncing (whether voiced along or not) is pervasive if not nearly universal in the making of professional music videos. Part of the nature of the beast.
Also, professional performers of all musical flavors (classical, folk, r&b, rock, latin AND pop) deserve their due credit for professionality as well as jazz musicians, whether they fit your stylistic leanings or not.

However, we don't know how much of the other takes are to be used. Maybe they ARE just patches/touchups...

Scott

[JVRaines]

But you are fooling yourselves: the use of multiple takes (with appropriate recorded click track replay and non-TOD genlocked cams) and the use of lip syncing (whether voiced along or not) is pervasive if not nearly universal in the making of professional music videos. Part of the nature of the beast.

Who said they weren't? Not me. We're trying to help Valentina make the best of what she has.

Also, professional performers of all musical flavors (classical, folk, r&b, rock, latin AND pop) deserve their due credit for professionality as well as jazz musicians, whether they fit your stylistic leanings or not.

I assume this pontification is aimed at the Bieber quip. No argument from me.

[budwzr]

Crop out the "instruments and fingering" and just show heads bobbing. In the really bad parts, cut to a shot of NewAwlins.

[newpball]

Crop out the "instruments and fingering" and just show heads bobbing.