Best way/tools to sync audio to movement of lips?

[J-E-T-S]

Hi to all,

I am embarking on a pretty labor intensive audio editing project. I have two drama series. They are the same series but dubbed in different languages, and one has poor video quality while the other has decent video. I want to take the audio from the poorly ripped series and sync it to the series with the better quality video (replacing the voice portions and if possible, keep the other special effects portions).

How do I go about this and what software do I need? I want to match the voice to the movement of the actors' lips. I know I might not get it perfect but I want it to be as close as possible.

Also, I don't want to re-encode the video at all.

Please help.

[J-E-T-S]

Anybody?

How about a video editing program that lets me edit both the video and audio on a time line and simulate playback to see how it does before finalizing it? This will give me a better chance to sync the audio to the video.

[jman98]

Your task is so labor intensive that few would want to do it. However, the following recent thread has someone trying a somewhat similar task and might be of interest to you. I am assuming that your copies are not the same frame rate. If they are the same frame rate then this is easy. Just demux the audio from the bad one and add it to the good one.
https://forum.videohelp.com/threads/343911-Using-NTSC-%2816-9%29-and-PAL-%284-3%29-sour...restore-a-film

[manono]

I agree with jman98, but there's not near enough information here. It could be very easy, or it could be very hard.

If easy, demux the audio from the one with bad video, add it to the one with the good video. This assumes both videos have the same framerate and length (the number of frames is the same between the two).

In what format are these two videos, what framerate, what length. Please post the text portion of what MediaInfo shows for the two videos.

And I may have misunderstood, but if you're saying you want to keep parts of both audio tracks, I'd say forget about it.

[J-E-T-S]

Hi!

The videos have different frame rates and different quality. The time is different as well because one is more complete while the other has missing parts. I guess I will have to mux bits and pieces at a time, since muxing the whole DVD will result in lagging. One series has Chinese audio and good video quality (the most complete series I have acquired). The other series has Vietnamese audio and very poor video quality (missing parts, poor ripping, etc.). I want to keep the Vietnamese audio and mux it with the good video from the Chinese set.

I was thinking if I had a program with a timeline where I can see the video and hear the audio at the same time as I'm moving the audio back and forth to sync it correctly, that would help a lot. Otherwise, as is the case with my editing of mkv files, it is a matter of guess work and that can get rather cumbersome.

At the moment, all I have is Goldwave for simple audio cut/paste job and TMPGenc to mux it with the video. I need some better software. I don't know what programs to buy.

[poisondeathray]

The videos have different frame rates and different quality. The time is different as well because one is more complete while the other has missing parts.

That sounds like a lot of work

What you want/need is a video editor with a timeline e.g. sony vegas pro, premiere pro

But many common end distribution formats are not supported by "pro" editors. e.g. mkv, rmvb common to Asia=> those types of files won't be imported natively, you need to convert them to something else (it doesn't really matter quality wise for the video, because you're only doing it for the audio, so you can use something lossy)

Also, I don't want to re-encode the video at all.

Once it's synced up, you can export only the audio, and remux that into the original video

[dragonkeeper]

The complexity of this endeavor lies in the source material. Is the source material actually DVD rips, or tv rips?

If DVD Rips:
If the video streams have the same frame rate demux the streams then mux to desired video stram and desired video and audio stream together using the apropriate tool like MKVmerge or virtual dub. (These tools can also be used shift the audio to bring the audio in sync with the video). THis will get you close, but will probably not be perfect unless your an extremely lucky fellow.

IF TV Rips:
The process will become more difficult, due to the fact the video will have been trimmed to remove commercials. And the two streams will not be trimmed exactly the same. You can usually tell segments where a commercial was cut out. It may be better to work with the individual segments by cutting the video up into segments using a frame accurate tool like avidemux. Then work with these individual segments to sync the good video with the good audio track. Then use avidemux to joing the video segments back together.

If you new to sync'ing audio and video this will be a pita especially in the case of TVRips. If its something you wasnt to spend that king of time on I would suggest buying series and rip yourself. A hour long SD series takes about 30 to encode, an hour long BD series @ 720P takes about 45 min to encode on a reasonably powered PC . This may be faster than trying to sync a video stream to another audio stream.

[J-E-T-S]

[QUOTE=poisondeathray;2146804]

But many common end distribution formats are not supported by "pro" editors. e.g. mkv, rmvb common to Asia=> those types of files won't be imported natively, you need to convert them to something else (it doesn't really matter quality wise for the video, because you're only doing it for the audio, so you can use something lossy)

Both sets have been ripped from 80s VHS format to DVD format. I'm assuming Vegas Pro and Premier can import both? I think I will also need a a tool or feature that can measure the audio length I would need to cut and paste, like a cookie cutter used in baking (you can use it to make the same shape dough regardless of where you press the cookie cutter in).

Can either program open both sets at the same time so I can switch back and forth between the two sets? That would be helpful.

I would also want to normalize the audio in the final version so that it's the same volume throughout.

[J-E-T-S]

The complexity of this endeavor lies in the source material. Is the source material actually DVD rips, or tv rips?

If DVD Rips:
If the video streams have the same frame rate demux the streams then mux to desired video stram and desired video and audio stream together using the apropriate tool like MKVmerge or virtual dub. (These tools can also be used shift the audio to bring the audio in sync with the video). THis will get you close, but will probably not be perfect unless your an extremely lucky fellow.

IF TV Rips:
The process will become more difficult, due to the fact the video will have been trimmed to remove commercials. And the two streams will not be trimmed exactly the same. You can usually tell segments where a commercial was cut out. It may be better to work with the individual segments by cutting the video up into segments using a frame accurate tool like avidemux. Then work with these individual segments to sync the good video with the good audio track. Then use avidemux to joing the video segments back together.

If you new to sync'ing audio and video this will be a pita especially in the case of TVRips. If its something you wasnt to spend that king of time on I would suggest buying series and rip yourself. A hour long SD series takes about 30 to encode, an hour long BD series @ 720P takes about 45 min to encode on a reasonably powered PC . This may be faster than trying to sync a video stream to another audio stream.

This series was made back in the early 80s and is very hard to find. I have spent years looking online for a good Vietnamese set but this is the best I can find after years of searching: one Vietnamese set with poor video and one Chinese set with good video.

[poisondeathray]

yes it will work for dvd, but they won't accept vob, you have to rewrap to mpeg2-ps (no quality loss, it's mpeg2 to begin with, not re-encoding) .eg. vob2mpeg

[J-E-T-S]

So I'm guessing a good plan is to cut out one portion at a time from the DVD, cut/paste/sync the audio, and then stitch the synced parts together?

Is avidemux more accurate in it's cutting than Nandub or Virtualdub? Is there any way to get around the keyframe in a video so you get a precise cut?

[poisondeathray]

It doesn't matter for precise cuts in your project , because you are doing this for audio (you're only using the visual video as a reference for the audio) . You said you don't want to re-encode the video.

But to answer the question if you were doing this for another project you need a frame accurate editor that "smart renders" (only reencodes a few frames around the cutsite of a GOP) for mpeg2 it would be e.g. videoredo, solveigmm video splitter, womble dvd video wizard. Vegas can also smart render mpeg2

Does the "good video" version have all good video parts or missing parts ? ie. Does the "bad" video have sections of video that you want to include in the final project ? or is the "bad" version only there for audio only?>

[J-E-T-S]

The Vietnamese version has poor video so I don't want any of that video; I just want to keep the audio from the Vietnamese version and mux it with the good video from the Chinese version (99% complete with good video).

[poisondeathray]

Right, so you are discarding the "bad" video - you're only using it for audio purposes

Therefore it doesn't matter whether or not your clips are precisely cut or not, because you're doing fine tuning and syncing in the editor anyways, and you're only exporting audio once it's done

How you organize the project is up to you. You might import the whole thing as 1 audio clip then make sub clips or you might cut them first and then import them. There are many different ways you can approach this , but all of them sound like lots of work especially if scenes don't match .

eg. If there are scenes in the "good" video that have no corresponding part in the bad video to get the audio from, you need to do something creative, or at minimum you might have to keep some of the original audio in those sections

[J-E-T-S]

Yes, it will indeed be a lot of work and I'm not sure if I will succeed but I love this series and they don't sell it anymore. I'm going to try these programs and go from there. Thank you all for your input.

[poisondeathray]

Make sure you check the framerate - they should be 25 if from Asia and PAL spec. You can use gspot or mediainfo

If they are both 25, then you shouldn't have to globally alter the audio

But there can be other considerations - sometimes the DVD's might be processed in bizarre manner, might be shot on film and then field blended - just be aware there can be many other problems

manono asked this earlier:

In what format are these two videos, what framerate, what length. Please post the text portion of what MediaInfo shows for the two videos.

You should answer those questions, very important

[J-E-T-S]

manono asked this earlier:

In what format are these two videos, what framerate, what length. Please post the text portion of what MediaInfo shows for the two videos.

You should answer those questions, very important

The Chinese version will arrive soon but the Vietnamese version has the following info:

Format : MPEG-PS
File size : 1 024 MiB
Duration : 30mn 7s
Overall bit rate mode : Variable
Overall bit rate : 4 751 Kbps

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : M=3, N=15
Duration : 30mn 7s
Bit rate mode : Variable
Bit rate : 4 401 Kbps
Maximum bit rate : 6 125 Kbps
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.425
Stream size : 948 MiB (93%)
Color primaries : SMPTE 170M
Transfer characteristics : SMPTE 170M
Matrix coefficients : SMPTE 170M

Audio
ID : 189 (0xBD)-128 (0x80)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Muxing mode : DVD-Video
Duration : 30mn 7s
Bit rate mode : Constant
Bit rate : 256 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Delay relative to video : -67ms
Stream size : 55.2 MiB (5%)

[poisondeathray]

Well that' s NTSC spec , I thought Vietnam was a PAL country, but I guess it's NTSC
http://shop.sandbag.uk.com/OneGiantLeap/PALNTSCInfo.html

If it's a TV show , most likely film rate if it's a NTSC production, but it's possible it was shot at 25 FPS then converted to 29.97.

If by "Chinese" you mean Hong Kong or China, it should be PAL spec - this would mean you would probably have to speedup the audio of the NTSC vietnamese audio to match the before even starting

If you are in the USA, most North American DVD players cannot play PAL spec DVD's - so might have to re-encode and reformat the video

But all this is just guessing, you need to post back when you have info on the other version

[J-E-T-S]

The Vietnamese version was ripped in the United States if that matters, so it's NTSC. I will post again after I get the other version. Thank you.