Audio Sync Issue after Deinterlacing and Converting to MP4

[Bermuda1]

Hi everyone,

I have an Audio Sync where picture and audio are not in line after deinterlacing and converting to MP4 using Avisynth+ and VirtualDub2. As you can see in the video file, the stroke of the keyboard does not perfectly align with the sound. This is not the case in the original .avi file. Is there perhaps an issue with my Avisynth+ script or should I use a different Deinterlacing method?

This is the script I am using:

SetFilterMTMode ("QTGMC", 2)
    FFMPEGSource2("Kammermusik-short.avi", atrack=1)
    AssumeTFF()
    QTGMC(preset="Slower", EdiThreads=4)
     Crop(10,0,-10,-4)
    Prefetch(14)

Any help or ideas on how to fix this (ideally, by adjusting my script) would be greatly appreciated! Thanks!

Bermuda1

[lollo]

Use AviSource() for .avi files, not FFMPEGSource2

Eventualy use the original audio (which is untouched anyhow) in the final output, with AudioDub() command

[Bermuda1]

Use AviSource() for .avi files, not FFMPEGSource2

Eventualy use the original audio (which is untouched anyhow) in the final output, with AudioDub() command

Thanks! So I replace FFMPEG with AviSource as per below? But where do I out the AudioDub() command in the script? OK like this?:

SetFilterMTMode ("QTGMC", 2)
video = AviSource("Kammermusik-short.avi")
audio = AviSource("Kammermusik-short.avi", audio=true)
AssumeTFF()
video = QTGMC(video, preset="Slower", EdiThreads=4)
video = Crop(video, 10,0,-10,-4)
AudioDub(video, audio)
Prefetch(14)

[lollo]

It should not be necessary because audio is not processed in your flow, but you can do the following:

video = AviSource("Kammermusik-short.avi")

cropped = video.crop(10, 0, -10, -4)

deinterlaced = cropped.AssumeTFF().QTGMC(preset="Slower", EdiThreads=4)

AudioDub(deinterlaced, video )

[Bermuda1]

It should not be necessary because audio is not processed in your flow, but you can do the following:

video = AviSource("Kammermusik-short.avi")

cropped = video.crop(10, 0, -10, -4)

deinterlaced = cropped.AssumeTFF().QTGMC(preset="Slower", EdiThreads=4)

AudioDub(deinterlaced, video )

I was told I need to deinterlace before cropping (vertically). Can I just switch the commands around in the script you posted?

[lollo]

I was told I need to deinterlace before cropping (vertically). Can I just switch the commands around in the script you posted?

Technically the deinterlacer has an easier task if you remove the head switching noise and the black borders.

However, the advantage in term of quality (on the borders of the picture) and performances is marginal, and cropping interlaced materials must follow certain rules (mod-2 for width and height in YUY2 color space) http://avisynth.nl/index.php/Crop, so to avoid any user's wrong manipulation you often see in the scripts the crop after the deinterlacing.
I do the opposite, if you prefer your way it's fine as well.

[Bermuda1]

Deleted.

[Bermuda1]

Alright, thanks so far!
So this is my new script.

SetFilterMTMode("QTGMC", 2)
AviSource("Kammermusik-short.avi")
AssumeTFF()
QTGMC(preset="Slower", EdiThreads=4)
Crop(10, 0, -10, -4)
Prefetch(14)

I dont get the AudioDub part yet. I dont have another audio source other than in the one video. Do i still need it keep original quality of the audio? Thanks!

[Bermuda1]

So one more question. Is it to be expected that the outcome using avisource rather than ffmpeg2source is of worse quality? I uploaded a comparison. On the left hand side, avisource, right hand side is ffmpeg2source. I feel like left hand side has far more noise.

[davexnet]

So one more question. Is it to be expected that the outcome using avisource rather than ffmpeg2source is of worse quality? I uploaded a comparison. On the left hand side, avisource, right hand side is ffmpeg2source. I feel like left hand side has far more noise.

There's is always an explanation, perhaps you should post a 30 second sample of your source
for analysis

[manono]

Perhaps the original audio has a delay? In any event, it's no big deal to fix it. But no, your script has nothing to do with it.

[Sharc]

So one more question. Is it to be expected that the outcome using avisource rather than ffmpeg2source is of worse quality? I uploaded a comparison. On the left hand side, avisource, right hand side is ffmpeg2source. I feel like left hand side has far more noise.

Shouldn't be a difference. Looks like something went wrong with the avisource (left) variant. See the horizontal double edges, for example the piano:


[Attachment 82489 - Click to enlarge]

Also, there seems to be some luma/chroma crosstalk. Is your S-video cable properly screened, or is it actually a composite signal?
Are the 2 compared frames of the same type (I,P,B)?

[Bermuda1]

I uploaded two short clips - really seems like something went wrong with the Avisource, but what? All settings were otherwise the same..
@Manono: original video audio (interlaced huffyuv avi) is fine, perfect match.

Also, using Handbrake Interlacer Decomb EEDI 2 Bob works fine too - no delay in audio/video.

@Sharc: I used a regular S-Video cable, not RCA.

[Sharc]

Can you upload a few seconds of the unprocessed *.avi capture?

Edit:
Both above are basically ok, but:
- The vFFMPEG is single rate deinterlaced (25fps)
- The vAviSource is bob deinterlaced (50fps).

I would continue with Avisource or LWLibavVideoSource, rather than your FFMPEGSource2 (there are various versions of it floating around).

[Bermuda1]

See attached.

[Bermuda1]

Edit: ok, now the 2 versions above look better (same).

The screenshot I provided were from the same videos. However, I agree, it looks like its only the very first frame that is off in the avisource video. I attached another screenshot 1 second into the videos, and I must say, I dont see a large difference anymore. Would you agree?

[Bermuda1]

Can you upload a few seconds of the unprocessed *.avi capture?

Edit:
Both above are basically ok, but:
- The vFFMPEG is single rate deinterlaced (25fps)
- The vAviSource is bob deinterlaced (50fps).

Ok, thats strange. I used the exact same command in avisynth to do the deinterlacing.

QTGMC(preset="Slower", EdiThreads=4)

Does the selection of either "AviSource" or "FFMPEG2Source" impact whether its single rate deinterlacing or bob deinterlacing?! Any way I can change this?

[Sharc]

You would have to study the FFMEGSource2() filter parameters of your version of it. As I wrote there are various versions floating around and they don't behave the same. Stick to AviSource or LWLibavVideoSource.

[Bermuda1]

You would have to study the FFMEGSource2() filter parameters of your version of it. As I wrote there are various versions floating around and they don't behave the same. Stick to AviSource or LWLibavVideoSource.

Will do! Thanks!

Edit 1:
For anyone interested, this is the final Avisynth+ script I now use:

SetFilterMTMode("QTGMC", 2)
AviSource("Kammermusik.avi")
AssumeTFF()
QTGMC(preset="Slower", EdiThreads=4)
Crop(10, 0, -10, -4)
Prefetch(14)

Edit 2:
I found some sources that give the following explanation regarding the change in fps depending on the scripts used (+fix to audio video sync issue also when using ffmpegsource2).

The difference in frame rates is due to how QTGMC operates and the absence of a frame rate control in the script. By default, QTGMC doubles the frame rate of the input video. This is because it's designed to create new frames during the deinterlacing process, effectively doubling the temporal resolution. However, when using the FFMPEG script, I also used a BilinearResize operation. While this script also doesn't explicitly control the frame rate, the addition of BilinearResize operations might be inadvertently causing the frame rate to revert to the original (25fps). This could be due to how some filters interact with the frame rate in certain AviSynth setups.
It was then suggested to add "FPSDivisor=2 to the QTGMC function to halve the output frame rate.
I tried it and it also did the trick to syncing audio and video + the fps are now at 50.

[jagabo]

DelayAudio() in seconds, float. Negative to advance the audio, positive to delay. For example, DelayAudio(-0.25), to advance audio by 250 ms.

[Sharc]

See attached.

Doing a few quick tests with your .avi of post#15 with the following script

<Sourcefilter>
AssumeTFF()
AssumeFPS(25)
SeparateFields()

gave the following results for different <SourceFilter>

- AviSource(): Total length 250 fields with fields 4...9 jerky, rest ok
- LWLibavVideoSource(): Total length 242 fields, ok
- ffms2(): Total length 242 fields, ok (ffms2 is the shortform of FFMPEGSource2)

Not sure why AviSource() produced the glitches at the beginning.

[jagabo]

The AVI headers says the video is exactly 5 minutes long. That would indicate 525 frames at 25 fps. But there are only 121 frames. ffprobe shows the first frame has a timestamp of 0.0 seconds. The second frame has a timestamp of 0.2 seconds. So some source filters are repeating the first frame for 200 ms (ie the first frame becomes 5 frames). But other source filters ignore the timestamps and play for the duration of only 121 frames. All the other frames play at 40 ms per frame (1/25 fps).


[Attachment 82496 - Click to enlarge]

[Sharc]

So this initial glitch and duration vs number of frames inconsitency may have been caused by the trimming process for the .avi snippet, rather than beeing a true capturing flaw, I assume? (For this clip and using Avisource() the first few frames could be trimmed off, trim(5,0)).
The audio may have to be trimmed accordingly to maintain A/V sync; to be verified.

Edit:
@Bermuda1, off the records: The darks are elevated (left). I would reduce them by about 16 notches (right). The black piano should be close to Y=16, means just above the lower brown bar in the waveform monitor. You can do it in post processing.

[Attachment 82498 - Click to enlarge]

Re. waveform monitor see jagabo's tutorial here:
https://forum.videohelp.com/threads/340804-colorspace-conversation-elaboration#post2121568
In case you are using the histogram for proc amp or level adjustments don't be fooled by the black side (or top/bottom) bars of the capture. Crop them off for the histogram.

[jagabo]

So this initial glitch and duration vs number of frames inconsitency may have been caused by the trimming process for the .avi snippet, rather than beeing a true capturing flaw, I assume?

That's my guess.

[Bermuda1]

Edit:
@Bermuda1, off the records: The darks are elevated (left). I would reduce them by about 16 notches (right). The black piano should be close to Y=16, means just above the lower brown bar in the waveform monitor. You can do it in post processing.

Yes, good catch. I will adjust in post-processing. Thanks!

[Sharc]

Here deinterlaced, darks corrected and some denoised:

AviSource("Beethoven_Klaviersonate-00.00.00.000-00.00.05.000.avi").trim(5,0).converttoYV16(interlaced=true)
assumeTFF()
assumefps(25)
QTGMC(preset="fast")
crop(12,0,-12,-8)
Smoothlevels(16,1.0,235,0,235) #darks shifted
MCDegrainSharp() #denoising (add a filter(s) of your choice here)
converttoYV12()
#now encode x264 with -sar 12/11

There's still aliasing and dotcrawl to be fixed/avoided .....

[Bermuda1]

Here deinterlaced, darks corrected and some denoised:

AviSource("Beethoven_Klaviersonate-00.00.00.000-00.00.05.000.avi").trim(5,0).converttoYV16(interlaced=true)
assumeTFF()
assumefps(25)
QTGMC(preset="fast")
crop(12,0,-12,-8)
Smoothlevels(16,1.0,235,0,235) #darks shifted
MCDegrainSharp() #denoising (add a filter(s) of your choice here)
converttoYV12()
#now encode x264 with -sar 12/11

There's still aliasing and dotcrawl to be fixed/avoided .....

Looks great already!

I will try getting rid of the dotcrawl and aliasing by amending your script as follows:

AviSource("Beethoven_Klaviersonate-00.00.00.000-00.00.05.000.avi").trim(5,0).converttoYV16(interlaced=true)
assumeTFF()
assumefps(25)
QTGMC(preset="fast")
crop(12,0,-12,-8)
Smoothlevels(16,1.0,235,0,235) #darks shifted

# Add dot crawl reduction
DotKillT()

# Add anti-aliasing
nnedi3_rpow2(2, nns=4)

MCDegrainSharp() #denoising (add a filter(s) of your choice here)
converttoYV12()

# Add additional anti-aliasing
FastLineDarkenMOD()

#now encode x264 with -sar 12/11

[Sharc]

A clean S-video signal is free of dotcrawl actually (main reason for preferring S-video over Composite), so there shouldn't be anything to fix.
If your capture has dotcrawl (what I think it has) it is either because of
- your "S-video" is in fact a composite signal
- your S-video cable is poorly screened, means it produces crosstalk between the Y and C wires.
Leaves me with a questionmark - or maybe the noise I am seeing is not really dotcrawl .....

[Sharc]

A simple but effective means to remove dotcrawl is to downscale-upscale horizontally

AviSource("Beethoven_Klaviersonate-00.00.00.000-00.00.05.000.avi").trim(5,0).converttoYV16(interlaced=true)
assumeTFF()
assumefps(25)
QTGMC(preset="fast")
crop(12,0,-12,-8)
Smoothlevels(16,1.0,235,0,235)
bilinearresize(width/2,height).bicubicresize(width,height)
MCDegrainSharp()
converttoYV12()
#now encode x264 using -SAR 12/11

[Bermuda1]

@Sharc: I keep getting the error "there is no function named MCDegrainSharp". I used the script you provided here: https://forum.videohelp.com/threads/408462-Is-this-classic-VHS-tape-degradation-or-can...ing-else/page2

What am I missing?

Edit: It works now. All I had to do was amend the extension from .avs to .avsi