Video stutters after deinterlacing (QTGMC)

[Name]

Hi all,

after capturing my first VHS, I wanted to deinterlace it with QTGMC, but the result is not appealing. The video stutters and the audio is out of sync.

VHS capture:
I captured the VHS with VirtualDub 1.9.11 in combination with an JVC S9500 and GV-USB2.
Video codec: HuffYUV (fastest)
Audio codec: PCM

The capture worked without any issue. I got 0 frames dropped and around 400 inserted over 40 min.

The AVI file is playing without any issue and the audio is in sync.


QTGMC:
Regarding the deinterlacing process, I used this project package:
https://forum.videohelp.com/threads/405720-FFmpeg-QTGMC-Easy%21

I adjusted the qtgmc.avs like this:

SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
#Define file name
filename = "D:\test.avi"

#QTGMC portable script
#Tested with [url=https://www.videohelp.com/software/Avisynth]Avisynth[/url]+ 3.7.2(r3661) 64 bits and [url=https://www.videohelp.com/software/ffmpeg]FFMPEG[/url] 5.1.2-full_build-www.gyan.dev 2022-09-26 
#To know more about AviSynth go to : [url]http://avisynth.nl/[/url]
#To know more about FFmpeg go to : [url]https://ffmpeg.org/[/url]

LoadPlugin("FFT3DFilter.dll")
#FFmpegSource v2.40 plugin : [url]http://avisynth.nl/index.php/FFMS2[/url]
#To know more bout loading video files in Avisynth : [url]http://avisynth.nl/index.php/FAQ_loading_clips[/url]
LoadPlugin("ffms2.dll")
#MaskTools2 v2.2.30 plugin : avisynth.nl/index.php/Masktools2
LoadPlugin("masktools2.dll")
#Rgtools v1.2 plugin : [url]http://avisynth.nl/index.php/RgTools[/url]
LoadPlugin("Rgtools.dll")
#MVTools v2.7.45 with depans v20210608 plugin : [url]http://avisynth.nl/index.php/MVTools[/url]
LoadPlugin("mvtools2.dll")
#Nnedi3 v0.9.4.60 Plugin : [url]http://avisynth.nl/index.php/Nnedi3[/url]
LoadPlugin("nnedi3.dll")
#Zs_RF_Shared v1.159 script : [url]http://avisynth.nl/index.php/Zs_RF_Shared[/url]
Import("Zs_RF_Shared.avsi")
#QTGMC v3.383s script : [url]http://avisynth.nl/index.php/QTGMC[/url]
Import("QTGMC.avsi")

#Opening of video file (the video file opened here is "in.mp4" and it has NO audio tack, you must replace "in.mp4" by the name of your video file)
#FFMpegSource2(filename)
#Opening of video file (the video file opened here is "in.mp4" and it as an audio track, you must replace "in.mp4" by the name of your video file)
FFMpegSource2(filename,atrack=1)

#Converion to YV12 color format: [url]http://avisynth.nl/index.php/ConvertToYV12[/url]
ConvertToYV12()
#Applying QTGMC in faster mode
QTGMC(Preset="Placebo")
# Enable MT using 16 cores (logical, not physical)!
SetFilterMTMode("QTGMC", 2)
Prefetch(15)

These are the lines I added:

• SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
• LoadPlugin("FFT3DFilter.dll")
• SetFilterMTMode("QTGMC", 2)
• Prefetch(15)

FFMPEG:
Finally, I used this FFMPEG command to run QTGMC:

.\ffmpeg -i "D:\QTGMC\qtgmc.avs" -c:v libx264 -preset veryslow -qp 0 -c:a pcm_s16le -aspect 4:3 output.mkv


Samples:

VHS capture: https://drive.google.com/file/d/1aVPHdJ-Z2KkOX8tujAbrNuERr054cymO

QTGMC: https://drive.google.com/file/d/1BRFZgjNWp6EeTykZabnt6pnxum655idN


The final QTGMC video stutters and the audio is out of sync.

What did I do wrong?

[Bernix]

qp 0? what bitrate do you get? try use -crf 18 instead if you can

[lollo]

The capture worked without any issue. I got 0 frames dropped and around 400 inserted over 40 min.

That's an issue! The rest comes later...

[Name]

qp 0? what bitrate do you get? try use -crf 18 instead if you can

I want to keep it as lossless as possible. Thats why I choose 0 and not any crf value.

AVI:

Format/Info                              : Audio Video Interleave
Format settings                          : PcmWaveformat
File size                                : 505 MiB
Duration                                 : 1 min 0 s
Overall bit rate                         : 70.5 Mb/s
Frame rate                               : 25.000 FPS
Writing application                      : Lavf61.1.100

Video
ID                                       : 0
Format                                   : [url=https://www.videohelp.com/software/HuffYUV]HuffYUV[/url]
Format version                           : Version 2
Codec ID                                 : HFYU
Duration                                 : 1 min 0 s
Bit rate                                 : 69.0 Mb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 5:4
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : Interlaced
Bits/(Pixel*Frame)                       : 6.652
Stream size                              : 494 MiB (98%)

Audio
ID                                       : 1
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : 1
Duration                                 : 1 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 1 536 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 11.0 MiB (2%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 17  ms (0.42 video frame)
Interleave, preload duration             : 42  ms

MKV:

Format                                   : Matroska
Format version                           : Version 4
File size                                : 538 MiB
Duration                                 : 1 min 0 s
Overall bit rate                         : 75.1 Mb/s
Frame rate                               : 49.068 FPS
Writing application                      : Lavf59.27.100
Writing library                          : Lavf59.27.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:4:4 Predictive@L4
Format settings                          : CABAC / 16 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 16 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 min 0 s
Bit rate                                 : 72.1 Mb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 49.068 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 3.542
Stream size                              : 516 MiB (96%)
Writing library                          : x264 core 164 r3099 e067ab0
Encoding settings                        : cabac=1 / ref=16 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=9 / psy=0 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=0 / threads=18 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc=cqp / mbtree=0 / qp=0
Default                                  : No
Forced                                   : No

Audio
ID                                       : 2
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : A_PCM/INT/LIT
Duration                                 : 1 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 1 536 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 11.0 MiB (2%)
Writing library                          : Lavc59.37.100 pcm_s16le
Default                                  : No
Forced                                   : No

That's an issue! The rest comes later...

But how can I prevent it? I even see the frame drops when watching the VHS normally. I assume the tape has an issue.

[lollo]

That's an issue! The rest comes later...

But how can I prevent it? I even see the frame drops when watching the VHS normally. I assume the tape has an issue.

If the inserted/dropped frames do not depend on external environment but on tape condition, the only hope is to fix a potential time base problem with an external TBC. If there is a physical damage on the tape, there is no solution.

In the sample you posted there are 28 inserted frames. You can navigate across them opening the file in VirtualDub and using the menu "Go -> Prev drop frame" and "Next drop frame" to check their consistency against what you see when playing the tape.

[Bernix]

crf 0 is lossless for 8 bit videos as well as cq 0. To the last bit But in any other case you should always prefer crf over cq. CQ is wastes of space.

[Name]

Maybe I didn't explain it well enough. I don't have the feeling that the video is stuttering like a game with low FPS.

When I watch the interlaced video, it gives me a "smooth" feeling and is comfortable to watch.

But when checking the deinterlaced version, it feels junky, like something would be off.
For example, when the people are leaving the bus (00:01) or when the guy is holding the blue strings (00:22).

This happens especially during motion.

Is it possible that the deinterlacing is putting together the wrong line pairs?

[lollo]

Is it possible that the deinterlacing is putting together the wrong line pairs?

Did you place AssumeTFF() in your script?

[Name]

This was the solution. No, AssumeTFF() was not added. Now I added it and it looks fine.

QTGMC with TFF: https://drive.google.com/file/d/1OqVKAeORIfOaxGEgGps1Mw1hRZFdYjYu

CQ is wastes of space.

Not saying that you are wrong, but in my small 1 min test video, there was no difference in file size between cq 0 and crf 0.

The difference I noticed was between AssumeBFF() and AssumeTFF().


What do you think about my capture and the final deinterlaced output?
Any feedback which I could take into account?

[davexnet]

What do you think about my capture and the final deinterlaced output?
Any feedback which I could take into account?

Something wrong with your link?

[Name]

Small typo. Now it should work.

[jagabo]

n my small 1 min test video, there was no difference in file size between cq 0 and crf 0.

That is correct. When you specify cfr 0 x264 automatically switches to cqp mode with a quantizer of 0. You can see it in the metadata (near the end of the line).

[Name]

The only problem I still couldn't fix is the audio part.

The AVI file has the video and audio in sync.
After deinterlacing, the audio is off.

Even when I mux the deinterlaced video stream and the AVI audio stream via mkvToolNix, in the end the audio is off again.

Any idea what I'm missing here?

[lollo]

The only problem I still couldn't fix is the audio part.

The AVI file has the video and audio in sync.
After deinterlacing, the audio is off.

It makes no sense (even if I doubt that with 400 inserted frames the a/v are in synch in the captured file).

Start with a simple

AviSource("D:\test.avi")
AssumeTFF()
QTGMC()

(no FFMpegSource2, no MT, no prefetch) and test

[jagabo]

Verify the number of frames before and after QTGMC. There should be twice as many after. Check the frame rate before and after. It should be twice as much after. Twice as many frames at twice the frame rate should be exactly the same running time.

[Name]

AviSource("D:\test.avi")

Using AviSource solved it.

I had the same issue like here, FFmpegSource2 works incorrectly with HuffYUV.

I guess it is done. I transferred my first VHS.

[Name]

If the inserted/dropped frames do not depend on external environment but on tape condition, the only hope is to fix a potential time base problem with an external TBC. If there is a physical damage on the tape, there is no solution.

I would like to come back to the dropped frames.

1) Can I find out what was causing the frame drops? When performing the capture, I can see that the CPU utilization is at around 20 % and at a certain point the preview window starts to stutter, and the dropped frames counter increases. My assumption was it is related to the tape itself.

2) How is VirtualDub handling dropped frames? At the beginning, I thought dropped frames will the compensated by adding duplicated frames to keep it at constant 25 FPS.
But when I checked the timing settings, I saw the option to adjust the audio to it. Which I can hear in the capture when I reach the problematic sections. The video stutters and the audio changes the pitch.

Are my VirtualDub settings fine?

[Attachment 84715 - Click to enlarge]

[lollo]

Your VirtualDub settings look ok; you may want to enable "Automatically disable ..." to experiment.

Dropping frames depends on tape conditions and/or on capturing (hardware/software) conditions.

The first if fixable only with an external TBC.

For the second you can test Vdub settings, fine tune your system (disable video preview while capturing, no internet, no antivirus, no processes running in background, capturing to a dedicated disk, etc; i do not even touch the mouse while capturing), explore other software (AmarecTV, VirtualVCR, GraphEdit).

In your sample you suffered of inserted frames, not dropped frames, which is even more difficult to debug, and which can be related/unrelated to frame drops.
Sometimes it depends on the tape (I found on one tape exactely the same frame inserted at the same exact timing across 5 different captures), sometime not...

[jagabo]

Disable audio playback in virtualdub while capturing. Also, don't compress audio while capturing.

[Name]

Thanks for all the input.

Sometimes it depends on the tape (I found on one tape exactely the same frame inserted at the same exact timing across 5 different captures), sometime not...

This sounds interesting. Because I noticed the stutter always at the same spot of the tape when I did some test captures.
This would already indicate that it is tape related, or not?

I never worked on the PC while I was capturing, so it would be strange that this issue always appeared at the same spot by chance.

[lollo]

This sounds interesting. Because I noticed the stutter always at the same spot of the tape when I did some test captures.
This would already indicate that it is tape related, or not?

yes

[Name]

Your VirtualDub settings look ok; you may want to enable "Automatically disable ..." to experiment.

If I check "Automatically disable resync when integrated audio/video capture is detected", does this conflict with the "Resync mode" from above, and should it be adjusted too?

[lollo]

No, it has the priority