QTGMC shimmer when deinterlacing a duplicate frame

[PRAGMA]

I am deinterlacing a 480i NTSC source with QTGMC that is interlaced on every frame (TFF).

clip = core.lsmas.LWLibavSource(r"L:\BDMVs\Tokyo.Mew.Mew.JPN.2xBD50.AVC.PCM2.0\Disc 1\DISC1_t12.mkv")
clip = QTGMC(clip, TFF=True, FPSDivisor=2, Preset="Placebo")
clip.set_output()

The output looks good, but it has a shimmer whenever it's doing a duplicate frame.


[Attachment 70373 - Click to enlarge]
(this is an APNG and should animate)

I understand that I should run TFM before deinterlacing to reduce the number of times deinterlacing is required.
However, I've begun to refrain from using TFM because it would often ignore not detect frames that had a tiny
amount of difference (very small combing), and then mark it as FieldBased=0 (progressive) and QTGMC then honors that.

For example, in the following image, it is in fact interlaced yet VFM has marked it as Combed==0, FieldBased==0.


[Attachment 70377 - Click to enlarge]

clip = core.vivtc.VFM(clip, order=1)
clip = core.text.FrameProps(clip)

While it is true that the 24fps sections of the source are seemingly being handled correctly, NTSC edits are happening over
some of the scenes. E.g., cross disolves, general overlays, e.t.c. This means if I run QTGMC it will skip some of the undetected frames
that it could not match.

One might think to just go through with FrameEval and set FieldBased back if VFMMatch is 0, but that won't work because
it sometimes matches with an interlaced frame.

I'm sure this has been brought up before but I have not found any threads about it.
I've attached the source I used. The scene example shown is 18s in (frame 553).

[poisondeathray]

It's QTGMC "wobbling" . Very common problem on progressive content, especially when QTGMC is used on anime and cartoons - because they often have many duplicate frames and pauses .

The underlying problem is QTGMC is temporal, and the motion vectors from mvtools2 are not perfect . You can reduce the severity of the artifacts somewhat by setting TR1=0, TR2=0. ie. reduce the temporal radius. But at the end of the day, QTGMC is a deinterlacer - so it interpolates from 1 field, instead of field matching both partner fields from a progressive frame. So QTGMC is always going to have some wobbling or shimmering issue compared to field matching (sometimes the severity might be acceptable)

I've begun to refrain from using TFM because it would often ignore not detect frames that had a tiny amount of difference (very small combing)

Did you play with the thresholds and settings ? For example a lower cthresh detects that frame above as interlaced (the interlaced overlay) and activates the deinterlacer (you can still specify QTGMC as the clip2 deinterlacer). TFM has more options than VFM, but I'm not sure if all the options have been ported to the vpy version

For relatively clean progressive content source like that , field matching is almost always going to be better than deinterlacing. Use TFM overrides if you have to

[PRAGMA]

I currently have it partially solved by just using VFM/TFM in the end and just ignore the few frames that are not detected. The frames I care about here are fine. But since some scenes/fades/e.t.c would use QTGMC, the issue technically does remain for when TFM cannot match.

[poisondeathray]

This example looks ok IVTCed . And adjusting the cthresh lower catches the combing . The scene fades do not have duplicates (the fade or overlays or transitions do not suddenly stop and repeat), so the "wobbling" issue with duplicates does not manifest there. I don't see an issue here

One issue on other anime is the transition/overlays can have 23.976p base content animation, but the overlay is 29.97p (or sometimes true interlaced, would be deinterlaced 59.94p) . If you use 23.976p the overlay would be choppy. If you use 59.94, the base animation has judder. You cannot satisfy both with typical frame rates .

[Selur]

Why use QTGMC on such a source at all?
Seems like a soft telecine source with a few interlaced frames.

clip = core.dgdecodenv.DGSource("....dgi",fieldop=1) # fieldop=1: Force Film
clip = core.tivtc.TFM(clip=clip)

seems to work fine,...
---------------

"If If you use 23.976p the overlay would be choppy."
then 60fps and live with duplicates might be best,..

[jagabo]

In AviSynth try:

function TFMBobN(clip v, int "pp", int "cthresh", int "MI")
{
    pp = default(pp, 6)
    cthresh=default(cthresh, 9)
    MI = default(MI, 80)

    n = nnedi3(v, field=-2, nns=4)
    ne = n.SelectEven()
    no = n.SelectOdd()

    v.GetParity() ? Interleave(TFM(v, field=1, clip2=ne, pp=pp, cthresh=cthresh, MI=MI), TFM(v, field=0, clip2=no, pp=pp, cthresh=cthresh, MI=MI)) : Interleave(TFM(v, field=0, clip2=no, pp=pp, cthresh=cthresh, MI=MI), TFM(v, field=1, clip2=ne, pp=pp, cthresh=cthresh, MI=MI))
}

TFMBobN(MI=20)

[Selur]

Here's also a VapourSynth version of that:

import vapoursynth as vs
core = vs.core

# port of AviSynth TFMBobN(..) by jagabo, see: https://forum.videohelp.com/threads/409273-QTGMC-shimmer-when-deinterlacing-a-duplicate-frame#post2687062
def TFMBobN(clip: vs.VideoNode,
            pp: int = 6,
            cthresh: int = 9,
            MI: int = 80,
            openCL: bool = False):
  field = clip.get_frame(0).props['_FieldBased']
  if openCL:
    n = core.nnedi3cl.NNEDI3CL(clip = clip, field=field+1,nns=4)
  else:
    n = core.nnedi3.nnedi3(clip=clip, field=fieldp1, nns=4)
  if field == 1: #BFF
    switchedField = 2
  else:  #TFF
    switchedField = 1
  odd = core.tivtc.TFM(clip=clip, field=switchedField, clip2=n[::2], PP=pp, cthresh=cthresh, MI=MI)
  even = core.tivtc.TFM(clip=clip, field=switchedField, clip2=n[1::2], PP=pp, cthresh=cthresh, MI=MI)
  return core.std.Interleave([odd,even])

which either uses znedi3 or nnedi3CL depending on whether openCL is set to False or True.

import TFMBobN
clip = TFMBobN.TFMBobN(clip,20)

Cu Selur

Ps.: also uploaded the script to https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/TFMBobN.py, btw. nnedi3 is also available for AviSynth.

[jagabo]

I also have TFMBobQ() which does the same except it uses QTGMC() instead of nnedi3 to generate clip2.

[Selur]

I also have TFMBobQ() which does the same except it uses QTGMC() instead of nnedi3 to generate clip2.

renamed the file and added TFMBobQ:

def TFMBobQ(clip: vs.VideoNode,
            pp: int = 6,
            cthresh: int = 9,
            MI: int = 80,
            openCL: bool = False):
  field = clip.get_frame(0).props['_FieldBased']
  q = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=(field == 2), opencl=openCL) 
  if field == 1: #BFF
    switchedField = 2
  else:  #TFF
    switchedField = 1
  odd = core.tivtc.TFM(clip=clip, field=switchedField, clip2=q[::2], PP=pp, cthresh=cthresh, MI=MI)
  even = core.tivtc.TFM(clip=clip, field=switchedField, clip2=q[1::2], PP=pp, cthresh=cthresh, MI=MI)
  return core.std.Interleave([odd,even])

see: https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/TFMBob.py

Cu Selur