QTGMC Stutter and Replaying Previous Frames

[Skiller]

Have you tried tweaking MPEG2Source?


Like so...

Mpeg2Source("VTS_Full.d2v",Info=3, cpu2="xxxxoo", moderate_h=20, moderate_v=40)

20 and 40 are the defaults.

Make sure the DGDecode version you are using is doing anything at all in terms of deblocking by setting moderate_h and moderate_v to a very low value such as 1. It should blur a lot. Then tweak the two settings to your liking.
And remember, badly damaged fields and frames such as the one on the first page will never look good again.

[nikkmann]

Is there a version of x64 mpegsource that still has the deblocking feature?

[jagabo]

Or try using deblock_qed which works for both 32 bit and 64 bit AviSynth:

# deblock QED for interlaced frames

function Deblock_QED_i ( clip clp, int "quant1", int "quant2", int "aOff1", int "bOff1", int "aOff2", int "bOff2", int "uv" )
{
    quant1 = default( quant1, 24 ) # Strength of block edge deblocking
    quant2 = default( quant2, 26 ) # Strength of block internal deblocking

    aOff1 = default( aOff1, 1 ) # halfway "sensitivity" and halfway a strength modifier for borders
    aOff2 = default( aOff2, 1 ) # halfway "sensitivity" and halfway a strength modifier for block interiors
    bOff1 = default( bOff1, 2 ) # "sensitivity to detect blocking" for borders
    bOff2 = default( bOff2, 2 ) # "sensitivity to detect blocking" for block interiors

    uv    = default( uv, 3 )    # u=3 -> use proposed method for chroma deblocking
                                # u=2 -> no chroma deblocking at all (fastest method)
                                # u=1|-1 -> directly use chroma debl. from the normal|strong deblock()

    last=clp
    par=getparity()
    SeparateFields().PointResize(width,height)
    Deblock_QED(last, quant1, quant2, aOff1, aOff2, bOff1, bOff2, uv)
    AssumeFrameBased()
    SeparateFields()
    Merge(SelectEven(),SelectOdd())
    par ? AssumeTFF() : AssumeBFF()
    Weave() 
}


Mpeg2Source("VTS_Full.d2v",Info=3)
deblock_qed_i(quant1=30, quant2=30)
TFM(d2v="VTS_Full.d2v")
TDecimate(Cycle=6,CycleR=1)
nnedi3_rpow2(rfactor=2,qual=2,pscrn=4,fapprox=0,fwidth=1440,fheight-1080,cshift="Spline36Resize")
Prefetch(22)

From what I've seen the worst of the blocking is in the intro sequence. The main body of the episodes don't look bad. You might use strong deblocking on the intro, weak deblocking on the rest. I'd add a dehalo filter and mild noise reduction after decimating and before upscaling. Maybe:

FineDehalo(rx=2.0, ry=2.0)
SMDegrain (tr=3, thSAD=200, refinemotion=true, contrasharp=false, PreFilter=4, mode=0, truemotion=true, plane=4, chroma=false)

And some sharpening after upscaling:

aWarpSharp2(depth=10)
CAS(0.5)

And since you're upscaling convert to rec.709 colors (early in the processing):

ColorMatrix(mode="rec.601->rec.709")

[TimA-C]

Just for completeness (is that a word?) don't forget about BlindPP(), which is a standalone part of DGDecode, and gives access to (I presume) the same de-ringing and de-blocking filters as when using MPeg2Source() but without having to use MPeg2Source().

You can find it's info here: https://www.rationalqm.us/dgmpgdec/DGDecodeManual.html

[nikkmann]

@jagabo,
Weird, my video doesn't look as clean as yours even though I used the same settings you pasted here. Its an improvement but smdegrain seems to cause purple blocky artifacting where it removes grain, unless I just have it in a bad place in my script.
Either way, it still doesn't come out nearly as clear as when I use VEAI. With that I'm able to completely remove all artifacting and noise, but it looks like it over-smooths details when I also try to upscale in VEAI. I might just render out the 4k video in avisynth using your settings (minus smdegrain) and then use VEAI to de-artifact it.
Here's my script if you see anything wrong with it (commented out smdegrain):

Mpeg2Source("VTS_Full.d2v",Info=3)
TFM(d2v="VTS_Full.d2v")
TDecimate(Cycle=6,CycleR=1)
ColorMatrix(mode="rec.601->rec.709")
deblock_qed(quant1=30, quant2=30)
FineDehalo(rx=2.0, ry=2.0)
#SMDegrain (tr=3,thSAD=200, refinemotion=true, contrasharp=false, PreFilter=4, mode=0, truemotion=true, plane=4, chroma=false)
nnedi3_rpow2(rfactor=4,qual=2,pscrn=4,fapprox=0,fw idth=2880,fheight=2160,cshift="Spline36Resize")
aWarpSharp2(depth=10)
CAS(0.5)
Prefetch(22)

[jagabo]

When reducing noise it's always a balance between getting rid of noise and losing detail or creating artifacts. I did notice a few places where the chroma was ghosting with smdegrain. You can play around with the tr (temporal radius, reducing it to 2 gives fewer chroma artifcts but also less effective noise reduction) and thSAD (denoising strength). You can degrain only the luma by changing plane=4 to plane=0. Or use different settings for the chroma and luma

luma = smdegrain(... plane=0)
chroma = smdegrain(... plane=3)
MergeChroma(luma, chroma)

Or use some other noise reducer.

[nikkmann]

yeah changing the plane to 0 fixed it.

Also, whats the point of converting the colors to rec 709? Is there some benefit to it/is there something wrong with the colorspace the original dvd uses?

[jagabo]

Also, whats the point of converting the colors to rec 709? Is there some benefit to it/is there something wrong with the colorspace the original dvd uses?

Some players always use rec.601 colors, some always 709 (mostly older players). Some players will use always use rec.601 for SD, always rec.709 for HD. Some players will use the flagged colors. So your best bet for getting the right colors is to use rec.601 for SD, rec.709 for HD and flag the colors. Then only the players that always use rec.601 will give the wrong colors.

[nikkmann]

Something I noticed with the de-telecining using TFM in Avisynth+, every couple of frames you'll get a "detelecined" frame with very obvious zig zagging on moving objects that used to be telescined (look at his body and head in the window).

[Attachment 66048 - Click to enlarge]

But, when playing the original video back through VLC player and using its automatic de-telescine feature, yes it adds extra frames, but the de-telescined frames do not have the same zig zagging issues as AVisnyth+

[Attachment 66049 - Click to enlarge]

How can I get rid of that on avisynth?

[rrats]

i've had the same problem. the way i solved it was by inserting it into virtualdub2, then bobbing it with proper field order and reinterlacing it.

[jagabo]

After field matching TFM examines the frame for any residual combing (this can be caused by various source issues). It then deinterlaces only those areas. Some types of picture content can look like combing and you want to avoid deinterlacing those. So it uses a threshold value to determine if deinterlacing is necessary. The default threhold of 9 sometimes misses very light combing as in that example. You can lower the threshold to catch residual combing using the cthresh variable:

TFM(cthresh=5)

You can provide your own deinterlaced frame for this too.

TFM(cthresh=5, clip2=QTGMC(FPSDivisor=2))

Using QTGMC here usually doesn't slow the processing down much because it's only used when residual combing is detected. Pixels from clip2 are only used in the areas that need deinterlacing.

[nikkmann]

Even after putting that in it still looks exactly the same. Even tried lowering cthresh down to 2 and didn't help. Here is my script:

Mpeg2Source("VTS_Full.d2v",Info=3)
TFM(d2v="VTS_Full.d2v",cthresh=5, clip2=QTGMC(FPSDivisor=2))
TDecimate(Cycle=6,CycleR=1)
ColorMatrix(mode="rec.601->rec.709")
deblock_qed(quant1=30,quant2=30)
nnedi3_rpow2(rfactor=4,qual=2,pscrn=4,fapprox=0,fw idth=2880,fheight=2160,cshift="Spline36Resize")
aWarpSharp2(depth=10)
CAS(0.5)
Prefetch(22)

[poisondeathray]

One way is to add post processing such as vinverse, or vinverse2 after TDecimate

You VLC screenshot looks like blend deinterlacing. Basically blur 2 fields together. This reduces artifacts, but reduces details as well. You can choose to blend deinterlace in avisynth too, and decimate the duplicates.

[nikkmann]

One way is to add post processing such as vinverse, or vinverse2 after TDecimate

You VLC screenshot looks like blend deinterlacing. Basically blur 2 fields together. This reduces artifacts, but reduces details as well. You can choose to blend deinterlace in avisynth too, and decimate the duplicates.

Is it possible to use this on telescined footage? Which script/plugin should I use for it?

[poisondeathray]

One way is to add post processing such as vinverse, or vinverse2 after TDecimate

You VLC screenshot looks like blend deinterlacing. Basically blur 2 fields together. This reduces artifacts, but reduces details as well. You can choose to blend deinterlace in avisynth too, and decimate the duplicates.

Is it possible to use this on telescined footage? Which script/plugin should I use for it?

Are you referring to vinverse ?
vinverse is for filtering residual combing after inverse telecine (in your script, directly after TDecimate)

http://avisynth.nl/index.php/Vinverse

[nikkmann]

Ah yep, so that fixed that issue. But something else I noticed. In VLC, for the duplicate frames caused by de-telescining, there is usually one really bad (lots of artifacting, blocking) looking duplicate and one normal looking duplicate (in-line with how the non-telescined portions of the movie look).

However, in Avisynth with TFM, it seems to only pull out the really bad looking de-telescined frames.

It looks like the movie uses 3 normal frames, then 3 telescined frames, and keeps alternating in that manner.

So say I have 3 telecined frames 1, 2, 3.

In VLC, they will be de-telescined with duplicates so, 1, 1, 2, 2, 3, 3, with the first duplicate being really bad quality and the second one being normal/good quality.

In Avisynth with TFM, you just get 1, 2, 3 but one or two of these frames will end up being the really bad quality ones.

Kind of hard to explain, but any ideas why this happens? And are there any other de-telescining plugins for Avisynth besides TFM?

[poisondeathray]

In general, decimation algorithms retain the most different frames, and discard the most similar duplicate frames .

But on poor or less clean sources, a "bad" frame with artifacts and noise will "look" more different than surrounding "good" frames. A decimation algorithm will tend to selectively choose the "bad" frame because it looks like a more "unique" frame when compared to it's neighbors

[nikkmann]

In general, decimation algorithms retain the most different frames, and discard the most similar duplicate frames .

On "normal" or clean sources, you want to discard the duplicates

But on less clean sources, a "bad" frame with artifacts and noise will "look" different than surrounding "good" frames. A decimation algorithm will selectively choose the "bad" frame because it looks like a more "unique" frame when compared to it's neighbors

That makes sense. I guess VLC just doesn't discard any frames. Anyway to fix this in TFM? Somehow tell it to keep the other frame, or not delete duplicates at all and have decimate handle all of that?

[jagabo]

Yes, TFM refuses to eliminate the combing in that frame (and others) unless you set cthresh to 0. But that has a very bad effect on the rest of the frame.

An alternative to vinverse() is Santiag(1,0).

You can use TFM(field=0) (the default is 1) to match the alternate field. But you will probably find that in some cases field=1 is cleaner, in other field=0 is cleaner. So you'll just end getting bad frames in different places.

[nikkmann]

Yes, TFM refuses to eliminate the combing in that frame (and others) unless you set cthresh to 0. But that has a very bad effect on the rest of the frame.

An alternative to vinverse() is Santiag(1,0).

You can use TFM(field=0) (the default is 1) to match the alternate field. But you will probably find that in some cases field=1 is cleaner, in other field=0 is cleaner. So you'll just end getting bad frames in different places.

Between Santiag and vinverse, which is better?

And from the last few comments above, do you know of a way to fix the TFM frame quality issue?

[jagabo]

Between Santiag and vinverse, which is better?

Santiag is meant for antialiasing. It was by accident I found it was very effective at eliminating small comb artifacts. vinverse smooths flat areas more. Santiag(1,0) smooths edges more. I think I'd choose Santiag here.

And from the last few comments above, do you know of a way to fix the TFM frame quality issue?

Not off the top of my head. It might be possible to compare the output of TFM(field=0) and TFM(field=1) and pick the frame with the least "noise". But the results of the two are not always the same film frame. Do you have some frame numbers where you saw this problem? So I don't have to search through the entire video...

[nikkmann]

Frame 28665 from VOB 1 (and the few frames after it) when using the below script. Like I said, these have duplicates when using the de-interlace mode in VLC that look much better:

Mpeg2Source("VTS_Full.d2v",Info=3)
TFM(d2v="VTS_Full.d2v",slow=0,pp=0)
vinverse()
TDecimate(Cycle=6,CycleR=1)
ColorMatrix(mode="rec.601->rec.709")
deblock_qed(quant1=30,quant2=30)
nnedi3_rpow2(rfactor=4,qual=2,pscrn=4,fapprox=0,fw idth=2880,fheight=2160,cshift="Spline36Resize")
aWarpSharp2(depth=10)
CAS(0.5)
Prefetch(22)

[nikkmann]

Any luck with it?

[jagabo]

I tried several things but nothing has help much yet. I'll try a few more things tomorrow...

[jagabo]

Try removing pp=0 from your TFM call. And changing slow to 1 or 2.

[nikkmann]

I actually didn't have those in there before and it looked slightly worse. Slow 1 (and 2) sometimes cause duplicate frames every 100 frames or so whereas slow 0 cause entirely unique frames with some horizontal telescine artifacts easily removed by vinverse.

And I just threw pp=0 in there because I didn't want any post-processing done since I was going to deblock it in another program after, but honestly I didn't notice a difference with it on or off.

Did those manage to help you?

[jagabo]

Oops, sorry. Most of the problems with slow=0 and pp=0 were caused by field numbers that I added to the video before TFM (to make it easier to see which fields were chosen). Back to the drawing board...

[nikkmann]

Gotcha. Thanks so much for doing all this. I wonder if this might be an issue not just with my movie, but something thats been caused for other movies as well that people just didn't know about.

[jagabo]

Using your script (minus the colormatrix and deblocking calls) I get this as frame 28665:


[Attachment 66187 - Click to enlarge]

Is that the frame you're talking about? When I encode the VOB with VLC I get an blend deinterlaced frame there.

Another thought: TFM matches fields. If one has more artifacts it will still use it. VLC uses a deinterlacer so it might not use the bad field (or may use only the bad field at times).

[nikkmann]

Yep, just to be clear here is what I'm talking about.

The image below is my output from Avisynth. You can see alot of, I guess you could call it chroma telecining, where the colors (in the lava for example) shift between red and green in horizontal lines.


[Attachment 66188 - Click to enlarge]

Now, for the same frame, when using VLC's automatic de-interlace mode, I get 2 duplicate frames here. The first one, shown below, is absolutely terrible quality.


[Attachment 66189 - Click to enlarge]

But the second duplicate frame, shown below, seems to be the same one that Avisynth produces, but without the "chroma" telescining, so it ends up looking cleaner.


[Attachment 66190 - Click to enlarge]

I want the output from Avisynth to be the same as the second duplicate frame from VLC (the last image in the post), and eliminate that residual "chroma" telescining, because it comes out looking alot better after upscaling in VEAI.