ResizersPack,MasksPack,PlaygroundPack, SmoothContrast, Logo & mod functions

[Dogway]

Sorry hello_hello, I've been having internet issues since the 19th, the whole day yesterday.

I don't have the version I fixed for you, since I didn't need it. So if you have it around pass it along to see all the changes I did perform.

I see problems with the newer versions you posted, at a glance there are some potential lsb clips that under yuy2 are treated the same as non-lsb (8-bit) clips. This is a known problem for several reasons, for example you will get artifacts at the bottom of the frame. To process them correctly you need to do some plane rearrangements that I included in SMDegrain as helper functions.

work fine here

ColorBars(width=720, height=480, pixel_type="rgb32")
ConvertToYUY2()
AddGrainC()
Interleaved2Planar
super_search = MSuper(rfilter=4, planar=true)

bv2 = super_search.MAnalyse(isb = true,  delta = 2, overlap= 4)
bv1 = super_search.MAnalyse(isb = true,  delta = 1, overlap= 4)
fv1 = super_search.MAnalyse(isb = false, delta = 1, overlap= 4)
fv2 = super_search.MAnalyse(isb = false, delta = 2, overlap= 4)

MDegrain2(MSuper(levels=1, planar=true), bv1, fv1, bv2, fv2, thSAD=300, thSADC=150, lsb=true, planar=true)
Planar2Interleaved
ConvertToYV16
DitherPost(mode=6)
ConvertToYUY2

If you employ ConvertToYV16 (haven't thoroughly tested) it might work but this is v2.6 exclusive. It's the same as your ConvertToYV16 mod in SMDegrain, it's either avs26 or the script craps together (not failsafe).

Also now that I'm at it syntaxis is wrong, add brackets to every function otherwise the script is prone to fail, memory issues (tested before).

[real.finder]

If you employ ConvertToYV16 (haven't thoroughly tested) it might work but this is v2.6 exclusive. It's the same as your ConvertToYV16 mod in SMDegrain, it's either avs26 or the script craps together (not failsafe).

Also now that I'm at it syntaxis is wrong, add brackets to every function otherwise the script is prone to fail, memory issues (tested before).

yes the new QTGMC for AVS26 only

about the brackets it was in avs mt only http://forum.doom9.org/showpost.php?p=1526746&postcount=292

and get fixed here http://forum.doom9.org/showpost.php?p=1568121&postcount=353

anyway, if I did another changes in the future I will add it to every function

[LouieChuckyMerry]

Hi Dogway, I hope you're well . I want to check out your modded QTGMC in progressive mode and compare it's denoising to SMDegrain on some older, very grainy sources, but I'm not sure if I'm using the "Lsb" feature correctly (I've been spoiled by SMDegrain ). My usual SMDegrain script is (feeding to 10-bit x264):

SMDegrain(TR=DependsOnSource,ThSAD=DependsOnSource,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
Dither_Out()

so would the QTGMC version be:

QTGMC(InputType=1,EZDenoise=DependsOnSource,ChromaMotion=False,Lsb=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
Dither_Out()

or would I need to send 16-bit to QTGMC:

Dither_Convert_8_To_16
QTGMC(InputType=1,EZDenoise=DependsOnSource,ChromaMotion=False,Lsb=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
Dither_Out()

I read through the documentation and the .avsi but I didn't find an answer (although it's probably there). Thanks for your help.

[Dogway]

I only use QTGMC's denoiser when I have to deinterlace, because QTGMC+External Denoiser harms the source too much. Actually my opinion is that QTGMC antialias too much, or to put it in another way the output has a very artificial touch (even on progressive mode). Another take is to recover all the lost grain from QTGMC (check for the settings) and then denoise with SMDegrain.

lsb in QTGMC is just for the internal processing, it neither inputs or outputs 16-bit, it only adds lsb=true to mdegrain (and dfttest if you enable lsbd=true) calls internally, then afterwards it's dithered to error diffusion so QTGMC works as it was supposed in 8-bit, you only gain some tiny detail... how much? I don't know.

[LouieChuckyMerry]

As usual I think I wasn't very clear, my apologies. I don't want to use QTGMC plus an external denoiser, I just want to replace the SMDegrain line in Post #63 with QTGMC in progressive mode (only denoising, no deinterlacing) to compare the results on some 1930-50's sources that are quite grainy, as I've read claims that QTGMC as only a denoiser does a reasonably good job of stabilizing the output on very grainy sources.

What I don't understand is how to use QTGMC to its fullest potential in the above script. My memory (ha ha ha) tells me that the SMDegrain call in Post #63 converts the source to 16-bit (Lsb=True) then passes the output in 16-bit (Lsb_Out=True), where it's converted to 8-bit for FastLineDarkenMod then reconverted to 16-bit for GradFun3 (Lsb_In=True) and passed as 16-bit (Lsb=True) to 10-bit x264. Since you mention that your QTGMC mod neither inputs nor outputs 16-bit (thanks for that information), would it be of any benefit to feed 16-bit input to QTGMC or would that be pointless? That is, which of these methods, in your opinion, would be more efficient-give better results:

Source
Dither_Convert_8_To_16
QTGMC(InputType=1,DenoisingSettingsTestedLater,Lsb=True,Lsbd=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
Dither_Out()

or

Source
QTGMC(InputType=1,DenoisingSettingsTestedLater,Lsb=True,Lsbd=True)
FastLineDarkenMod()
Dither_Convert_8_To_16
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
Dither_Out()

Or is there an even better way? Thanks again.

[Dogway]

I understood you the first time. I only stated that I don't like QTGMC denoising, and that I only use that feature when I have to deinterlace either way.

As I said QTGMC neither inputs or outputs 16-bit so you can't feed 16-bit input to QTGMC, you only gain a minimal detail depth when you enable lsb, but it is an internal process hidden from the user.

The second example is the correct one, but as I said, I wouldn't denoise with QTGMC unless you like this type of artificial finish.

[LouieChuckyMerry]

Thanks for the clarification and advice, Dogway. I've not found any denoiser as good as SMDegrain (not even close) from an ease of use-speed-quality of output standpoint, but as I dig into the sources for my oldest movies I'm finding that their lack of consistency noise-wise on a scene-to-scene basis (due to the use of different film stocks for different lighting scenarios?) is causing me to pick the middle ground with regards to ThSAD, so that some scenes are undercleaned, some scenes are overcleaned, and some scenes are just-right cleaned (I feel like Goldilocks, ha ha). This isn't a problem with modern movies since they typically have a "consistency of appearance" such that a well-chosen ThSAD leaves them looking uniformly excellent. Anyway, as I mentioned above, I remember reading claims that QTGMC does reasonably well with these older, inconsistent sources so I thought I'd check it out. Thanks again for the scripting help . Perhaps I'll become a master of Trim() when QTGMC proves lacking as a denoiser .

On a related note, do you have any plans for a special version of SMDegrain, maybe something like SMDegrainLPF, where you run an analysis pass to generate a statistics file measuring luma per frame (lpf) then use that .stats file for the encoding pass such that SMDegrainLPF uses a per-frame ThSAD (and-or other settings) based on each frames luma value? That would be awesome.

[Dogway]

There's no shortcut to manual scene based trimmings. The reason you might like QTGMC is because it has an internal prefilter, what this does is level out grainy scenes and clean scenes before motion analysis, where SAD discretion is evaluated. So there won't be more or less denoised frames when denoising is performed. In QTGMC this is a dumb approach, a stuck strong prefilter, strong denoising, and strong antialias (detail destruction).

You can do the same but at a finer level with SMDegrain, use a prefilter of your choice or one of the presets, prefilter=4 (KNLmeans) being a good candidate for grainy sources, and play with the other settings. You might like QTGMC because it destroys mosquito noise, but that's only a side-effect of the strong anti-alias. As I explained the best approach IMO is to do all your spatial denoising and masking work before and feed this as prefilter and let SMDegrain do the fine job.

On the dynamic thSAD I have studied the option but for different reasons, I dabbled with the idea to apply a 'dynamic' dynamic range expansion prefilter as an addition to the current fixed TV->PC luma expansion, this benefits of better detail preservation specially on low contrast areas. This requires probably dynamic thSAD but I don't want to use runtime filters... (nor run analysis passes of any sorts). I have checked RT_Stats as an option... still unsure of what to do.

[LouieChuckyMerry]

There's no shortcut to manual scene based trimmings. The reason you might like QTGMC is because it has an internal prefilter, what this does is level out grainy scenes and clean scenes before motion analysis, where SAD discretion is evaluated. So there won't be more or less denoised frames when denoising is performed. In QTGMC this is a dumb approach, a stuck strong prefilter, strong denoising, and strong antialias (detail destruction).

Happy Friday! and I really appreciate the explanation, thanks. And now I know why QTGMC is sooo slooow...

You can do the same but at a finer level with SMDegrain, use a prefilter of your choice or one of the presets, prefilter=4 (KNLmeans) being a good candidate for grainy sources, and play with the other settings. You might like QTGMC because it destroys mosquito noise, but that's only a side-effect of the strong anti-alias. As I explained the best approach IMO is to do all your spatial denoising and masking work before and feed this as prefilter and let SMDegrain do the fine job.

Is that how prefilters work? I've never really checked them out because my usual +/- 3.5 fps for HD material gets cut in half . However, maybe that's the answer for these older sources so I'll runs some tests (and dream of an actual desktop setup with an Octo-Core® processor ).

On the dynamic thSAD I have studied the option but for different reasons, I dabbled with the idea to apply a 'dynamic' dynamic range expansion prefilter as an addition to the current fixed TV->PC luma expansion, this benefits of better detail preservation specially on low contrast areas. This requires probably dynamic thSAD but I don't want to use runtime filters... (nor run analysis passes of any sorts). I have checked RT_Stats as an option... still unsure of what to do.

You could always do this as something separate from SMDegrain, but with SMDegrain as its core, so that SMDegrain is still a super-functional, self-contained death denoiser and SMDegrainDR (DynamicRange, or some such) is its own beast. That way if it eventually required runtime filters or an analysis pass it wouldn't diminish SMDegrain in any way. Just a thought.

[LouieChuckyMerry]

Happy Friday, Dogway, I hope you've been well . I'm attacking my Futurama DVD's (S1.E1-FuturamaTestClip) and had questions about LinearResize and FastLineDarkenMod. Currently my script is:

LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\DGIndexNV\DGDecodeNV.dll")
DGSource("SourcePath.dgi")
### Deinterlace-Match Fields-Decimate ###
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\avisynth_plugin\TIVTC.dll")
AssumeTFF()
TFM(Chroma=False,PP=0) 
AssumeBFF()
TFM(Chroma=False) 
TDecimate()
### Adjust Color ###
SmoothTweak(Saturation=0.9)
### Crop ###
Crop(8,0,-8,0)
### Overall Temporal Denoise ###
SMDegrain(TR=1,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,Lsb=True,Lsb_Out=True)
### Resize ###
LinearResize(640,480,Kernel="Spline64",Lsb_In=True,Lsb_Out=True)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=20,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=3,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Deband ###
GradFun3(Radius=16,SMode=2,StaticNoise=True,Lsb_In=True,Lsb=True)
### Preview Source OR Send 16-bit Output To x264 10-bit ###
# Trim()
# DitherPost()
Dither_Out()

I'm wondering:

1) Is there a way to improve the sharpness of the resize? Reading the LinearResize.avsi led me to "Spline64", which from what I've read, is (technically) better than "Spline36", but perhaps you know an even better way?

2) The FastLineDarkenMod-aWarpSharp line I copied from the script for my Simpsons DVD's that you helped me with ages ago, so I'm sure it can be improved for the different Futurama source. I've read the FastLineDarkenMod.avsi and the aWarpSharp Wiki but I just can't seem to wrap my head around all the settings and how they combine with each other, and, more importantly, how the FastLineDarkenMod and aWarpSharp filters work in conjunction. Any enlightenment would be greatly appreciated.

3) I'm pretty happy with the rest of the script, but please feel free to critique it given the test clip; I'm certain that you've forgotten more about video encoding than I actually know .

Thanks for your time.

[Dogway]

In practice spline64 has almost invisible improvement for a greater performance hit, so stick to spline36. Or, in case you are downscaling (not sure what your source resolution is) you can use a trick suggested by Didée using bicubic and b=-.5,c=.25. If that's not enough you might want to sharpen independently.

As for FastLineDarkenMod, it depends whether you perceive lineart a bit bland or not. After all the filtering it mostly needs a bit of punch, so be careful with strength. I use aWarpSharp2 for those sources that are very blurry, because sharpeners can't do good enough without creating heavy ringing, so adding some clever and subtle aWarpSharp2 can improve the overall sharpeness, but it's easy to overdo so be subtle again here.

Overall looks good, I will have it a look later if I have time. Have a good one!

[LouieChuckyMerry]

They're NTSC DVD's so the source resolution is 720x480(4:3). Thanks for the information about Spline64 vs Spine36 (I couldn't see a difference frame-to-frame on a couple tests I ran, now I know why ), and I'll try Didée's trick ASAP. The sources do seem a bit blurry, at least to my eyes, and I'd really like to correct that if possible. It would be awesome if you have the time to look; I'm sure you'd improve the FastLineDarkenMod-aWarpSharp2 settings if you did. Thanks again and Happy Saturday!

Edit: I'm having problems with the BicubicResize line, sorting out the 8 bit-16 bit details, and I also can't figure out how to use Dither_Resize16 with BicubicResize as the kernel (not yet, at least), but I found Debicubic, which has an Lsb_InOut setting and its "Parameters and sampling pattern try to mimic Avisynth's BicubicResize." Using the line:

Debicubic(Target_Width=640,Target_Height=480,B=0.5,C=0.25,Lsb_InOut=True)

makes a remarkable difference in quality with respect to the blurriness. Thank you! Now to figure out Dither_Resize16...

EditEdit: I'm a bit slow, but I finally realized that I can use LinearResize with BicubicResize thusly:

LinearResize(640,480,Kernel="Bicubic",A1=0.5,A2=0.25,Lsb_In=True,Lsb_Out=True)

to maintain the 16 bit chain.

EditEditEdit: I ran the test clip with the above LinearResize with Kernel="Bicubic" and it looks, frame-by-frame, exactly like the results with LinearResize and the default Spline36, which is rather blurry compared to the results with Debicubic. Honestly, it looks like BicubicResize wasn't even used. Now I'm more confused than usual . Most likely I did something incorrectly...

[Dogway]

Mind you, minus "b" (a1=-.5). It's sharper than spline36, although very slightly. You can also use the fh, and fv settings, but in my experience it leads to aliasing, so I would refrain from using it.
For me it's rather sharp already so you would have to play with some settings, maybe turn contrasharp to 50, or 60 instead of the default internal contrasharp.

The source looks good, it has a bit of gibbs noise so I would use what I suggested before for the simpsons, use before smdegrain.

Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(KNLMeansCL(D=0, A=1, h=7.0,lsb_inout=false),Mask,Luma=True)

[LouieChuckyMerry]

Ahhh, thanks for pointing out the minus; I didn't notice that, and it certainly makes it sharper than without it. If you're interested, I ran the test clip with the exact same script (no Gibb's noise block, though) except for the resizer, using LinearResize with Bicubic as the kernel, Dither_Resize16 with Bicubic as the kernel, and Debicubic; the "-" and "+" indicate whether the "B" (or "A1") is positive or negative. One of them stands out to me, and I'd be very curious for your opinion as to which looks best; the six screenshots are here.

Unfortunately I can't use KNLMeansCL due to OpenCL compiler errors (I've tried to sort it on the KNLMeansCL thread at Doom9 but eventually deemed it not worth the effort because your SMDegrain is vastly superior as a denoiser anyway). Would you then recommend the following:

### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(DFTTest(),Mask,Luma=True)

or is there a better (faster ) method?

Lastly--no no, really--would you suggest any tweaks to the FastLineDarkenMod-aWarpSharp2 line to improve the results for Futurama, as it's different from the Simpsons source and I just copied that line from your recommendation for The Simpsons? The seemingly endless settings combinations left my brain a bit softer than usual, but I'm sure there's improvement to be had. Thanks again, Dogway .

[Dogway]

I found "Linear(Bicubic)-" the best balanced. Debicubics had some nasty ringing and aliasing (oversharpening).

In your example I didn't spot any gibbs noise, so whatever you did stick with it. If you want at any point use the gibbs denoise block, I recommend something like knlmeans. Dfttest is not quite like that since it's a frequency domain denoiser. You can use minblur which is more similar, or better yet "Dither_bilateral16" (test the different wrappers, etc).

I can't recommend you anything for the FastLineDarkenMod-aWarpSharp2, that's down to very personal tastes. I wouldn't touch it, maybe lowering the "blur" setting a little bit, but it's very neat-picking.

[LouieChuckyMerry]

I agree with your assessments: Linear(Bicubic)- is definitely the winner.

As for the lack of Gibbs noise, all credit goes to SMDegrain; I used the same script I posted in #70 except for the different resizers (and upping the SmoothTweak to 1.1). I'm quite happy about that because the encoding speed will more than double, from ~2.5 fps to ~5.5 fps. Maybe I'll go bother Khanattila again and try to finally sort out using KNLMeansCL. And I'll run a few test with a lower Blur just to see the difference.

As always, many thanks for your kind help, Dogway, I really appreciate it . Be well.