Help With Suitable Settings For hqdn3d Filter

[meeshu]

I had a script that had good settings for hqdn3d but have since misplaced it! I've tried adjusting some of settings, but have yet to find the best yet.

To save me time in testing literally hundreds if not thousands of setting combinations, would someone suggest best settings to use with the hqdn3d filter to clean up color banding from the enclosed video clip, please?

Thank you.

https://u.pcloud.link/publink/show?code=XZ5zqiVZuLgufLMPTBX5iwGFw5A8GY3YLfp7

[jagabo]

You need multiple HQDN3D calls to reduce that chroma banding/flickering. It may lead to chroma blurring in some shots.

LWLibavVideoSource("LR_color_banding.avi") 
ConvertToYV12()
HQDN3D(ls=0.0, cs=0.0, lt=5.0, ct=5.0, y=3, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)

[meeshu]

Thank you very much for the comment!

I suspected that additional passes may be required.

Thanks for the sample script as well!

I'll run further tests using your script as a base, and make changes to settings as required.

[jagabo]

Note that those setting are specific to that clip. They are not general purpose settings.

[meeshu]

Note that those setting are specific to that clip. They are not general purpose settings.

Thank you.

I have noticed that the original video where the sample clip was taken from does not appear to be successfully processed in removing the color banding!

I'll try to upload another sample clip from the original video, and maybe it can then be seen why the original video is not apparently being processed properly.

Screenshot from original video showing that color banding is not being removed (completely).

Original video on the left, processed video on the right of the screenshot.



[Attachment 74373 - Click to enlarge]

This is the script I'm using -

SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
AviSource("F:\Liver Run Mk2.avi")
ConvertToYV12(interlaced=false, matrix="Rec601")
A=last

HQDN3D(ls=0.0, cs=0.0, lt=5.0, ct=5.0, y=3, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)
HQDN3D(ls=0.0, cs=0.0, lt=0.0, ct=5.0, y=2, u=3, v=3, mt=true)
AssumeFPS(30000,1001)
PreFetch(4)
B=last

StackHorizontal(A, B)

[jagabo]

Note that multi-frame temporal filters can take several frames to fully kick in. So the first several frames of the clip, and last several frames of the clip, won't have as good results as the middle frames.

And as I said, that sequence is specific to the clip you provided. If you have another sequence where it doesn't work well you'll have to supply a sample.

[meeshu]

Thanks.

So I'll delete the SetFilterMTMode command(?).

Here is another sample clip taken from the original video.

https://u.pcloud.link/publink/show?code=XZF0UiVZvWtcd7I20w0wI1Ar1T4X28gdwfqX

[Selur]

I think https://forum.videohelp.com/threads/409421-Removing-Color-Banding-(Moire-Pattern-) was where this source was discussed before,...

Cu Selur

[jagabo]

Here is another sample clip taken from the original video.

https://u.pcloud.link/publink/show?code=XZF0UiVZvWtcd7I20w0wI1Ar1T4X28gdwfqX

The same filter sequence will work but you'll need to crank each ct up to 10. Though there is more blurring of the chroma at the edges of his face when his head moves. Reducing the number of calls to HQDN3D will reduce that blurring but will also reduce the effectiveness of the color stripe removal.

An Alternate is to use just the first call to HQDN3D but crank ct up to 50.

Using a motion compensated temporal noise reduction filter instead might work better. SMDegrain for example:

MergeChroma(last, SMDegrain(thsad=3000, tr=4, PreFilter=2, plane=3))

[Selur]

MergeChroma(last, SMDegrain(thsad=3000, tr=4, PreFilter=2, plane=3))

Why the MergeChroma?
I mean, shouldn't plane=3 already restrict SMDeGrain to only filtering the chroma? (3 - both chromas)

[jagabo]

I originally used two SMDegrain's with different settings, one for luma one for chroma. When I replaced the one for luma I forgot to remove the MergeChroma.

[Selur]

Ah, okay, just wanted to make sure I wasn't missing something.

[meeshu]

I think https://forum.videohelp.com/threads/409421-Removing-Color-Banding-(Moire-Pattern-) was where this source was discussed before,...

Cu Selur

Correct! This is basically from the same original video. But I had lost the modified script, and forgotten what settings I modified/used.

The current script I'm testing with (for actual processing/dubbing) is -

AviSource("F:\Liver Run Mk2.avi")
ConvertToYV12(interlaced=false, matrix="Rec601")
hqdn3d(ls=0.00,cs=0.00,lt=4.00,ct=13.00)
hqdn3d(ls=0.00,cs=0.00,lt=4.00,ct=13.00)
hqdn3d(ls=0.00,cs=0.00,lt=2.00,ct=7.00)
AssumeFPS(30000,1001)
PreFetch(4)

Return last

I don't know about the y, u, v parameters within hqdn3d function, so I didn't use them. The temporal values I chose were largely guesswork but partly based on other script settings.

The resulting processed video clip seemed to be fairly good (to me) with good removal of color banding.

https://u.pcloud.link/publink/show?code=XZMhaiVZ6jr72zaIvvFyRCergm9r6S4w63yX

[meeshu]

Here is another sample clip taken from the original video.

https://u.pcloud.link/publink/show?code=XZF0UiVZvWtcd7I20w0wI1Ar1T4X28gdwfqX

The same filter sequence will work but you'll need to crank each ct up to 10. Though there is more blurring of the chroma at the edges of his face when his head moves. Reducing the number of calls to HQDN3D will reduce that blurring but will also reduce the effectiveness of the color stripe removal.

An Alternate is to use just the first call to HQDN3D but crank ct up to 50.

Using a motion compensated temporal noise reduction filter instead might work better. SMDegrain for example:

MergeChroma(last, SMDegrain(thsad=3000, tr=4, PreFilter=2, plane=3))

As mentioned in my previous post I modified the script by increasing the ct parameters from hqdnd3. The resulting processed clip appeared to quite good, but a bit blurry.

Now, tried to test using SMDegrain, but got error message -


[Attachment 74387 - Click to enlarge]

Using this script -

AviSource("F:\LR_color_banding.avi")
ConvertToYV12(interlaced=false, matrix="Rec601")
A=last

SMDegrain(thsad=3000, tr=4, PreFilter=2, plane=3)
PreFetch(4)
B=last

StackHorizontal(A, B)

What is wrong with the SMDegrain command here?

[jagabo]

It's probably a version problem. Either of SMDegrain or one of its support filters. I've tried SMDegrain 4.5.0d and 3.1.2d. Both worked. What version are you using?

[meeshu]

According to the text header of SMDegrain.avsi, it is version 3.1.2d.

[jagabo]

line 879 of SMDegrain 3.1.2d reads:

    IsAvsNeo ? eval(MidStr(VersionString(),20,4)) : IsAvsPlus ? eval(MidStr(VersionString(),17,4)) : 0

Are you using AviSynth+? Or an older AviSynth?

[meeshu]

line 879 of SMDegrain 3.1.2d reads:

    IsAvsNeo ? eval(MidStr(VersionString(),20,4)) : IsAvsPlus ? eval(MidStr(VersionString(),17,4)) : 0

Are you using AviSynth+? Or an older AviSynth?

I'm using AviSynth+ version 3.7.3 from this downloaded file - AviSynthPlus_3.7.3_20230715_vcredist

Please note that I'm currently having some difficulty with my computer, and I may not be able to do further testing for a day or so(?)

[meeshu]

Back to this slightly older topic. Still can't get SMDegrain to run without error.

Line 879 of the script SMDegrain.avsi reads -

IsAvsNeo ? eval(MidStr(VersionString(),20,4)) : IsAvsPlus ? eval(MidStr(VersionString(),17,4)) : 0

and line 75 of the script SMDegrain.avsi reads -

sisphbd = AvsPlusVersionNumber > 2294

Screenshot of SMDegrain script header -


[Attachment 77081 - Click to enlarge]

Running AviSynth+ version 3.7.3


Any further ideas on how to get SMDegrain to run?

EDIT: please refer to post #14 for the error message when trying to run this script.