floating chroma shift issue?

[cinephil]

1x_BleedOut_Compact_300k_net_g.onnx is placed in folder next to mlrt_ncnn.dll. Not sure if it's not like the placing or the file itself.

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\ffms\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi", fpsnum=30000, fpsden=1001, threads=1, colorspace="YUV420P8")
ConvertBits(32)
ConvertToPlanarRGB()
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\mlrt_ncnn.dll")
mlrt_ncnn(network_path="1x_BleedOut_Compact_300k_net_g.onnx")
# assuming 8bit 4:2:0 output
ConvertBits(8)
ConverttoYv12()

[Attachment 81984 - Click to enlarge]

[Selur]

Have you tried using an absolute path?
Or create a 'models' folder next to the dll and put the model in it?

[poisondeathray]

If you specify a path that is not in the same location as the plugin , use builtin=False

- builtin\
    Whether the models are in the same location with the plugin.\
    Default: True.

I use absolute path, with builtin=False for the avs version

[cinephil]

It's loading with 'models' folder next to the dll BIG thanks, Selur!
Now will try it out.
And one more question about this "model" thing - is there any parameters can be adjusted, or it only works as is?

[poisondeathray]

And one more question about this "model" thing - is there any parameters can be adjusted, or it only works as is?

You cannot change the output of a specific model directly. You can only change the model (or train a different one)

Indirectly, you can composite versions (eg. overlay with opacity %, perhaps with original, or different versions) , and or use pre/post filtering to modulate the outcome

[Selur]

Also be aware that most of these models are spatial only filters (no temporal influence), so depending on input and the model used you can get temporal instabilities.

[cinephil]

Did someone run in this behaviour of MeGui(x64) - that it don't show in the preview window actual colors from the script? In MeGui(x32) I was able to see picture that correspond to the output file, but in MeGui(x64) preview and output don't match anymore

[Attachment 81989 - Click to enlarge]

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi")
ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0) #adjust gamma and contrast
Tweak(bright=16, coring=false)  #adjust brightness
Tweak(Sat=1.2, startHue=80, endHue=135) # "red" pixels (boost up)
Tweak(Sat=1.2, startHue=325, endHue=15) # "blue" pixels (boost up)
ConvertBits(8)
ConvertToYV12()

Also FFMS2(x64) acts kind of different from what it was in AviSynth2.5.8(x32).

In AviSynth2.5.8(x32) I runned script like this:

LoadPlugin("M:\edit\MeGUI-2944-32\tools\ffms\ffms2.dll")
FFVideoSource("M:\edit\SAMPLE\video\05-5D[JP]+.avi")
ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0) #adjust gamma and contrast
Tweak(bright=16, coring=false)  #adjust brightness
Tweak(Sat=1.2, startHue=80, endHue=135) # "red" pixels (boost up)
Tweak(Sat=1.2, startHue=325, endHue=15) # "blue" pixels (boost up)

and have correct preview and no warnings that color space incorrect.

But with MeGUI-6666.2308_x64 and AviSynthPlus 20230715 3.7.3 if I run internal FFMS2 from AVS script creator, then I getting this in the script:

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\ffms\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi", fpsnum=30000, fpsden=1001, threads=1, colorspace="YUV420P8")

And if I use external plugin, like this:

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi")

then is ask-

[Attachment 81990 - Click to enlarge]

But niether way preview don't match end result.

[jagabo]

The difference between the two images is a matter of luma levels: full vs. limited. Check to see if the videos have the levels flagged. Check to see if the encoders are flagging the levels. And whether the players/editors are following the levels flag.

[cinephil]

...following the levels flag...

It seems like nowhere, not in the video, not in the encoder or player is not set(or forced) levels flag.
--------------
But anyway I was able to make some test with BleedOut_Compact_300k and MergeChroma/aWarpSharp2.

MergeChroma/aWarpSharp2 - works fast and result looks good.

AddAutoloadDir("M:\edit\MeGUI-2944-32\tools\avs\plugins")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi")
ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0) #adjust gamma and contrast
Tweak(bright=16, coring=false)  #adjust brightness
Tweak(Sat=1.2, startHue=80, endHue=135) # "red" pixels (boost up)
Tweak(Sat=1.2, startHue=325, endHue=15) # "blue" pixels (boost up)
ConvertToYV12()
FFT3DFilter(sigma=0.3, plane=0, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.6, interlaced=true) # Y - noise reduction+sharp
FFT3DFilter(sigma=0.7, plane=3, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.9, interlaced=true) #UV - noise reduction+sharp
SeparateFields() 
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

[Attachment 82043 - Click to enlarge]

[Attachment 82044 - Click to enlarge]

[Attachment 82045 - Click to enlarge]

BleedOut_Compact_300k - works slow (like 3 times slower), but result looks better. Although the picture has some tint (bluish?) on it (especially noticeable in the end titles).

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\fft3dfilter.dll")
FFT3DFilter(sigma=0.3, plane=0, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.6, interlaced=true) # Y - noise reduction+sharp
FFT3DFilter(sigma=0.7, plane=3, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.9, interlaced=true) #UV - noise reduction+sharp
ConvertBits(32)
ConvertToPlanarRGB()
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\mlrt_ncnn.dll")
mlrt_ncnn(network_path="1x_BleedOut_Compact_300k_net_g.onnx")
ConvertBits(8) # assuming 8bit 4:2:0 output
ConverttoYv12() # assuming 8bit 4:2:0 output
ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0) #adjust gamma and contrast
Tweak(bright=16, coring=false)  #adjust brightness
Tweak(Sat=1.2, startHue=80, endHue=135) # "red" pixels (boost up)
Tweak(Sat=1.1, startHue=325, endHue=15) # "blue" pixels (boost up)

[Attachment 82047 - Click to enlarge]

[Attachment 82046 - Click to enlarge]

[Attachment 82048 - Click to enlarge]

There is still some ghosts left. But fighting with them - guess I will lose more picture details than gain.
----------------------

And more to the colors -> Is it typical that Histogram looks like this after YUV->RGB->YUV conversion? Don't I lose too much information?

YUV (after MergeChroma/aWarpSharp2 script)

[Attachment 82050 - Click to enlarge]
YUV->RGB->YUV (after BleedOut_Compact_300k script)

[Attachment 82051 - Click to enlarge]

[poisondeathray]

BleedOut_Compact_300k - works slow (like 3 times slower), but result looks better. Although the picture has some tint (bluish?) on it (especially noticeable in the end titles).

It's quite common for some models to sometimes shift the colors slightly. You can make small adjustment

And more to the colors -> Is it typical that Histogram looks like this after YUV->RGB->YUV conversion? Don't I lose too much information?

Banding in histogram (y waveform) is from manipulation of levels in 8bit . The 1st mergechroma script has banding too after only coloryuv/tweak are used and there are breaks in the histogram. The difference is because FFT3DFilter is applied afterwards and smooths the result (if you comment out the FFT3DFilter(s) , you will see similar banding result in the waveform in the mergechroma script)

What some some people do are color correct/manipulate a higher bit depths , dither on the down conversion



Also, I would IVTC beforehand for progressive filters like avs-mlrt. You can re-interlace later if you need to . The quality difference might be minimal on this type of lower quality source but there would be significant differences visible on higher quality sources. Similarly, the YUV/RGB conversions are mixed up between progressive and interlaced. e.g. ConvertToYV12 should be interlaced=true , if interlaced. Same with RGB conversions with interlaced=true vs. false. Or else you will be prone to chroma artifact errors, including chroma ghosting, notching artifacts (less visible on low quality sources)

[poisondeathray]

Or else you will be prone to chroma artifact errors, including chroma ghosting, notching artifacts (less visible on low quality sources)

e.g.

No IVTC , progressive RGB conversion + Bleedout, (IVTC after for display screenshot). Notice the chroma notched ghosting artifacts on the dropped cup. Basically anything with motion will have these type of issues because the chroma is treated as progressive, when it's actually still in organized in fields (interlaced)

[Attachment 82053 - Click to enlarge]

IVTC + Bleedout

[Attachment 82054 - Click to enlarge]

[cinephil]

It's quite common for some models to sometimes shift the colors slightly. You can make small adjustment

I kinda suck with colors... I manage the picture look somewhat good with ColorYUV/Tweak. But how to get rid of that tint and don't ruin everything that was in good balance?

Banding in histogram (y waveform) is from manipulation of levels in 8bit . The 1st mergechroma script has banding too after only coloryuv/tweak are used and there are breaks in the histogram. The difference is because FFT3DFilter is applied afterwards and smooths the result (if you comment out the FFT3DFilter(s) , you will see similar banding result in the waveform in the mergechroma script)

What some some people do are color correct/manipulate a higher bit depths , dither on the down conversion

I see...

The problem is - if I make ColorYUV/Tweak before convert to RGB, then the result looks like as if I didn't do it at all.

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\VR.5\SAMPLE\video\05-5D[JP]+.avi")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\fft3dfilter.dll")
ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0) #adjust gamma and contrast
Tweak(bright=16, coring=false)  #adjust brightness
Tweak(Sat=1.2, startHue=80, endHue=135) # "red" pixels (boost up)
Tweak(Sat=1.1, startHue=325, endHue=15) # "blue" pixels (boost up)
ConvertBits(32)
ConvertToPlanarRGB(interlaced=true)
SeparateFields() 
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\mlrt_ncnn.dll")
mlrt_ncnn(network_path="1x_BleedOut_Compact_300k_net_g.onnx")
Weave()
ConvertBits(8) # assuming 8bit 4:2:0 output
ConverttoYv12(interlaced=true) # assuming 8bit 4:2:0 output
FFT3DFilter(sigma=0.3, plane=0, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.6, interlaced=true) # Y - noise reduction+sharp
FFT3DFilter(sigma=0.7, plane=3, bw=16, bh=16, ow=8, oh=8, ncpu=4, sharpen=0.9, interlaced=true) #UV - noise reduction+sharp

[Attachment 82086 - Click to enlarge]

Also, I would IVTC beforehand for progressive filters like avs-mlrt. You can re-interlace later if you need to . The quality difference might be minimal on this type of lower quality source but there would be significant differences visible on higher quality sources. Similarly, the YUV/RGB conversions are mixed up between progressive and interlaced. e.g. ConvertToYV12 should be interlaced=true , if interlaced. Same with RGB conversions with interlaced=true vs. false. Or else you will be prone to chroma artifact errors, including chroma ghosting, notching artifacts (less visible on low quality sources)

I didn't pay attention to it. Thanks!

Video had some parts with true 29.97fps (without 3:2 pull down). In this example - the first part (with computer generated images) without pull down, and the second part is with pull down. Don't know if I managed to save it correctly with all this conversions and SeparateFields/Weave process.

[Attachment 82087 - Click to enlarge]
[Attachment 82088 - Click to enlarge]