Porting ChubbyRain2 To Vapoursynth

[The 14th Doctor]

Apologies if this is wrong thread or I sound like I'm begging here but I'm stumped on how to handle this since I have no python nor real coding experience.

Is there anybody out there that would know how to port ChubbyRain2 to Vapoursynth? Or at the very least, know a way to load the avsi file into vapoursynth? I only know how to load the required dll files ChubbyRain needs to use.

Alas this isn't a dll, otherwise I would have known what to do. DeCross works great as a Derainbow filter but it doesn't catch them all, I'd rather use it and ChubbyRain2 to deal with rainbows while not sucking out/changing the original colors of a video which other filters like regular Derainbow and RainbowSmooth do.


Thanks.

[Selur]

You can't load an Avisynth script directly inside a Vapoursynth script.

Here's a !quick and untested! rewrite (base on the http://avisynth.nl/images/ChubbyRain2.avsi version)

# requires: 
#   BiFrost https://github.com/dubhatervapoursynth/vapoursynth-bifrost
#   Temporal soften: https://github.com/dubhatervapoursynth/vapoursynth-temporalsoften
#   CNR2: https://github.com/dubhatervapoursynth/vapoursynth-cnr2
def ChubbyRain2(c, th=10, radius=10, show=False, sft=10, interlaced=False):
    if interlaced:
        c = core.std.SeparateFields(c)

    u = core.std.ShufflePlanes(c, 1, vs.GRAY)
    v = core.std.ShufflePlanes(c, 2, vs.GRAY)

    uc = core.std.Convolution(u, matrix=[1, -2, 1], mode="v")
    vc = core.std.Convolution(v, matrix=[1, -2, 1], mode="v")

    cc = core.std.Convolution(c, matrix=[1, 2, 1], mode="v", divisor=4, planes=[0, 1, 2])
    cc = core.bifrost.Bifrost(cc, interlaced=False)
    cc = core.cnr2.Cnr2(cc)
    cc = core.focus.TemporalSoften(cc, radius=radius, luma_threshold=0, chroma_threshold=sft, scenechange=2, mode=2)

    rainbow = core.std.Expr([uc, vc], expr=f"x y + {th} > 256 0 ?")
    rainbow = core.resize.Point(rainbow, width=c.width, height=c.height)

    for _ in range(3):
        rainbow = core.std.Maximum(rainbow)

    output = core.std.MaskedMerge(c, cc, rainbow)

    if show:
        return rainbow

    if interlaced:
        output = core.std.DoubleWeave(output)
        output = core.std.SelectEvery(output, cycle=2, offsets=0)

    return output

A more modern version should
a. automatically adjust the expression, when high bit depth is used
b. use modern filters (akarin instead of std.Expr, zsmooth.TemporalSoften instead of focus.TemporalSoften, zsmooth.Cnr4 instead of cnr2.Cnr2)
(shouldn't be easy to adjust)

off to work now, but I can look at it after work some more.

Cu Selur

[The 14th Doctor]

Thank you for your reply, will test it out (or wait until, you tweak it)

I see, thought there was a way to load avs scripts but guess not. Really your only option is to remake in Vapoursynth it seems...

At least you can load avisynth dlls, so those kind of filters can work.

[The 14th Doctor]

You can't load an Avisynth script directly inside a Vapoursynth script.

Here's a !quick and untested! rewrite (base on the http://avisynth.nl/images/ChubbyRain2.avsi version)

# requires: 
#   BiFrost https://github.com/dubhatervapoursynth/vapoursynth-bifrost
#   Temporal soften: https://github.com/dubhatervapoursynth/vapoursynth-temporalsoften
#   CNR2: https://github.com/dubhatervapoursynth/vapoursynth-cnr2
def ChubbyRain2(c, th=10, radius=10, show=False, sft=10, interlaced=False):
    if interlaced:
        c = core.std.SeparateFields(c)

    u = core.std.ShufflePlanes(c, 1, vs.GRAY)
    v = core.std.ShufflePlanes(c, 2, vs.GRAY)

    uc = core.std.Convolution(u, matrix=[1, -2, 1], mode="v")
    vc = core.std.Convolution(v, matrix=[1, -2, 1], mode="v")

    cc = core.std.Convolution(c, matrix=[1, 2, 1], mode="v", divisor=4, planes=[0, 1, 2])
    cc = core.bifrost.Bifrost(cc, interlaced=False)
    cc = core.cnr2.Cnr2(cc)
    cc = core.focus.TemporalSoften(cc, radius=radius, luma_threshold=0, chroma_threshold=sft, scenechange=2, mode=2)

    rainbow = core.std.Expr([uc, vc], expr=f"x y + {th} > 256 0 ?")
    rainbow = core.resize.Point(rainbow, width=c.width, height=c.height)

    for _ in range(3):
        rainbow = core.std.Maximum(rainbow)

    output = core.std.MaskedMerge(c, cc, rainbow)

    if show:
        return rainbow

    if interlaced:
        output = core.std.DoubleWeave(output)
        output = core.std.SelectEvery(output, cycle=2, offsets=0)

    return output

A more modern version should
a. automatically adjust the expression, when high bit depth is used
b. use modern filters (akarin instead of std.Expr, zsmooth.TemporalSoften instead of focus.TemporalSoften, zsmooth.Cnr4 instead of cnr2.Cnr2)
(shouldn't be easy to adjust)

off to work now, but I can look at it after work some more.

Cu Selur

You'll probably tweak the code further but I tested it out, looks great and way better at keeping the original color unlike DFM Derainbow.

Thank you.

First image is with no derainbow filters, second is with ChubbyRain2 + DeCross


[Attachment 93505 - Click to enlarge]
[Attachment 93506 - Click to enlarge]

[Selur]

Here's a more comprehensive port, which supports newer filters and should work fine with high bitdeph content too:

# requires:
#   BiFrost: https://github.com/dubhatervapoursynth/vapoursynth-bifrost
#   CNR2 (used as fallback when zsmooth is not installed):
#            https://github.com/dubhatervapoursynth/vapoursynth-cnr2
#   TemporalSoften (used as fallback when zsmooth is not installed):
#            https://github.com/dubhatervapoursynth/vapoursynth-temporalsoften
# optional, used automatically when present:
#   akarin:  https://github.com/AkarinVS/vapoursynth-plugin      (faster Expr; falls back to std.Expr)
#   zsmooth: https://github.com/adworacz/zsmooth                 (Cnr4 + faster TemporalSoften)
#   scenechange: https://github.com/Tatsh/scenechange            (feeds Cnr4's scene-change detection)
#   vs-miscfilters-obsolete (fallback for scene-change if scenechange isn't present):
#            https://github.com/vapoursynth/vs-miscfilters-obsolete
def ChubbyRain2(c, th=10, radius=10, show=False, sft=10, interlaced=False):
    if interlaced:
        c = core.std.SeparateFields(c)

    bits = c.format.bits_per_sample
    is_float = c.format.sample_type == vs.FLOAT
    # th/sft are tuned on an 8bit (0-255) scale; scale to the clip's actual range
    peak = 1.0 if is_float else (1 << bits) - 1
    th_scaled = th / 255 if is_float else th * peak // 255

    u = core.std.ShufflePlanes(c, 1, vs.GRAY)
    v = core.std.ShufflePlanes(c, 2, vs.GRAY)

    uc = core.std.Convolution(u, matrix=[1, -2, 1], mode="v")
    vc = core.std.Convolution(v, matrix=[1, -2, 1], mode="v")

    cc = core.std.Convolution(c, matrix=[1, 2, 1], mode="v", divisor=4, planes=[0, 1, 2])
    cc = core.bifrost.Bifrost(cc, interlaced=False)

    if hasattr(core, 'zsmooth'):
        # Cnr4 needs external scene-change detection; Cnr2 did this internally via scdthr=10.0.
        # Reproduce that ~10% threshold with whichever scene-change plugin is available.
        have_scd = hasattr(core, 'scd') and not is_float
        if have_scd:
            # scd.Detect's thresh is an absolute 0-254 (x2^(bits-8)) luma-diff value, not a
            # percentage, so scale a 10% threshold onto that range.
            thresh = max(1, round(0.10 * 254 * (1 << max(bits - 8, 0))))
            cc = core.scd.Detect(cc, thresh=thresh)
            use_scenechange = True
        elif hasattr(core, 'misc'):
            cc = core.misc.SCDetect(cc, threshold=0.10)
            use_scenechange = True
        else:
            use_scenechange = False

        # mode/tmode/radius/sense/str below reproduce Cnr2's defaults
        # (mode="oxx", scdthr=10.0, ln/un/vn=35/47/47, lm/um/vm=192/255/255)
        cc = core.zsmooth.Cnr4(cc, mode="oxx", tmode=2, radius=2, sense=[35, 47, 47], str=[192, 255, 255], scenechange=use_scenechange)
    else:
        cc = core.cnr2.Cnr2(cc)  # defaults: mode="oxx", scdthr=10.0, ln=35, lm=192, un=47, um=255, vn=47, vm=255

    if hasattr(core, 'zsmooth'):
        # zsmooth takes one threshold per plane and scales it itself (scalep=True)
        cc = core.zsmooth.TemporalSoften(cc, radius=radius, threshold=[0, sft, sft], scenechange=2, scalep=True)
    else:
        # dubhater's focus.TemporalSoften; effectively 8bit-only, so no manual scaling here
        cc = core.focus.TemporalSoften(cc, radius=radius, luma_threshold=0, chroma_threshold=sft, scenechange=2, mode=2)

    expr = f"x y + {th_scaled} > {peak} 0 ?"
    # prefer akarin, fallback to std.Expr
    rainbow = core.akarin.Expr([uc, vc], expr) if hasattr(core, 'akarin') else core.std.Expr([uc, vc], expr)
    rainbow = core.resize.Point(rainbow, width=c.width, height=c.height)

    for _ in range(3):
        rainbow = core.std.Maximum(rainbow)

    output = core.std.MaskedMerge(c, cc, rainbow)

    if show:
        return rainbow

    if interlaced:
        output = core.std.DoubleWeave(output)
        output = core.std.SelectEvery(output, cycle=2, offsets=0)

    return output

Note:

• It will produce slightly different outputs than the previous version, since it will prefer CNR4 (instead of CNR2), but that should work even better.
• It should be faster since zsmooth and akarin are used which, especially on larger resolutions and multithreaded systems are usually faster.

=> hope that helps with your rainbows

Cu Selur