VideoHelp Forum




+ Reply to Thread
Results 1 to 4 of 4
  1. I want to use vsmlrt to remove rainbows and slight colour bleed in my source using https://openmodeldb.info/models/1x-BleedOut-Compact
    But I'm not sure how to use this plugin. There's no parameter doc anywhere and I'm not sure where to put this .pth file. I have downloaded vsmlrt and dragged and dropped into my plugins folder but not sure what do from there

    This is my script so far
    Code:
    import vapoursynth as vs
    import fvsfunc as fvf
    import kagefunc as kgf
    import vsTAAmbk as taa
    import functools
    import mvsfunc as mvf
    import lvsfunc as lvf
    import havsfunc as haf
    import vsmlrt
    from vsutil import *
    from nnedi3_rpow2 import *
    core = vs.core
    
    def postprocess(n, f, clip, deinterlaced):
       if f.props['_Combed'] > 0:
          return deinterlaced
       else:
          return clip
    
    def decimate(src):
        src = core.vivtc.VFM(src, 1)
        return core.vivtc.VDecimate(src)
    
    
    def detelecine(src):
        matched = core.vivtc.VFM(src, 1)
        src = core.nnedi3.nnedi3(src, field=0)
        return core.std.FrameEval(matched, functools.partial(postprocess, clip=matched, deinterlaced=src), prop_src=matched)
    
    src = core.dgdecodenv.DGSource(r"E:\VapWork\Hare\VTS_02_1.dgi")
    src = core.tcomb.TComb(src, mode=2)
    src = core.bifrost.Bifrost(src, interlaced=True)
    src = haf.Stab(src, range=5, mirror=0, dxmax=8, dymax=8)
    src = decimate(src)
    src = core.reduceflicker.ReduceFlicker(src, strength=3, aggressive=1)
    src = fvf.Depth(src, 16)
    
    crop = core.std.CropRel(src, 10, 8, 2)
    scaled = core.resize.Spline36(crop, 720, 540)
    
    aamask = kgf.retinex_edgemask(scaled).std.Binarize(12000).std.Inflate()
    aa = taa.TAAmbk(scaled, aatype='Nnedi3', mclip=aamask)
    dehalo = haf.HQDeringmod(aa, darkthr=0, sharp=0, mthr=54)
    
    dpir = lvf.deblock.vsdpir(dehalo, strength=40, mode='deblock', cuda=True, matrix="6")
    denoise = haf.SMDegrain(dpir, tr=3, thSAD=220, RefineMotion=True)
    line_mask = kgf.retinex_edgemask(denoise, sigma=1)
    mask_outer = iterate(line_mask, core.std.Maximum, 3)
    dpir_merge = core.std.MaskedMerge(dehalo, denoise, mask_outer)

    How do I use this .pth file and call it to vapoursynth? Any help is appreciated thank you!
    Last edited by hihflappy; 5th Feb 2025 at 11:37.
    Quote Quote  
  2. There's no parameter doc anywhere and ...
    Sure, there is: Since I use vsort I followed https://github.com/AmusementClub/vs-mlrt/tree/master/vsort (which is linked on the main page)
    Docs for the other methods are also linked on the main page.

    I'm not sure where to put this .pth file.
    You can't use .pth files, you would need to use VSGAN for .pth files.
    You only can use .onnx files with vsmlrt, but you can convert .pth to .onnx using https://github.com/chaiNNer-org/chaiNNer.

    Here's an example how I use vs-mlrt with the an .onnx model.
    Code:
    # Imports
    import vapoursynth as vs
    # getting Vapoursynth core
    import site
    import sys
    import os
    core = vs.core
    # Import scripts folder
    scriptPath = 'F:/Hybrid/64bit/vsscripts'
    sys.path.insert(0, os.path.abspath(scriptPath))
    os.environ["CUDA_MODULE_LOADING"] = "LAZY"
    # loading plugins
    core.std.LoadPlugin(path="F:/Hybrid/64bit/vs-mlrt/vstrt.dll")
    core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
    core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
    # Import scripts
    from importlib.machinery import SourceFileLoader
    vsmlrt = SourceFileLoader('vsmlrt', 'F:/Hybrid/64bit/vs-mlrt/vsmlrt.py').load_module()
    import validate
    # Source: 'G:\TestClips&Co\files\test.avi'
    # Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual
    # Loading G:\TestClips&Co\files\test.avi using LWLibavSource
    clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
    frame = clip.get_frame(0)
    # setting color matrix to 470bg.
    clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
    # setting color transfer (vs.TRANSFER_BT601), if it is not set.
    if validate.transferIsInvalid(clip):
      clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
    # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
    if validate.primariesIsInvalid(clip):
      clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
    # setting color range to TV (limited) range.
    clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
    # making sure frame rate is set to 25fps
    clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
    # making sure the detected scan type is set (detected: progressive)
    clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
    # changing range from limited to full range for vsVSMLRT
    clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
    # setting color range to PC (full) range.
    clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
    # adjusting color space from YUV420P8 to RGBH for vsVSMLRT
    clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="full")
    # resizing using VSMLRT, target: 640x352
    from vsmlrt import Backend
    clip = vsmlrt.inference([clip],network_path="F:/Hybrid/64bit/onnx_models/1x_BleedOut_Compact_300k_net_g.onnx", backend=Backend.TRT(fp16=True,device_id=0,num_streams=1,verbose=True,use_cuda_graph=False,workspace=1073741824,builder_optimization_level=3,engine_folder="J:/TRT")) # 640x352
    # changing range from full to limited range for vsVSMLRT
    clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
    # adjusting output color from: RGBH to YUV420P10 for NVEncModel
    clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
    # set output frame rate to 25fps (progressive)
    clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
    # output
    clip.set_output()
    Here's also an example how I use it using VSGAN:
    Code:
    # Imports
    import vapoursynth as vs
    # getting Vapoursynth core
    import sys
    import os
    core = vs.core
    # Import scripts folder
    scriptPath = 'F:/Hybrid/64bit/vsscripts'
    sys.path.insert(0, os.path.abspath(scriptPath))
    # loading plugins
    core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
    core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
    # Import scripts
    import validate
    # Source: 'G:\TestClips&Co\files\test.avi'
    # Current color space: YUV420P8, bit depth: 8, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: MPEG-4 Visual
    # Loading G:\TestClips&Co\files\test.avi using LWLibavSource
    clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
    frame = clip.get_frame(0)
    # setting color matrix to 470bg.
    clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
    # setting color transfer (vs.TRANSFER_BT601), if it is not set.
    if validate.transferIsInvalid(clip):
      clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
    # setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
    if validate.primariesIsInvalid(clip):
      clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
    # setting color range to TV (limited) range.
    clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
    # making sure frame rate is set to 25fps
    clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
    # making sure the detected scan type is set (detected: progressive)
    clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
    # changing range from limited to full range for VsVSGANFilter
    clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
    # setting color range to PC (full) range.
    clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_FULL)
    # adjusting color space from YUV420P8 to RGBH for VsVSGANFilter
    clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="full")
    # filtering using VSGANFilter
    from vsgan import ESRGAN
    vsgan = ESRGAN(clip=clip,device="cuda")
    model = "F:/Hybrid/64bit/vsgan_models/1x_BleedOut_Compact_300k_net_g.pth"
    vsgan.load(model)
    vsgan.apply() # 640x352
    clip = vsgan.clip
    # changing range from full to limited range for VsVSGANFilter
    clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
    # no resizing since resolution is already archived
    # adjusting output color from: RGBH to YUV420P10 for NVEncModel
    clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
    # set output frame rate to 25fps (progressive)
    clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
    # output
    clip.set_output()
    Hope that helps a bit. Will not exactly work like in my setup since I use a portable Vapoursynth&Python.

    Cu Selur
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
  3. Thanks for responding, how do I convert it? Also it seems I need an engine file?

    EDIT: I tried VSGAN, but it seems RGB is not even compatible with even half my filters
    Last edited by hihflappy; 23rd Jan 2025 at 19:37.
    Quote Quote  
  4. I tried VSGAN, but it seems RGB is not even compatible with even half my filters
    vs-mlrt and vsgan both use RGH and RGBS. If your filters do not support these, convert to a color format that is supported before applying them,...
    how do I convert it?
    https://github.com/upscayl/upscayl/wiki/Model-Conversion-Guide/306dae7c1251659b979a30a...a2c616ddf8a3e8
    Also it seems I need an engine file?
    Engine files get automatically generated (by vs-mlrt) when using TRT as backend.

    Cu Selur
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
Visit our sponsor! Try DVDFab and backup Blu-rays!