View Profile
View Forum Posts
Private Message
Visit Homepage
Sorry, this is going to be lengthy,...
Starting out with a portable Vapoursynth.
I downloaded all the stuff from https://github.com/AmusementClub/vs-mlrt/releases an extracted into a folder named vs-mlrt, which gave me:
So now I started some testing with the following base script:Code:I:\Hybrid\64bit\vsfilters\vs-mlrt\models I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt.py I:\Hybrid\64bit\vsfilters\vs-mlrt\vsncnn.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsort I:\Hybrid\64bit\vsfilters\vs-mlrt\vsort.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vstrt.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\models\anime-segmentation I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan I:\Hybrid\64bit\vsfilters\vs-mlrt\models\dpir I:\Hybrid\64bit\vsfilters\vs-mlrt\models\oidn I:\Hybrid\64bit\vsfilters\vs-mlrt\models\ppocr I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2 I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x I:\Hybrid\64bit\vsfilters\vs-mlrt\models\anime-segmentation\isnet_is.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-conservative-up2x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-conservative-up3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-denoise3x-up2x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-denoise3x-up3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-no-denoise3x-up2x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\pro-no-denoise3x-up3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up2x-latest-conservative.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up2x-latest-denoise1x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up2x-latest-denoise2x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up2x-latest-denoise3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up2x-latest-no-denoise.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up3x-latest-conservative.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up3x-latest-denoise3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up3x-latest-no-denoise.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up4x-latest-conservative.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up4x-latest-denoise3x.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\cugan\up4x-latest-no-denoise.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\dpir\drunet_color.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\dpir\drunet_deblocking_color.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\dpir\drunet_deblocking_grayscale.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\dpir\drunet_gray.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\oidn\rt_ldr.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\ppocr\ml_PP-OCRv3_det.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\animejanaiL1-sharp.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\animejanaiL2-sharp.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\animejanaiL2-std.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\animejanaiL3-sharp.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\animejanaiL3-std.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\realesr-animevideov3.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\RealESRGANv2-animevideo-xsx2.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\RealESRGANv2\RealESRGANv2-animevideo-xsx4.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.0.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.2.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.3.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.4.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.5.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\rife\rife_v4.6.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\ukbench I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10 I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art\noise1_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art\noise2_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art\noise3_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb\noise0_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb\noise1_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb\noise2_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb\noise3_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\anime_style_art_rgb\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise0_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise0_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise1_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise1_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise2_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise2_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise3_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\noise3_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\cunet\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo\noise0_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo\noise1_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo\noise2_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo\noise3_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\photo\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\ukbench\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb\noise0_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb\noise1_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb\noise2_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb\noise3_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_anime_style_art_rgb\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo\noise0_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo\noise1_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo\noise2_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo\noise3_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upconv_7_photo\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10\noise0_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10\noise1_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10\noise2_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10\noise3_scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\models\waifu2x\upresnet10\scale2.0x_model.onnx I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cublas64_11.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cublasLt64_11.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cudart64_110.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cudnn64_8.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cudnn_adv_infer64_8.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cudnn_cnn_infer64_8.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cudnn_ops_infer64_8.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cufft64_10.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cufftw64_10.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\cupti64_2022.3.0.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\msvcp140.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvblas64_11.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvinfer.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvinfer_builder_resource.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvinfer_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvonnxparser.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvparsers.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvrtc-builtins64_118.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\nvrtc64_112_0.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\trtexec.exe I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\vcruntime140.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\vcruntime140_1.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsmlrt-cuda\zlibwapi.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsort\onnxruntime.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsort\onnxruntime_providers_cuda.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsort\onnxruntime_providers_shared.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\cache.json I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_auto_batch_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_auto_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_c.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_hetero_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_intel_cpu_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_intel_gpu_plugin.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_ir_frontend.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_onnx_frontend.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_ov_c.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\openvino_tensorflow_fe.dll I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\plugins.xml I:\Hybrid\64bit\vsfilters\vs-mlrt\vsov\tbb.dll
Staring with Onnx, replacing '#### MAGIC ####' with:Code:# Imports import vapoursynth as vs # getting Vapoursynth core core = vs.core # Loading Plugins core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll") # source: 'G:\TestClips&Co\test.avi' # current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive # Loading G:\TestClips&Co\test.avi using LWLibavSource clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0) # Setting detected color matrix (470bg). clip = core.std.SetFrameProps(clip, _Matrix=5) # Setting color transfer info (470bg), when it is not set clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5) # Setting color primaries info (), when it is not set clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5) # Setting color range to TV (limited) range. clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1) # making sure frame rate is set to 25 clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive # adjusting color format to RGBS for custom section clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited") # side note: since my input is already mod32, I do not need any padding #### MAGIC #### # Output clip.set_output()
and uncommenting the lines I want to test, works fine.Code:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vsort.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend # DPIR through Onnx #from vsmlrt import DPIR, DPIRModel #clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.ORT_CUDA) # GPU # clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.ORT_CPU) # CPU # Waifu2x through Onnx #from vsmlrt import Waifu2x, Waifu2xModel #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.ORT_CUDA) # GPU #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.ORT_CPU) # CPU # CUGAN through Onnx #from vsmlrt import CUGAN #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.ORT_CUDA) #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.ORT_CPU) # RealESRGAN through Onnx #from vsmlrt import RealESRGAN, RealESRGANModel #clip = RealESRGAN(clip, model=RealESRGANModel.animevideo_xsx2, backend=Backend.ORT_CUDA) #clip = RealESRGAN(clip, model=RealESRGANModel.animevideo_xsx2, backend=Backend.ORT_CPU) # RealESRGANv2 through Onnx #from vsmlrt import RealESRGANv2, RealESRGANv2Model #clip = RealESRGANv2(clip, model=RealESRGANv2Model.animevideo_xsx2, backend=Backend.ORT_CUDA) #clip = RealESRGANv2(clip, model=RealESRGANv2Model.animevideo_xsx2, backend=Backend.ORT_CPU) # RIFE through Onnx #from vsmlrt import RIFE, RIFEModel, RIFEMerge #clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.ORT_CUDA) #clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.ORT_CPU)
testing OpenVINO, by replacing '#### MAGIC ####' with:
Works.Code:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vsov.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend # DPIR through OpenVINO #from vsmlrt import DPIR, DPIRModel #clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.OV_GPU) #clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.OV_CPU) # Waifu2x through OpenVINO #from vsmlrt import Waifu2x, Waifu2xModel #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.OV_GPU) #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.OV_CPU) # CUGAN through OpenVINO #from vsmlrt import CUGAN #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.OV_GPU) #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.OV_CPU) # RealESRGAN through OpenVINO #from vsmlrt import RealESRGAN, RealESRGANModel #clip = RealESRGAN(clip, model=RealESRGANModel.animevideo_xsx2, backend=Backend.OV_GPU) #clip = RealESRGAN(clip, model=RealESRGANModel.animevideo_xsx2, backend=Backend.OV_CPU) # RealESRGANv2 through OpenVINO #from vsmlrt import RealESRGANv2, RealESRGANv2Model #clip = RealESRGANv2(clip, model=RealESRGANv2Model.animevideo_xsx2, backend=Backend.OV_CPU) #clip = RealESRGANv2(clip, model=RealESRGANv2Model.animevideo_xsx2, backend=Backend.OV_GPU) # RIFE through Onnx from vsmlrt import RIFE, RIFEModel, RIFEMerge #clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.OV_CPU)
Using:
fails with:Code:clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.OV_GPU)
(I got an NVIDIA RTX 4080, an Intel Arc A380 and an AMD Raphael C1 igpu from my Ryzen 9 7950X)Code:clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.OV_GPU) File "i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py", line 945, in RIFE output0 = RIFEMerge( File "i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py", line 827, in RIFEMerge return inference_with_fallback( File "i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py", line 1444, in inference_with_fallback raise e File "i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py", line 1423, in inference_with_fallback return _inference( File "i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py", line 1342, in _inference clip = core.ov.Model( File "src\cython\vapoursynth.pyx", line 2612, in vapoursynth.Function.__call__ vapoursynth.Error: operator (): Operation: GridSample_55 of type GridSample(op::v0) is not supported
I also tried:
but that gave me:Code:clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.OV_GPU(device_id=1))
=> So here's my first question: How do I get this (RIFE + GPU + OpenVINO) working?Code:vapoursynth.Error: operator (): Config for device with 1 ID is not registered in GPU plugin
testing NCNN, by replacing '#### MAGIC ####' with:
works,Code:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vsncnn.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend # DPIR through NCNN #from vsmlrt import DPIR, DPIRModel #clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.NCNN_VK) # Waifu2x through NCNN #from vsmlrt import Waifu2x, Waifu2xModel #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.NCNN_VK) # CUGAN through NCNN #from vsmlrt import CUGAN #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.NCNN_VK) # RealESRGAN through NCNN #from vsmlrt import RealESRGAN, RealESRGANModel #clip = RealESRGAN(clip, model=RealESRGANModel.animevideo_xsx2, backend=Backend.NCNN_VK)
but using:
sadly simply causes vsViewer and vspipe to close with:Code:# RIFE through NCNN from vsmlrt import RIFE, RIFEModel, RIFEMerge clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.NCNN_VK)
=> So here's my second question: How do I get this (RIFE + GPU + NCNN) working?Code:GridSample not supported yet! # align_corners=1 # mode=bilinear # padding_mode=border GridSample not supported yet! # align_corners=1 # mode=bilinear # padding_mode=border GridSample not supported yet! # align_corners=1 # mode=bilinear # padding_mode=border GridSample not supported yet! # align_corners=1 # mode=bilinear # padding_mode=border Unsupported split axis ! [0 NVIDIA GeForce RTX 4080] queueC=2[8] queueG=0[16] queueT=1[2] [0 NVIDIA GeForce RTX 4080] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0 [0 NVIDIA GeForce RTX 4080] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 NVIDIA GeForce RTX 4080] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1 [1 Intel(R) Arc(TM) A380 Graphics] queueC=1[1] queueG=0[1] queueT=2[1] [1 Intel(R) Arc(TM) A380 Graphics] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0 [1 Intel(R) Arc(TM) A380 Graphics] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [1 Intel(R) Arc(TM) A380 Graphics] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1 [2 AMD Radeon(TM) Graphics] queueC=1[2] queueG=0[1] queueT=2[1] [2 AMD Radeon(TM) Graphics] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0 [2 AMD Radeon(TM) Graphics] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [2 AMD Radeon(TM) Graphics] subgroup=64 basic=1 vote=1 ballot=1 shuffle=1 layer GridSample not exists or registered
testing TRT, by replacing '#### MAGIC ####' with:
works.Code:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vstrt.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend # DPIR through TRT #from vsmlrt import DPIR, DPIRModel #clip = DPIR(clip, strength=5, model=DPIRModel.drunet_color,backend=Backend.TRT) # Waifu2x through TRT #from vsmlrt import Waifu2x, Waifu2xModel #clip = Waifu2x(clip,noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.TRT) # CUGAN through TRT #from vsmlrt import CUGAN #clip = CUGAN(clip, noise=-1, scale=2, backend=Backend.TRT) # RIFE through TRT #from vsmlrt import RIFE, RIFEModel, RIFEMerge #clip = RIFE(clip, multi=2, model=RIFEModel.v4_6, backend=Backend.TRT)
Here's an example on how to load a custom model:
Code:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vsort.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend clip = vsmlrt.inference([clip],network_path=r"I:\Hybrid\64bit\vsfilters\vs-mlrt\models\custom\2x_Futsuu_Anime_Compact_130k_net_g_32bit.onnx", backend = Backend.ORT_CUDA())Cu SelurCode:core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/vs-mlrt/vstrt.dll") import imp vsmlrt = imp.load_source('vsmlrt', 'i:/Hybrid/64bit/vsfilters/vs-mlrt/vsmlrt.py') from vsmlrt import Backend clip = vsmlrt.inference([clip],network_path=r"I:\Hybrid\64bit\vsfilters\vs-mlrt\models\custom\2x_Futsuu_Anime_Compact_130k_net_g_32bit.onnx", backend = Backend.TRT())
Ps.: hope this help others that want to try vs-mlrt.
+ Reply to Thread
Results 1 to 9 of 9
-
Last edited by Selur; 5th May 2023 at 12:36.
users currently on my ignore list: deadrats, Stears555, marcorocchini -
Converting the onnx models to .engine models will again significantly increase the speed.
-
Yes, it does, using vsmlrt.py, the onnx models automatically get converted to .engine files.Converting the onnx models to .engine models will again significantly increase the speed.
Question is: Why does RIFE not work with NCNN and OpenVINO (for me)?
Update:
Okay, it's known that NCNN isn't working with RIFE (https://github.com/AmusementClub/vs-mlrt/releases/tag/v12).
-> so only OpenVINO is unknown.
Cu SelurLast edited by Selur; 5th May 2023 at 14:19.
users currently on my ignore list: deadrats, Stears555, marcorocchini -
source: https://github.com/AmusementClub/vs-mlrt/releases/tag/model-20220923Specifically, ov-gpu and ncnn-vk do not support RIFE due to missing gridsample op.
=> It was a known restriction, which I simply didn't know of.
(Thanks to hooke007 over at GitHub for finding this!)
Cu Selurusers currently on my ignore list: deadrats, Stears555, marcorocchini -
Do you think it could increase speed, if you convert to .eninge before? So it has not to be done while execute the command.Originally Posted by Selur
Yes, it does, using vsmlrt.py, the onnx models automatically get converted to .engine files.Converting the onnx models to .engine models will again significantly increase the speed.
How could I convert to .engine file or how do you convert pytorch into .onnx ? In "chaiNNer" you could convert it from PyTorch into ONNX.
And when i look on your last two codes, then the moddel is called "..._32bit.onnx". Why it is called like that. Did you have to change sth for beeing 32bit?
On the Githublink from Post 1, did you use the nightlybuild or which one did you use for downloading ?Last edited by Platos; 5th May 2023 at 20:29.
-
You can only use vstrt with engine files. Whether you manually convert the onnx file to an engine file or whether vsmlrt.py does it for you doesn't change the speed.Do you think it could increase speed, if you convert to .eninge before? So it has not to be done while execute the command.
I used chaiNNer.How could I convert to .engine file or how do you convert pytorch into .onnx ? In "chaiNNer" you could convert it from PyTorch into ONNX.
I called it like that since I used fp32 when converting and saving it in chaiNNer.And when i look on your last two codes, then the moddel is called "..._32bit.onnx". Why it is called like that. Did you have to change sth for beeing 32bit?
I used the nightly.On the Githublink from Post 1, did you use the nightlybuild or which one did you use for downloading ?
Cu Selurusers currently on my ignore list: deadrats, Stears555, marcorocchini -
Okay, the OpenVINO issue is fixed, I somehow used an older vsov.dll, using the one from v13.1 and that works again.
Cu Selurusers currently on my ignore list: deadrats, Stears555, marcorocchini -
Ok, thanks, i will try. I guess i dont need a "new" Vapoursynth installation in portable, when i already have one ? Or can i install a portable one without killing the other or viseversa ?
And nice you fixed it. I heard Intel GPUs are quite good for this stuff. But i dont have one, so i go for the nvidia stuff.
So only for clarification: The TRT Version is the "fast" one, right? Why did you try the NCNN Version or the ONNX Version (the OpenVINO is for your IntelGPU i know) ?
And so you did test RIFE with TRT, the FPS-Interpolation ? Did you get good speed compared to without TRT? I Use Interpolation for Films with "real" charakters, so the non animated stuff.Last edited by Platos; 6th May 2023 at 06:17.
-
NoI guess i dont need a "new" Vapoursynth installation in portable, when i already have one ?
Mine isn't, but I bought it for it's hardware decoder and encoder.I heard Intel GPUs are quite good for this stuff. But i dont have one, so i go for the nvidia stuff.
yesThe TRT Version is the "fast" one, right?
to know whether they work, TensorRT requires a newer NVIDIA card, the others will work on older NVIDIA cards too.Why did you try the NCNN Version or the ONNX Version (the OpenVINO is for your IntelGPU i know) ?
Didn't do any benchmarks, just checked that it worked.And so you did test RIFE with TRT, the FPS-Interpolation ? Did you get good speed compared to without TRT?
Cu Selurusers currently on my ignore list: deadrats, Stears555, marcorocchini
QuoteSimilar Threads
-
Fresh x264 10-bit vs Ice Lake hevc qsv 10-bit
By sophisticles in forum Video ConversionReplies: 5Last Post: 7th Jan 2021, 16:06 -
What has better quality - DTS Core @ 24 bit or FLAC @ 16 bit
By vash1 in forum AudioReplies: 1Last Post: 1st Dec 2020, 08:45 -
10-bit colour - does it improves 8-bit videos and games?
By KENJI512 in forum Newbie / General discussionsReplies: 1Last Post: 27th Apr 2020, 17:55 -
10-bit to 8-bit Video File Conversions - the easy (free) way!
By VideoWiz in forum Video ConversionReplies: 10Last Post: 6th Feb 2020, 03:24 -
Help with running 32-bit DLL in 64-bit AVSynth with MP_Pipeline.
By Vitality in forum Video ConversionReplies: 1Last Post: 12th Jan 2019, 16:26