Hybrid(Windows/Linux/Mac): Input -> x264/x265/Xvid/VP8/VP9

[Platos]

Ahh, now it works. Sorry for the problem. I did not now (or forgot), that i have to deinstall hybrid before i use a new dev.

But i have another problem ^^

When i use a fp16 model wit vsmlrt and TensorRT RTX Backend, i got an error. When i use TensorRT, i do not get an error. I tried it with 32Bit checkbox enabled and disabled.

Failed to evaluate the script:
Python exception: module 'onnx.helper' has no attribute 'float32_to_bfloat16'

Traceback (most recent call last):
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 2415, in tensorrt_rtx
model = convert_float_to_float16(model, keep_io_types=not fp16_io)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\onnxconverter_common\float16.py", line 220, in convert_float_to_float16
raise ValueError(
ValueError: The model is already converted to float16, if convert again, the model might be wrong. 
If you are sure to convert again, please set check_fp16_ready=False.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "vapoursynth.pyx", line 3524, in vapoursynth._vpy_evaluate
File "vapoursynth.pyx", line 3525, in vapoursynth._vpy_evaluate
File "C:\Users\Gaming-Tower\Documents\Hybrid\Tempfolder\tempPreviewVapoursynthFile17_40_23_932.vpy", line 55, in 
clip = vsmlrt.inference([clip],network_path="C:/Users/Gaming-Tower/Videos/$$$AI-Models$$$/SISR Models/[StinkyBread]/2x-AnimeFilm/2x_animefilm_light_161k_fp16_dynamic.onnx", backend=Backend.TRT_RTX(fp16=True,device_id=0,num_streams=2,verbose=True,use_cuda_graph=False,workspace=1073741824,builder_optimization_level=3,engine_folder="C:/Users/Gaming-Tower/Documents/Hybrid/Engine-Files")) # 3840x2160
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 3113, in inference
return inference_with_fallback(
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 3087, in inference_with_fallback
raise e
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 3064, in inference_with_fallback
ret = _inference(
^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 2977, in _inference
engine_path = tensorrt_rtx(
^^^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 2418, in tensorrt_rtx
from modelopt.onnx.autocast import convert_to_f16, configure_logging
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\modelopt\onnx\__init__.py", line 23, in 
from . import quantization
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\modelopt\onnx\quantization\__init__.py", line 19, in 
from .int4 import quantize as quantize_int4
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\modelopt\onnx\quantization\int4.py", line 30, in 
import onnx_graphsurgeon as gs
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\onnx_graphsurgeon\__init__.py", line 1, in 
from onnx_graphsurgeon.exporters.onnx_exporter import export_onnx
File "c:\program files\hybrid\64bit\vapoursynth\Lib\site-packages\onnx_graphsurgeon\exporters\onnx_exporter.py", line 134, in 
np.uint16, onnx.helper.float32_to_bfloat16
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'onnx.helper' has no attribute 'float32_to_bfloat16'

This is the error then. Do you need more information?
Is this the same installation-error? Should i completly deinstall hybrid, delete all hybrid folders and reinstall ? I did now "only" a normal deinstallation with the normal deinstaller.

[Selur]

RTX sometimes works with vs-mlrt and sometimes it does not, form what I gather it's a problem of the onnx and torch versions that are used.
The vs-mlrt addon in Hybrid uses the v15.16 release which currently is the latest version.
=> nothing I can do about it, you report it over at https://github.com/AmusementClub/vs-mlrt/issues and maybe they can fix it.

Cu Selur

[Platos]

Ok, i did open a issue there. Thanks

[Platos]

When i use MultiModel in vsmlrt and one of them is a VSR model, i get a crash. When i use only one model and set corret the "MultiInput" number to 5, it works. But when i use multimodel and the other model is not a VSR model, i get a crash. I guess this is because then hybrid thinks the second model also uses 5 frames at once, right ? Im pretty sure (Because it works in Vapourkit)

[Selur]

You can use the FilterQueue to have multiple model calls with full control over the settings and order. Which is why Other->vsMLRT has no 'Multi' option.
The Multi-option under Resizer->VSMLRT is meant to chain multiple models that can be called with the same parameters.

I guess this is because then hybrid thinks the second model also uses 5 frames at once, right ?

No need to guess.
Hybrid allows you to see the generated script, so it's easy to see what is happening.

Because it works in Vapourkit

¯\_(ツ)_/¯
Never used it, but if it does what you want, simply use that instead of Hybrid.

Cu Selur

[Platos]

Ahh, ok thanks. Then i use "other", thanks. I did not know that vsmlrt is also there.

And no, i like Hybrid more, i wan't to use hybrid

thank you

Edit: How is that done with "other", in terms of the resize? When i use on "crop/resize" double the resolution and then usw first a 1x model and then a 2x model, does it then resize already with the 1x model? Or when/how do i tell hybrid, when it does the actual upscale with double resolution ? Because on frame-> Resize i can see the resolution which is applied, but on other i can not see it. Do you know how that works ?

[Selur]

You can always look at the generated script in the Vapoursynth Script View to see what Hybrid does.
Together with the Gimmick-options (Filtering->Vapousynth->Misc->UI->Show 'Gimmick'-controls, you can set for the filters whether the source should be resized before the filter is applied,...
If you enable all 'hidden'-elements there are a lot of otherwise hidden option available.

Combined with the filter queue you can do a lot of stuff,... alternatively you could also write a custom code in custom section,...
=> there are tons of options, you just have to look around an experiment a bit and use the Vapoursynth Script View to see what happens. (I tried to add a lot of comments inside the scripts so that they are easier to understand, but they require a basic understanding and motivation,..)

Cu Selur

[Platos]

Ok, thank you. I will have a look.

I did now find out, that using DAAMod AntiAliasing before Upscaling makes the output looks less Aliasied (which is clear) but also the Upscaling-model can upscale "better" (it looks like a higher upscale). At least on my (Comic) source now. I will need to test on more sources.

[Selur]

Always depends on the source, sometimes even applying AA before deinterlacing on separated fields can make sense,...

[Platos]

Question: There are different Resizermethodeds:

Scenario: I want to upscale with a 2x model with vsmlrt and then resize it to 1440p.

My plan was to resize with spline36 or DPID or Bicbuic (so all three and then compare).

But: On the crop/resize tab there is "only" spline and bicubic, no DPID and also no spline 36. I wonder, on WHICH resize the resize-methods from crop/resize get applied.

Edit: I found it out:

- When i'am right, the resize-method used in crop/resize is only used, if you do not use the resizer in Filtering->Vapoursynth->Frame->Resize.
- When you use The vsmlrt resizer and you set the same resolution in crop/resize as the source, then no resize (with spline or sth like that) is applyied. Only Upscaling.
- When you use higher or lower resolution in crop/resize, then you use the resize-method from the Filtering->Vapoursynth->Frame->Resize Section where you have to chose the Resizer in the drop down menue.
(That also means, if you do only want to resize in an encode, you can only use the methods on the crop/resize tabs, unless you do a custom script, which also works. With a code like this: "clip = core.resize.Spline36(clip, width=2560, height=1440)")

-> I like spline36 most for 2160p to 1440p downscaling.

At least that's my interpretation, when reading the vapoursynth-code-preview.

[Selur]

- When i'am right, the resize-method used in crop/resize is only used, if you do not use the resizer in Filtering->Vapoursynth->Frame->Resize.

Correct

- When you use The vsmlrt resizer and you set the same resolution in crop/resize as the source, then no resize (with spline or sth like that) is applyied. Only Upscaling.

When you use VSMLRT as Resizer under Vapoursynth->Frame->Resize->Resizer and don't change the resize resolution, Hybrid will undo any resizing done by the used model to match the resize resolution.
In example:

# resizing using VSMLRT, target: 640x352
clip = vsmlrt.inference([clip],network_path="F:/Hybrid/64bit/onnx_models/4x_BSRGAN.onnx", backend=Backend.TRT(fp16=True,device_id=0,bf16=False,num_streams=1,verbose=True,use_cuda_graph=False,workspace=1073741824,builder_optimization_level=3,engine_folder="J:/TRT")) # 2560x1408
# resizing 2560x1408 to 640x352
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS)
clip = core.fmtc.resample(clip, w=640, h=352, kernel="spline64", interlaced=False, interlacedd=False)

the resizer here is the one you select as 'Resizer' unter 'VSLMRT'.

- When you use higher or lower resolution in crop/resize, then you use the resize-method from the Filtering->Vapoursynth->Frame->Resize Section where you have to chose the Resizer in the drop down menue.

You can chose a specific resizer to adjust the output resolution caused by VSMLRT to your desired target resolution,...

Resizer:
Choose the resizer which will be used in case VLMLRT does not perfectly hit the desired resolution.

is what the tool-tip says, which seem accurate.

(That also means, if you do only want to resize in an encode, you can only use the methods on the crop/resize tabs, unless you do a custom script, which also works. With a code like this: "clip = core.resize.Spline36(clip, width=2560, height=1440)")

Since your assumtions are wrong, so is your conclusion, aside from that if you don't overwrite the Resizer unter Crop/Resize through Avisynth/Vapoursynth->Frame->Resize->Resizer the selected resizer will be used.

At least that's my interpretation, when reading the vapoursynth-code-preview.

=> you might want to try that a few more times, to really understand what is happening.

Question ...

your whole post has no question in it,...

Cu Selur