Hybrid(Windows/Linux/Mac): Input -> x264/x265/Xvid/VP8/VP9

[Selur]

Okay,.. input has "Pixel aspect ratio : 1.000" according to MediaInfo. (resolution is 960x720)
Vapoursynth script:

# Imports
import vapoursynth as vs
import os
import sys
os.environ["CUDA_MODULE_LOADING"] = "LAZY"
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vs-mlrt/vstrt.dll")
import site
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
from importlib.machinery import SourceFileLoader
vsmlrt = SourceFileLoader('vsmlrt', 'C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py').load_module()
# source: 'C:\Users\Gaming-Tower\Videos\avatar herr der elemente tests\Avatar.Der.Herr.der.Elemente.S01E01.Der.Junge.im.Eisberg.German.DL.FS.720p.BluRay.x264-PL3X\test kurz1\test.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 960x720, fps: 23.976, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading C:\Users\Gaming-Tower\Videos\avatar herr der elemente tests\Avatar.Der.Herr.der.Elemente.S01E01.Der.Junge.im.Eisberg.German.DL.FS.720p.BluRay.x264-PL3X\test kurz1\test.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Gaming-Tower/Videos/avatar herr der elemente tests/Avatar.Der.Herr.der.Elemente.S01E01.Der.Junge.im.Eisberg.German.DL.FS.720p.BluRay.x264-PL3X/test kurz1/test.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
# Setting detected color matrix (709).
clip = core.std.SetFrameProps(clip, _Matrix=1)
# Setting color transfer info (709), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=1)
# Setting color primaries info (), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 23.976
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from YUV420P8 to RGBS for vsVSMLRT
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="full")
# resizing using VSMLRT
from vsmlrt import Backend
clip = vsmlrt.inference([clip],network_path="C:/Program Files/Hybrid/64bit/onnx_models/2x_LD-Anime_Compact_330k_net_g.onnx", backend=Backend.TRT(fp16=True, device_id=0,num_streams=2,verbose=True,use_cuda_graph=False, workspace=1 << 30))
# resizing 1920x1440 to 1440x1080
# changing range from full to limited range
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1440, h=1080, kernel="spline64", interlaced=False, interlacedd=False)
# adjusting output color from: RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 23.976fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

960x720 (=DAR 4/3) -> becomes 1920x1440 (=DAR 4/3) due to "2x_LD-Anime_Compact_330k_net_g.onnx" thanks to the 2x and then it's resized to =1440, h=1080 (=DAR 4/3).
-> seems fine so far.

NVEncC call:

"C:\Program Files\Hybrid\64bit\NVEncC.exe" --y4m -i - --fps 23.976 --codec h265 --profile main10 --level auto --tier high --sar 1:1 --lookahead 16 --output-depth 10 --multipass "2pass-full" --vbr 5000 --max-bitrate 800000 --gop-len 0 --ref 3 --bframes 3 --bref-mode disabled --no-b-adapt --mv-precision Q-pel --preset quality --colorrange limited --colormatrix bt709 --cuda-schedule sync --output "C:\Users\Gaming-Tower\AppData\Local\Temp\test_new_1_2023-06-17@11_29_51_8710_04.265"

uses '--sar 1:1' which seems fine too.

Muxing call:

"C:\Program Files\Hybrid\64bit\mkvmerge.exe" --ui-language en -o "C:\Users\Gaming-Tower\Videos\avatar herr der elemente tests\Avatar.Der.Herr.der.Elemente.S01E01.Der.Junge.im.Eisberg.German.DL.FS.720p.BluRay.x264-PL3X\test kurz1\2023-06-17@11_29_51_8710__06.mkv" --global-tags "C:\Users\Gaming-Tower\AppData\Local\Temp\mkvtags_2023-06-17@11_29_51_8710__06.xml" -d 0 --default-track 0:yes --default-duration "0:24000/1001fps" --aspect-ratio-factor 0:1/1 --no-chapters --compression -1:none --forced-track 0:yes --field-order 0:0 --no-audio --no-subtitles "C:\Users\Gaming-Tower\AppData\Local\Temp\test_new_1_2023-06-17@11_29_51_8710_04.265" --language 0:de --default-track 0:yes --sync 0:-53 --forced-track 0:no -a 0 --compression -1:none --no-video --no-subtitles --no-chapters "C:\Users\Gaming-Tower\AppData\Local\Temp\iId_1_aid_0_lang_de_DELAY_-53ms_2023-06-17@11_29_51_8710_01.ac3" --default-track 0:no --sync 0:-74 --forced-track 0:no -a 0 --compression -1:none --no-video --no-subtitles --no-chapters "C:\Users\Gaming-Tower\AppData\Local\Temp\iId_2_aid_1_DELAY_-74ms_2023-06-17@11_29_51_8710_02.dts"

uses:
"--aspect-ratio-factor 0:1/1"
which should be fine.


=> don't see an issue with the used calls.
I did update mkvtoolnix, so there might be an issue with mkvmerge.
Try:
a. mp4 as container
b. whether using ffmpeg instead of mkvmerge as muxer works fine

Cu Selur

Ps.: I'm having some issues with mkv too, see: https://forum.doom9.org/showpost.php?p=1988463&postcount=1875 (not totally sure it's related), for me the issue was only with anamorphic content.

[Platos]

i tried mp4 and it does not happen there while it does with mkv.

how do i use ffmpeg instead of mkvmerge with hybrid ?

[Selur]

Give me a few minutes, seems like there is some mixup with mkvmerge binaries.
-> will send you a new link once I repackaged and uploaded a new dev

[Platos]

Works now, thanks!

[Platos]

Hello

I have a question about an error with RIFEmlrt. I tried to interpolate a video, but i got an error:

When i use the vapoursynth preview i get this:

2023-08-24 11:26:43.566
Failed to evaluate the script:
Python exception: vsmlrt.RIFEMerge: tile size must be divisible by 32 (1920, 808)

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate
File "C:\Users\Gaming-Tower\AppData\Local\Temp\tempPreviewVapoursynthFile11_26_42_167.vpy", line 42, in 
clip = vsmlrt.RIFE(clip, multi=3, model=46, backend=Backend.TRT(fp16=True,device_id=0,verbose=True,use_cuda_graph=False, num_streams=3, workspace=1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 1024, in RIFE
output0 = RIFEMerge(
^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 888, in RIFEMerge
raise ValueError(
ValueError: vsmlrt.RIFEMerge: tile size must be divisible by 32 (1920, 808)

I guess the resolution of 808 is the problem. Can i do sth to get it work?

Below is also a debug output.

[Selur]

Not really,... (aside from adding borders before the call and removing them afterwards through a custom section, in my current dev version Hybrid should do this automatically -> will send you a link).

[Platos]

Thanks for your answer.

I tried it, but it did not work. Do i have to change sth in settings or so ?

Vapoursynth preview does also say the same as far as i can see:

2023-08-24 14:20:19.897
Failed to evaluate the script:
Python exception: vsmlrt.RIFEMerge: tile size must be divisible by 32 (1920, 808)

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate
File "C:\Users\Gaming-Tower\AppData\Local\Temp\tempPreviewVapoursynthFile14_20_18_548.vpy", line 42, in 
clip = vsmlrt.RIFE(clip, multi=3, model=46, backend=Backend.TRT(fp16=True,device_id=0,verbose=True,use_cuda_graph=False, num_streams=3, workspace=1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 1024, in RIFE
output0 = RIFEMerge(
^^^^^^^^^^
File "C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py", line 888, in RIFEMerge
raise ValueError(
ValueError: vsmlrt.RIFEMerge: tile size must be divisible by 32 (1920, 808)

But here the debug output:

[Selur]

Okay, seems like Hybrid is making a mistake when calculating the mod.
Video starts out as 1920x804 and Hybrid adds borders to get to 1920x808
-> I'll try to figure out where it goes wrong, will report back.

Cu Selur

[Selur]

Found the problem happened when more than modX/2 needed to be added to fulfill modX.
-> send you a new link via pm

Cu Selur

[Platos]

Sorry, i answered in the wrong thread yesterday:

Thanks, it worked now. Could interpolate the whole video.

Thank you very much!

[Platos]

Hi, i have a question:

Is it possible to upscale a folder with pictures in hybrid (no video/no gif)? Because when i try to drag and drop it says it can not identify framecount.

[Selur]

Hybrid can take image sequences as input, but not via drag&drop.
Hybrid can not resize a folder or random single images.

Cu Selur

[Platos]

Ok thank you. I tried it with ChaiNNer. Found out, you can use a batch-like function to upscale images with same model. It's called Image File Iterator.

BTW: Can you somehow "add" an additional Audio-Stream to a Video without reencoding the others? So let's say i have a Video with English 7.1 and i want to add German 5.1 Audio to this Video. Can i do that with hybrid without "touching" (reencoding) the other Audio-Stream ?

So in simple words: Add one Audio without touching the rest of the file. Like "muxing".

[Selur]

Sure, set audio handling to custom, add the audio streams with passthrough and load and add the new audio in the audio tab.

Cu Selur

[Platos]

Nice, thank you!

[Selur]

Small heads-up for the Linux users: I probably won't build any new Linux builds since it's more and a hassle to build all the dependencies (especially VS plugins), Hybrid then will be Windows only.

[Platos]

I have a question about handling the upscaling-factor on ai-models on upscaling:

When i have a 4-x model (its named 4x-wood-bc1), 720p source and i want a 2160p output, how does hybrid handle this? I mean, when i have the same situation but i want to have 1080p, then hybrid resize the video down to 1080p right? because the 4x model does upscale 4 times anyway right?

So but when i use 2160p in this situation as output, then hybrid wont resize at all, right ?

[Selur]

Yes, if your model upscales by a factor of X and you want a different resolution than the model outputs, Hybrid will use the selected 'Resizer' to adjust to the selected resolution. (This is true for all resizers that can't freely resize.)

Cu Selur

Ps.: 1280x720 upscaled with a 4x model would result in 5120x2880 so if you aim for 3840x2160, Hybrid would still need to use the selected 'Resizer' for compensation.

[Platos]

Thanks, but when i have a 4x model and i use 4x resolution of the input on the "crop/resize" tab, then it will not resize, even if it is actually called "resize" or how can i make sure that it does not resize?

[Selur]

Correct, the additional 'Resizer'-step is only used when necessary.

[Platos]

Ok, thank you very much!

[mindphasar]

Why was Linux support removed ?. it was very convenient to use it instead of the complex syntax of ffmpeg, will there be a return to Linux again? it was useful to have similar software on the Linux side. It also worked very well with Debian Testing.

[Selur]

Linux support was removed since I couldn't quickly figure out how to build, L-SMASH-Works.
It might return if someone comes on how to fix https://github.com/Selur/hybrid-vapoursynth-addon,
also see https://github.com/Selur/hybrid-vapoursynth-addon/issues/5

Cu Selur

[locotus]

Get this error when trying to change configuration: Filteringmodeldo Set 'vsKNLMVSLMeans apply onlytoeveryoffset'

Here's the debug file:

[Selur]

Try resetting your defaults. If you are loading an old configuration, load it and resave it.

[locotus]

I guess resetting defaults means reset in the Config section, that I did
but it didn't work. another thing is the error window is not closing and had to kill the program

Thanks.

[Selur]

Problem is that your 'mine-lfmod-fast.json' contains some values which do not exist in Hybrid anymore.
Normally loading the config, clicking away all the popups and saving the model should help. (Those popups will appear quite a few times.)
If that for some unknown reason does not work, you could also edit the config file with a text editor and simply remove the lines that cause the problem.

Cu Selur

[locotus]

Thanks Selur, working now.