Hybrid(Windows/Linux/Mac): Input -> x264/x265/Xvid/VP8/VP9

[Selur]

Yeah, should be easily fixable, I accidentally added the name not the index.
=> Uploaded a new dev that should fix it.

Cu Selur

[Platos]

Thanks again. Works !

[Selur]

Thanks for the feedback.

[Platos]

I have a (two) Question about Avisynth SMDegrain:

- In Tooltip it says "tr 4+ requires Dither's MVTools2 mod". Is this already integrated or where can i get that?
- Is it possible, that Avisynth tr 4 does not help much? When i compare Vapoursynth tr1,tr2,tr3 they all have huge differences. But Avisynth tr4 does have nearly no difference to tr3. Avisynth tr1 has also a huge difference to avisynth 3. But from 3 to 4 there isn't much difference anymore. Is this normal?

[Selur]

Yes, everything needed is included.
Yes, it's normal that with increasing temporal radius the increase in the degraining effect is lower and the change of getting ghosting increases.
Usually going beyond 3 is a bad idea, better increase SAD value or use another filter.

[Platos]

Ok, thanks!

[mwesten]

I've encountered a few instances where a minor glitch or a hiccup in the source video causes the encoding process in Hybrid to abruptly abort and remux with no errors reported, leaving me with a outputted container file that has full duration audio but only partially completed video.
This occurs when using VIVTC or TIVTC to inverse telecine, but does not occur when I turn those off.
I've tried other software ranging from handbrake, to avidemux, to staxrip, and these will simply passthru the glitch/hiccup and continue encoding the video to completion. (StaxRip successfully completes even with TIVTC turned on)
Is there some setting or method to configure in Hybrid so that it continues encoding the video to completion?
Example attached:

[october262]

I've encountered a few instances where a minor glitch or a hiccup in the source video causes the encoding process in Hybrid to abruptly abort and remux with no errors reported, leaving me with a outputted container file that has full duration audio but only partially completed video.
This occurs when using VIVTC or TIVTC to inverse telecine, but does not occur when I turn those off.
I've tried other software ranging from handbrake, to avidemux, to staxrip, and these will simply passthru the glitch/hiccup and continue encoding the video to completion. (StaxRip successfully completes even with TIVTC turned on)
Is there some setting or method to configure in Hybrid so that it continues encoding the video to completion?
Example attached:

see if this thread helps - https://forum.videohelp.com/threads/401305-TIVTC-settings

[mwesten]

I've encountered a few instances where a minor glitch or a hiccup in the source video causes the encoding process in Hybrid to abruptly abort and remux with no errors reported, leaving me with a outputted container file that has full duration audio but only partially completed video.
This occurs when using VIVTC or TIVTC to inverse telecine, but does not occur when I turn those off.
I've tried other software ranging from handbrake, to avidemux, to staxrip, and these will simply passthru the glitch/hiccup and continue encoding the video to completion. (StaxRip successfully completes even with TIVTC turned on)
Is there some setting or method to configure in Hybrid so that it continues encoding the video to completion?
Example attached:

see if this thread helps - https://forum.videohelp.com/threads/401305-TIVTC-settings

I skimmed through that thread, I'm not seeing how that helps with this.
Ultimately I'm trying to figure out how to get Hybrid to not abort the video encode when it encounters this particular spot in the video.
I assume it should be possible since StaxRip is able to do so even with TIVTC turned on.

[Sharc]

No problem here with your sample file using Hybrid's TIVTC (Avisynth). Maybe you submit the debug output to Selur's forum.

[mwesten]

No problem here with your sample file using Hybrid's TIVTC (Avisynth). Maybe you submit the debug output to Selur's forum.

Ah okay. It may be a vapoursynth issue specifically since that's what I was using. The StaxRip build on my other PC uses avisynth.

[Selur]

It usually depends on the source filter, that said using your 'video-glitch-example-clip.mkv' sample with Vapoursynth and different source filters (DGDecNV, Bestsource(cpu), Bestsource(cuda), LWLibavSource(cpu), LWLibavSource(cuda), FFMS2) and TIVTC, I had no problem.
=> maybe check if you can produce the problem on your system with that sample if you do, share a debug output, so I can see what your script looked like.

Cu Selur

[mwesten]

=> maybe check if you can produce the problem on your system with that sample if you do, share a debug output, so I can see what your script looked like.

Cu Selur

I ran it twice using VIVTC (Vapoursynth) and TIVTC (Vapoursynth)

[Selur]

Ah, you are on Linux and you use an old version (2024.09.29.1).
=> Try disabling BestSource and if you use BestSource update the used version.

[mwesten]

=> Try disabling BestSource

Where? How?

[Selur]

Disable "Filtering->Vapoursynth->Misc->Source->Prefer BestSource"

[mwesten]

Thanks. That worked.

[Platos]

When i use RIFE the black bars of the Video gets brighter. When i use RIFEmlrt it does not happen.

How can this be?

I used this video (i know this is heavy grain source. Im Degraining atm): https://uploadnow.io/f/1Cv0dLn

And another Question: When i use crf 14 on x265 and use RIFE, why do i get same File-size ? i thought crf does hold each frame at specific quality-level. Should it not be then ~2.5x filsize from 24fps to 60 FPS ? Or how can i match same Quality per Frame, when using RIFE ? Just lower crf number or using sth else (variable bitrate or quantisizer) ?

[Selur]

When i use RIFE the black bars of the Video gets brighter. When i use RIFEmlrt it does not happen.

How can this be?

Can't reproduce this here.

source and RIFE interpolated output both have no change in luma range here.
Probably some TV vs PC luma scale issue. My guess is that for some reason the output luma range is wrongly signaled.
(Here with x265, Hybrid properly signals '--range limited' for your source.)

And another Question: When i use crf 14 on x265 and use RIFE, why do i get same File-size ?

Compared to not using crf 14 or compared to the original?
Non-lossless reencoding will always potentially destroy details and create compression artifacts.

Cu Selur

[Platos]

Thanks for your answer!

I tried it again (one time with scene change, one time without and with different RIFE-Models). I used this code and it still happens (this is default hybrid settings beside of x265, crf 14 and RIFE-settings):

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import validate
# Source: 'C:\Users\Gaming-Tower\Videos\Test_kurz.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, format: AVC
# Loading C:\Users\Gaming-Tower\Videos\Test_kurz.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Gaming-Tower/Videos/Test_kurz.mkv", format="YUV420P8", stream_index=0, cache=0, fpsnum=24000, fpsden=1001, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
clip = core.misc.SCDetect(clip=clip,threshold=0.070)
# adjusting color space from YUV420P8 to RGBS for vsRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")
# adjusting frame count&rate with RIFE, target fps: 60fps
clip = core.rife.RIFE(clip, model=58, fps_num=60, fps_den=1, sc=True) # new fps: 60
# adjusting output color from: RGBS to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 60fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60, fpsden=1)
# output
clip.set_output()

Did you compare inside Vapoursynth-Preview or sth like that or really the resulting files? Because i can't see it in Vapoursynth-Preview. Only on resulting files. When i open 2 hybrid instances and compare original and RIFE preview, i can't see any difference. And best way i think is to make both videos fullscreen and then with alt+tab switch. Because on small windows it's not so easy to see.

When i use VS-MLRT-RIFE i have this vapoursaynth-code (and it does not happen there):

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import site
from fractions import Fraction
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
os.environ["CUDA_MODULE_LOADING"] = "LAZY"
# loading plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vs-mlrt/vsort.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/akarin.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
from importlib.machinery import SourceFileLoader
vsmlrt = SourceFileLoader('vsmlrt', 'C:/Program Files/Hybrid/64bit/vs-mlrt/vsmlrt.py').load_module()
from vsmlrt import Backend
import validate
# Source: 'C:\Users\Gaming-Tower\Videos\Test_kurz.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 23.976fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709, format: AVC
# Loading C:\Users\Gaming-Tower\Videos\Test_kurz.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Gaming-Tower/Videos/Test_kurz.mkv", format="YUV420P8", stream_index=0, cache=0, fpsnum=24000, fpsden=1001, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 709.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT709)
# setting color transfer (vs.TRANSFER_BT709), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT709)
# setting color primaries info (to vs.PRIMARIES_BT709), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT709)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
clip = core.misc.SCDetect(clip=clip,threshold=0.100)
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=4, bottom=4) # add borders to archive mod 32 (vsRIFEmlrt) - 1920x1088
# adjusting color space from YUV420P8 to RGBS for vsRIFEmlrt
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")
# adjusting frame count&rate with RIFE (mlrt)
clip = vsmlrt.RIFE(clip, multi=Fraction(2), model=418, backend=Backend.ORT_CUDA(fp16=False,device_id=0)) # new fps: 47.952
clip = core.std.Crop(clip=clip, left=0, right=0, top=4, bottom=4) # removing borders (vsRIFEmlrt) -  1920x1080
# adjusting output color from: RGBS to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 47.952fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=48000, fpsden=1001)
# output
clip.set_output()

And about the other question: I mean when i use crf 14 with and without RIFE, i get more or less the same filesize compared to crf 14 (RIFE is even a bit smaller). But i thought crf does make the frames match a specific quality. So when i use RIFE crf 14, shouldnt it use (more or less) the same filesize per frame (and therefore 2.5x times more from 24fps to 60fps) ? Because i got hard compression-artifacts on this scene above with crf 14 + RIFE. I had to use crf 12. But it is not so important. Other problem is more important

[Platos]

Edit: Actually i only use RIFE because mlrt-RIFE can not make 60FPS from 24FPS. It's also way slower than mlrt-RIFE.

Anyway, didn't you had earlier this implemented or am i dreaming ? https://github.com/HolyWu/vs-rife
Because this can make 60FPS from 24FPS. But maybe it is slower, i don't know.

[Selur]

1. I compared the Preview, checked the encoding calls and looked at an encoded output. (comparing source to reencode with RIFE)
2. Hybrid does support:
a. RIFE = https://github.com/styler00dollar/VapourSynth-RIFE-ncnn-Vulkan
b. TorchRIFFE = https://github.com/HolyWu/vs-rife (assuming torch-add-on is present)
c. RIFEmlrt = https://github.com/AmusementClub/vs-mlrt/blob/b136ab7a1dbf78f5be1d10d982e3080a20549e9f...smlrt.py#L1164 (assuming vs-mlrt-add-on is present) All of them can create 60fps from 24fps fine here.

=> My guess is either:
a. your way of comparing the outputs is flawed (i.e. media player is not behaving as you expected) or
(I used a Vapoursynth script to compare the outputs)
b. your encoding and/or muxing calls signals the wrong or no luma range

But i thought crf does make the frames match a specific quality.

That is only true when you have the same source, adding frames changes the source and thus what is the effect of cfr XY.

. So when i use RIFE crf 14, shouldnt it use (more or less) the same filesize per frame (and therefore 2.5x times more from 24fps to 60fps) ?

No, that is not how crf does work. A constant rate factor does not work as you make it out to be.
The same crf will output different qualities per frame if the encoding settings (or the source) changes.

Cu Selur

Ps.: https://www.researchgate.net/publication/4289294_Improved_Rate_Control_and_Motion_Esti...r_H264_Encoder is probably the paper to read for crf.

Here's how comparing (ncnn, mlrt, torch) RIFE looks like to me:

using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Source: 'G:\Output\RIFE_ncnn.mp4'
clip1 = core.lsmas.LWLibavSource(source="G:/Output/RIFE_ncnn.mp4", format="YUV420P10", stream_index=0, cache=0, fpsnum=60, prefer_hw=0).text.Text("ncnn")
clip2 = core.lsmas.LWLibavSource(source="G:/Output/RIFE_mlrt.mp4", format="YUV420P10", stream_index=0, cache=0, fpsnum=60, prefer_hw=0).text.Text("mlrt")
clip3 = core.lsmas.LWLibavSource(source="G:/Output/RIFE_torch.mp4", format="YUV420P10", stream_index=0, cache=0, fpsnum=60, prefer_hw=0).text.Text("mlrt")
clip = core.std.StackHorizontal([clip1, clip2, clip3])
# output
clip.set_output()

=> can't spot a difference in the luma range (argh, typo, last one should be torch, but too lazy to fix)

[Platos]

Oh wait what? RIFEmlrt can now finally use specific framerate instead of just a multiplicator ? Because this was not possible in past. I thought vsmlrt can still not use that because of this: https://github.com/AmusementClub/vs-mlrt/issues/59

So is it really only mlrtRIFE which does the 60FPS? Or is hybrid just using 2x mlrtRIFE and then sth else for the rest to reach 60FPS?

If so, then i'm fine with using finally mlrtRIFE instead of normal RIFE. (But i tested it with VLC, MPC and Windows internals movie-players and all did show brighter black bars. Maybe it's because of my new Display, which has now very good black-values. Maybe you need good black values, to even see the difference. But i don't know.)

And thanks for clarification for crf. I used crf 14 with RIFE, which looked fine for most scenes, but on these black scenes i got these horrible compression-blocking artifacts which you can see on Streaming often.

[Platos]

[Attachment 86256 - Click to enlarge]
[Attachment 86257 - Click to enlarge]

If it's interesting for you, here are 2 screenshots made directly in MPC

Maybe you have to open the pictures in a new tab and compare fullscreen. But im ok with using mlrtRIFE now. Now, when i know it works with 60FPS.

Edit: The darker one is with mlrtRIFE, the other with normal RIFE.

[Selur]

RIFEmlrt can now finally use specific framerate instead of just a multiplicator ?

multi, can be a fraction, look at the code you posted:

multi=Fraction(2)

I was added in the end of 2023, see: https://github.com/AmusementClub/vs-mlrt/releases/tag/v14.test3

[Platos]

Nice!

Then my Problem is gone

[Platos]

Hi

I think sth is wrong with torchRIFE. No matter what SceneChange-value i take, it does not work. When i use mlrtRIFE with 0.07 scene changes, it works fine. But when i use torchRife with 0.07 it does screw up every scene change. I tried 0.05, 0.04, 0.03, 0.01, 0.1, 0.15 and it does not change the behavior.

Then there is a second problem: When i change Cuda Streams it crashes. When i use FP16 it crashes. When i use Ensamble it crashes and when i use TensorRT it crashes (i mean on torchRIFE). I can only change Scale and scene change and model and factor. Vapoursynth Preview does not work on these cases. It happens on all videos for me. I tried x264 and x265, it does not work on both.

[Selur]

If the preview does not work, the encoding will not work either.

Did a quick test with RIFEmlrt and had no issue with it. (Using 'DML (GPU)' and 'TorchRT' as backend with&without FP16 and Ensamble dis-/enabled and 1-3 streams)
Worked fine here.
=> not knowing what version you are using and what gpu drivers you are using, I would suggest updating your drivers.

TorchRIFE does have problems here too.
with TensorRT it crashes and sc_threshold doesn't seem to work either. => Will look into it later and report back.
Cleaning up my engine folder helped with the crash, bat scene change doesn't seem to work here either.
The generated code does seem fine. Created an issue entry over at: https://github.com/HolyWu/vs-rife/issues/74 (my guess atm. is that it's some torch incompatibility with the version I use in the troch-addon)

Cu Selur

[Selur]

DOH, thanks to HolyWu I found the problem. Hybrid falsely assumes sc=True is the default, when it's sc=False.

Uploaded a dev version which fixes this. For me, both TorchRIFE and RIFEmlrt work fine with this version.
(Make sure to clear your engine-folder, on torch- or mlrt-updates; and sometimes driver updates. )

Cu Selur

[Platos]

mlrt Rife did work fine for me (but torchRife did not).

When mlrtRIFE and torchRIFE both use tensorRT, are the .engine files mixed in the same folder? So can i see which .engine file is from mlrtRIFE or torchRIFE?

And ok, thanks. I will try the dev-version.