Hybrid(Windows/Linux/Mac): Input -> x264/x265/Xvid/VP8/VP9

[Selur]

That is really strange, since I use Qt6.4.2 to build Hybrid.
The AppImage should set:
LD_LIBRARY_PATH
QT_PLUGIN_PATH
QML2_IMPORT_PATH
and use the libraries that come with the AppImage
Strange, I tried the image on my Ubuntu system and an ArchLinux system and had no problem.
=> I'll try to look into it.

[Selur]

Uploaded another AppImage , does this behave differently?

Cu Selur

[mindphasar]

Uploaded another AppImage , does this behave differently?

Cu Selur

Thank you for the new upload. Good news: Now the software runs perfectly under Debian testing with AppImage. But there is another problem: it cannot read a mp4 file with messagge: "Found no video or audio data in input" . I tested several music videos with different sizes in mp4 format (x264) and same problem.
I'm sorry, but Unfortunately, Debian testing is quite troublesome compared to ArchLinux or Ubuntu.

[Selur]

Would need a debug output to know what is happening.
This usually only happens if:
a. there is a problem with mediainfo
b. there is a problem with the media itself, i.e. if it is corrupt or encrypted.
c. if the container contains something that can't be handled

Cu Selur

[mindphasar]

Would need a debug output to know what is happening.
This usually only happens if:
a. there is a problem with mediainfo
b. there is a problem with the media itself, i.e. if it is corrupt or encrypted.
c. if the container contains something that can't be handled

Cu Selur

How can I help you with the log using the software?

[Selur]

Create and share a debug log. read this if you are unsure how to create a debug output.

[WinSpirit]

With HandBrakeCLI for windows, since version 1.10.xxx, the AV1 encoder of Radeon ( vce_av1 ) didn't work at all. It's working well before with 1.9.xxx .

[Selur]

Wrong thread? I don't see the connection to Hybrid on "Debian Testing" here,....

[WinSpirit]

Yeah! My bad, sorry

[Compositor]

How can I instruct Hybrid to use the GPU instead of the CPU to do the encoding? I got a new machine and I would really like to do encoding on the GPU instead of the CPU on this machine if possible. The GPU is a RTX 5070 Ti running on an AMD 78003D.

[Selur]

Use NVEnc as encoder

[Compositor]

OK, found NVEnc and tried it out. I hope someone can help me with this part. I had a profile for encoding with X265 that gave results that I liked and I had been using ever since. From what I can see, NVEnc has almost completely different options and even the options that look similar don't always accept the same range of values as for X265. What, if any, of my X265 profile can I adapt to an NVEnc profile that can give similar quality results?

Here is the commandline generated by my X265 profile

--input -
--fps 25/1
--output-depth 10
--y4m
--profile main10
--level-idc 4.1
--limit-tu 4
--me star
--subme 7
--merange 44
--max-merge 5
--no-early-skip
--rskip 0
--no-strong-intra-smoothing
--no-open-gop
--opt-ref-list-length-pps
--keyint 240
--min-keyint 6
--bframes 8
--weightb
--b-intra
--ref 5
--rc-lookahead 70
--qpmax 32
--crf 16.00
--opt-qp-pps
--qpfile GENERATED_QP_FILE
--limit-refs 0
--rd 6
--rd-refine
--psy-rd 1.20
--aq-mode 3
--vbv-maxrate 50000
--vbv-bufsize 50000
--deblock=-2:-2
--no-repeat-headers
--no-mcstf
--range full
--colormatrix smpte170m
--sar 1:1
--output OUTPUTFILE

When it comes to X265 encodes, I find that the "no open GOP" option is a must to produce videos can be joined properly. Does NVEnc's "GOP len 0" mean the same thing?
And does NVEnc's "constant quality" mean the same as X265's "constant rate factor"? From my first experiment, 16.00 for both resulted in a bigger video file from NVEnc than X265.

[Selur]

You used: "--no-open-gop --keyint 240" which should be not using 'strict' and setting gop size to 240.

And does NVEnc's "constant quality" mean the same as X265's "constant rate factor"? From my first experiment, 16.00 for both resulted in a bigger video file from NVEnc than X265.

Yes, both options refer to similar concepts, but do not exactly mean the same. Depending on the other settings setting a constant rate factor will result in different output.

There is no exact/good matching x265 to NVEnc H265.

[Compositor]

There is no exact/good matching x265 to NVEnc H265.

I thought as much but is there anything from the x265 profile worth carrying over to NVEnc?

[Selur]

not really, best play a bit around with NVEnCs H265 to find new settings.

[Compositor]

Does changing anything on the "Frame" tab of the NVEnc tab affect the quality of the output video or just how much processing the encoder does?

[Compositor]

After some fiddling, this is the NVEnc profile I've got:

NVEnc
--y4m
-i -
--fps 23.976
--codec h265
--profile main10
--level auto
--tier high
--sar 1:1
--lookahead 16
--output-depth 10
--vbr 0
--vbr-quality 18.00
--aq
--aq-strength 3
--aq-temporal
--gop-len 240
--ref 7
--bframes 5
--bref-mode auto
--lookahead 16
--mv-precision Q-pel
--lookahead-level 0
--preset quality
--colorrange limited
--colormatrix bt709
--colorrange limited
--cuda-schedule sync
--keyfile GENERATED_KEY_FILE
--output "H:\Temp\00004_new.265

Am I doing things right? I'm aiming for quality over performance settings but these settings result in processing that's over 3X faster than the playback fps and it only uses one third of the GPU's video encoding capacity. If this is indeed quality settings, I'm interested in suggestions for pushing the quality even higher because my GPU has the power to spare.

When I was looking for the best X265 profile to use, I kept encoding the same video with varying settings, not matter how they looked, I never felt satisfied. Then, I came to the one I posted earlier, and the video just looked right. I used that profile ever since.

So far, I haven't gotten the same feeling from any of the NVEnc encodes I used. Even the one I posted here feels just "ok".

[Selur]

Does changing anything on the "Frame" tab of the NVEnc tab affect the quality of the output video or just how much processing the encoder does?

Everything you change that does not just change an output flag changes how much the encoder is processing.


Settings seem fine to me.

[Compositor]

Is the chroma quantizer offset actually set by Hybrid when using NVEnc to encode? I see that changing the value doesn't add or change anything in the commandline window, unlike for anything else.

[Selur]

No, that's a bug. => uploaded a new dev (Hybrid_dev_2025.12.17-75452) which should fix it.

Cu Selur

[Compositor]

That did fix it. Question, I read that the spatial and temporal AQ are automatically set by NVEnc if not specified by the user. But Hybrid's help on those AQ is "en/disable AQ" I'm sure setting a number for AQ instructs NVEnc to use that level of AQ, right? But if both AQ are unchecked, NVEnc outright doesn't use AQ? Then, how to instruct NVEnc to automatically decide how to apply AQ?

[Selur]

--aq
Enable adaptive quantization in frame (spatial). (Default: off) Default depends on --preset and --tune.

--aq-temporal
Enable adaptive quantization between frames (temporal). (Default: off) Default depends on --preset and --tune.

--aq-strength <int>
Specify the AQ strength. (1 (weak) - 15 (strong), 0 = auto) Default depends on --preset and --tune.

see: https://github.com/rigaya/NVEnc/blob/master/NVEncC_Options.en.md
=> enabling adaptive quantization and setting the strength to 0 should do what you want.

[Ol Bob]

I received an error 1 for different files and now I don't receive an error, it just seems to never start processing. I had been processing videos for a few days, then it just stopped working with an error 1 code. Tried different files, uninstalled/reinstalled and deleting temp folders.

Win Versions: 2025.11.09.1 and 2026.01.02.1
Win 11 / 128 Ram / i9-14900K / NVIDIA 4090
Debug output attached.

[Selur]

Try:
0. uninstall Hybrid will all it's settings (check that the Hybrid folder and the se
1. clearing your temp folder
2. removing old jobs and debug output files.
3. creating a new debug output
the debug output contains the creating of one job, but the failed processing of another,...

Cu Selur

Ps.: Why are you using Avisynth and not Vaporusynth?

[Ol Bob]

Seems to be working again. Instead of just deleting the AppData/Local/Temp files, I also deleted all the Windows temp folders.

I noticed the previous job in the debug output, but couldn't figure out where Hybrid was finding the files. So, each time I uninstalled Hybrid, I made sure and deleted the Hybrid folder and deleted files in the AppData/Local/Temp folder. I also tried installing Hybrid on a different drive and it still found older jobs. Hybrid must have been scanning a different temp folder in Windows. Thank you for the assistance.

As for using Avisynth, a long time ago I was getting a glitch each second of my video when removing pulldown using Vapoursynth in Hybrid. When I searched around for others having the same issue, it was mentioned to use Avisynth in Hybrid instead of Vapoursynth. That fixed the issue and I guess it's just a habit now.

Hybrid is an incredible program that will do most anything I need when it comes to video.