Hybrid(Windows/Linux/Mac): Input -> x264/x265/Xvid/VP8/VP9

[Selur]

Happy that got resolved.
If you use crf encoding, this can also happen if your source is rather dark.

Cu Selur

[Compositor]

A quick update on my situation from this post. Today, I was finally able to perform a definitive test to find out what was going on. I encoded a new version of a video I had encoded using the 11.28 version 2 weeks ago. The earlier encode was 647mb for the video stream, the new encode is just 370mb. Since they were both encoded from the same Bluray raw, the issue is in the settings or the encoder. Will look into it more, but have to go now.

[Selur]

If you can create two debug outputs (one for reach method), it should be easy to tell whats happening.

[Compositor]

If it was just a matter of different settings on the same version of Hybrid, it should be that easy. But it looks like I need to test the same settings on different Hybrid versions instead. But I still have install file of 2022.11.28 and 2022.3.20 with me.

[Selur]

Okay, different settings (encoding&filter) will result in different results.

[Compositor]

I did a test by encoding the same source video twice. One using Hybrid 2022.03.20, the version I used most of last year with satisfactory results, the other using 2022.12.12, the test version you gave me last time. Both used the same encode settings. I uninstalled one version along with removing the settings before installing the other for the test. Both tests used the same copy of the encode settings I exported years ago as backup. The 3.20 encode was 113mb while the 12.12 encode was only 78.8mb.

The debug report for both encode jobs are included here.

[Selur]

=> 2022.12.12 version uses '--sbrc' the 2022.03.22 does not.
sbr = segment based rate control is under "x265->Quantization->Adpative Quantizatuin->SBRC" (it's only available when adaptive quantization is turned on)
From the looks of it, it's not set my any preset or tune, so you must have set it manually yourself, and it didn't come from some default. (in Hybrid it's disabled by default)

Cu Selur

[Compositor]

Let me see if I got this right. If '--sbrc' appears in the commandline, it means sbrc is on, correct? But in my Hybrid UI, SBRC is unchecked, but that line is in the commandline. But if I check the option, the '--sbrc' disappears from the commandline.
You wouldn't happened to have flipped which action turns on sbrc, did you?

[Selur]

LOL, you are right I totally overlooked that.
-> will fix.

[Compositor]

I did a few more tests, not encodes. It may not be as simple as a flipped behavior. On starting Hybrid, SBRC is indeed disabled when no adaptive quantization is selected, sbrc also doesn't appear in the commandline. However, when I select an AQ mode, SBRC is automatically set in the commandline- even though the check box isn't checked. After this, the check box indeed works in the opposite manner from what it's supposed to. Since SBRC is automatically triggered when an AQ mode is selected, SBRC is automatically selected when the user loads a setting that uses an AQ mode, whether they had chosen to use SBRC or not.

This explains why another of my settings produced an encode of a size as what I was expecting. That particular setting doesn't use AQ unlike my preferred one.

[Selur]

However, when I select an AQ mode, SBRC is automatically set in the commandline- even though the check box isn't checked.

That does not happen here, before:

and after changing mode:


Disabling minimize command line:
Before:

x265 --frame-threads 0 --wpp --no-pmode --no-pme --log-level 2 --input - --output-depth 10 --y4m --profile main10 --high-tier --min-cu-size 8 --ctu 32 --qg-size 32 --no-opt-cu-delta-qp --tu-intra-depth 1 --tu-inter-depth 1 --limit-tu 0 --max-tu-size 32 --no-hme --me hex --subme 2 --merange 57 --no-analyze-src-pics --limit-modes --no-rect --no-amp --max-merge 3 --no-early-skip --rskip 1 --temporal-mvp --rdpenalty 0 --no-tskip --strong-intra-smoothing --no-constrained-intra --no-open-gop --gop-lookahead 0 --scenecut 40 --scenecut-bias 5 --scenecut-aware-qp 0 --no-hist-scenecut --opt-ref-list-length-pps --keyint 250 --min-keyint 0 --max-ausize-factor 1 --radl 0 --bframes 4 --no-weightb --no-b-intra --bframe-bias 0 --b-adapt 2 --b-pyramid --ref 3 --weightp --rc-lookahead 20 --slices 1 --lookahead-threads 0 --lookahead-slices 0 --qpstep 4 --qpmin 0 --qpmax 69 --qblur 0.50 --cplxblur 20.0 --crf 18.00 --opt-qp-pps --no-rc-grain --cbqpoffs -2 --crqpoffs -2 --ipratio 1.40 --pbratio 1.30 --nr-intra 0 --nr-inter 0 --qpfile GENERATED_QP_FILE --limit-refs 0 --rd 3 --no-fast-intra --ssim-rd --no-rd-refine --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --signhide --no-splitrd-skip --qcomp 0.60 --qp-adaptation-range 1 --no-aq-motion --aq-mode 0 --no-hevc-aq --no-const-vbv --vbv-maxrate 0 --vbv-bufsize 0 --vbv-init 0.90 --vbv-end 0.000 --min-vbv-fullness 50.00 --max-vbv-fullness 80.00 --no-hrd --no-aud --info --no-idr-recovery-sei --deblock=-1:-1 --sao --selective-sao 4 --no-sao-non-deblock --limit-sao --no-repeat-headers --no-temporal-layers --log2-max-poc-lsb 8 --no-fades --no-psnr --no-ssim --no-interlace --range limited --colormatrix bt709 --no-single-sei --no-hdr10 --output "J:\tmp\sample-clip.mov.265"

after switching mode:

x265 --frame-threads 0 --wpp --no-pmode --no-pme --log-level 2 --input - --output-depth 10 --y4m --profile main10 --high-tier --min-cu-size 8 --ctu 32 --qg-size 32 --no-opt-cu-delta-qp --tu-intra-depth 1 --tu-inter-depth 1 --limit-tu 0 --max-tu-size 32 --no-hme --me hex --subme 2 --merange 57 --no-analyze-src-pics --limit-modes --no-rect --no-amp --max-merge 3 --no-early-skip --rskip 1 --temporal-mvp --rdpenalty 0 --no-tskip --strong-intra-smoothing --no-constrained-intra --no-open-gop --gop-lookahead 0 --scenecut 40 --scenecut-bias 5 --scenecut-aware-qp 0 --no-hist-scenecut --opt-ref-list-length-pps --keyint 250 --min-keyint 0 --max-ausize-factor 1 --radl 0 --bframes 4 --no-weightb --no-b-intra --bframe-bias 0 --b-adapt 2 --b-pyramid --ref 3 --weightp --rc-lookahead 20 --slices 1 --lookahead-threads 0 --lookahead-slices 0 --qpstep 4 --qpmin 0 --qpmax 69 --qblur 0.50 --cplxblur 20.0 --crf 18.00 --opt-qp-pps --no-rc-grain --cbqpoffs -2 --crqpoffs -2 --ipratio 1.40 --pbratio 1.30 --nr-intra 0 --nr-inter 0 --qpfile GENERATED_QP_FILE --limit-refs 0 --rd 3 --no-fast-intra --ssim-rd --no-rd-refine --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --signhide --no-splitrd-skip --qcomp 0.60 --qp-adaptation-range 1 --no-aq-motion --aq-mode 1 --aq-strength 1 --no-sbrc --no-cutree --no-const-vbv --vbv-maxrate 0 --vbv-bufsize 0 --vbv-init 0.90 --vbv-end 0.000 --min-vbv-fullness 50.00 --max-vbv-fullness 80.00 --no-hrd --no-aud --info --no-idr-recovery-sei --deblock=-1:-1 --sao --selective-sao 4 --no-sao-non-deblock --limit-sao --no-repeat-headers --no-temporal-layers --log2-max-poc-lsb 8 --no-fades --no-psnr --no-ssim --no-interlace --range limited --colormatrix bt709 --no-single-sei --no-hdr10 --output "J:\tmp\sample-clip.mov.265"

=> only '--no-sbrc' (not '--sbrc') was added, which is correct.

--aq-mode 0 --no-hevc-aq

changed to

--aq-mode 1 --aq-strength 1--no-sbrc --no-cutree

Cu Selur

[Compositor]

You fixed it? I'll be happy to try it out if you give me a link.

EDIT: I encoded the video from this post again. This time I checked the SBRC option and it disappeared from the commandline. The result was 113mb, like the 3.20 encode. I attached the debug report for your reference.

[Selur]

Send you a link, try if that solves the issue for you.

[Compositor]

This version does seem to properly enable or disable SBRC as set by the user. It doesn't appear on the commandline until the option is actually checked.
I am encoding the video again. Two versions, one with and one without SBRC checked or on the commandline. I'm calling it a night now. I'll let you know how it turns out in a few hours.

[Selur]

Good night.

[Compositor]

Good news. No SBRC was 113mb and SBRC was 78.8. Exactly as intended. Attaching the debug report for your reference. Because I was asleep, both runs are in the same report.
Glad that was settled. I'm working on the series that made me want to do encoding in the first place and I finally got proper raws to do it with. So I want them looking their best.

[Selur]

I'm happy too, that this issue was resolved.

Cu Selur

[ampersand]

I'm using Hybrid's QTGMC deinterlacer on some DVDs I have and the results are great, but there's one thing on my mind that's been niggling away at me:

I have QTGMC set to bob (which doubles the framerate) because when I tested it with the regular framerate the problem I had (jerky video at times) was still present even though very much less than before (without QTGMC). Now, the question is: if I'm using constant kbps when encoding, does that mean that if the framerate is doubled, the quality is effectively half that which it would be with the regular framerate? So a 1700 kbps video at 50 fps will have the quality of a 850kbps at 25 fps?

[Selur]

About the 'jerky video': I would need a sample to suggest anything.
About 'double frame rate/count => double the amount of bit rate required': No, bob deinterlacing does not double the mount of unique data it does create double the raw data, but lots of this data is not unique. Encoding formats have come a long way in the last 20+years and they have become really good at finding duplicate data and the likes.

Cu Selur

[sophisticles]

Selur,

Any chance of adding support for braw?

https://github.com/AkBKukU/braw-decode

It looks like it should be relatively easy since someone has already used the sdk to make a program that decodes braw and pipes it to ffmpeg.

[Selur]

I doubt that:

braw-decode sample.braw | ffmpeg -y $(braw-decode -f sample.braw) -c:v hevc_nvenc output.mov

can work on Windows. (The '$(braw-decode -f sample.braw)' part will not work on a Windows terminal.)
Also, neither Vapoursynth nor Avisynth could be used with this.
To use it in Hybrid you would have to call first.

Seems like
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20191214115705.25497-1-onemda@gmail.com/#49242
didn't make it into libav, otherwise you could open it in Avisynth and Vapoursynth.

=> don't really see how this could be integrated into Hybrid, without creating huge temp files, which would kill the purpose of the whole thing.

Cu Selur

[Compositor]

Is it possible to disable the creation of the Global Tags track when creating mkvs? While I can discard it after encoding, it would be better if it wasn't created in the first place.

[Selur]

No, global tags are always written since the 'Writing application'/'Encoding Gui' info is always set and there are no plans to change this.

[ampersand]

I set up a DVD rip to use 2-pass with 1700kbps bitrate, but the resulting file has a video bitrate of less than 700kbps?
Other DVD files work fine, is it possible the presence of 2 audio files in this one is confusing Hybrid?

Setup:

x265 --input - --output-depth 10 --y4m --profile main10 --limit-modes --no-open-gop --opt-ref-list-length-pps --pass 1 --no-slow-firstpass --bitrate 1700 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --qpfile GENERATED_QP_FILE --limit-refs 0 --ssim-rd --rdoq-level 2 --psy-rdoq 1.00 --aq-mode 0 --deblock=-2:-2 --no-sao --selective-sao 0 --no-repeat-headers --range limited --colormatrix bt709 --no-strong-intra-smoothing --stats "D:\Download\Temp\Hybrid\ABCD_generated.stats" --multi-pass-opt-analysis --multi-pass-opt-distortion --analysis-reuse-file "D:\Download\Temp\Hybrid\ABCD_generated.analysis" --output NUL

x265 --input - --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --pass 2 --bitrate 1700 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --rdoq-level 2 --psy-rdoq 1.00 --aq-mode 0 --deblock=-2:-2 --no-sao --selective-sao 0 --no-repeat-headers --range limited --colormatrix bt709 --no-strong-intra-smoothing --stats "D:\Download\Temp\Hybrid\ABCD_generated.stats" --multi-pass-opt-analysis --multi-pass-opt-distortion --analysis-reuse-file "D:\Download\Temp\Hybrid\ABCD_generated.analysis" --output "D:\Download\Temp\Hybrid\ABCD.265"

[Compositor]

is it possible the presence of 2 audio files in this one is confusing Hybrid?

Perhaps. This seems easy enough to test, discard or disregard one of the audio streams and encode again to see what the result is like.

[Selur]

I doubt it's related to the audio count, more likely to the fact that Avisynth&Vapoursynth only support using the first program chain.
Would need a debug output of the source analysis and the job creation to know what's causing the issue.

Cu Selur

[ampersand]

I doubt it's related to the audio count, more likely to the fact that Avisynth&Vapoursynth only support using the first program chain.
Would need a debug output of the source analysis and the job creation to know what's causing the issue.

Cu Selur

All I have is this, from the "Log" window in Hybrid:

(I replaced the video name with "ABCD")

Analyzing 'ABCD.mkv'.
Checking a/v ids with FFMpeg VideoAnalyser,...
Grabbing audio ids for D:\Handbrake\ABCD\ABCD.mkv.
-> Found the following audio ids: 0, 1
Analyzing audio streams,...
Analyzing audio stream with id '0'.
Analyzing audio stream with MPlayer AudioAnalyser for id '0',...
Grabbing audio data and delay infos,...
Analyzing audio stream with id '1'.
Analyzing audio stream with MPlayer AudioAnalyser for id '1',...
Grabbing audio data and delay infos,...
Grabbing video ids,...
-> Video ids found: 0
Analyzing video streams...
Analyzing video stream (0).
=> Detected stretch factor for track 0 is 4/5.
Prefering container (23.976) over stream frame rate (29.970)
Notice: Video stream raw and container lenght differ, other streched streams might exist.
Extracting time codes from mkv for track with id 0.
Adjusting frame count (125039) to time code entry count (125041).
starting auto routines for source number: 1
Set inputpath to: D:\Handbrake\ABCD
No input color matrix found, guessing by format and resolution,..
selecting 'bt709' since input is MPEG-2 and TV scale.
-> Assuming color matrix to be: bt709
-> finished auto routines for source number: 1
Input is completely analysed,...
Set inputpath to: D:\Handbrake\ABCD
No input color matrix found, guessing by format and resolution,..
selecting 'bt709' since input is MPEG-2 and TV scale.
-> Assuming color matrix to be: bt709
Creating job for current source
Current input: D:\Handbrake\ABCD\ABCD.mkv
-> Creating jobs for source: (1)
-> Generating calls for: D:\Handbrake\ABCD.mkv
adding 5 video related calls for source: 1
createJobs for D:\Handbrake\ABCD.mkv
creating audio subjobs,...
creating attachment extraction subjobs
creating video subjobs
creating muxing subjobs
optimizing the subJobs
-> finished creating subjobs for current source
added new job with id 2023-02-23@11_19_26_4710
starting 2023-02-23@11_19_26_4710_01_audio@11:19:51.403 - D:\Handbrake\ABCD.mkv
2023-02-23@11_19_26_4710_01_audio finished after 00:00:02.546
starting 2023-02-23@11_19_26_4710_03_video@11:19:54.017 - D:\Handbrake\ABCD.mkv
2023-02-23@11_19_26_4710_03_video finished after 00:48:50.047
starting 2023-02-23@11_19_26_4710_04_video@12:08:44.071 - D:\Handbrake\ABCD.mkv
2023-02-23@11_19_26_4710_04_video finished after 01:14:45.063
delete D:\Download\Temp\Hybrid\ABCD_generated_2023-02-23@11_19_26_4710_04.stats
delete D:\Download\Temp\Hybrid\ABCD_generated_2023-02-23@11_19_26_4710_04.analysis
delete D:\Download\Temp\Hybrid\ABCD_generated_2023-02-23@11_19_26_4710_04.stats.cutree
starting 2023-02-23@11_19_26_4710_09_muxing@13:23:29.553 - D:\Handbrake\ABCD.mkv
2023-02-23@11_19_26_4710_09_muxing finished after 00:00:15.499
delete D:\Download\Temp\Hybrid\mkvtags_2023-02-23@11_19_26_4710__09.xml
delete D:\Download\Temp\Hybrid\timecodeV2_2023-02-23@11_19_26_4710.tc
delete D:\Download\Temp\Hybrid\2023-02-23@11_19_26_4710_05.265
delete D:\Download\Temp\Hybrid\iId_3_aid_0_lang_en_2023-02-23@11_19_26_4710_01.ac3
delete D:\Download\Temp\Hybrid\iId_4_aid_1_lang_en_2023-02-23@11_19_26_4710_02.ac3
Job 2023-02-23@11_19_26_4710 finished!

[Selur]

That doesn't help me at all, but according to that the input is an .mkv not a DVD.

Prefering container (23.976) over stream frame rate (29.970)

this might cause issues if following that the length is calculated.
https://www.selur.de/support describes how to create a debug output.

Cu Selur

[ampersand]

The length in the resulting file is same as the input. When I create the debug file I'll post a pastebin here.
The mkv is a result of makemkv, sourced from a DVD.

[Selur]

The question is whether Hybrid correctly identifies the frame count and length, since those are used to adjust the file size during 2pass encoding if 2pass 'Target size' is used.
If 2pass 'Bitrate' is used, Hybrid does not adjust the bit rate and it's up to the encoder to archive the right size.

If 2pass 'Target size' is used. It's mainly a question whether Hybrid correctly calculates the audio size.
If you first set a 'Target size' and then set the audio, the audio will cause a reduction in the size and thus bit rate of the video.