MeGUI x264 Avisynth encodings keep coming out at different filesizes

[AndersonJames]

I'm using MeGUI 2913 from a fresh "install" (MeGUI doesn't actually install anything). I use FFMS as the indexer but the problem mentioned below will also happen with DGDecodeNV. The latest versions of x264, MKV Merge, etc are used.

I have these 2 short clips:
Wrestlemania Clip is 1080p 30fps
Raw Clip is 1080p 30fps

They are both from the same streaming service and both have a similar bitrate. The Raw clip is native HD. The Wrestlemania clip is an upscale that still looks like SD.

I'm just encoding the clips to x264 CRF 20-26 so I can see how they look at different bitrates.
I'm using the same encoding settings for both files. The problem is that the "Raw" clip once encoded has a different filesize when using the same CRF setting in multiple encodings. I found a way to trigger the bug, encode to CRF 20 Slow for example. Then encode the source to CRF 21 Slow. Then encode the source to CRF 20 Slow again and then to CRF 21 again. You can keep doing that as often as you want. Every time I get a different file size for the same CRF setting using the same settings. For example:

CRF 20 Slow: 91.4 MB, 96.5 MB, 103 MB 
CRF 21 Slow: 103 MB, 91.4 MB, 49.4 MB

Strangely this bug doesn't effect the "Wrestlemania" file which always encodes to the same filesize. Also note that as you can see from the above numbers, sometimes a higher CRF has the same or higher filesize than 1 CRF lower which doesn't make any sense. A more rare example is the CRF 21 sample which gave me a 49.4 MB file which is way too low a filesize of almost 50% less than CRF 20 (and has bitrate of 2.9 Mbps). So you can see why I'm concerned.

These are my x264 flags:

--bluray-compat --pulldown double --keyint 60 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 
--open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1

I'm using keyint 60 just for testing so I can seek to certain points in the video in MPC-HC when watching the videos.

I use this Avisynth script to convert from 30 fps to 29.97 fps:

<input>
AssumeFPS(30000,1001, sync_audio=true)
SSRC(48000)
Spline36Resize(1280,720)

Would a MeGUI log file be of any use to you to find out the issue? I couldn't see anything useful mentioned but then again I don't know what I'm supposed to be looking for.

Attached is my MeGUI install and the 2 source clips. Will be easier for you to just use that rather than me giving a long list of settings for you to use. That way you'll only need to change the following:

Options > Main tab> Default output directory.
Options > External Programs tab. Change NeroAACEnc location to wherever you put MeGUI. Disable DGINdex if you don't have a paid licence for it.

To change the CRF setting, in the Main window click on the "One Click" button on the bottom of the window. Then on the Video tab click on "Config" next to Encoder. Change the number next to Quality then click on the Update button and OK then click on Update again and OK. You can now drag the folder of files into the Input section of the One Click window. After you OK the message it will ask you to enter a prefix for output file names if you want. You might want to just call it "CRF 20 Slow" for example to avoid any confusion in trying to tell your encodings apart.

I've been using MeGUI version 2525 for years and I've never had this problem until MeGUI 2913. Please help!

I've also just done a CRF 20 and CRF 21 encoding of the Raw full episode. Here's the result (log is attached):
CRF 20: 5.68 GB, 6.4 Mbps
CRF 21: 8.05 GB, 9.14 Mbps

Clearly there's something wrong as the CRF 21 encoding should have a lower filesize not a far higher filesize than the CRF 20 file.

[KarMa]

to be edited

[Selur]

Had a first short look:

Every time I get a different file size for the same CRF setting using the same settings.

Different file sizes when encoding different clips with the same settings is to be expected.
Different file sizes when encoding the same clip with the same settings when '--non-deterministic' is to be expected.
let's see if any of this is the case here,....

Looking at logfile-20-100-15_19-09-57.log and the first 4 encoder (got not more time atm.) I see:
1.
a crf 20 encode of Raw 2020-01-06 Clip.mkv

custom command line: --bluray-compat --pulldown double --keyint 60 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
Job command line: "cmd.exe" /c ""Z:\MeGUI\tools\ffmpeg\ffmpeg.exe" -loglevel level+error -i "L:\WWE Test Clips\fzvhiocz.bpy\Raw 2020-01-06 Clip.mkv.avs" -strict -1 -f yuv4mpegpipe - | "Z:\MeGUI\tools\x264\x264.exe" --level 4.1 --bluray-compat --preset slow --crf 20.0 --keyint 60 --weightp 2 --slices 4 --vbv-bufsize 15000 --vbv-maxrate 15000 --qpfile "L:\WWE Test Clips\fzvhiocz.bpy\Raw 2020-01-06 Clip.mkv.qpf" --pulldown double --open-gop --sar 1:1 --frames 4172 --output "L:\WWE Test Clips\fzvhiocz.bpy\Raw 2020-01-06 Clip_Video.264" --stdin y4m -"

with:

LoadPlugin("Z:\MeGUI\tools\ffms\ffms2.dll")
FFVideoSource("L:\WWE Test Clips\Raw 2020-01-06 Clip.mkv", cachefile="L:\WWE Test Clips\fzvhiocz.bpy\Raw 2020-01-06 Clip.mkv.ffindex", fpsnum=30, fpsden=1, threads=1)
AssumeFPS(30000,1001, sync_audio=true)
SSRC(48000)
Spline36Resize(1280,720)

resulting in:

---[Information] [15/10/2020 19:13:40] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
---[Information] [15/10/2020 19:13:40] x264 [info]: using SAR=1/1
---[Information] [15/10/2020 19:13:40] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
---[Information] [15/10/2020 19:13:40] x264 [info]: profile Progressive High, level 4.1, 4:2:0, 8-bit
---[Information] [15/10/2020 19:15:14] x264 [info]: frame I:93    Avg QP:18.67  size:120152
---[Information] [15/10/2020 19:15:14] x264 [info]: frame P:1103  Avg QP:21.64  size: 46014
---[Information] [15/10/2020 19:15:14] x264 [info]: frame B:2976  Avg QP:25.44  size: 13026
---[Information] [15/10/2020 19:15:14] x264 [info]: consecutive B-frames:  3.4%  3.2%  4.1% 89.4%
---[Information] [15/10/2020 19:15:14] x264 [info]: mb I  I16..4:  5.0% 65.3% 29.7%
---[Information] [15/10/2020 19:15:14] x264 [info]: mb P  I16..4:  2.9% 12.4%  2.7%  P16..4: 39.5% 24.3% 12.0%  0.0%  0.0%    skip: 6.3%
---[Information] [15/10/2020 19:15:14] x264 [info]: mb B  I16..4:  0.4%  1.2%  0.3%  B16..8: 42.2% 12.1%  2.8%  direct: 4.2%  skip:36.9%  L0:38.3% L1:44.3% BI:17.4%
---[Information] [15/10/2020 19:15:14] x264 [info]: 8x8 transform intra:66.9% inter:57.0%
---[Information] [15/10/2020 19:15:14] x264 [info]: direct mvs  spatial:99.6% temporal:0.4%
---[Information] [15/10/2020 19:15:14] x264 [info]: coded y,uvDC,uvAC intra: 66.9% 78.7% 52.1% inter: 20.4% 28.0% 8.5%
---[Information] [15/10/2020 19:15:14] x264 [info]: i16 v,h,dc,p: 22% 43% 10% 26%
---[Information] [15/10/2020 19:15:14] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 12% 20% 12%  6%  9%  9% 11%  9% 13%
---[Information] [15/10/2020 19:15:14] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 19%  8%  7% 11% 10% 12%  8% 13%
---[Information] [15/10/2020 19:15:14] x264 [info]: i8c dc,h,v,p: 38% 33% 17% 13%
---[Information] [15/10/2020 19:15:14] x264 [info]: Weighted P-Frames: Y:0.5% UV:0.2%
---[Information] [15/10/2020 19:15:14] x264 [info]: ref P L0: 85.0%  9.8%  5.1%  0.2%
---[Information] [15/10/2020 19:15:14] x264 [info]: ref B L0: 96.8%  3.2%
---[Information] [15/10/2020 19:15:14] x264 [info]: ref B L1: 96.8%  3.2%
---[Information] [15/10/2020 19:15:14] x264 [info]: kb/s:5786.70
---[Information] [15/10/2020 19:15:14] encoded 4172 frames, 43.71 fps, 5786.70 kb/s
---[Information] [15/10/2020 19:15:14] Constant Quality Mode: Quality 20 computed...
---[Information] [15/10/2020 19:15:14] Video Bitrate Obtained (approximate): 5786 kbit/s

2.
a crf 20 encode of WrestleMania 2002 Clips.mkv

custom command line: --bluray-compat --pulldown double --keyint 60 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
Job command line: "cmd.exe" /c ""Z:\MeGUI\tools\ffmpeg\ffmpeg.exe" -loglevel level+error -i "L:\WWE Test Clips\tlksbx5t.rsv\WrestleMania 2002 Clips.mkv.avs" -strict -1 -f yuv4mpegpipe - | "Z:\MeGUI\tools\x264\x264.exe" --level 4.1 --bluray-compat --preset slow --crf 20.0 --keyint 60 --weightp 2 --slices 4 --vbv-bufsize 15000 --vbv-maxrate 15000 --pulldown double --open-gop --sar 1:1 --frames 1350 --output "L:\WWE Test Clips\tlksbx5t.rsv\WrestleMania 2002 Clips_Video.264" --stdin y4m -"

with:

LoadPlugin("Z:\MeGUI\tools\ffms\ffms2.dll")
FFVideoSource("L:\WWE Test Clips\WrestleMania 2002 Clips.mkv", cachefile="L:\WWE Test Clips\tlksbx5t.rsv\WrestleMania 2002 Clips.mkv.ffindex", fpsnum=30, fpsden=1, threads=1)
AssumeFPS(30000,1001, sync_audio=true)
SSRC(48000)
Spline36Resize(1280,720)

resulting in:

---[Information] [15/10/2020 19:14:10] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
---[Information] [15/10/2020 19:14:10] x264 [info]: using SAR=1/1
---[Information] [15/10/2020 19:14:10] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
---[Information] [15/10/2020 19:14:10] x264 [info]: profile Progressive High, level 4.1, 4:2:0, 8-bit
---[Information] [15/10/2020 19:14:46] x264 [info]: frame I:29    Avg QP:16.70  size: 90057
---[Information] [15/10/2020 19:14:46] x264 [info]: frame P:366   Avg QP:22.59  size: 35967
---[Information] [15/10/2020 19:14:46] x264 [info]: frame B:955   Avg QP:25.62  size: 11129
---[Information] [15/10/2020 19:14:46] x264 [info]: consecutive B-frames:  3.6%  3.6%  7.8% 85.0%
---[Information] [15/10/2020 19:14:46] x264 [info]: mb I  I16..4: 24.0% 62.9% 13.1%
---[Information] [15/10/2020 19:14:46] x264 [info]: mb P  I16..4:  3.2% 13.9%  1.6%  P16..4: 29.0% 19.7%  9.4%  0.0%  0.0%    skip:23.3%
---[Information] [15/10/2020 19:14:46] x264 [info]: mb B  I16..4:  0.4%  1.7%  0.3%  B16..8: 30.1% 10.5%  2.4%  direct: 5.0%  skip:49.5%  L0:36.8% L1:43.2% BI:20.0%
---[Information] [15/10/2020 19:14:46] x264 [info]: 8x8 transform intra:71.1% inter:80.7%
---[Information] [15/10/2020 19:14:46] x264 [info]: direct mvs  spatial:99.0% temporal:1.0%
---[Information] [15/10/2020 19:14:46] x264 [info]: coded y,uvDC,uvAC intra: 64.5% 68.2% 39.3% inter: 22.6% 24.9% 2.2%
---[Information] [15/10/2020 19:14:46] x264 [info]: i16 v,h,dc,p: 47% 32%  6% 14%
---[Information] [15/10/2020 19:14:46] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 19% 14%  6%  9% 10% 10%  9% 11%
---[Information] [15/10/2020 19:14:46] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 20%  6%  6% 12% 12% 11%  9%  9%
---[Information] [15/10/2020 19:14:46] x264 [info]: i8c dc,h,v,p: 43% 29% 15% 12%
---[Information] [15/10/2020 19:14:46] x264 [info]: Weighted P-Frames: Y:11.7% UV:11.2%
---[Information] [15/10/2020 19:14:46] x264 [info]: ref P L0: 78.4% 13.1%  7.7%  0.8%
---[Information] [15/10/2020 19:14:46] x264 [info]: ref B L0: 97.1%  2.9%
---[Information] [15/10/2020 19:14:46] x264 [info]: ref B L1: 96.9%  3.1%
---[Information] [15/10/2020 19:14:46] x264 [info]: kb/s:4689.26
---[Information] [15/10/2020 19:14:46] encoded 1350 frames, 35.97 fps, 4689.26 kb/s
---[Information] [15/10/2020 19:14:46] Constant Quality Mode: Quality 20 computed...
---[Information] [15/10/2020 19:14:46] Video Bitrate Obtained (approximate): 4689 kbit/s

3. a crf 20 encode of Raw 2020-01-06 Clip.mkv

custom command line: --bluray-compat --pulldown double --keyint 60 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
Job command line: "cmd.exe" /c ""Z:\MeGUI\tools\ffmpeg\ffmpeg.exe" -loglevel level+error -i "L:\WWE Test Clips\awx250mu.051\Raw 2020-01-06 Clip.mkv.avs" -strict -1 -f yuv4mpegpipe - | "Z:\MeGUI\tools\x264\x264.exe" --level 4.1 --bluray-compat --preset slow --crf 20.0 --keyint 60 --weightp 2 --slices 4 --vbv-bufsize 15000 --vbv-maxrate 15000 --qpfile "L:\WWE Test Clips\awx250mu.051\Raw 2020-01-06 Clip.mkv.qpf" --pulldown double --open-gop --sar 1:1 --frames 4172 --output "L:\WWE Test Clips\awx250mu.051\Raw 2020-01-06 Clip_Video.264" --stdin y4m -"

with:

LoadPlugin("Z:\MeGUI\tools\ffms\ffms2.dll")
FFVideoSource("L:\WWE Test Clips\Raw 2020-01-06 Clip.mkv", cachefile="L:\WWE Test Clips\awx250mu.051\Raw 2020-01-06 Clip.mkv.ffindex", fpsnum=30, fpsden=1, threads=1)
AssumeFPS(30000,1001, sync_audio=true)
SSRC(48000)
Spline36Resize(1280,720)

resulting in:

---[Information] [15/10/2020 19:26:14] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
---[Information] [15/10/2020 19:26:14] x264 [info]: using SAR=1/1
---[Information] [15/10/2020 19:26:14] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
---[Information] [15/10/2020 19:26:14] x264 [info]: profile Progressive High, level 4.1, 4:2:0, 8-bit
---[Information] [15/10/2020 19:27:49] x264 [info]: frame I:93    Avg QP:18.67  size:120152
---[Information] [15/10/2020 19:27:49] x264 [info]: frame P:1103  Avg QP:21.64  size: 46014
---[Information] [15/10/2020 19:27:49] x264 [info]: frame B:2976  Avg QP:25.44  size: 13024
---[Information] [15/10/2020 19:27:49] x264 [info]: consecutive B-frames:  3.4%  3.2%  4.1% 89.4%
---[Information] [15/10/2020 19:27:49] x264 [info]: mb I  I16..4:  5.0% 65.3% 29.7%
---[Information] [15/10/2020 19:27:49] x264 [info]: mb P  I16..4:  2.9% 12.4%  2.7%  P16..4: 39.5% 24.3% 12.0%  0.0%  0.0%    skip: 6.3%
---[Information] [15/10/2020 19:27:49] x264 [info]: mb B  I16..4:  0.4%  1.2%  0.3%  B16..8: 42.2% 12.1%  2.8%  direct: 4.2%  skip:36.9%  L0:38.3% L1:44.3% BI:17.4%
---[Information] [15/10/2020 19:27:49] x264 [info]: 8x8 transform intra:66.9% inter:57.0%
---[Information] [15/10/2020 19:27:49] x264 [info]: direct mvs  spatial:99.6% temporal:0.4%
---[Information] [15/10/2020 19:27:49] x264 [info]: coded y,uvDC,uvAC intra: 66.9% 78.7% 52.1% inter: 20.4% 28.0% 8.5%
---[Information] [15/10/2020 19:27:49] x264 [info]: i16 v,h,dc,p: 22% 43% 10% 26%
---[Information] [15/10/2020 19:27:49] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 12% 20% 12%  6%  9%  9% 11%  9% 13%
---[Information] [15/10/2020 19:27:49] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 19%  8%  7% 11% 10% 12%  8% 13%
---[Information] [15/10/2020 19:27:49] x264 [info]: i8c dc,h,v,p: 38% 33% 17% 13%
---[Information] [15/10/2020 19:27:49] x264 [info]: Weighted P-Frames: Y:0.5% UV:0.2%
---[Information] [15/10/2020 19:27:49] x264 [info]: ref P L0: 84.9%  9.8%  5.1%  0.2%
---[Information] [15/10/2020 19:27:49] x264 [info]: ref B L0: 96.9%  3.1%
---[Information] [15/10/2020 19:27:49] x264 [info]: ref B L1: 96.8%  3.2%
---[Information] [15/10/2020 19:27:49] x264 [info]: kb/s:5786.36
---[Information] [15/10/2020 19:27:49] encoded 4172 frames, 43.42 fps, 5786.36 kb/s
--[Information] [15/10/2020 19:27:49] Final statistics
---[Information] [15/10/2020 19:27:49] Constant Quality Mode: Quality 20 computed...
---[Information] [15/10/2020 19:27:49] Video Bitrate Obtained (approximate): 5786 kbit/s

4. a crf 20 encode of WrestleMania 2002 Clips.mkv

custom command line: --bluray-compat --pulldown double --keyint 60 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
Job command line: "cmd.exe" /c ""Z:\MeGUI\tools\ffmpeg\ffmpeg.exe" -loglevel level+error -i "L:\WWE Test Clips\1muuhg4b.dbh\WrestleMania 2002 Clips.mkv.avs" -strict -1 -f yuv4mpegpipe - | "Z:\MeGUI\tools\x264\x264.exe" --level 4.1 --bluray-compat --preset slow --crf 20.0 --keyint 60 --weightp 2 --slices 4 --vbv-bufsize 15000 --vbv-maxrate 15000 --pulldown double --open-gop --sar 1:1 --frames 1350 --output "L:\WWE Test Clips\1muuhg4b.dbh\WrestleMania 2002 Clips_Video.264" --stdin y4m -"

with:

---[NoImage] LoadPlugin("Z:\MeGUI\tools\ffms\ffms2.dll")
---[NoImage] FFVideoSource("L:\WWE Test Clips\WrestleMania 2002 Clips.mkv", cachefile="L:\WWE Test Clips\1muuhg4b.dbh\WrestleMania 2002 Clips.mkv.ffindex", fpsnum=30, fpsden=1, threads=1)
---[NoImage] AssumeFPS(30000,1001, sync_audio=true)
---[NoImage] SSRC(48000)
---[NoImage] Spline36Resize(1280,720)

resulting in:

---[Information] [15/10/2020 19:26:43] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
---[Information] [15/10/2020 19:26:43] x264 [info]: using SAR=1/1
---[Information] [15/10/2020 19:26:43] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
---[Information] [15/10/2020 19:26:43] x264 [info]: profile Progressive High, level 4.1, 4:2:0, 8-bit
---[Information] [15/10/2020 19:27:19] x264 [info]: frame I:29    Avg QP:16.70  size: 90057
---[Information] [15/10/2020 19:27:19] x264 [info]: frame P:366   Avg QP:22.59  size: 35967
---[Information] [15/10/2020 19:27:19] x264 [info]: frame B:955   Avg QP:25.62  size: 11129
---[Information] [15/10/2020 19:27:19] x264 [info]: consecutive B-frames:  3.6%  3.6%  7.8% 85.0%
---[Information] [15/10/2020 19:27:19] x264 [info]: mb I  I16..4: 24.0% 62.9% 13.1%
---[Information] [15/10/2020 19:27:19] x264 [info]: mb P  I16..4:  3.2% 13.9%  1.6%  P16..4: 29.0% 19.7%  9.4%  0.0%  0.0%    skip:23.3%
---[Information] [15/10/2020 19:27:19] x264 [info]: mb B  I16..4:  0.4%  1.7%  0.3%  B16..8: 30.1% 10.5%  2.4%  direct: 5.0%  skip:49.5%  L0:36.8% L1:43.2% BI:20.0%
---[Information] [15/10/2020 19:27:19] x264 [info]: 8x8 transform intra:71.1% inter:80.7%
---[Information] [15/10/2020 19:27:19] x264 [info]: direct mvs  spatial:99.0% temporal:1.0%
---[Information] [15/10/2020 19:27:19] x264 [info]: coded y,uvDC,uvAC intra: 64.5% 68.2% 39.3% inter: 22.6% 24.9% 2.2%
---[Information] [15/10/2020 19:27:19] x264 [info]: i16 v,h,dc,p: 47% 32%  6% 14%
---[Information] [15/10/2020 19:27:19] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 19% 14%  6%  9% 10% 10%  9% 11%
---[Information] [15/10/2020 19:27:19] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 20%  6%  6% 12% 12% 11%  9%  9%
---[Information] [15/10/2020 19:27:19] x264 [info]: i8c dc,h,v,p: 43% 29% 15% 12%
---[Information] [15/10/2020 19:27:19] x264 [info]: Weighted P-Frames: Y:11.7% UV:11.2%
---[Information] [15/10/2020 19:27:19] x264 [info]: ref P L0: 78.4% 13.1%  7.7%  0.8%
---[Information] [15/10/2020 19:27:19] x264 [info]: ref B L0: 97.1%  2.9%
---[Information] [15/10/2020 19:27:19] x264 [info]: ref B L1: 96.9%  3.1%
---[Information] [15/10/2020 19:27:19] x264 [info]: kb/s:4689.26
---[Information] [15/10/2020 19:27:19] encoded 1350 frames, 35.75 fps, 4689.26 kb/s
--[Information] [15/10/2020 19:27:19] Final statistics
---[Information] [15/10/2020 19:27:19] Constant Quality Mode: Quality 20 computed...
---[Information] [15/10/2020 19:27:19] Video Bitrate Obtained (approximate): 4689 kbit/s

so:
1. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.71 fps, 5786.70 kb/s
3. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.42 fps, 5786.36 kb/s
my guess is there is still some non-deterministic element even though '--non-deterministic' is not used when using multiple threads.
This could be reported to the devs after making sure the source of the non determinism isn't the avisynth script. (no clue why the whole audio part is in there)

2. crf 20 encode of WrestleMania 2002 Clips.mkv -> encoded 1350 frames, 35.97 fps, 4689.26 kb/s
4. crf 20 encode of WrestleMania 2002 Clips.mkv -> encoded 1350 frames, 35.75 fps, 4689.26 kb/s
-> same results.

I've also just done a CRF 20 and CRF 21 encoding of the Raw full episode. Here's the result:
CRF 20: 5.68 GB, 6.4 Mbps
CRF 21: 8.05 GB, 9.14 Mbps

Clearly there's something wrong as the CRF 21 encoding should have a lower filesize not a far higher filesize than the CRF 20 file.

Yes, that does seem like there is something wrong, but I got no time to look at the log file more.


Cu Selur

[AndersonJames]

Thanks for looking. I've reported it to the devs as well as posting here. I've added a new log file in the attachment of my original post for when I encoded the full length video to make things more clear.

[hello_hello]

AndersonJames,
Your custom command line colorimetry arguments aren't making it to the MeGUI command line. You need to remove the quotes.

--colorprim bt709 --transfer bt709 --colormatrix bt709

I have no explanation as to what's going on, but from your log file, this probably explains one anomaly.

--[Information] [16/10/2020 01:46:01] custom command line: --bluray-compat --pulldown double --keyint 30 --vbv-maxrate 15000 --vbv-bufsize 15000 --level 4.1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
--[Information] [16/10/2020 01:46:01] Job command line: "cmd.exe" /c ""Z:\MeGUI\tools\ffmpeg\ffmpeg.exe" -loglevel level+error -i "L:\0 = To Encode\cvmrlifo.w55\CRF 21 = Raw 2020-01-06.avs" -strict -1 -f yuv4mpegpipe - | "Z:\MeGUI\tools\x264\x264.exe" --level 4.1 --bluray-compat --preset slow --crf 17 --keyint 30 --weightp 2 --slices 4 --vbv-bufsize 15000 --vbv-maxrate 15000 --pulldown double --open-gop --sar 1:1 --frames 226773 --output "L:\0 = To Encode\cvmrlifo.w55\Raw 2020-01-06_Video.264" --stdin y4m -"

[AndersonJames]

Well spotted. I'll do some more testing now I've removed the quotes. I got those flags from here: http://www.x264bluray.com/home/720p-encoding
Why do they have the quotes on them if that means the commands will be ignored due to the quotes?

[AndersonJames]

Didn't make any difference. Problem still remains. It's not Avisynth that's the issue as I used an older MeGUI version on both Windows 7 with older Avisynth plugins and on Windows 10 with latest plugins and didn't get the problem in either. It's MeGUI 2913 that is causing the problem somehow.

[poisondeathray]

so:
1. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.71 fps, 5786.70 kb/s
3. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.42 fps, 5786.36 kb/s
my guess is there is still some non-deterministic element even though '--non-deterministic' is not used when using multiple threads.
This could be reported to the devs after making sure the source of the non determinism isn't the avisynth script. (no clue why the whole audio part is in there)

It's purposefully non deterministic with VBV and multithreading according to dev's


https://forum.doom9.org/showthread.php?t=152950

[AndersonJames]

If I encode to MKV instead of M2TS then the problem is fixed. Why would TSmuxer which MeGUI uses to mux the video cause massively different filesizes?

The same TSMuxer is used in MeGUI 2525 as in MeGUI 2913 so I'm still not sure what's causing the problem.

[poisondeathray]

If I encode to MKV instead of M2TS then the problem is fixed. Why would TSmuxer which MeGUI uses to mux the video cause massively different filesizes?

The same TSMuxer is used in MeGUI 2525 as in MeGUI 2913 so I'm still not sure what's causing the problem.

If you're referring to m2ts vs mkv container (in general) , a transport stream typically adds about 4-8% overhead

[AndersonJames]

so:
1. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.71 fps, 5786.70 kb/s
3. crf 20 encode of Raw 2020-01-06 Clip.mkv -> encoded 4172 frames, 43.42 fps, 5786.36 kb/s
my guess is there is still some non-deterministic element even though '--non-deterministic' is not used when using multiple threads.
This could be reported to the devs after making sure the source of the non determinism isn't the avisynth script. (no clue why the whole audio part is in there)

It's purposefully non deterministic with VBV and multithreading according to dev's


https://forum.doom9.org/showthread.php?t=152950

But if I use the exact same settings but encode to MKV then I get the same file size every time. I only get the different file size problem when encoding to M2TS. Also I don't think non-deterministic would cause a filesize difference of a few GBs in the full length video would it? CRF 21 sometimes gives a 50% smaller filesize than CRF 20 for example. This is obviously wrong surely?

I also encoded to M2TS in MeGUI 2525 and the filesize is exactly the same every time in that. So why would it be different in MeGUI 2913?

I don't think I'm using multi-threading in x264 anyway. How do I check?

If I encode to MKV instead of M2TS then the problem is fixed. Why would TSmuxer which MeGUI uses to mux the video cause massively different filesizes?

The same TSMuxer is used in MeGUI 2525 as in MeGUI 2913 so I'm still not sure what's causing the problem.

If you're referring to m2ts vs mkv container (in general) , a transport stream typically adds about 4-8% overhead

I'm aware of that. That's not the problem.

[poisondeathray]

I don't think non-deterministic would cause a filesize difference of a few GBs in the full length video would it? CRF 21 sometimes gives a 50% smaller filesize than CRF 20 for example. This is obviously wrong surely?

Yes, 50% size difference is a problem . Did you check the file, was it playable etc... ?

My reply was to Selur about non determinism

Selur's test didn't reproduce a difference that large. His results are in line with expectations

I don't think I'm using multi-threading in x264 anyway. How do I check?

You're automatically using it, unless you explicitly enter threads=1 . (--threads 1)

If you look at mediainfo (view=>text) of the output file look under threads . It's normally "auto" and set to 1.5* the number of logical cores

[AndersonJames]

I don't know what could be causing the issue. The same TSMuxer is used in MeGUI 2525 (which didn't have the problem) as in MeGUI 2913.
I tried swapping the x264.exe from MeGUI 2525 into MeGUI 2913 and the problem remains.
If MeGUI is just a GUI and it's x264, TSmuxer and FFMS that are the real tools then I don't know what's causing the problem.

[Selur]

It's purposefully non deterministic with VBV and multithreading according to dev's
https://forum.doom9.org/showthread.php?t=152950

That just sounds wrong -> opened a new thread over at doom9s.
May be it's just me not getting what '--non-deterministic' is meant to do when x264 is purposefully non deterministic with VBV and multithreading.

Cu Selur

[hello_hello]

Well spotted. I'll do some more testing now I've removed the quotes. I got those flags from here: http://www.x264bluray.com/home/720p-encoding
Why do they have the quotes on them if that means the commands will be ignored due to the quotes?

I checked via a command prompt and it seems x264 will accept them with or without the quotes.
x264.exe --fullhelp implies any argument requiring literal text should have quotes because the default values are shown that way.
ie --range "auto"

I've never thought about it before, so I tried --me "umh", and x264 was happy with that.

MeGUI will accept both --me "umh" and --me umh as a custom command line argument (I didn't try encoding but it appears in the command line section at the bottom of the encoder configuration), so I guess it could be considered a bug that it doesn't do the same for --colormatrix etc.

BTW, I encoded your samples a couple of times today using your command line (MeGUI 2913) and the bitrate was identical each time.
In your case the Avisynth output is being piped to x264 by ffmpeg, whereas for me it's not (32 bit Windows), so as going back to a previous x264 version didn't fix the problem, maybe try a different ffmpeg version. I've no idea if ffmpeg could be the cause, but it probably doesn't hurt to try.
I have a vague recollection that for 32 bit MeGUI on 64 bit Windows there's an option to use the 32 bit version of x264. If I'm remembering correctly and it's still the case, I assume MeGUI would stop involving ffmpeg.

[Selur]

Quotes are only necessary if your option contains spaces, otherwise it should work fine without the quotes.
So for most of the option using quotes is unnecessary. (I usually only use quotes when specifying a file name.)

[AndersonJames]

Well spotted. I'll do some more testing now I've removed the quotes. I got those flags from here: http://www.x264bluray.com/home/720p-encoding
Why do they have the quotes on them if that means the commands will be ignored due to the quotes?

I checked via a command prompt and it seems x264 will accept them with or without the quotes.
x264.exe --fullhelp implies any argument requiring literal text should have quotes because the default values are shown that way.
ie --range "auto"

I've never thought about it before, so I tried --me "umh", and x264 was happy with that.

MeGUI will accept both --me "umh" and --me umh as a custom command line argument (I didn't try encoding but it appears in the command line section at the bottom of the encoder configuration), so I guess it could be considered a bug that it doesn't do the same for --colormatrix etc.

BTW, I encoded your samples a couple of times today using your command line (MeGUI 2913) and the bitrate was identical each time.

Did you encode to M2TS as that's the file format that has the problem. MKV doesn't have the problem.

[Selur]

Did you encode to M2TS as that's the file format that has the problem. MKV doesn't have the problem.

the question is does the size of the raw videos differ,.. otherwise it's an x264 problem,...

[hello_hello]

Did you encode to M2TS as that's the file format that has the problem. MKV doesn't have the problem.

Only raw AVC, but when the format is MKV, is x264 writing raw AVC first or is it writing directly to an MKV? There's an option under External Programs "x264: use external muxer for MKV". When it's unchecked, x264 writes directly to MKV, when it's checked, the output is raw AVS and then it's muxed, which I assume is what happens when the output is M2TS. You could try changing that option, but I'm not sure why it'd effect the encoding, although for reasons I forget, writing directly to MKV isn't 100% Bluray compatible.
That's at least what happens when adding a video encoding job directly to the job queue and MKV is the selected format. I don't use OneClick.

I edited my previous post to add some thoughts about ffmpeg piping you may have missed before replying.