i7-9700K but extremely low performance enconding of FFMPEG/x264?

Member

[Attachment 47603 - Click to enlarge]

also using tmpgenc Video Mastering Work seems that the CPU wont' be used over 19/20%. It's strange because using the HP the CPU is used at 100%

Quote

23rd Dec 2018 16:43 #3

Member

Check another ffmpeg build , also check x264.exe directly

Did you make sure it's operating as expected with other system benchmark testing suites ?

Check heat/ temps/fans to ensure no throttling , and make sure it's not on energy saving/lower power modes where clock speed and/or cores can be disabled

Quote

23rd Dec 2018 17:27 #4

Member

ok, however I confirm that the CPU usage from the batch that recall ffmpeg (of course 64bit) is 20/22% instead of 98/100 of the HP

[Attachment 47605 - Click to enlarge]

Quote

23rd Dec 2018 17:45 #5

Member

please I wonder:

my CPU is:
Code:
Intel Core i7-9700K Unlocked | 3.60~4,90GHz | 12MB | 8C/8T | TDP 95W | Coffee Lake R
8 phisical core? and 8 virtual? boh

however, theorically, what is the suitable setting of
Code:
-threads
?

"-threads 16" ?

Quote

23rd Dec 2018 18:14 #6

jagabo

Member

Originally Posted by marcorocchini
my CPU is:
Code:
Intel Core i7-9700K Unlocked | 3.60~4,90GHz | 12MB | 8C/8T | TDP 95W | Coffee Lake R
8 phisical core? and 8 virtual?
8 physical cores, no hyperthreading (no virtual cores).
Originally Posted by marcorocchini
however, theorically, what is the suitable setting of
Code:
-threads
?

"-threads 16" ?
x264 usually likes 1.5x the number of cores (physical + virtual). So 12 in your case. You have some other bottleneck. Even my old quad core i5 2500K gets about 17 fps with 1080i video.

Quote

23rd Dec 2018 18:23 #7

jagabo

Member

Does Task Manager show 8 CPU graphs when you have View -> CPU History -> One Graph Per CPU selected?

Quote

23rd Dec 2018 18:57 #8

Member

exactly I don't know, in this moment I cannot find "View -> CPU History -> One Graph Per CPU selected?"

[Attachment 47606 - Click to enlarge]

however it seems that the problem is a bios limitation current over each CPU. I try to change some settings and seems to improve the speed encoding but the fan goes crazy, and emits a very strong heat, in this case encoding speed become 47 fps (is a encoding 4:2:2 extrafine seamless quality)

by the task manager seems: 8 core and 8 logical core. So need I to specify a -threads XY value?

Quote

23rd Dec 2018 19:03 #9

Member

Originally Posted by marcorocchini

however it seems that the problem is a bios limitation current over each CPU. I try to change some settings and seems to improve the speed encoding but the fan goes crazy, and emits a very strong heat, in this case encoding speed become 47 fps (is a encoding 4:2:2 extrafine seamless quality)

by the task manager seems: 8 core and 8 logical core. So need I to specify a -threads XY value?

i7-9700k does not have HT . It's 8core/8thread

So what's the problem ? 47fps is faster than 17/18fps on the old laptop

If loud fans and heat are a problem maybe you shouldn't have bought a laptop . Maybe a desktop with liquid cooling, big radiator, slow quiet fans

Quote

23rd Dec 2018 19:26 #10

Member

Originally Posted by poisondeathray

Originally Posted by marcorocchini

however it seems that the problem is a bios limitation current over each CPU. I try to change some settings and seems to improve the speed encoding but the fan goes crazy, and emits a very strong heat, in this case encoding speed become 47 fps (is a encoding 4:2:2 extrafine seamless quality)

by the task manager seems: 8 core and 8 logical core. So need I to specify a -threads XY value?

i7-9700k does not have HT . It's 8core/8thread

So what's the problem ? 47fps is faster than 17/18fps on the old laptop

If loud fans and heat are a problem maybe you shouldn't have bought a laptop . Maybe a desktop with liquid cooling, big radiator, slow quiet fans

7

no the fan is not a big problem, only are a little noisy. However the setting of -Threads whhat have to be?

HT = Hyper Cat Treading?

Quote

23rd Dec 2018 21:01 #11

jagabo

Member

Originally Posted by marcorocchini

encoding speed become 47 fps

That sounds about right for your CPU.

Originally Posted by marcorocchini

no the fan is not a big problem, only are a little noisy. However the setting of -Threads whhat have to be?

I already told you. It's also the default in x264 (and ffmpeg by extension). Look at a MediaInfo report of one of the files you created. It shows the x264 metatdata which includes the number of threads that were used.

Quote

24th Dec 2018 06:23 #12

pandy

Member

Code:
@set /a cput=((%NUMBER_OF_PROCESSORS%/2)*3)
and explicitly call this by -threads %cput%

IMHO you should also set power management profile to high performance, noisy fan and heat are directly linked with CPU load.
I warned you in previous thread about heat produced during encoding - you should provide additional help to efficiently cool notebook, i recall that more advanced users advised same.

Quote

24th Dec 2018 11:18 #13

Member

That is why I don't use ffmpeg anymore for video encoding.

Just for reference, with my setup it pulled 70 fps to encode a 1080p@60 file with your settings:

Code:

ffmpeg -y -threads 9 -i cputest.mov -pix_fmt yuv422p -c:v libx264 -profile:v high422 -level:v 4.1 -g 33 -bf 2 -crf 19 -flags +ildct+ilme -top 1 cputest-sample1.MP4
frame=15629 fps= 70 q=27.0 size=   93696kB time=00:04:20.36 bitrate=2948.0kbits
[libx264 @ 00000130384d1fc0] kb/s:2929.54

Using x264 v2538 it was around 50 fps:

Code:

x264_2538.exe --preset slow --sar 1:1 --threads 9 --input-csp yuv422p --profile high422 --level 4.1 --crf 19 --bframes 2 --keyint 60 --no-interlaced --videoformat component --range pc --vbv-maxrate 4000 --vbv-bufsize 8000 --colorprim bt709 --transfer bt709 --colormatrix bt709 --output-csp i422 --output "cputest-sample2.MP4" "cputest.avs"
encoded 15640 frames, 50.12 fps, 1759.33 kb/s

The difference in bitrate reflected the final size, x264=55.5MB and ffmpeg=91.2MB

Code:

Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp42/mp41/isom)
File size                                : 55.6 MiB
Duration                                 : 4 min 20 s
Overall bit rate                         : 1 789 kb/s
Encoded date                             : UTC 2018-12-24 16:00:11
Tagged date                              : UTC 2018-12-24 16:00:11

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:2@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 4 min 20 s
Bit rate                                 : 1 783 kb/s
Maximum bit rate                         : 3 804 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 60.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.014
Stream size                              : 55.4 MiB (100%)
Writing library                          : x264 core 146 r2538 121396c
Encoding settings                        : cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=8 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=60 / keyint_min=6 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=4000 / vbv_bufsize=8000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 2018-12-24 16:00:11
Tagged date                              : UTC 2018-12-24 16:00:11
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

ffmpeg (N-91473-gc5329d64b1)

Code:

Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 92.3 MiB
Duration                                 : 4 min 20 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 970 kb/s
Writing application                      : Lavf58.17.101

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:2@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 4 min 20 s
Bit rate                                 : 2 930 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 60.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.024
Stream size                              : 91.1 MiB (99%)
Writing library                          : x264 core 155 r2901 7d0ff22
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=0 / keyint=33 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=33 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box                  : avcC

Also related with threads, you set it up to be "9" latter you see that it used "12" among other things you might setup and ffmpeg doesn't follow.
Quality wise, I see no difference between the two.

Click on the PC icon to see my setup.

Last edited by amaipaipai; 24th Dec 2018 at 12:56.

Quote

24th Dec 2018 11:29 #14

pandy

Member

Originally Posted by amaipaipai

That is why I don't use ffmpeg anymore for video encoding.

Just for reference, with my setup it pulled 70 fps to encode a 1080p@60 file with your settings
Using x264 v2538 it was around 50 fps

Why different encoder settings was used?

Code:

cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=8 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=60 / keyint_min=6 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=4000 / vbv_bufsize=8000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00

Code:

cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=0 / keyint=33 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=33 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Quote

24th Dec 2018 11:39 #15

JVRaines

Member

Originally Posted by pandy

Originally Posted by amaipaipai

That is why I don't use ffmpeg anymore for video encoding.

Just for reference, with my setup it pulled 70 fps to encode a 1080p@60 file with your settings
Using x264 v2538 it was around 50 fps

Why different encoder settings was used?

Indeed. Hardly makes it a fair comparison.

Quote

24th Dec 2018 12:11 #16

Member

Originally Posted by pandy

Why different encoder settings was used?

Go figure, this is what I was talking about, you set one thing with ffmpeg and you get another.

Quote

24th Dec 2018 12:17 #17

Member

Originally Posted by amaipaipai

Also related with threads, you set it up to be "9" and it show it used "12" among other things you might setup and it doesn't follow.

Are you taking about threads ?

(lib)x264 threads are not the same thing as ffmpeg threads. ffmpeg does other things, filters, audio etc...those can be threaded too

If you wanted 9 libx264 threads
Code:
-x264opts threads=9
Or what else are you talking about ? You should be able to get what you want out of ffmpeg libx264 (with some very minor exceptions, a few things are missing compared to x264 cli)

Quote

24th Dec 2018 12:41 #18

pandy

Member

Originally Posted by amaipaipai

Originally Posted by pandy

Why different encoder settings was used?

Go figure, this is what I was talking about, you set one thing with ffmpeg and you get another.

Well... i see that ffmpeg was forced to perform interlaced encoding where x264 was set to progressive encoding, also seem you missed x264 mapping to ffmpeg https://sites.google.com/site/linuxencoding/x264-ffmpeg-mapping and not using at all private encoder settings (trough -x264opts and trough -x264-params)

for example:
Code:
@SET x264opts="crf=%vq%:level=4.1:qpmin=8:cabac=1:keyint=infinite:no-deblock=1:no-mbtree=1:me=dia:rc-lookahead=5:ref=3:subme=0:no-weightb=1:weightp=0:trellis=0:no-scenecut=1:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709"

@ffmpeg.exe -hide_banner -v 32 -stats -y -color_range 2 -loop 1 -r 1 -i "%fimage%" -i "%faudio%" -vf %vproc% -c:v libx264 -preset faster -tune stillimage -profile:v high -level:v 4.1 -x264opts %x264opts% -x264-params %x264opts% -color_range 2 -c:a libfdk_aac -b:a 8k -af %aproc% -sn -dn -shortest -movflags faststart -f mp4 "%~n2_%~n1.mp4"
try to copy encoder setting 1:1 and compare results

Quote

24th Dec 2018 12:54 #19

Member

Originally Posted by poisondeathray

Are you taking about threads ?

Yes, what I meant to say was if you set "A" ffmpeg does "B", if you set threads=9 latter after encoding you see that it was encoded with threads=12, this is just a small example. I don't trust ffmpeg because of that.
Originally Posted by pandy
Well... i see that ffmpeg was forced to perform interlaced encoding where x264 was set to progressive encoding, also seem you missed x264 mapping to ffmpeg https://sites.google.com/site/linuxencoding/x264-ffmpeg-mapping and not using at all private encoder settings (trough -x264opts and trough -x264-params)

for example:
Code:
@SET x264opts="crf=%vq%:level=4.1:qpmin=8:cabac=1:keyint=infinite:no-deblock=1:no-mbtree=1:me=dia:rc-lookahead=5:ref=3:subme=0:no-weightb=1:weightp=0:trellis=0:no-scenecut=1:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709"

@ffmpeg.exe -hide_banner -v 32 -stats -y -color_range 2 -loop 1 -r 1 -i "%fimage%" -i "%faudio%" -vf %vproc% -c:v libx264 -preset faster -tune stillimage -profile:v high -level:v 4.1 -x264opts %x264opts% -x264-params %x264opts% -color_range 2 -c:a libfdk_aac -b:a 8k -af %aproc% -sn -dn -shortest -movflags faststart -f mp4 "%~n2_%~n1.mp4"
try to copy encoder setting 1:1 and compare results
Looks interesting

Quote

24th Dec 2018 13:09 #20

Member

Originally Posted by amaipaipai

Yes, what I meant to say was if you set "A" ffmpeg does "B", if you set threads=9 latter after encoding you see that it was encoded with threads=12, this is just a small example. I don't trust ffmpeg because of that.

But that was just user error or misunderstanding on your end . ffmpeg threads are not the same thing as libx264 threads

The syntax for the switches are slightly different but you should be able to get what you want, or translate x264cli to ffmpeg libx264 commands

Or do you have any other examples where you get "B" instead of "A"

Quote

24th Dec 2018 13:25 #21

Member

Originally Posted by pandy

Well... i see that ffmpeg was forced to perform interlaced encoding where x264 was set to progressive encoding, also seem you missed x264 mapping to ffmpeg https://sites.google.com/site/linuxencoding/x264-ffmpeg-mapping and not using at all private encoder settings (trough -x264opts and trough -x264-params)

for example:

Code:

@SET x264opts="crf=%vq%:level=4.1:qpmin=8:cabac=1:keyint=infinite:no-deblock=1:no-mbtree=1:me=dia:rc-lookahead=5:ref=3:subme=0:no-weightb=1:weightp=0:trellis=0:no-scenecut=1:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709"

@ffmpeg.exe -hide_banner -v 32 -stats -y -color_range 2 -loop 1 -r 1 -i "%fimage%" -i "%faudio%" -vf %vproc% -c:v libx264 -preset faster -tune stillimage -profile:v high -level:v 4.1 -x264opts %x264opts% -x264-params %x264opts% -color_range 2 -c:a libfdk_aac -b:a 8k -af %aproc% -sn -dn -shortest -movflags faststart -f mp4 "%~n2_%~n1.mp4"

try to copy encoder setting 1:1 and compare results

This gives a lot of errors, what are the other variables?
This are the changes I've made so I just drag and drop the file over the .bat:

Code:

cd %~dp0
@SET x264opts="crf=%vq%:level=4.1:qpmin=8:cabac=1:keyint=infinite:no-deblock=1:no-mbtree=1:me=dia:rc-lookahead=5:ref=3:subme=0:no-weightb=1:weightp=0:trellis=0:no-scenecut=1:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709"

@SET fimage=%1

@ffmpeg.exe -hide_banner -v 32 -stats -y -color_range 2 -r 1 -i "%fimage%" -vf %vproc% -c:v libx264 -preset faster -tune stillimage -profile:v high -level:v 4.1 -x264opts %x264opts% -x264-params %x264opts% -color_range 2 -c:a copy -sn -dn -shortest -movflags faststart -f mp4 "%~n2_%~n1.mp4"
pause

It returns:

Code:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\encode\cputest.mov':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isom
    creation_time   : 2018-12-22T13:45:10.000000Z
  Duration: 00:04:20.74, start: 0.000000, bitrate: 543 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 510 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      creation_time   : 2018-12-22T13:43:15.000000Z
      handler_name    : GPAC ISO Video Handler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 25 kb/s (default)
    Metadata:
      creation_time   : 2018-12-22T13:35:06.000000Z
[NULL @ 000001eec6610980] Unable to find a suitable output format for 'libx264'
libx264: Invalid argument

By the way, there is no need for audio just to make this test, we can go just with video for now.

Last edited by amaipaipai; 24th Dec 2018 at 13:36.

Quote

24th Dec 2018 14:26 #22

Member

The output is garbage at 1 fps and a 4 min video became a 4h long video with 14.4 kb/s bitrate, ffmpeg is a waste of everybody time.

Code:

Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 27.9 MiB
Duration                                 : 4 h 20 min
Overall bit rate                         : 15.0 kb/s
Writing application                      : Lavf58.17.101

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 4 h 20 min
Bit rate                                 : 14.4 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 1.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.007
Stream size                              : 26.8 MiB (96%)
Writing library                          : x264 core 155 r2901 7d0ff22
Encoding settings                        : cabac=1 / ref=3 / deblock=0:-3:-3 / analyse=0x3:0x113 / me=dia / subme=0 / psy=1 / psy_rd=2.00:0.70 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=0 / open_gop=0 / weightp=0 / keyint=infinite / keyint_min=1 / scenecut=0 / intra_refresh=0 / rc=crf / mbtree=0 / crf=19.0 / qcomp=0.60 / qpmin=8 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.20
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

Code:

cd %~dp0
@SET fimage=%1
@SET vq=19
@SET x264opts="crf=%vq%:level=4.1:qpmin=8:cabac=1:keyint=infinite:no-deblock=1:no-mbtree=1:me=dia:rc-lookahead=5:ref=3:subme=0:no-weightb=1:weightp=0:trellis=0:no-scenecut=1:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709"

@ffmpeg.exe -hide_banner -v 32 -stats -y -color_range 2 -r 1 -i "%fimage%" -c:v libx264 -preset faster -tune stillimage -profile:v high -level:v 4.1 -x264opts %x264opts% -x264-params %x264opts% -color_range 2 -c:a copy -sn -dn -shortest -movflags faststart -f mp4 "%~n2_%~n1.mp4"
pause

Sorry @pandy, I've something better to do, maybe next year.

Quote

24th Dec 2018 15:07 #23

Member

What are you trying to do ? Did you want to match your x264 settings and convert them to ffmpeg libx264 syntax?

But there are problems with your x264 settings;
Code:
x264_2538.exe --preset slow --sar 1:1 --threads 9 --input-csp yuv422p --profile high422 --level 4.1 --crf 19 --bframes 2 --keyint 60 --no-interlaced --videoformat component --range pc --vbv-maxrate 4000 --vbv-bufsize 8000 --colorprim bt709 --transfer bt709 --colormatrix bt709 --output-csp i422 --output "cputest-sample2.MP4" "cputest.avs"
encoded 15640 frames, 50.12 fps, 1759.33 kb/s
1) You should be using L4.2 for 1080p60 , L4.1 is "illegal" for 1080p60 and might cause issues with some players

2) If you use --range pc, that will actually expand the range to pc levels (ie. change actual data from Y 16-235 => 0-255) if --input-range was tv or auto, like it will be for an avs in your commandline . So it's not just a flag in that case , you will get the wrong levels

But if your script was full range data, and you are wanting to encode full range, you must include --input-range pc as well. That would be full range data, encoded as full range, flagged as full range for x264cli

Or did you want to convert his settings to x264cli syntax to compare?

Eitherway - if you want to do a proper comparison, you should do "apples to apples" . So if you use avs as input for x264, you should use avs as input for ffmpeg. If you are using direct video input for ffmpeg you should use --demuxer lavf for x264 with direct video input, you should be using same settings, etc... . I don't know what you're trying to show with completely different settings here...

If something doesn't work as expected, please report it; because dev's have to know about it in order to fix it. But I don't see any proper commandlines or comparisons. Before making a premature judgement , make sure your tests and methodology are valid first

Quote

24th Dec 2018 15:48 #24

Member

Originally Posted by poisondeathray

What are you trying to do ? Did you want to match your x264 settings and convert them to ffmpeg libx264 syntax?

Hi!
Pandy ask me to, I don't use ffmpeg for video encoding.

Originally Posted by poisondeathray

But there are problems with your x264 settings;
Code:
x264_2538.exe --preset slow --sar 1:1 --threads 9 --input-csp yuv422p --profile high422 --level 4.1 --crf 19 --bframes 2 --keyint 60 --no-interlaced --videoformat component --range pc --vbv-maxrate 4000 --vbv-bufsize 8000 --colorprim bt709 --transfer bt709 --colormatrix bt709 --output-csp i422 --output "cputest-sample2.MP4" "cputest.avs"
encoded 15640 frames, 50.12 fps, 1759.33 kb/s
Originally Posted by poisondeathray

1) You should be using L4.2 for 1080p60 , L4.1 is "illegal" for 1080p60 and might cause issues with some players

I'm aware it's illegal.
Those are me trying to match the OP settings. Since the issue it's not the legality of the OP settings and standards, I did any way.

Originally Posted by poisondeathray

2) If you use --range pc, that will actually expand the range to pc levels (ie. change actual data from Y 16-235 => 0-255) if --input-range was tv or auto, like it will be for an avs in your commandline . So it's not just a flag in that case , you will get the wrong levels

That was my setup mixed up with the OP.

Originally Posted by poisondeathray

But if your script was full range data, and you are wanting to encode full range, you must include --input-range pc as well. That would be full range data, encoded as full range, flagged as full range for x264cli

Yes, I use this with my scripts.

Originally Posted by poisondeathray

Or did you want to convert his settings to x264cli syntax to compare?

At first I did to try to compare to show the discrepancy.

Originally Posted by poisondeathray

Eitherway - if you want to do a proper comparison, you should do "apples to apples" . So if you use avs as input for x264, you should use avs as input for ffmpeg. If you are using direct video input for ffmpeg you should use --demuxer lavf for x264 with direct video input, you should be using same settings, etc...

The avs is just a "DirectShowSource" so the x264 could understand the file, ffmpeg can handle direct file input that I have.

Originally Posted by poisondeathray

I don't know what you're trying to show with completely different settings here...

The idea was to show that ffmpeg assume options you didn't choose, like the thread option, I set up 9 and ffmpeg assume 12, I didn't setup lookahead and ffmpeg choose lookahead=2 and as you saw it, is a complete different settings even choosing the ones you want.

Originally Posted by poisondeathray

If something doesn't work as expected, please report it; because dev's have to know about it in order to fix it. But I don't see any proper commandlines or comparisons. Before making a premature judgement , make sure your tests and methodology are valid first

I pass.
This issue is 10 years old at least, the developers are well aware of it, because of this you can't have control since your settings are ignored by ffmpeg. This is why you can't compare it and this is not a premature judgment, people are trusting this peace of **** and are not aware of this. I just don't use it anymore, last time I use it was in 2006 and looks like still has the same issues now in 2018.

But hey, this is what I think about it, I don't trust it.
If you have time, try to match the OP settings with x264 and see what you get, in general you get a bigger file with no improvements to justify the extra size.

Sorry the rush, I'm cooking the Christmas dinner.
Merry Christmas for you poisondeathray and for everybody, happy holidays everybody.

Last edited by amaipaipai; 24th Dec 2018 at 15:58.

Quote

24th Dec 2018 16:31 #25

Member

Originally Posted by amaipaipai

Those are me trying to match the OP settings. Since the issue it's not the legality of the OP settings and standards, I did any way.

Just pointing out that he's encoding 1080i25 mbaff (or call it 1080i50 if you want, either way it's 25 frames per second interlaced), which is L4.1.

You chose a poor example with 1080p60.0 to compare (progressive , wrong frame rate)

Originally Posted by poisondeathray

Eitherway - if you want to do a proper comparison, you should do "apples to apples" . So if you use avs as input for x264, you should use avs as input for ffmpeg. If you are using direct video input for ffmpeg you should use --demuxer lavf for x264 with direct video input, you should be using same settings, etc...

The avs is just a "DirectShowSource" so the x264 could understand the file, ffmpeg can handle direct file input that I have.

I understand, but best practices is to compare something "apples to apples" as close to as possible, or ceteris paribus

It negatively impacts the comparison ; avisynth adds at least a bit of overhead , it's a frameserver, consumes resources like memory , cpu cycles. You want to reduce as many variables and differences as possible, otherwise your results are biased and there might be other factors affecting your observations

Originally Posted by poisondeathray

I don't know what you're trying to show with completely different settings here...

The idea was to show that ffmpeg assume options you didn't choose, like the thread option, I set up 9 and ffmpeg assume 12, I didn't setup lookahead and ffmpeg choose lookahead=2 and as you saw it, is a complete different settings even choosing the ones you want.

What you see is the direct result of your commandline. x264 and libx264 have set default values for things that you don't explicitly include. For example, if you don't set specify a default preset, it will use "medium" . You explicily set --preset slow for your x264cli commandline

I already explained the threads - You chose 9 ffmpeg threads. You did not specify libx264 threads in your commandline , so it chose "auto." Which is 1.5*cpu cores. I'm assuming you have 8, so 8 * 1.5 = 12

If you wanted to set libx264, you need to use the command I posted earlier with -x264opts

For lookahead threads, the "auto" setting is 1/6 of regular threads. Since you had 12 threads for libx264 (because you didn't set it properly, 12/6 = 2)

Originally Posted by poisondeathray

If something doesn't work as expected, please report it; because dev's have to know about it in order to fix it. But I don't see any proper commandlines or comparisons. Before making a premature judgement , make sure your tests and methodology are valid first

I pass.
This issue is 10 years old at least, the developers are well aware of it, because of this you can't have control since your settings are ignored by ffmpeg. This is why you can't compare it and this is not a premature judgment, people are trusting this peace of **** and are not aware of this. I just don't use it anymore, last time I use it was in 2006 and looks like still has the same issues now in 2018.

ffmpeg has MANY issues, but not the ones your describe in this scenario with ffmpeg libx264. If you have an example where a setting is ignored, please post it

From what I see, it reacted appropriately according to your commandline.

In short, I'm asking for concrete examples where something isn't passed or settings are ignored - so we can help improve it if there is a problem. That's all

But hey, this is what I think about it, I don't trust it.

That's ok, but I like basing my trust on facts and evidence

If you have time, try to match the OP settings with x264 and see what you get, in general you get a bigger file with no improvements to justify the extra size.

You're using capped crf encoding with vbv (restricting the bitrate --vbv-maxrate 4000 --vbv-bufsize 8000) . That's the main reason why your filesize is lower

Merry Christmas for you poisondeathray and for everybody, happy holidays everybody.

Merry Christmas

Last edited by poisondeathray; 24th Dec 2018 at 16:41.

Quote

24th Dec 2018 17:10 #26

Member

One of the op's files from another post
https://forum.videohelp.com/threads/384431-FFMPEG-why-I-cannot-f-concat-this-2-video-f...e2#post2493088

C0056.MXF

Not an exciting "cat" video

I looped it 8x

AVS script

Code:

ffvideoSource("C0056.MXF")
loop(8)
#2904 frames

Code:

ffmpeg -i C0056.avs -pix_fmt yuv422p -c:v libx264 -profile:v high422 -level:v 4.1 -g 33 -bf 2 -crf 19 -flags +ildct+ilme -top 1 -an ffmpeg_libx264.mp4

Code:

x264k --input-csp i422 --output-csp i422 --level 4.1 --keyint 33 --bframes 2 --crf 19 --tff -o x264cli.mp4 C0056.avs

Note, I used slightly different x264 libraries r2935 vs. r2833 , and the r2833 has a fade-compensate patch (but set to 0) , but the results are essentially the same. Same filesize. 784MiB. Some differences in the muxer, and mp4 profile, but the point is the same encoding settings passed if you look at the metadata

Because I did not specify threads, "auto" was used. This is a 4+4HT = 8 cores . So 8*1.5 =12 threads. And look ahead threads is 12/6 =2 . Which is what you get.

ffmpeg libx264

Code:

Complete name                            : ffmpeg_libx264.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 784 MiB
Duration                                 : 1 min 56 s
Overall bit rate                         : 56.6 Mb/s
Writing application                      : Lavf58.24.101

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:2@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 56 s
Bit rate                                 : 56.1 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 1.081
Stream size                              : 784 MiB (100%)
Writing library                          : x264 core 157 r2935 545de2f
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=0 / keyint=33 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=33 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box                  : avcC

x264cli

Code:

Complete name                            :x264cli.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp42/mp41/isom)
File size                                : 784 MiB
Duration                                 : 1 min 56 s
Overall bit rate                         : 56.6 Mb/s

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:2@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 56 s
Bit rate                                 : 56.1 Mb/s
Maximum bit rate                         : 61.3 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 1.081
Stream size                              : 784 MiB (100%)
Writing library                          : x264 core 150 r2833kMod df79067
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / fade_compensate=0.00 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=0 / keyint=33 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=33 / rc=crf / mbtree=1 / crf=19.0000 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box                  : avcC

Quote

24th Dec 2018 18:15 #27

Member

That looks great poisondeathray, thank you very much to take your time to do all this, much appreciated!!!

ffmpeg has MANY issues, but not the ones your describe in this scenario with ffmpeg libx264. If you have an example where a setting is ignored, please post it

The threads is one of them, if you are telling ffmpeg to use this amount of it, ffmpeg has to limit it self to that. Reasons can be many, control the system heat, resources if I'm doing something else, etc. You said that this must be set in "x264opts", so what the point for ffmpeg to have this settings in the first place if it can't pass it to x264 or whatever encoder the users chooses.

It can be frame rate or any other setting, ffmpeg ignores the user input unless the user understand that the settings he want, must be pass throughout the use of "xxxops" or similar options. That was the whole deal at the time.

You're using capped crf encoding with vbv (restricting the bitrate --vbv-maxrate 4000 --vbv-bufsize 8000) . That's the main reason why your filesize is lower

Yes, that come from the very first script I saw, is the one I use with my Blu-Ray player.

Quote

24th Dec 2018 18:45 #28

Member

Originally Posted by amaipaipai

The threads is one of them, if you are telling ffmpeg to use this amount of it, ffmpeg has to limit it self to that. Reasons can be many, control the system heat, resources if I'm doing something else, etc. You said that this must be set in "x264opts", so what the point for ffmpeg to have this settings in the first place if it can't pass it to x264 or whatever encoder the users chooses.

It can be frame rate or any other setting, ffmpeg ignores the user input unless the user understand that the settings he want, must be pass throughout the use of "xxxops" or similar options. That was the whole deal at the time.

Another way of looking at it is you didn't use the correct commandline

And you can see it does pass when you use the correct commandline , try the -x264opts threads=9 command.

threads=9 / lookahead_threads=1

And as expected, the lookahead threads is 1, because of the formula 9/6 rounds to 1

Well, the assumption is the user is supposed to understand how to use ffmpeg. A developer is just going to say "RTFM - It's your problem if you didn't read the instructions" . The documentation sucks to be fair, but at least it has improved quite a bit from 10 years ago. But you need to provide constructive criticism if you want things improved. You need to build a valid case with strong reasoning about why things should be done a certain way, or provide examples of something not working as expected

ffmpeg can run multiple processes . On a server, the same video might be split off into 4 or 5 different encoding processes simultaneously . A global threads setting might not be appropriate. You might want the sd encode to get 2 threads, the hd thread to get 4 threads, the uhd encode to get 8 threads and so forth. You might even run different encoders simultaneously, a SD DVD encode with MPEG2 concurrently with a web vp9 version. Maybe the x265 version needs more threads than the x264 version. You might want to leave some threads for audio and filter pre processing, etc.. So the encoder specific commands are for that specific encoder . You can't control all that with a single -threads setting

Quote

24th Dec 2018 19:40 #29

Member

Originally Posted by poisondeathray

Another way of looking at it is you didn't use the correct commandline
And you can see it does pass when you use the correct commandline , try the -x264opts threads=9 command.

I understand that, I agree with you!
x264 has a "threads" settings so as ffmpeg, the point that I'm trying to make you understand is that it's redundant and ffmpeg is full of that redundancy.

Originally Posted by poisondeathray

Well, the assumption is the user is supposed to understand how to use ffmpeg. A developer is just going to say "RTFM - It's your problem if you didn't read the instructions" . The documentation sucks to be fair, but at least it has improved quite a bit from 10 years ago. But you need to provide constructive criticism if you want things improved. You need to build a valid case with strong reasoning about why things should be done a certain way, or provide examples of something not working as expected

ffmpeg can run multiple processes . On a server, the same video might be split off into 4 or 5 different encoding processes simultaneously . A global threads setting might not be appropriate. You might want the sd encode to get 2 threads, the hd thread to get 4 threads, the uhd encode to get 8 threads and so forth. You might even run different encoders simultaneously, a SD DVD encode with MPEG2 concurrently with a web vp9 version. Maybe the x265 version needs more threads than the x264 version. You might want to leave some threads for audio and filter pre processing, etc.. So the encoder specific commands are for that specific encoder . You can't control all that with a single -threads setting

Yes, the ffmpeg documentation is another issue on its own as for the help. Do a "ffmpeg -h fullhelp >ffmpeg-help.txt", look for "libx264 AVOptions", now try to find the "threads" settings, there is none, how a regular user should know that "threads" go inside -x264opts? As you can see it's not the case that we don't read the documentation, maybe it's online:
https://www.ffmpeg.org/ffmpeg-all.html#libx264_002c-libx264rgb

No, it's not there either.

The only place I could find "threads" for x264 was inside ffmpeg source code:
https://www.ffmpeg.org/doxygen/4.1/libx264_8c_source.html

Inside "AVCodecDefault x264_defaults" line 1038.

As you can see, it's not a RTFM issue, it's a lack of leadership, it's a lack of organization, if you don't organize your own documentation I imagine how mess up the code is.

I'm not a software engineer, but since ffmpeg has redundant settings why not do something like this:

check if user set "threads", if yes check if the encoder uses "threads", if yes use the definition made by user, if not, use defaults.
Again, I'm not a programmer but you get the idea.

I'm not trying to build a case, to be scientific about it or to ask for any changes, I'm long gone with ffmpeg, I did help development years ago and I'm done, I don't use it and don't care about it anymore.

Again, thank you for your time and effort to shed some light on some issues, I hope the forum members make a very goo use of it.

You have a very good Christmas poisondeathray, you're a very nice guy!

Last edited by amaipaipai; 24th Dec 2018 at 19:50.

Quote

24th Dec 2018 20:31 #30