ffmpeg: how to encode @ target size and 2passes?

[marcorocchini]

Hi, but sorry I'm a cat

please consider my commandline:

ffmpeg.exe -threads 12 -y -i input.avi -i input.wav -pix_fmt yuv420p -vf yadif=1,scale=interl=1 -c:v libx264 -crf 19 -profile:v high -level:v 4.2 -g 33 -bf 2 -c:a ac3 -b:a 320k -ar 48000 -aspect 16:9 out.mp4

My source is 4:2:2 uncompressed intelarced video, my target is progressive 4:2:0 .mp4 file

I would like change the commandline so that:

the target size become 4.7 GB and the encoding process is 2 passes. How I have to do? thanks

[sneaker]

https://trac.ffmpeg.org/wiki/Encode/H.264#twopass

[pandy]

Not possible to set target size directly (or perhaps more correct - i have no clue how to create such batch) - you must calculate "manually" bitrate to match expected target size.

@set OUTDIR="E:\out"
@if not exist %OUTDIR% (mkdir %OUTDIR%)
@set cdir=%CD%


@set gop=240
@set vbitrate=2732
@set /a vmaxrate=20000-1
@set /a vbuffsiz=%vmaxrate%*1
@set qf=23

@SET x264opts1="bitrate=%vbitrate%:vbv_maxrate=%vmaxrate%:vbv_bufsize=%vbuffsiz%:level=4.0:keyint=%gop%:keyint_min=6:qpmin=4:cabac=1:threads=auto:lookahead_threads=auto:sliced_threads=0:slices=%slice%:me=tesa:subme=10:bframes=5:b_adapt=2:trellis=2:merange=32:fast_pskip=0:tff=1:interlaced=0:fake-interlaced=1:no_psnr=1:no_ssim=1:bluray_compat=1:open-gop=0:pic_struct=1:aud=1:nal_hrd=vbr:force_cfr=1:fullrange=off:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709:pass=1:deblock=-2,-2:aq_mode=3:aq_strength=1.05:psy=1:psy_rd=1.15,0.15:ipratio=1.25:8x8dct=1"
@SET x264opts2="bitrate=%vbitrate%:vbv_maxrate=%vmaxrate%:vbv_bufsize=%vbuffsiz%:level=4.0:keyint=%gop%:keyint_min=6:qpmin=4:cabac=1:threads=auto:lookahead_threads=auto:sliced_threads=0:slices=%slice%:me=tesa:subme=10:bframes=5:b_adapt=2:trellis=2:merange=32:fast_pskip=0:tff=1:interlaced=0:fake-interlaced=1:no_psnr=1:no_ssim=1:bluray_compat=1:open-gop=0:pic_struct=1:aud=1:nal_hrd=vbr:force_cfr=1:fullrange=off:overscan=show:colorprim=bt709:transfer=bt709:colormatrix=bt709:pass=2:deblock=-2,-2:aq_mode=3:aq_strength=1.05:psy=1:psy_rd=1.15,0.15:ipratio=1.25:8x8dct=1"

@set vproc="crop=1920:800:0:140,pp=ac/dr,hqdn3d=2,smartblur=lr=1.75:ls=-0.50:lt=-3.5:cr=0.5:cs=0.250:ct=0.5,hqdn3d=2,scale=iw:ih:sws_flags=neighbor:sws_dither=0:in_range=1:out_range=1,zscale=d=ordered:f=spline36:r=full:w=(iw*3)/4:h=ih,format=pix_fmts=yuv420p"
@set aproc="pan=stereo|FL<FL+1.414FC+0.5BL+0.5SL+0.5BC+0.25LFE|FR<FR+1.414FC+0.5BR+0.5SR+0.5BC+0.25LFE,firequalizer=gain='if(gte(f,21),0,-INF)+if(lte(f,15450),0,-INF)',dynaudnorm=p=1/sqrt(2):m=100:s=12,firequalizer=gain='if(gte(f,21),0,-INF)+if(lte(f,13450),0,-INF)'"


@ffmpeg.exe -y -threads %cput% -hide_banner -loglevel 24 -stats -avoid_negative_ts 1 -i "%1" -map 0:0 -flags:v:0 +cgop -c:v:0 libx264 -b:v:0 %vbitrate%k -preset:v:0 veryslow -tune:v:0 film -profile:v:0 high -level:v:0 4.0 -x264opts:v:0 %x264opts1% -x264-params:v:0 %x264opts1% -pass:v:0 1 -passlogfile:v:0 log_%~n1 -color_range:v:0 2 -pix_fmt:v:0 yuv420p -vf:v:0 %vproc% -flags:a +qscale -map 0:1 -c:a:0 ac3 -b:a:0 384k -ac:0 6 -filter_complex [0:a:0]%aproc%[a2c] -map [a2c]:a:1 -c:a:1 libmp3lame -q:a:1 6 -map 0:s:language:eng -c:s copy -f matroska NUL
@ffmpeg.exe -y -threads %cput% -hide_banner -loglevel 24 -stats -avoid_negative_ts 1 -i "%1" -map 0:0 -flags:v:0 +cgop -c:v:0 libx264 -b:v:0 %vbitrate%k -preset:v:0 veryslow -tune:v:0 film -profile:v:0 high -level:v:0 4.0 -x264opts:v:0 %x264opts2% -x264-params:v:0 %x264opts2% -pass:v:0 2 -passlogfile:v:0 log_%~n1 -color_range:v:0 2 -pix_fmt:v:0 yuv420p -vf:v:0 %vproc% -flags:a +qscale -map 0:1 -c:a:0 ac3 -b:a:0 384k -ac:0 6 -filter_complex [0:a:0]%aproc%[a2c] -map [a2c]:a:1 -c:a:1 libmp3lame -q:a:1 6 -map 0:s:language:eng -c:s copy -f matroska "%OUTDIR%\%~n1.mkv"

[_Al_]

batch script to calculate 2pass average could be something like this:

rem call this subroutine to get 2pass average value
call :get_average_bitrate "my_file.mp4"



:get_average_bitrate
SET Mediainfo="C:tools\Mediainfo CLI\Mediainfo.exe"

rem 34400000 is something less than 4.3GB so get whatever number here for your desired overall size
SET "volume=34400000"

%Mediainfo% --Inform=Video;%%Duration%% "%~1" > "temp.tmp"
set /p duration_miliseconds=<"temp.tmp"
%Mediainfo% --Inform=Video;%%Duration/String%% "%~1" > "temp.tmp"
set /p duration_string=<"temp.tmp"
echo video runtime is %duration_string%
set /A duration_seconds=%duration_miliseconds% / 1000
set /a average=%volume%/%duration_seconds% 
echo average bitrate is %average%kbps
goto :eof

[marcorocchini]

mmmm thanks but is a little complicated for a cat, however

my source video file is 190GB (uncompressed YUY2 1920x1080 25fps video file)

General
Complete name : V:\sfilata2017\ESP\virtual.avi
Format : AVI
Format/Info : Audio Video Interleave
Format profile : OpenDML
File size : 181 GiB
Duration : 31mn 18s
Overall bit rate : 830 Mbps

Video
ID : 0
Format : YUV
Codec ID : YUY2
Codec ID/Info : YUV 4:2:2 as for UYVY but with different component ordering within the u_int32 macropixel
Duration : 31mn 18s
Bit rate : 830 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:2
Compression mode : Lossless
Bits/(Pixel*Frame) : 16.004
Stream size : 181 GiB (100%)

my audio source is 360 MB:

General
Complete name : V:\sfilata2017\ESP\virtual.wav
Format : Wave
File size : 344 MiB
Duration : 31mn 18s
Overall bit rate mode : Constant
Overall bit rate : 1 536 Kbps

Audio
Format : PCM
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : 1
Duration : 31mn 18s
Bit rate mode : Constant
Bit rate : 1 536 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Stream size : 344 MiB (100%)

My target is to optimize encoding and fit the video and audio in one .mp4 file of 4,7 GB to put it in una dvd-data.
But I need also deinterlace and double framerate, so that from 25fps ---> become a 50fps. And scale from 4:2:2 to 4:2:0

after a lot of proofs I have empirically find that CRF 19.5 produce a final .mp4 file of:

ffmpeg64bitmar2017.exe -threads 12 -y -i V:\sfilata2017\ESP\virtual.avi -i V:\sfilata2017\ESP\virtual.wav -pix_fmt yuv420p -vf yadif=1,scale=interl=1 -c:v libx264 -crf 19.5 -profile:v high -level:v 4.2 -g 33 -bf 2 -c:a ac3 -b:a 320k -ar 48000 -aspect 16:9 V:\sfilata2017\ESP\highcrf19.mp4

General
Complete name : V:\sfilata2017\ESP\sfilataGatti.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 4.30 GiB
Duration : 31mn 18s
Overall bit rate : 19.7 Mbps
Writing application : Lavf57.56.100

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.2
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 31mn 18s
Bit rate : 19.4 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 50.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.187
Stream size : 4.23 GiB (98%)
Writing library : x264 core 148 r2744 b97ae06
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=2 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=33 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=33 / rc=crf / mbtree=1 / crf=18.5 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Audio
ID : 2
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Format settings, Endianness : Big
Codec ID : ac-3
Duration : 31mn 18s
Bit rate mode : Constant
Bit rate : 320 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Delay relative to video : 40ms
Stream size : 71.6 MiB (2%)

Quality video seems good but I wonder:

1) in my proof I have try to use the CRF, not the VBR 2 passes: in general can CRF be the same quality or better of the VBR 2 passes?
2) if yes, is there a way to predict the the CRF value (To get to know use 19.5 I had to try 18,19,20,21 ... a lot of time lost) to get a specific target size?
3) if I need to scale from 4:2:2 to 4:2:0 is correct the use of

-vf yadif=1,scale=interl=1

to avoiding the chroma ghost as is?

[pandy]

mmmm thanks but is a little complicated for a cat, however

my source video file is 190GB (uncompressed YUY2 1920x1080 25fps video file)

Quality video seems good but I wonder:

1) in my proof I have try to use the CRF, not the VBR 2 passes: in general can CRF be the same quality or better of the VBR 2 passes?
2) if yes, is there a way to predict the the CRF value (To get to know use 19.5 I had to try 18,19,20,21 ... a lot of time lost) to get a specific target size?
3) if I need to scale from 4:2:2 to 4:2:0 is correct the use of

-vf yadif=1,scale=interl=1

to avoiding the chroma ghost as is?

2 Pass may provide better bitrate distribution across video, CRF is quality focus and quality is quite vaguely described by number - check http://slhck.info/video/2017/02/24/crf-guide.html .
Resize shall be done after deinterlacing and as such resizer shall be in progressive not interlace mode. Consider to use EDI deinterlacer, zscale resizer and bigger GOP (1 sec is quite safe, 2 -5 sec is acceptable).

Personally i calculate 2 pass target bitrate by adding video and audio bitrates and around 2 - 3% overhead for container (quite valid for matroska - other containers may have higher overhead) - PS3 as player has nasty limitation - file must be 4GB big and as such it is tricky to encode.

[sneaker]

2 Pass may provide better bitrate distribution across video

Not better distribution than CRF. About equal.

[marcorocchini]

about the 4:2:2-->4:2:0 scaling: I have to include the

scale=interl=1

in commandline or not?

already there is specify the

-pix_fmt yuv420p

line: is this enough to avoid the chroma ghost?

[pandy]

scale=interl=1

is for scaling non progressive video (fields are separated and scaled then relaced) - if you scaling progressive video (scaling deinterlaced video) it should be set to 0.

Not sure what you understand by chroma ghost.

@sneaker
disagree - bitrate allocation for complex video is usually better - important for situations where every bit counts, in normal use case (ie crf=20) probably no gain but for case where crf going to be 26 or higher it may be way better.

btw 2 pass will be faster where exact target size must be hit - it is almost impossible to hit exact crf value in less than 2 passes.

[sneaker]

CRF and 2pass are based on the same algorithm. 2pass is not better. If anything CRF is better because it doesn't have to constantly adjust for the slight target bitrate<>actual bitrate mismatches.

[pandy]

CRF and 2pass are based on the same algorithm. 2pass is not better. If anything CRF is better because it doesn't have to constantly adjust for the slight target bitrate<>actual bitrate mismatches.

Is better in terms of matching desired target size - you never know desired crf in advance - after 2 pass your crf is usually lower than selected manually.

[sneaker]

Don't confuse "final ratefactor" and CRF, and don't take them as measures of quality.

[_Al_]

To hit target volume is not scientific measure of quality or anything else, it is just stuffing whatever content into limited space with consequences - either ending up wasting space (hey, it looks good, 2pass is great! ) or having not enough (hey , it does not look that good, but I saved space, CRF 18 takes too much )
This is the purpose of 2pass, not advantage.

Of course CRF 26 is crap, you know it WILL be crap. This is scientific prove that CRF make sense. You get steady results.

If insisting on some sort of "economy" encodings or encoding streaming video, in both cases you start to "starve" encoder in some way, use limiting buffer technique together with lowerCRF , not CRF 26. Scenes that x264 gives lots of bitrate would look good anyway (for your purpose) and you boost low light, color gradient scenes at the same time.

[jagabo]

Here we go again...

CRF and 2-pass deliver almost the same quality (assuming other settings are the same) when the final bitrates match. In my experience, 2-pass gives slightly higher peak bitrates and slightly lower minimum bitrates. But not so much that you'll really notice any difference in picture quality at playback.

You should use CRF when you want to be assured of the quality of the video but don't particularly care about the exact size. That is, you select the quality and the encoder delivers whatever file size is needed for that quality.

You should use 2-pass when you want to be assured of the size of the video but don't particularly care about the exact quality. That is, you select the file size, and the encoder delivers whatever quality it can for that size.

Chroma ghosting is probably the result of incorrect 4:2:2 to 4:2:0 conversion -- using a progressive algorithm on interlaced video.

[marcorocchini]

can I predict the file size in the case of a CRF encoding? Is there a tool or FFMPEG function that can estimate the final size without enconding?

I have find a little more quality in 2 pass VBR encoding but each time I don't know what is my target size: because of this, I assume as final size the correspondant size of CRF-encoding.

For example I have 20 minutes of my video cat files, and I encode it in CRF achieving a final size of 1,2 GB

Using the myFFmpeg tool I encode the same video in VBR-2pass @ 1,2GB target size mode with the same x264 options and seems that quality is a little better. But is a time expensive/wasting to encode previously in CRF mode only to get the file-size. So I wonder if is there a way to get this parameter (approx) previously without a real encoding

[sneaker]

You are wasting your time.

[jagabo]

Using the myFFmpeg tool I encode the same video in VBR-2pass @ 1,2GB target size mode with the same x264 options and seems that quality is a little better. But is a time expensive/wasting to encode previously in CRF mode only to get the file-size. So I wonder if is there a way to get this parameter (approx) previously without a real encoding

Encoding the video in 2-pass mode will not get you better quality. So just call the job done after the CRF encode.

[marcorocchini]

quality is practically the same but seems to me (that I'm a Cat:- ) that VBR-2 passes preserve a little more detail @ the exactly same size of the CRF. I have do some proof using as source the original image, and comparing in a CRT monitor the VBR-2 passes and CRF results

[jagabo]

I have do some proof using as source the original image, and comparing in a CRT monitor the VBR-2 passes and CRF results

I'd love to see your examples. I've run dozens of tests with all types of material, clean, noisy, SD, HD, live, animated, etc. and run visual ABX comparisons, PSNR and SSIM analysis, and have never seen significant differences. Yes, sometimes you see a frame or detail that's a little better in 2-pass, but every one of those there is another frame/detail that's better in the CRF encode. In the end they balance out.

[pandy]

2 pass usually deal better with noise or grain...

As i said earlier - i use 2 pass to match final file size in case where file size is main limitation (like 4GB-1 FAT32 limit) - in other cases i prefer crf or fixed qp.

And interlaced video before resize shall be always deinterlaced - resizing of interlaced video is possible only in limited situations (resizing by half efficiently work as deinterlace technique called field blending)..

[marcorocchini]

I cannot understan if the line -vf scale=interl=1 I ha to be use even encoding from 4:2:2 --> 4:2:0 or also in the case if the 4:2:2 --> 4:2:0 conversion that include the yadif.. so that my commandline

ffmpeg.exe -threads 12 -y -i input.avi -i input.wav -pix_fmt yuv420p -vf yadif=1,scale=interl=1 -c:v libx264 -crf 19 -profile:v high -level:v 4.2 -g 33 -bf 2 -c:a ac3 -b:a 320k -ar 48000 -aspect 16:9 out.mp4

that do conversion 4:2:2 --> 4:2:0 and do deinterlce+ double framerate

.. I wonder: is corrrect to keep

-vf yadif=1,scale=interl=1

or in enaught only

-vf yadif=1

?

my target is a 4:2:0 final file, and my source is 4:2:2

[jagabo]

Why do you bother asking questions like that when you can run a 10 second test and figure it out for yourself?

[jagabo]

If you really think 2-pass is better: you can run CRF on the first pass, then use the bitrate of the first pass on the second pass. You'll have to use --slow-firstpass though.

[pandy]

I cannot understan if the line -vf scale=interl=1 I ha to be use even encoding from 4:2:2 --> 4:2:0 or also in the case if the 4:2:2 --> 4:2:0 conversion that include the yadif.. so that my commandline

ffmpeg.exe -threads 12 -y -i input.avi -i input.wav -pix_fmt yuv420p -vf yadif=1,scale=interl=1 -c:v libx264 -crf 19 -profile:v high -level:v 4.2 -g 33 -bf 2 -c:a ac3 -b:a 320k -ar 48000 -aspect 16:9 out.mp4

that do conversion 4:2:2 --> 4:2:0 and do deinterlce+ double framerate

.. I wonder: is corrrect to keep

-vf yadif=1,scale=interl=1

or in enaught only

-vf yadif=1

?

my target is a 4:2:0 final file, and my source is 4:2:2

Once again... after deinterlacer (in your case yadif) video is no longer interlaced as such resize flag shall be progressive (and usually interlace flag mean separate fields and resize them separately then fold fields back to recreate interlace - such approach is usually violation for sampling theorem and usually it looks like garbage so theory match practice).

Different problem is chroma sample location in 422 vs 420 and interlace vs progressive.

[jagabo]

Not to mention that there's no scaling taking place so it doesn't matter at all.

[poisondeathray]

I'm not a cat, but I think he's asking about the scaling of the U,V channels when 422=>420. ie. Is -pix_fmt yuv420p applied before or after yadif ? If before, you need interlaced scaling. If after, you need progressive scaling. It's easy enough to test.

Alternatively, you can specify a linear chain using format, instead of -pix_fmt separately. That way you ensure the proper order. -vf yadif=1,format=pix_fmts=yuv420p

[marcorocchini]

mmmm thanks c**s

so if there is the 4:2:2-->4:2:0 color depth conversion but not the interlace --> there is no necessary the -vf scale=interl=1 line to avoid the chroma ghost