improve encoding speed in ffmpeg/x264?

[marcorocchini]

ffmpeg64bitjuly2017.exe -threads 12 -y -i source422.mxf -pix_fmt yuv420p -vf scale=interl=1 -c:v libx264 -profile:v main -level:v 4.1 -g 33 -bf 2 -crf 22 -flags +ildct+ilme -top 1 -aspect 16:9 -an x264july2017high.MP4

UFF .. cats, I'm force to encode @ 17/18fps

is the a way to improve encoding speed in ffmpeg/x264?

Thanks

[poisondeathray]

Use a faster preset . The faster the preset, the lower the compression ratio (lower quality at given bitrate)

e.g

-preset:v superfast

[jagabo]

Two settings that effect encoding speed a lot with x264:
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#me
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#subme

As far as the presets are concerned, when using CRF encoding, ultrafast and superfast deliver high bitrates and are only useful for realtime compression. Veryfast through placebo deliver lower bitrates of course, but there's usually only about a 20 percent difference, and the size doesn't decrease linearly -- there are small peaks and valleys. But quality increases noticeably up to about slow or veryslow. veryslow and beyond deliver video that many devices cannot handle (mainly too many b-frames and reference frames). Though, if you're limiting the settings to level 4.1 that shouldn't be an issue.

[sneaker]

Why are you limiting to -profile main?
Nowadays high profile works pretty much everywhere.

Two settings that effect encoding speed a lot with x264:
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#me
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#subme

It's preferable to change the presets instead. Not only will they change me/subme, they will also change the other options to "match" the chosen subme/me so that the time/compression tradeoff is optimal for a given time.

TAs far as the presets are concerned, when using CRF encoding, ultrafast and superfast deliver high bitrates

Those two don't use mbtree and CRF is very different with(out) mbtree. That's why they're often wildly different from the rest of the presets. Sometimes they even produce smaller files than the other presets.

veryslow and beyond deliver video that many devices cannot handle (mainly too many b-frames and reference frames). Though, if you're limiting the settings to level 4.1 that shouldn't be an issue.

Many bframes aren't usually a problem.
With ref frames and level you have to be careful. If you are using x264cli ("x264.exe") you can simply set e.g. --level 4.1 and the number of references is automatically reduced. This is not automatic if you use (lib)x264 via ffmpeg! Then you need to manually set e.g. --x264-params ref=4.
But as you say is mostly a problem with presets slow to placebo. medium and lower/faster use not more than 3 (unless combined with tune animation or manually set higher).

[pandy]

@set x264opts="qp=16:vbv_maxrate=20000:qpmin=4:level=4.0:ref=1:bframes=0:b_adapt=0:no_mbtree=1:no_mixed_refs=1:no_8x8dct=0:chroma_me=0:no_chroma_me=1:subme=1:me=dia:me_range=4:no_deblock=1:trellis=0:weightp=0:no_weightb=1:rc_lookahead=0:open_gop=0:keyint=1:min_keyint=1:sliced_threads=0:slices=%slice%:tff=1:interlaced=1:bluray_compat=1:pic_struct=1:aud=1:nal_hrd=vbr:force_cfr=1:cabac=0:threads=auto:no_psnr=1:no_ssim=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:stitchable=1:constrained_intra=0:psy=0:no_psy=1:no_scenecut=1:no_deblock=1:aq_mode=0:ipratio=1.0:chroma_qp_offset=0"
@ffmpeg.exe -y -hide_banner -i %1 -intra -flags:v +ilme+ildct+cgop+low_delay+qscale -me_method 0 -g 1 -top 1 -c:v libx264 -profile:v high -level:v 4.0 -x264opts %x264opts% -x264-params %x264opts% -an -dn -sn -f mpegts

Above should provide real time encoding on moderate speed CPU.

[sneaker]

keyint=1? qp? vbv maxrate without bufsize? merange=4? no psy? ipratio=1.0? (bluray-compat, hrd, ...)

There is no need to re-invent the wheel. We have presets to make this task easier and less error prone.

[jagabo]

With ref frames and level you have to be careful. If you are using x264cli ("x264.exe") you can simply set e.g. --level 4.1 and the number of references is automatically reduced. This is not automatic if you use (lib)x264 via ffmpeg!

I didn't realize ffmepg didn't do the same.

[pandy]

keyint=1? qp? vbv maxrate without bufsize? merange=4? no psy? ipratio=1.0? (bluray-compat, hrd, ...)

There is no need to re-invent the wheel. We have presets to make this task easier and less error prone.

Perhaps presets are tailored for BD ripped 24fps movies... this usage case was different and verified for speed (real time video generation and encoding on moderate speed CPU - few year old server blade from trash pile).
btw buffsize is limited by level, BD compatibility to enforce particular encoder limitation/config (target is consumer HW decoder with maximum level 4.1 supported but 4.0 preffered - it was not about video quality but particular functionality).

With ref frames and level you have to be careful. If you are using x264cli ("x264.exe") you can simply set e.g. --level 4.1 and the number of references is automatically reduced. This is not automatic if you use (lib)x264 via ffmpeg!

I didn't realize ffmepg didn't do the same.

for sure you will receive warning...

[sneaker]

Perhaps presets are tailored for BD ripped 24fps movies... this usage case was different and verified for speed (real time video generation and encoding on moderate speed CPU - few year old server blade from trash pile).

The presets are also "verified". You know, by the guys who developed x264. And all the users who have tested them.

btw buffsize is limited by level

The level setting does not set bufsize for you. Your combination of settings is not working the way you intend to. No vbv. Which also makes e.g. HRD not work.


Your settings are just insane. keyint 1 will hurt compression very badly. If someone asks for faster settings you don't throw the most insane combination you can think of at him. Maybe he will be happy with preset faster, veryfast etc.

BD compatibility to enforce particular encoder limitation/config (target is consumer HW decoder with maximum level 4.1 supported but 4.0 preffered

That doesn't explain anything. We set level through level and enforce bitrates and ref through vbv and ref options. bluray-compat setting doesn't ensure level compatibility!

[pandy]

The presets are also "verified". You know, by the guys who developed x264. And all the users who have tested them.

Seriously doubt on this - AFAIK HW decoders testing is quite limited in case of x264 (but i don't blame x264 developers for this - it is quite obvious that they access to HW decoders may be somehow limited).

The level setting does not set bufsize for you. Your combination of settings is not working the way you intend to. No vbv. Which also makes e.g. HRD not work.

True (buffer size was defined only accidentally removed during edit - removed some syntax due presence variable).
Bellow full line taken from script:

@SET x264opts="qp=%qpval%:vbv_maxrate=%vmaxrate%:vbv_bufsize=%vbuffsiz%:vbv_init=.5:qpmin=4:level=4.0:ref=1:bframes=0:b_adapt=0:no_mbtree=1:no_mixed_refs=1:no_8x8dct=0:chroma_me=0:no_chroma_me=1:subme=1:me=dia:me_range=4:no_deblock=1:trellis=0:open_gop=0:keyint=1:min_keyint=1:sliced_threads=0:slices=%slice%:interlaced=0:bluray_compat=1:pic_struct=1:aud=1:nal_hrd=vbr:force_cfr=1:cabac=0:threads=auto:no_psnr=1:no_ssim=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:stitchable=1:constrained_intra=0:psy=0:no_psy=1:no_scenecut=1:no_deblock=1:aq_mode=0:ipratio=1.0:chroma_qp_offset=0"

Your settings are just insane. keyint 1 will hurt compression very badly. If someone asks for faster settings you don't throw the most insane combination you can think of at him. Maybe he will be happy with preset faster, veryfast etc.

Maybe he will be happy - he is free to edit, use or not - in my case i don't care about compression but i do care about CPU load as on this machine audio and video is generated in real time, in real time audio video is encoded, real time multiplexed and real time injected to headend multiplexer over IP and it have special purpose - series of such services is used to verify channel switching in automated test system - this is not about compression quality or efficiency - it is about creating repeatable test scenario where HW decoder can be verified in objective way in particular case - keyint=1 is to remove GOP limitation.
FYI With this setting average blade machine was able to create 2 1080i25 services.

That doesn't explain anything. We set level through level and enforce bitrates and ref through vbv and ref options. setting doesn't ensure level compatibility!

It is not about level but used codec syntax and configuration compatible with HW decoders (as you know BD require some limitations or rather particular configuration for H.264 stream).

Hope it is clear now - btw as i understand Marco - he pursue some "AVC INTRA like' settings.

[poisondeathray]

Hope it is clear now - btw as i understand Marco - he pursue some "AVC INTRA like' settings.

Probably doesn't want "intra like" settings, the first post has -g 33 -bf 2

[sneaker]

The presets are also "verified". You know, by the guys who developed x264. And all the users who have tested them.

Seriously doubt on this - AFAIK HW decoders testing is quite limited in case of x264 (but i don't blame x264 developers for this - it is quite obvious that they access to HW decoders may be somehow limited).

I meant "verified" in the sense that the presets are optimal points on the speed compression curve. Why should any given preset be compatible with your specific hardware out-of-the-box out of thousands different H.264 devices? It should comply with the H.264 specs. You can then use the other options to make it compatible with your specific hardware. x264 has the tools for that.

Maybe he will be happy - he is free to edit, use or not

He doesn't know ...

- in my case i don't care about compression but i do care about CPU load as on this machine audio and video is generated in real time, in real time audio video is encoded, real time multiplexed and real time injected to headend multiplexer over IP and it have special purpose - series of such services is used to verify channel switching in automated test system - this is not about compression quality or efficiency - it is about creating repeatable test scenario where HW decoder can be verified in objective way in particular case

Right. A very very particular use case. Why recommend it to someone who just asked for "faster"? He didn't ask for your scenario. And he didn't say "as fast as possible no matter cost" or "compatible with pandy's hardware".
That's like recommending someone who asked for a "decent car" an amphibious vehicle just because you happen to live in the middle of a lake.

It is not about level but used codec syntax and configuration compatible with HW decoders (as you know BD require some limitations or rather particular configuration for H.264 stream).

Your settings are way beyond BD limitations.

Hope it is clear now - btw as i understand Marco - he pursue some "AVC INTRA like' settings.

I didn't see that. Did he say that in a different thread? (wouldn't surprise me with how much he's over the place...)

P.S.: x264cli can create actual AVC-INTRA.

[pandy]

I meant "verified" in the sense that the presets are optimal points on the speed compression curve. Why should any given preset be compatible with your specific hardware out-of-the-box out of thousands different H.264 devices? It should comply with the H.264 specs. You can then use the other options to make it compatible with your specific hardware. x264 has the tools for that.

Optimal for general usage like movie encoding not for edition or quick access with 1 frame accuracy... AFAIR there is no ready to use profile like AVC INTRA but rather switch to force some hacks.

He doesn't know ...

oh common - he can try at least, he can ask once again - this is iterative process and i have impression that you slightly underestimate Marco.

Right. A very very particular use case. Why recommend it to someone who just asked for "faster"? He didn't ask for your scenario. And he didn't say "as fast as possible no matter cost" or "compatible with pandy's hardware".
That's like recommending someone who asked for a "decent car" an amphibious vehicle just because you happen to live in the middle of a lake.

now i have impression that you just trying to find something that doesn't exist...

Your settings are way beyond BD limitations.

but not beyond HW decoders core's used also not in BD products in real RT environment.

P.S.: x264cli can create actual AVC-INTRA.

Yes, but AVC INTRA has own limitations/quirks/demands that's why i've wrote 'AVC INTRA like' - (BD HW decoders are not compliant with AVC INTRA for sure).

[marcorocchini]

[Attachment 42382 - Click to enlarge] sorry but I'm a *** and very ignorant

however my answer was bad, because I need to specify more. For example, I mean if is there a way to improove speed at the same quality and overall at the same target size. Seems that trellis force to the CPU to a greater computational complexity but seems that it "improove" image quality

The question is certainly more complex than the initial question

In my specific case is not important the decoding of the encoded files because when I deliver to my catStation it have a mediation system (called "vantage") that re-code all to uncompress and deliver to the editing stations in uncompressed mode, so I can encode to a maximum complexity without the problem of the playback

[sneaker]

however my answer was bad, because I need to specify more. For example, I mean if is there a way to improove speed at the same quality and overall at the same target size.

Not much except increase keyint (-g) and maybe tune for specific source (e.g. -tune film). Or downscale the video. (but then you need to deinterlace which costs time if done with a good deinterlacer and scaler)

[marcorocchini]

for my specific case I need a "balanced" and "rational" encoding that put in balance the speed and quality depending on the hardware of the machine.

and the commandline that I have tryed one by one the options of pandy is:

v:\automazioneclip\core\ffmpeg64bit332.exe -threads 12 -y -f conCAT -safe 0 -i conCATList.txt -filter_complex "[0:1] [0:2] amerge" -vf scale=interl=1 -pix_fmt yuv420p -c:v libx264 -profile:v main -level:v 4.1 -x264-params me_range=4:subme=1:rc_lookahead=1:chroma_qp_offset=0:8x8dct=1 -g 100 -bf 2 -crf 22.8 -flags +ildct+ilme -top 1 -c:a ac3 -b:a 320k -ar 48000 -aspect 16:9 out.mp4

In my proof seems that encoding become average 62 FPS instead of 20, and the quality loss is minimal

[sneaker]

I recommend using the presets (and high profile) instead of playing around with the detailed options. Your settings are not using the CPU in the best way.