Which codec and settings to use with FFmpeg for temporary transcoding?

[konstantinusz]

I would like to upload about 200 videos to a video sharing website. Unfortunately the videos are rather dark and the usual flash player of the website doesn't allow setting brightness or gamma, so I would like to transcode my videos and tuning the gamma before uploading. This needs a re-encode because gamma and brightness cannot be set in bitstream / container level.

I would like to do the transcoding as fast as possible. I don't care the resulting video size so bitrate 3000-6000 kbps for an SD size video (eg 512x384 - 960x544) is not a problem for me, because uploading speed is very high and I think website deletes the uploaded files after transcoding them to their native mp4 format. However transcoding speed is much more important for me, I would like to achieve a 3x multiplyer at least, which means to transcode a 30 min video in 10 minutes or less.

My command line for ffmpeg looks like:

ffmpeg -i http://example.com/video.avi -c:v <magic-codec> -flags <magic-settings> -q:v 1 -c:a copy -vf "lutyuv=y=gammaval(1/1.65)" output.mkv

Which codec and settings will do the trick? I tried mpeg2video with "-qscale 2" but I could achieve only a 2x multiplier for a 512x384 video.

Or doesn't any web service exist (lets say ffmpegbox.com), where I can post a video slice and some filters together with it and I get back a h264 compressed slice in super fast time ?

[Baldrick]

Any faster with a constant bitrate?

-b:v

mpeg2 is probably best for bitrates around 3000-6000kbps and fast encoding.

[sneaker]

mpeg2? What year is it?

I would recommend using x264 with preset that achieves your speeds. Now the question is why encoding on your PC is so slow? Something going wrong or is your CPU 10+ years old?

[pandy]

Constant quantization will be always faster than bitrate, search for multi thread codec - h.264 is a good candidate - with preset ultrafast, tune fastdecode and gop size it can be faster than mpeg2 inside ffmpeg - you can try to tweak multi thread mpeg2 encoding - there is few options...

Look at those options - i would disable motion search, force intra coding and force multi thread capabilities.

Encoder mpeg2video [MPEG-2 video]:
    General capabilities: delay threads 
    Threading capabilities: slice
    Supported framerates: 1/1 2/1 3/1 4/1 5/1 6/1 8/1 9/1 10/1 12/1 15/1 16/1 18/1 20/1 24/1 25/1 30/1 32/1 36/1 40/1 45/1 48/1 50/1 60/1 72/1 75/1 80/1 90/1 96/1 100/1 120/1 150/1 180/1 200/1 240/1 750/1001 800/1001 960/1001 1000/1001 1200/1001 1250/1001 1500/1001 1600/1001 1875/1001 2000/1001 2400/1001 2500/1001 3000/1001 3750/1001 4000/1001 4800/1001 5000/1001 6000/1001 7500/1001 8000/1001 10000/1001 12000/1001 15000/1001 20000/1001 24000/1001 30000/1001 60000/1001
    Supported pixel formats: yuv420p yuv422p
mpeg2video encoder AVOptions:
  -gop_timecode      <string>     E..V.... MPEG GOP Timecode in hh:mm:ss[:;.]ff format. Overrides timecode_frame_start.
  -intra_vlc         <boolean>    E..V.... Use MPEG-2 intra VLC table. (default false)
  -drop_frame_timecode <boolean>    E..V.... Timecode is in drop frame format. (default false)
  -scan_offset       <boolean>    E..V.... Reserve space for SVCD scan offset user data. (default false)
  -timecode_frame_start <int64>      E..V.... GOP timecode frame start number, in non-drop-frame format (from -1 to I64_MAX) (default -1)
  -non_linear_quant  <boolean>    E..V.... Use nonlinear quantizer. (default false)
  -alternate_scan    <boolean>    E..V.... Enable alternate scantable. (default false)
  -seq_disp_ext      <int>        E..V.... Write sequence_display_extension blocks. (from -1 to 1) (default auto)
     auto                         E..V....
     never                        E..V....
     always                       E..V....
  -mpv_flags         <flags>      E..V.... Flags common for all mpegvideo-based encoders. (default 0)
     skip_rd                      E..V.... RD optimal MB level residual skipping
     strict_gop                   E..V.... Strictly enforce gop size
     qp_rd                        E..V.... Use rate distortion optimization for qp selection
     cbp_rd                       E..V.... use rate distortion optimization for CBP
     naq                          E..V.... normalize adaptive quantization
     mv0                          E..V.... always try a mb with mv=<0,0>
  -luma_elim_threshold <int>        E..V.... single coefficient elimination threshold for luminance (negative values also consider dc coefficient) (from INT_MIN to INT_MAX) (default 0)
  -chroma_elim_threshold <int>        E..V.... single coefficient elimination threshold for chrominance (negative values also consider dc coefficient) (from INT_MIN to INT_MAX) (default 0)
  -quantizer_noise_shaping <int>        E..V.... (from 0 to INT_MAX) (default 0)
  -error_rate        <int>        E..V.... Simulate errors in the bitstream to test error concealment. (from 0 to INT_MAX) (default 0)
  -qsquish           <float>      E..V.... how to keep quantizer between qmin and qmax (0 = clip, 1 = use differentiable function) (from 0 to 99) (default 0)
  -rc_qmod_amp       <float>      E..V.... experimental quantizer modulation (from -FLT_MAX to FLT_MAX) (default 0)
  -rc_qmod_freq      <int>        E..V.... experimental quantizer modulation (from INT_MIN to INT_MAX) (default 0)
  -rc_eq             <string>     E..V.... Set rate control equation. When computing the expression, besides the standard functions defined in the section 'Expression Evaluation', the following functions are available: bits2qp(bits), qp2bits(qp). Also the following constants are available: iTex pTex tex mv fCode iCount mcVar var isI isP isB avgQP qComp avgIITex avgPITex avgPPTex avgBPTex avgTex.
  -rc_init_cplx      <float>      E..V.... initial complexity for 1-pass encoding (from -FLT_MAX to FLT_MAX) (default 0)
  -rc_buf_aggressivity <float>      E..V.... currently useless (from -FLT_MAX to FLT_MAX) (default 1)
  -border_mask       <float>      E..V.... increase the quantizer for macroblocks close to borders (from -FLT_MAX to FLT_MAX) (default 0)
  -lmin              <int>        E..V.... minimum Lagrange factor (VBR) (from 0 to INT_MAX) (default 236)
  -lmax              <int>        E..V.... maximum Lagrange factor (VBR) (from 0 to INT_MAX) (default 3658)
  -ibias             <int>        E..V.... intra quant bias (from INT_MIN to INT_MAX) (default 999999)
  -pbias             <int>        E..V.... inter quant bias (from INT_MIN to INT_MAX) (default 999999)
  -rc_strategy       <int>        E..V.... ratecontrol method (from 0 to 1) (default ffmpeg)
     ffmpeg                       E..V.... default native rate control
     xvid                         E..V.... libxvid (2 pass only)
  -motion_est        <int>        E..V.... motion estimation algorithm (from 0 to 2) (default epzs)
     zero                         E..V....
     epzs                         E..V....
     xone                         E..V....
  -force_duplicated_matrix <boolean>    E..V.... Always write luma and chroma matrix for mjpeg, useful for rtp streaming. (default false)
  -b_strategy        <int>        E..V.... Strategy to choose between I/P/B-frames (from 0 to 2) (default 0)
  -b_sensitivity     <int>        E..V.... Adjust sensitivity of b_frame_strategy 1 (from 1 to INT_MAX) (default 40)
  -brd_scale         <int>        E..V.... Downscale frames for dynamic B-frame decision (from 0 to 3) (default 0)
  -skip_threshold    <int>        E..V.... Frame skip threshold (from INT_MIN to INT_MAX) (default 0)
  -skip_factor       <int>        E..V.... Frame skip factor (from INT_MIN to INT_MAX) (default 0)
  -skip_exp          <int>        E..V.... Frame skip exponent (from INT_MIN to INT_MAX) (default 0)
  -skip_cmp          <int>        E..V.... Frame skip compare function (from INT_MIN to INT_MAX) (default dctmax)
     sad                          E..V.... Sum of absolute differences, fast
     sse                          E..V.... Sum of squared errors
     satd                         E..V.... Sum of absolute Hadamard transformed differences
     dct                          E..V.... Sum of absolute DCT transformed differences
     psnr                         E..V.... Sum of squared quantization errors, low quality
     bit                          E..V.... Number of bits needed for the block
     rd                           E..V.... Rate distortion optimal, slow
     zero                         E..V.... Zero
     vsad                         E..V.... Sum of absolute vertical differences
     vsse                         E..V.... Sum of squared vertical differences
     nsse                         E..V.... Noise preserving sum of squared differences
     dct264                       E..V....
     dctmax                       E..V....
     chroma                       E..V....
  -sc_threshold      <int>        E..V.... Scene change threshold (from INT_MIN to INT_MAX) (default 0)
  -noise_reduction   <int>        E..V.... Noise reduction (from INT_MIN to INT_MAX) (default 0)
  -mpeg_quant        <int>        E..V.... Use MPEG quantizers instead of H.263 (from 0 to 1) (default 0)
  -ps                <int>        E..V.... RTP payload size in bytes (from INT_MIN to INT_MAX) (default 0)
  -mepc              <int>        E..V.... Motion estimation bitrate penalty compensation (1.0 = 256) (from INT_MIN to INT_MAX) (default 256)
  -mepre             <int>        E..V.... pre motion estimation (from INT_MIN to INT_MAX) (default 0)

Multi thread ffmpeg options:

  -slices            <int>        E..V.... set the number of slices, used in parallelized encoding (from 0 to INT_MAX) (default 0)
  -thread_type       <flags>      ED.VA... select multithreading type (default slice+frame)
     slice                        ED.V....
     frame                        ED.V....

[raffriff42]

Easy decision IMHO:

-c:v libx264 -crf 18.0 -preset veryfast

[pandy]

Easy decision IMHO:

-c:v libx264 -crf 18.0 -preset veryfast

qp will be faster (no decision on bitrate)

[sneaker]

Constant quantization will be always faster than bitrate

Single pass will be faster than multi pass.

h.264 is a good candidate - with preset ultrafast, tune fastdecode and gop size

Why would you recommend fastdecode or "gop size" (I assume you mean small gop size)? He wants fast encoding, not hurt compression unnecessarily. Just choose preset. (preset ultrafast implies fastdecode anyways)

[sneaker]

qp will be faster (no decision on bitrate)

Negligible. The resulting bitrate difference between qp and crf will have a stronger effect on fps so crf might be faster depending on the source, though I don't have any statistics at hand.

[pandy]

Constant quantization will be always faster than bitrate

Single pass will be faster than multi pass.

Of course - i never recommend dual pass especially for fixed qp - this will be bizzare recommendation as fixed qp produce pure VBR.

h.264 is a good candidate - with preset ultrafast, tune fastdecode and gop size

Why would you recommend fastdecode or "gop size" (I assume you mean small gop size)? He wants fast encoding, not hurt compression unnecessarily. Just choose preset. (preset ultrafast implies fastdecode anyways)

CABAC off, GOP=1 (intra) - yes - ultrafast partially imply fastdecode but i assume ultrafast and fastdecode provide fastest encode/decode.

qp will be faster (no decision on bitrate)

Negligible. The resulting bitrate difference between qp and crf will have a stronger effect on fps so crf might be faster depending on the source, though I don't have any statistics at hand.

I've made some tests in past and there is small difference as such where speed is main goal i will always recommend fixed qp.

[sneaker]

CABAC off, yes it partially imply fastdecode

--preset ultrafast fully implies --tune fastdecode. (See x264 --fullhelp.)

but i assume ultrafast and fastdecode provide fastest encode/decode.

--tune fastdecode implies fast decode. Not encode. OP isn't interested in fast decoding.

[pandy]

CABAC off, yes it partially imply fastdecode

--preset ultrafast fully implies --tune fastdecode. (See x264 --fullhelp.)

but i assume ultrafast and fastdecode provide fastest encode/decode.

--tune fastdecode implies fast decode. Not encode. OP isn't interested in fast decoding.

Both leading to simplify syntax (features) used by codec i.e. faster encoding and decoding... and yes fastdecode is redundant (but it will not hurt anything if used)

- ultrafast:
 --no-8x8dct --aq-mode 0 --b-adapt 0
 --bframes 0 --no-cabac --no-deblock
 --no-mbtree --me dia --no-mixed-refs
 --partitions none --rc-lookahead 0 --ref 1
 --scenecut 0 --subme 0 --trellis 0
 --no-weightb --weightp 0 

- fastdecode:
 --no-cabac --no-deblock --no-weightb
 --weightp 0

[konstantinusz]

mpeg2? What year is it?

I would recommend using x264 with preset that achieves your speeds. Now the question is why encoding on your PC is so slow? Something going wrong or is your CPU 10+ years old?

Yes, my PC is 14 years old, I bought it in 2002 and still using it... Pretty slow though, maybe I should by a laptop instead. This is why I do the encoding / transcoding stuff on a VPS: about 4x faster than on home machine.

[konstantinusz]

Thx for the replies, but how can I set -qscale or -c:v for h264? ffmpeg says that quantizer is not supported any more, only -crf settings, constant rate factor.
Other question: in VirtualDub when compressing with xvid, one can set a quantizer like 1.5 or 2.5. In this case quantizers are chosen in a way that their mean will be 1.5 or 2.5 - 1 or 2 and 2 or 3 repsectively.

Is it possible also for h264 to set a real quantizer or it can be only integer number?

[pandy]

Thx for the replies, but how can I set -qscale or -c:v for h264? ffmpeg says that quantizer is not supported any more, only -crf settings, constant rate factor.
Other question: in VirtualDub when compressing with xvid, one can set a quantizer like 1.5 or 2.5. In this case quantizers are chosen in a way that their mean will be 1.5 or 2.5 - 1 or 2 and 2 or 3 repsectively.

At first why you have problem with INT quantizer? Fractional qp is outcome of averaging series INT qp values - consequentive frames can be only INT - qscale may be disabled but AFAIR there is way to force ffmpeg trough flag to use qscale - not sure if it works or not - i use qp trough internal x264 params.

-flags:v +qscale

Is it possible also for h264 to set a real quantizer or it can be only integer number?

try to use x264aprams and x264opts (only INT accepted) - working example bellow:

@setlocal
@set FFMPG=D:\Media\FF
@set PATH=%PATH%;%FFMPG%
@set FC_CONFIG_DIR=%FFMPG%\conf
@set FONTCONFIG_FILE=%FFMPG%\conf\fonts.conf
@set FONTCONFIG_PATH=%FFMPG%\conf

@SET qp=8

@SET x264opts="qp=%qpval%:8x8dct=0:cabac=0:me_range=4:chroma_me=0:keyint=1:open_gop=0:pic_struct=1:aud=1:force_cfr=1:threads=auto:no_psnr=1:no_ssim=1:fullrange=on:overscan=show:colorprim=undef:transfer=undef:colormatrix=YCgCo:stitchable=1"

@ffmpeg -y -hide_banner -color_range 2 -benchmark -i %1 -c:a copy -c:v libx264 -preset ultrafast -tune zerolatency -profile:v high -x264opts %x264opts% -x264-params %x264opts% -vf "scale=iw:ih:sws_flags=sws_flags:sws_dither=0:in_range=1:out_range=1,format=pix_fmts=yuv420p" -color_range 2 -colorspace 8 -f matroska "%~n1_.mkv"

@endlocal
@pause

[konstantinusz]

Thx, and what is the relation between the "qp" parameter (8 here) and the -qscale (or -q:v) parameter value, which is the quentizer, and it is best to set to 2 ?

[pandy]

Thx, and what is the relation between the "qp" parameter (8 here) and the -qscale (or -q:v) parameter value, which is the quentizer, and it is best to set to 2 ?

This is same thing only different way to pass this to encoder. Just replace 8 by 2 however in my opinion 8 should give you good quality already and i would not go bellow 4.