Documentation for QSV enabled filters?

[BartZLederman]

I have a PC running Windows which is QSV capable. I've found the codec and at least one QSV filter which really speed up decoding and encoding AVC (x264) videos.

But the experience is extremely aggravating, as there is no proper list anywhere that I can find on what the filters and arguments actually are.

The few examples I've been able to find in various places (including stackoverflow) do not work. I've done a cut-and-paste to make sure I'm not typing them in wrong.

The closest I've come to any explanation of vpp_qsv is this:

https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_vpp_qsv.c

It's a start, but it has no examples of what the arguments actually do or how to use them.

By experimentation, I've found that this works:

ffmpeg -hide_banner -init_hw_device qsv=qsv -hwaccel qsv -i "input.mts" -c:a aac -c:v h264_qsv -preset veryslow -vf "vpp_qsv=cw=628:cx=0:w=640:h=480" -q:v 30 "output.mp4"

I can add other things to the filter. For example:

"vpp_qsv=deinterlace=2:cw=628:cx=0:w=640:h=480 "

(By the way, the value for "deinterlace" is mandatory, if you don't enter a value it will not use the default. I've also found that "saturation" appears to be just a placeholder, no matter what value I enter the saturation doesn't change.)

The above examples work. However, this will NOT work:

"vpp_qsv=cw=628:cx=0:w=640:h=480:async_depth=4 "

even though there are examples posted that say it will, and the web page posted above with the source code says that async_depth is a valid parameter.

[Parsed_vpp_qsv_0 @ 0000000005bff8c0] Option 'async_depth' not found
[AVFilterGraph @ 0000000007327dc0] Error initializing filter 'vpp_qsv' with args 'cw=628:cx=0:w=640:h=480:async_depth=4'
Error reinitializing filters! Failed to inject frame into filter network: Option not found Error while processing the decoded data for stream #0:0

I also have been unable to find good examples of using more than one QSV enabled filter at the same time.

I'm using the newest version of ffmpeg I can find:

I:\ffmpeg -version ffmpeg version 4.4-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers built with gcc 10.2.0 (Rev6, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml 2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enab le-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --ena ble-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-lib zimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang - -enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine - -enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libv orbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100

and the output of "ffmpeg -filters" says I have the various QSV enabled filters on my system.

I have searched dozens and dozens of sites including Intel and ffmpeg.org and I cannot find a clear explanation of how the various QSV enabled filters work, with examples that will actually execute on my system. Could someone please say where the to look? Someone developed these filters, so someone somewhere must know how they work and why async_depth doesn't work.

[poisondeathray]

But the experience is extremely aggravating, as there is no proper list anywhere that I can find on what the filters and arguments actually are.

The full help has a list (you can argue it's not a "proper list", with illustrated examples and a 24/7 help line , but this is ffmpeg, better get used to it )

(By the way, the value for "deinterlace" is mandatory, if you don't enter a value it will not use the default. I've also found that "saturation" appears to be just a placeholder, no matter what value I enter the saturation doesn't change.)

Default is "0" which means "off". Many ffmpeg parameters and filters are structured like this. Parameter=value

Look in the -fullhelp
vpp_qsv AVOptions:
deinterlace <int> ..FV....... deinterlace mode: 0=off, 1=bob, 2=advanced (from 0 to 2) (default 0)
bob 1 ..FV....... Bob deinterlace mode.
advanced 2 ..FV....... Advanced deinterlace mode.
.
.
.
You have to enable saturation as a vpp_qsv procamp option . The hue/saturation/contrast/brightness should be indented in the list

vpp_qsv AVOptions:
  deinterlace       <int>        ..FV....... deinterlace mode: 0=off, 1=bob, 2=advanced (from 0 to 2) (default 0)
     bob             1            ..FV....... Bob deinterlace mode.
     advanced        2            ..FV....... Advanced deinterlace mode. 
  denoise           <int>        ..FV....... denoise level [0, 100] (from 0 to 100) (default 0)
  detail            <int>        ..FV....... enhancement level [0, 100] (from 0 to 100) (default 0)
  framerate         <rational>   ..FV....... output framerate (from 0 to DBL_MAX) (default 0/1)
  procamp           <int>        ..FV....... Enable ProcAmp (from 0 to 1) (default 0)
  hue               <float>      ..FV....... ProcAmp hue (from -180 to 180) (default 0)
  saturation        <float>      ..FV....... ProcAmp saturation (from 0 to 10) (default 1)
  contrast          <float>      ..FV....... ProcAmp contrast (from 0 to 10) (default 1)
  brightness        <float>      ..FV....... ProcAmp brightness (from -100 to 100) (default 0)
  transpose         <int>        ..FV....... set transpose direction (from -1 to 6) (default -1)
     cclock_hflip    0            ..FV....... rotate counter-clockwise with horizontal flip
     clock           1            ..FV....... rotate clockwise
     cclock          2            ..FV....... rotate counter-clockwise
     clock_hflip     3            ..FV....... rotate clockwise with horizontal flip
     reversal        4            ..FV....... rotate by half-turn
     hflip           5            ..FV....... flip horizontally
     vflip           6            ..FV....... flip vertically
  cw                <string>     ..FV....... set the width crop area expression (default "iw")
  ch                <string>     ..FV....... set the height crop area expression (default "ih")
  cx                <string>     ..FV....... set the x crop area expression (default "(in_w-out_w)/2")
  cy                <string>     ..FV....... set the y crop area expression (default "(in_h-out_h)/2")
  w                 <string>     ..FV....... Output video width (default "cw")
  width             <string>     ..FV....... Output video width (default "cw")
  h                 <string>     ..FV....... Output video height (default "w*ch/cw")
  height            <string>     ..FV....... Output video height (default "w*ch/cw")
  format            <string>     ..FV....... Output pixel format (default "same")
  async_depth       <int>        ..FV....... Internal parallelization depth, the higher the value the higher the latency. (from 0 to INT_MAX) (default 0)

eg.

-vf "vpp_qsv=format=nv12,vpp_qsv=procamp=1:saturation=2"

"vpp_qsv=cw=628:cx=0:w=640:h=480:async_depth=4 "

even though there are examples posted that say it will, and the web page posted above with the source code says that async_depth is a valid parameter.

[Parsed_vpp_qsv_0 @ 0000000005bff8c0] Option 'async_depth' not found
[AVFilterGraph @ 0000000007327dc0] Error initializing filter 'vpp_qsv' with args 'cw=628:cx=0:w=640:h=480:async_depth=4'
Error reinitializing filters! Failed to inject frame into filter network: Option not found Error while processing the decoded data for stream #0:0

a value of 1 for async_depth works for me, any other value besides default does not. Maybe it's used for multiple output's or filter_complex branches with more than a simple single linear filter chain. I didn't test it any father.

"vpp_qsv=format=nv12,vpp_qsv=procamp=1:saturation=2:async_depth=1"

It's also listed as an encoder option, but using it separately out of the filter chain does not work for me

h264_qsv encoder AVOptions:
-async_depth <int> E..V....... Maximum processing parallelism (from 1 to INT_MAX) (default 4).

I also have been unable to find good examples of using more than one QSV enabled filter at the same time.

eg.

 "vpp_qsv=format=nv12,vpp_qsv=procamp=1:saturation=2,vpp_qsv=detail=100"

Like everything ffmpeg, just try them out. If it doesn't work, check again or post a bug report if you 're certain it's ffmpeg and not a command issue

[BartZLederman]

I should have been more specific about 'more than one QSV filter.

Your example is multiple instances of the same filter "vpp_qsv".

I was looking for something like using "deinterlace_qsv=2, scale_qsv=..."

At the moment both of these operations can be done with a single vpp_qsv filter, but if you want to add something like overlay_qsv then you're stuck.

I'm asking this more in the hope that there will be more QSV enabled filters in the future, especially as Intel is expanding it's graphic support.

Actually, this is more of a problem with Cuda enabled filters: I haven't found a way to use both a Cuda filter and a non-accelerated filter on the same pass, but this should perhaps go in a separate topic.

[poisondeathray]

Actually, this is more of a problem with Cuda enabled filters: I haven't found a way to use both a Cuda filter and a non-accelerated filter on the same pass, but this should perhaps go in a separate topic.

The data has to be uploaded to cuda filter, and downloaded to "cpu" filters

hwupload_cuda, hwdownload


Often the increased latency for passing data back/forth cancels out speed advantages . So if you can tidy up the order of operations, you can make it less slow. Going back and forth multiple times is less ideal

e.g yadif_cuda, and cpu scale using swscale

ffmpeg -hwaccel cuda -i input.ext -vf hwupload_cuda,yadif_cuda=mode=1:parity=tff,hwdownload,format=nv12,scale=640:480 -c:v h264_nvenc -b:v 4M -an output.mp4