quicksync via ffmpeg

[vhelp]

So here's my story, its sort of a two-part question: for quicksync support in ffmpeg and how to increase the encoding speed through ffmpeg.

I don't use ffmpeg for quicksync. But after searching for how to encode videos with audio in one sweep through QSEnvc through what I thought was with --avqsv and failed, I started searching around for other alternatives. Thus, ffmpeg came to mind and I searched around for how to do it, and it should be pretty simple, like every other audio/video processing. However, I can't seem to find an ffmpeg build that includes quicksync support. Even though it is enabled { --enable-libmfx } in every build i've tested so far, but they don't work! I don't get it. I mean, whats the point of compiling it into ffmpeg if its not gonna work? Are anyone using quicksync via ffmpeg ? And no one is reporting its failure ? I don't know. Anyway.

The last build I could find, after searching around again, was ffmpeg 4.9.2 (N-72805-g913685f the last date is 2015-06-09) but it is encoding video very slowly at about 38 fps versus QSEnc at 250 fps, both are using the same avs script. I thought that the reason for the slowness was maybe a bug in that build, so I began searching yet again, but so far I have not been able to locate any more working version of ffmpeg that supports quicksync, even though all the builds have support, including today's that I just downloaded, include the { --enabled-libmfx } support.

Does anyone have any ideas ?

Here is the last generated output I just made on an encode:

usage: { -c:v h264_qsv -q:v 1 }

ffmpeg version N-72805-g913685f Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 41.100 / 56. 41.100
  libavformat    56. 36.100 / 56. 36.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 16.101 /  5. 16.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avisynth, from 'movie.avs':
  Duration: 00:00:10.68, start: 0.000000, bitrate: 3 kb/s
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 720x480, 23.98 fps, 23.98 tbr, 23.98 tbn, 23.98 tbc
    Stream #0:1: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
Output #0, matroska, to 'c:\dst\video.20170211.5thelement.vhs.pv4662.950q.dell.inspire.sandybridge.i3-2370M_QSV2.62_api-1.4_[u1.cqp.01.12.12_0720x0480].0558.ffmpeg.h264.movie22.mkv':
  Metadata:
    encoder         : Lavf56.36.100
    Stream #0:0: Video: h264 (h264_qsv) (H264 / 0x34363248), nv12, 720x480, q=2-31, 8 kb/s, 29.97 fps, 1k tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc56.41.100 h264_qsv
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
frame=   19 fps=0.0 q=0.0 size=    2423kB time=00:00:00.03 bitrate=601431.3kbits/s    
frame=   36 fps= 34 q=0.0 size=    4765kB time=00:00:00.03 bitrate=1182815.3kbits/s    
frame=   55 fps= 35 q=0.0 size=    7226kB time=00:00:00.03 bitrate=1793794.7kbits/s    
frame=   73 fps= 35 q=0.0 size=    9575kB time=00:00:00.03 bitrate=2307017.9kbits/s    
frame=   92 fps= 36 q=0.0 size=   12034kB time=00:00:00.03 bitrate=2899366.1kbits/s    
frame=  111 fps= 35 q=0.0 size=   14522kB time=00:00:00.03 bitrate=3499027.8kbits/s    
frame=  131 fps= 36 q=0.0 size=   17229kB time=00:00:00.03 bitrate=4276953.2kbits/s    
frame=  148 fps= 35 q=0.0 size=   19606kB time=00:00:00.03 bitrate=4867082.2kbits/s    
frame=  169 fps= 36 q=0.0 size=   22524kB time=00:00:00.03 bitrate=5591298.7kbits/s    
frame=  185 fps= 35 q=0.0 size=   24737kB time=00:00:00.03 bitrate=5960255.3kbits/s    
frame=  203 fps= 35 q=0.0 size=   27199kB time=00:00:00.03 bitrate=6752041.5kbits/s    
frame=  221 fps= 35 q=0.0 size=   29833kB time=00:00:00.03 bitrate=7405886.5kbits/s    
frame=  235 fps= 35 q=0.0 size=   32210kB time=00:00:00.03 bitrate=7995915.6kbits/s    
frame=  252 fps= 34 q=0.0 size=   34943kB time=00:00:00.03 bitrate=8674415.3kbits/s    
frame=  256 fps= 34 q=0.0 Lsize=   36095kB time=00:00:00.03 bitrate=8960266.7kbits/s    
video:36092kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.008401%

[poisondeathray]

Works ok in ffmpeg with zeranoe's windows ffmpeg builds here with proper speeds . I tested avs input directly wiht a 32bit build 2017.01.12

What is your avs script ? Any slow filters other bottlenecks ? According to your log it's YUY2 input , and you're encoding YV12 so maybe some additional unoptimized conversion routines going on in the ffmpeg version ? Maybe try converting in the script

But after searching for how to encode videos with audio in one sweep through QSEnvc through what I thought was with --avqsv and failed,

For qsvenc, --avqsv is for HW decode /encode . So you can't use avs input with it. Direct input with valid accelerated formats will work for video/audio transcode using --avqsv

[vhelp]

--avqsv is unrecognized.

i don't use any filtering in avs script. these sources are from UT codec captures of vhs content. and i don't use any filtering for vhs content.

i'm confused. if i understood you correctly, you are saying that i should not be using avs script ?

i need to ivtc these sources. i have a template where i make changes to the start and end points, and i make constant changes to these when i want to make various test encodes from different positions in the video.

here is the script that i've been using in regular encodes successfully. snipped most unnecessary junk from original as it was very length from my edits and things.
(depending on the clip, i will REMark in or out what i need to process with)

# vhs capture
    
  v = "g:\caps\video.2017-02-11-sat-1004pm.pv4662.950q.D-R4.T5E.avi" 
# v = "c:\cap\cap.PAN.PV-4662.to.HVR950Q.5thelement.02.D-R4.02.avi"
  
  LoadPlugin(  "i:\plugins\TIVTC.dll") # usage; tfm(order=-1).tdecimate()
  loadplugin(  "i:\plugins\ColorMatrix.dll")
  
  AVIsource(v) 
  
# LoadPlugin( "i:\PLUGINS\LSMASHSource.dll" ) 
# LWLibavVideoSource(v, cache=true , seek_mode=0 )
  
# LoadPlugin( "i:\PLUGINS\ffms2.32b.dll" )
# FFVideoSource(v, fpsnum=24000, fpsden=1001) 
# FFVideoSource(v, fpsnum=23976, fpsden=1000) 
# FFVideoSource(v, fpsnum=29970, fpsden=1000) 
# FFVideoSource(v, fpsnum=50000, fpsden=1000) 
  
  start  =  00018 # 00019 # 23639 # 00868 # 00002 # 8087# 00282 # 00032 # 14636 # 08869 # 16894 # 06599 # 19441 # 00000  # 00000  # 42934 # 1404 # 00000
  end    =  23481 # 29210 # 46977 # 13282 # 56968 # 8657# 01015 # 10086 # 72240 # 13823 # 27953 # 23095 # 02029 # 03419  # 45203  # 06513 # 2356 
  length = 099999
  
  a=trim( start, end ) 
  last=a
  
  separateFields( ).selectevery(2, 1,0).weave # i need this for analog captures 
# trim( 00004,0000).separateFields().selectevery( 10, 0,1,2,3, 4,5, 6,9).weave( ).trim( 0000, 0)  
# trim( 00000,0000).tfm(order=-1).tdecimate() 
  trim( 00000,0000).separateFields().selectevery( 05, 0,1,2,3).weave( ).trim(0,0).trim( 0000,100) 
  
# ConvertToYUY2( interlaced=false ) 
# ConvertToYV12( interlaced=false ) 
  trim( 0,0 )            # optional--because some clips won't open w/out this hack. i don't know why.

[vhelp]

ok, i did go to zeranoe's and downloaded today. it does not work for me. maybe it is a processor issue.

https://ffmpeg.zeranoe.com/builds/ --> and downloaded this version https://ffmpeg.zeranoe.com/builds/win32/static/ffmpeg-20170208-3aae1ef-win32-static.zip

but the one in my first post works with avs scripts, its just very slow. here's the parameters for that encode from that old build, and encodes successfully, just slow:

ffmpeg -i "movie.avs" -c:a aac -strict experimental -c:v h264_qsv -q:v 1 -y "movie.mkv"

[pandy]

ffmpeg.exe -f lavfi -i testsrc2=s=uhd2160:r=50 -pix_fmt nv12 -flags +cgop -c:v hevc_qsv -preset:v 7 -profile:v 1 -load_plugin 2 -level:v 5.1 -b:v 38M -maxrate:v 40M-1 -minrate:v 1M -bufsize:v 19M-1 -qmin:v 4 -qmax:v 69 -g:v 250 -bf:v 3 -refs:v 5 -bitrate_limit 1 -async_depth 4 -vf "scale=s=uhd2160,pad=3840:2160:(ow-iw)/2:(oh-ih)/2,fps=fps=50,setpts=N/(50*TB)" -c:a copy -f mpegts - | ffplay.exe -analyzeduration 512Ki -probesize 512Ki -i - -vf "scale=iw/4:-2:sws_flags=lanczos"

Seem you missing load_plugin, above worked for me fine (but currently have no HW and SW to verify this claim).

Bellow something you may find useful:

ffmpeg -? encoder=h264_qsv 
--------------- 
Encoder h264_qsv [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration)]:
    General capabilities: delay 
    Threading capabilities: none
    Supported pixel formats: nv12 qsv
h264_qsv encoder AVOptions:
  -async_depth       <int>        E..V.... Maximum processing parallelism (from 0 to INT_MAX) (default 4)
  -avbr_accuracy     <int>        E..V.... Accuracy of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -avbr_convergence  <int>        E..V.... Convergence of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -preset            <int>        E..V.... (from 1 to 7) (default medium)
     veryfast                     E..V....
     faster                       E..V....
     fast                         E..V....
     medium                       E..V....
     slow                         E..V....
     slower                       E..V....
     veryslow                     E..V....
  -vcm               <int>        E..V.... Use the video conferencing mode ratecontrol (from 0 to 1) (default 0)
  -rdo               <int>        E..V.... Enable rate distortion optimization (from -1 to 1) (default -1)
  -max_frame_size    <int>        E..V.... Maximum encoded frame size in bytes (from -1 to 65535) (default -1)
  -max_slice_size    <int>        E..V.... Maximum encoded slice size in bytes (from -1 to 65535) (default -1)
  -bitrate_limit     <int>        E..V.... Toggle bitrate limitations (from -1 to 1) (default -1)
  -mbbrc             <int>        E..V.... MB level bitrate control (from -1 to 1) (default -1)
  -extbrc            <int>        E..V.... Extended bitrate control (from -1 to 1) (default -1)
  -adaptive_i        <int>        E..V.... Adaptive I-frame placement (from -1 to 1) (default -1)
  -adaptive_b        <int>        E..V.... Adaptive B-frame placement (from -1 to 1) (default -1)
  -b_strategy        <int>        E..V.... Strategy to choose between I/P/B-frames (from -1 to 1) (default -1)
  -cavlc             <int>        E..V.... Enable CAVLC (from 0 to 1) (default 0)
  -idr_interval      <int>        E..V.... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
  -pic_timing_sei    <int>        E..V.... Insert picture timing SEI with pic_struct_syntax element (from 0 to 1) (default 1)
  -single_sei_nal_unit <int>        E..V.... Put all the SEI messages into one NALU (from -1 to 1) (default -1)
  -max_dec_frame_buffering <int>        E..V.... Maximum number of frames buffered in the DPB (from 0 to 65535) (default 0)
  -look_ahead        <int>        E..V.... Use VBR algorithm with look ahead (from 0 to 1) (default 1)
  -look_ahead_depth  <int>        E..V.... Depth of look ahead in number frames (from 0 to 100) (default 0)
  -look_ahead_downsampling <int>        E..V.... (from 0 to 2) (default unknown)
     unknown                      E..V....
     off                          E..V....
     2x                           E..V....
  -int_ref_type      <int>        E..V.... Intra refresh type (from -1 to 65535) (default -1)
     none                         E..V....
     vertical                     E..V....
  -int_ref_cycle_size <int>        E..V.... Number of frames in the intra refresh cycle (from -1 to 65535) (default -1)
  -int_ref_qp_delta  <int>        E..V.... QP difference for the refresh MBs (from -32768 to 32767) (default -32768)
  -recovery_point_sei <int>        E..V.... Insert recovery point SEI messages (from -1 to 1) (default -1)
  -trellis           <flags>      E..V.... Trellis quantization (default 0)
     off                          E..V....
     I                            E..V....
     P                            E..V....
     B                            E..V....
  -profile           <int>        E..V.... (from 0 to INT_MAX) (default unknown)
     unknown                      E..V....
     baseline                     E..V....
     main                         E..V....
     high                         E..V....
  -a53cc             <int>        E..V.... Use A53 Closed Captions (if available) (from 0 to 1) (default 0)

--------------- 
ffmpeg -? encoder=hevc_qsv 
--------------- 
Encoder hevc_qsv [HEVC (Intel Quick Sync Video acceleration)]:
    General capabilities: delay 
    Threading capabilities: none
    Supported pixel formats: nv12 qsv
hevc_qsv encoder AVOptions:
  -async_depth       <int>        E..V.... Maximum processing parallelism (from 0 to INT_MAX) (default 4)
  -avbr_accuracy     <int>        E..V.... Accuracy of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -avbr_convergence  <int>        E..V.... Convergence of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -preset            <int>        E..V.... (from 1 to 7) (default medium)
     veryfast                     E..V....
     faster                       E..V....
     fast                         E..V....
     medium                       E..V....
     slow                         E..V....
     slower                       E..V....
     veryslow                     E..V....
  -vcm               <int>        E..V.... Use the video conferencing mode ratecontrol (from 0 to 1) (default 0)
  -rdo               <int>        E..V.... Enable rate distortion optimization (from -1 to 1) (default -1)
  -max_frame_size    <int>        E..V.... Maximum encoded frame size in bytes (from -1 to 65535) (default -1)
  -max_slice_size    <int>        E..V.... Maximum encoded slice size in bytes (from -1 to 65535) (default -1)
  -bitrate_limit     <int>        E..V.... Toggle bitrate limitations (from -1 to 1) (default -1)
  -mbbrc             <int>        E..V.... MB level bitrate control (from -1 to 1) (default -1)
  -extbrc            <int>        E..V.... Extended bitrate control (from -1 to 1) (default -1)
  -adaptive_i        <int>        E..V.... Adaptive I-frame placement (from -1 to 1) (default -1)
  -adaptive_b        <int>        E..V.... Adaptive B-frame placement (from -1 to 1) (default -1)
  -b_strategy        <int>        E..V.... Strategy to choose between I/P/B-frames (from -1 to 1) (default -1)
  -cavlc             <int>        E..V.... Enable CAVLC (from 0 to 1) (default 0)
  -load_plugin       <int>        E..V.... A user plugin to load in an internal session (from 0 to 2) (default hevc_sw)
     none                         E..V....
     hevc_sw                      E..V....
     hevc_hw                      E..V....
  -load_plugins      <string>     E..V.... A :-separate list of hexadecimal plugin UIDs to load in an internal session (default "")
  -profile           <int>        E..V.... (from 0 to INT_MAX) (default unknown)
     unknown                      E..V....
     main                         E..V....
     main10                       E..V....
     mainsp                       E..V....

Ok, seem load_plugin is only for hevc - sorry - didn't tested h264_qsv - it works fine for hevc.

[vhelp]

that is so coincidental. i just found that while googling just now.. { C:\tools\ffmpeg\ffmpeg.20170208 -h encoder=h264_qsv }

[vhelp]

but i'm encoding to h264 since my laptop processor is <sandybridge>

but at the moment, i am not satisfied with hevc video. i find that hardware based encoders are flawed in the quality dept. iow, i'm very disappointed in it.

anyway. i am more than satisfied with h264 via hardware encoder on this laptop and would like to get audio included in it. since qsvenc does not support video+audio in one line, i am trying to get ffmpeg working. but i want to get it working as fast as qsvenc. but i can't get the latest ffmpeg to work on my laptop with the script i am using. it needs to work with avs scripts.

[vhelp]

this code (below) works (as seen in post # 1) in an old build from 2015, but it will not work in the latest ffmpeg builds:

ffmpeg -i "movie.avs" -c:a aac -strict experimental -c:v h264_qsv -q:v 1 -y "movie.mkv"

[raffriff42]

Is QuickSync working at all for you? I keep mine disabled, so can't test.

https://www.bandicam.com/support/tips/intel-quick-sync/

To use Intel® Quick Sync Video encoder, you have to activate the Intel Quick Sync Video by
connecting a video cable to the monitor(s) or adding a display device...

https://mirillis.com/en/products/tutorials/action-tutorial-intel-quick-sync-setup_for_desktops.html

Setup Intel Quick Sync Video on Desktop PC with PCI-E graphics card

This tutorial shows how to setup Intel® Quick Sync Video on desktop PC without need
to connect a monitor to Intel HD graphics card.

The tutorial is designed for all desktop computers with Intel® processors supporting
Intel® Quick Sync Video technology and with additional NVIDIA or AMD PCI-E graphics card installed...

[vhelp]

hi raff, yes. i use it every day, pretty much. but i use it through the qsvencC.exe cli.

but qsvenc does not support video+audio in one line. after encode to video, you have re-mux the audio portion with the video to get a new video/audio file. thats just to many steps.

but i can simplify it by using ffmpeg:

ffmpeg -i "movie.avs" -c:a aac -strict experimental -c:v h264_qsv -q:v 1 -y "movie.mkv"

but the above is from an old ffmpeg build from 2015, see post # 1 for an output screenshot of a successful encode through ffmpeg+qsv, but the encoding process is very slow, 38 fps w/ ffmpeg while 250 fps when using the qsvenc cli app.

theory:

poisondeathray said that it works for him, but he is probably using an newer generation intel chipset (prob Haswel), while i am using a laptop cpu is <sandybridge> quicksync media sdk v1.4 version. it only supports h264 encoding, no hevc. i am suspecting that ffmpeg changed their code, this time to limit search for only Intel 4th generation or higher cpu's. so anything below that (my sandybridge for instance) will just throw errors, like i'm getting now.

note: the only way to initiate hardware quicksync encoder through ffmpeg is by using the following parameter { -c:v h264_qsv }

still, i hope my theory is wrong.

[poisondeathray]

--avqsv is for rigaya's qsvencc , not ffmpeg . It's full HW decode+encode . It cannot read avs

IIRC , I read somewhere that newer ffmpeg dropped older GPU support for QS , that might explain why only older ffmpeg builds work for you and why it's slower

[vhelp]

bummer. but thanks.

i wonder how OBS does it. they can screen capture+encode both video/audio at the same time. there must be another way to do this. maybe through some awkward piping.... encode video+ffmpeg to mux audio into video all in one pass. i don't know. i'm just not gonna sit here and capture video, edit, encode, mux audio into video, delete old file, etc.

[pandy]

Definitely ffmpeg is active project in a good and bad way - saw many times regressions in new builds - accordingly to Intel ffmpeg pdf's command line is simple and straightforward so i would expect or problem with QSV libs itself in your OS or with ffmpeg - definitely i would not give up with QSV but just try or private ffmpeg build or different builds from zeranoe for example.

[vhelp]

--avqsv is for rigaya's qsvencc , not ffmpeg . It's full HW decode+encode . It cannot read avs

muxing audio into video via ffmpeg is getting tiresome.

@ poisondeathray, do you know how i can use the { --avqsv } parameter to encode video+audio in one shot through qsvenc ?

I realize qs is slower than nvidia and amd but at this point, i don't really care, plus i'm comfortable w/ the quality i get from qs via lossless mode. I just want a video+audio file.

Thanks if you know the answer!

[poisondeathray]

--avqsv is for rigaya's qsvencc , not ffmpeg . It's full HW decode+encode . It cannot read avs

muxing audio into video via ffmpeg is getting tiresome.

@ poisondeathray, do you know how i can use the { --avqsv } parameter to encode video+audio in one shot through qsvenc ?

I realize qs is slower than nvidia and amd but at this point, i don't really care, plus i'm comfortable w/ the quality i get from qs via lossless mode. I just want a video+audio file.

Thanks if you know the answer!

Yes , it has to be a supported format by qsv natively . So you can't use .avs

e.g. for stream copy audio, use --audio-copy

"QSVEncC" --avqsv -i "INPUT.mp4" --audio-copy -o "OUTPUT_audiocopy.mp4"

eg. for re-encoding with AAC , enter aac and the bitrate in kbps

"QSVEncC" --avqsv -i "INPUT.mp4" --audio-codec aac --audio-bitrate 256 -o "OUTPUT_256kbpsAAC.mp4"

[vhelp]

success

@ pois, thanks for the help, though your method only encodes audio. I was looking to encode video+audio in one shot through qsvenc

However, after some more time spent through researching and trial/error, i've made up with success. Now is possible to Quicksync encode video+audio in one step with the use of:

{ -audio-source 'audio_file' } and { --audio-copy }

QSVEncC  --audio-source "audio.mp4" --audio-copy  -i "movie.avs" --codec h264 -u 1 --cqp 0 -o "video-and-audio.mp4"

Rather than encode+convert_audio, the above was fastest when using --audio-copy, provided you already encoded the audio through another means, ie ffmpeg.

Its very late for me. Tomorrow i will try and find a "best solution" for encoding audio, one that is fastest and highest quality (not lossless, nor wav). Then use the above params in quicksync.

Finally, I can encode video+audio in one step through quicksync via qsvenc.

Thanks for the nudge that helped to push me in this find

[poisondeathray]

success

@ pois, thanks for the help, though your method only encodes audio. I was looking to encode video+audio in one shot through qsvenc

Yes, it encodes audio and video in 1 step (or encodes video and stream copies audio in 1 step)

For the 3rd time, the video input CANNOT be .avs when using --avqsv. It tells you that in the instructions. The avqsv reader only supports quicksync supported formats. It has to be a format natively supported by quicksync such as AVC, MPEG2 etc... . Again, this works in 1 step