VideoHelp Forum

+ Reply to Thread
Results 1 to 17 of 17
Thread
  1. Hello,

    I would like to share with you some feedbacks regarding some tests I made today.

    My first goal was to compress BD videos in a "decent" quality and the most important in an "acceptable" size. Decent and acceptable are of course subjective, and the following is based only on my criterias.
    Most important, I fixed arbitrarily 4000KB/s as bitrate of my target videos, (of course all are 1080p), it means for example moreless 3Gb for a 1h30 movie (including sound tracks).
    So I ripped one of my BD (The Hunger games - catching fire), in passthrough mode, then extracted approximately 6 mins of the video (with fast motion) : 8349 frames for a exact duration of 5mins48sec22.
    Then I converted this little video in AVC and HEVC, with different tools and finally compare to the reference, using ffmpeg :

    ffmpeg -i "compressed file" -i "reference file" -lavfi "ssim;[0:v][1:v]psnr" -f null

    My computer : core I7 + video card nvidia GTX1060 + SSD disk

    Here are the results (average PSNR and SSIM) :

    1) DVDFAB 10

    - AVC 1-pass cpu encoding : PSNR 25.86 / SSIM 0.910
    - AVC 1-pass cuda decode / cpu encoding : PSNR 25.86 / SSIM 0.910
    - AVC 1-pass nvenc : PSNR 25.85 / SSIM 0.910
    - AVC 2-pass cpu : PSNR 25.86 / SSIM 0.910
    - HEVC 1-pass nvenc : PSNR 46.05 / SSIM 0.985
    - HEVC 2-pass cpu : PSNR 41.11 / SSIM 0.985

    2) Hybrid 2018.02.18.1

    - HEVC Preset VBR_HQ / 5 Ref frames NvencC : PSNR 48.82 / SSIM 0.986

    3) Staxrip 1.7.0.0

    - HEVC Preset Quality 2-pass nvenc : PSNR 46.85 / SSIM 0.986
    - HEVC Preset VBR_HQ 1-pass nvenc : PSNR 46.85 / SSIM 0.986

    4) VidCoder 3.8 Bete x64 :

    - AVC 2-pass / cpu / preset slow : PSNR 17.69 / SSIM 0.788
    - AVC 2-pass / cpu / preset medium : PSNR 17.69 / SSIM 0.789
    - AVC 2-pass / cpu / preset fast : PSNR 17.69 / SSIM 0.790
    - AVC 2-pass / cpu / preset faster : PSNR 17.69 / SSIM 0.791
    - AVC 2-pass / cpu / preset very fast : PSNR 17.69 / SSIM 0.791
    - AVC 2-pass / cpu / preset superfast : PSNR 17.69 / SSIM 0.791

    5) MediaCoder x64

    - HEVC 2-pass nvenc : PSNR 25.86 / SSIM 0.910

    With all of that, I understand that :

    At a relative low birate (4000 kB/s), HEVC has better results than AVC (PSNR / SSIM), I think that the difference between AVC & HEVC decrease when bitrate increase.

    But I can't understand why :

    - NVENC seems to give better results than cpu conversion in HEVC (strange )
    - Vidcoder presets seems not to be working (see PSNR / SSIM) (slow identical than veryfast ???)

    Am I missing something ?, I'm not a "video Guru", and what I understand is that it's better to compress in HEVC/NVENC at 4000 kB/s than AVC/Cpu ???

    Thanks for your comments and advices, have fun
    Quote Quote  
  2. 1) check your actual bitrates or filesizes . Entering 4000kb/s does not mean the encoder actually achieves it. Some might undershoot or overshoot by a large margin
    2) check your encoding settings, or post your encoding settings. There can be significant differences between the fastest and slowest encoding settings when looking at metrics.
    3) try x265 with staxrip or hybrid . I think dvdfab uses different hevc implementation for cpu
    4) check that your frames are aligned .

    There are inconsistencies in your results, that suggest testing or implementation problems
    a) why mediacoder hevc nvenc results so different than staxrip or dvdfab ? You would expect nvenc to get similiar results if you used the same settings (did you use the same settings ?) . Large deviations like that (PSNR is on a log scale) , suggest other problems like non aligned frames (mismatched comparison, wrong frames) , such as decoding error, encoding errors
    b) The vidcoder results suggest other problems - You should not get identical values for all presets


    4Mb/s for 1080p is not a lot of bitrate, you would expect hevc to perform better than avc (any variety of avc, even x265). But you would expect x265 to perform better than nvenc hevc, so I would look closer at the results and testing methodology

    Beware that PSNR and SSIM are objective measures, but they have limited value when looking at "quality". There is quite a deviation from human perception (the correlation coefficient is low) . Something that has a higher PSNR or SSIM score might look lower quality to the human eye. Also they are easy to "trick" with various cheats. But they are useful for tracking trends , determining lossless (or not), and mismatched frames . The point is do not rely on them alone, look at other things as well
    Quote Quote  
  3. You're right, something's wrong with my params on VidCoder as I made some x264 conversion tests with the same video source, but using staxrip :

    The results are very close to HEVC , and going decreasing from medium to ultrafast. Ultrafast is clearly below in terms of quality.

    staxrip
    params : ffmpeg - x264 / Video bitrate : 4000
    Decoder AviSynth/VapourSynth/Codec x264/Mode two-passMode/Preset : see below/Tune : none

    Medium
    frame= 8349 fps=165 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=6.87x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 000002854efdff60] SSIM Y:0.984256 (18.028917) U:0.991660 (20.788447) V:0.991650 (20.783000) All:0.986723 (18.768843)
    [Parsed_psnr_1 @ 000002854efe0ba0] PSNR y:45.920822 u:50.262068 v:50.274143 average:46.949112 min:41.865453 max:55.046981

    Fast
    frame= 8349 fps=184 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=7.66x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 0000021cf4a37400] SSIM Y:0.982606 (17.596001) U:0.991005 (20.459966) V:0.991129 (20.520204) All:0.985426 (18.364308)
    [Parsed_psnr_1 @ 0000021cf4a37c20] PSNR y:45.297783 u:50.005923 v:50.076992 average:46.385262 min:41.170569 max:54.204602

    Faster
    frame= 8349 fps=188 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=7.83x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 000002c77a19dc60] SSIM Y:0.983651 (17.865057) U:0.990503 (20.224268) V:0.990605 (20.271015) All:0.985952 (18.523853)
    [Parsed_psnr_1 @ 000002c77a19d360] PSNR y:45.464179 u:49.651846 v:49.711940 average:46.472085 min:41.016494 max:53.454734

    Veryfast
    frame= 8349 fps=185 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=7.72x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 0000013e4228dac0] SSIM Y:0.983172 (17.739747) U:0.990644 (20.289265) V:0.990739 (20.333535) All:0.985679 (18.440211)
    [Parsed_psnr_1 @ 0000013e4228d840] PSNR y:45.115843 u:49.583852 v:49.641313 average:46.166984 min:40.952628 max:53.697213

    Superfast
    frame= 8349 fps=189 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=7.86x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 000001d8532dd500] SSIM Y:0.981904 (17.424233) U:0.989996 (19.998372) V:0.990133 (20.058030) All:0.984624 (18.131667)
    [Parsed_psnr_1 @ 000001d8532dde00] PSNR y:44.542789 u:49.212233 v:49.291750 average:45.625294 min:38.689497 max:53.985116

    Ultrafast :
    frame= 8349 fps=200 q=-0.0 Lq=-0.0 size=N/A time=00:05:48.22 bitrate=N/A speed=8.34x
    video:8610kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    [Parsed_ssim_0 @ 0000020dce79d500] SSIM Y:0.970589 (15.314898) U:0.988611 (19.434986) V:0.988319 (19.325367) All:0.976548 (16.298143)
    [Parsed_psnr_1 @ 0000020dce79dd20] PSNR y:42.394233 u:48.611007 v:48.532806 average:43.660776 min:37.020178 max:52.256512

    I'll retry tomorrow with VidCoder/Handbrake checking the params . Thanks & regards
    Quote Quote  
  4. Hi,

    Didn't want to open a new thread so I'd woken this up.
    Please have a look at the attached print-screen and tell me what's wrong every ~24 frames.
    Image Attached Thumbnails Click image for larger version

Name:	videohelp.jpg
Views:	27
Size:	1.18 MB
ID:	62882  

    Quote Quote  
  5. Originally Posted by neveryman View Post
    Hi,

    Didn't want to open a new thread so I'd woken this up.
    Please have a look at the attached print-screen and tell me what's wrong every ~24 frames.


    A BD typically has keyframes every second. A 23.976p BD every 24 frames.

    A lossy re-encode typically has max keyframe intervals much larger, for compression purposes

    Keyframes generally are higher in quality other frametypes, and require more bitrate - hence the dip in quality on the re-encoded versions . So on most of those dips, you're actually comparing a high quality , high bitrate keyframe, to a lower quality, low bitrate other frame (b,p)

    If you set max keyframes to every second to match the BD, you'd need higher bitrate to achieve a simliar level of "quality" that you have now. ie. you'd need a larger filesize for roughly similar visual quality overall
    Quote Quote  
  6. It may sound ridiculous but I guess the "original" was encoded in a fancy way I can't figure out. In the picture I put the encoding parameters side by side. Is there anything able to create the situation you see in the previous picture?

    Thank you.

    Image
    [Attachment 62888 - Click to enlarge]
    Last edited by neveryman; 12th Jan 2022 at 18:22. Reason: cosmetic
    Quote Quote  
  7. Its the same explanation as the previous post. Nothing fancy.

    The "original" has keyint 24 . The re-encode has keyint 240 .

    The max GOP size of the "original" is 24, the max GOP size of re-encode is 240

    --keyint is the maximum keyframe interval. The encoder will be forced to place an new I frame every 24 frames (or earlier if there is a scene change detected, if scenecut is enabled)

    Because the re-encode places lower quality p and b frames with higher quantizers at most those 24 frame positions relative to the source, you see dips in the quality measurement at those positions as seen in the metrics (Whether or not you actually "see" the dips in normal viewing is another discussion)


    (The "original" is not the actual original BD either - it's not using BD compliant settings, and uses a low bitrate and x264)
    Quote Quote  
  8. Reencoded with ... / keyint=24 / keyint_min=13 / ... quality check graph looks absolutely the same.
    Besides, I'm using Handbrake SW/HW encoders for other conversion, with other sources, some very high quality, some very poor quality. I never encountered such an issue.

    (the resulted video looks very fine, I'm just curios what can generate those dips.)
    Quote Quote  
  9. Those are fairly large dips; do the PNSR ,SSIM graphs dip the same magnitude ?

    If so, it might be error in measurement method. e.g. slightly off timestamps , jitter in mkv container in the re-encodes, causing different frames to be compared

    You can index them using frame accurate method e.g. avisynth, and measure those to rule out if this is an issue
    Quote Quote  
    1. Took an ISO
    2. Identified the main title
    3. extracted the video stream with Avidemux, MakeMKV, Mkvmerge, FFmpeg
    4. MakeMKV and Mkvmerge output files almost identical. Avidemux, smaller, FFmpeg larger
    5. MakeMKV, Mkvmerge and FFMPEG had the same PSNR etc.
    6. I guess it is a problem with the filler data
    7. I also guess that FFmetrics (using FFmpeg's PSNR) do/es a lousy job in evaluating metrics "objectively"
    Is there a better way to do this evaluation?

    LE: funny thing. Encoding with Handbrake those 4 different files, same settings, of course, the re-encoded files are very similar. And absolutely the same quality parameters. Which means the useful data in the stream is correctly taken out regardless of the ripping method. Just the 'associated" garbage is different each time.

    Image
    [Attachment 62949 - Click to enlarge]
    Last edited by neveryman; 17th Jan 2022 at 17:52. Reason: Later test
    Quote Quote  
  10. You would expect remuxing to give the same result

    You showed the VMAF chart, but how low does the PSNR (db) go on those dips ? If the delta is very large, it suggests an error in measurement or encoding

    You can try lossless encoding, if it's not lossless, then it could be the timestamps issue (comparing different frames) - ie. whatever procedure you're using to either encode or decode or analyze is flawed (wrong frame), then you have to examine it more closely at each stage

    BTW , FFmpeg supports libvmaf as well . FFmpeg metrics are "objective" - ie. repeatable if the timestamps are ok, and the container timebase differences are corrected for. Otherwise you get unexpected results (wrong timestamps, wrong frames compared)
    Quote Quote  
  11. For encoders comparison I use MSU VQMT. Lot of metrics, free for personal use, but has limitation - resolution must be below 720p ( 1264x712 works fine). It allows comparison of 2 encodes vs original on the same plot. The csv-data also avail, including min, max, average.
    To automate things I've composed a handy batch file that generates different encodes along with .avs scripts for each of them. So I just drop test clip on the shortcut, and it does the rest. Finally I load avs-scripts into the tool for analysis.

    To get you the idea of the batch-script:
    Code:
    @echo off
    TITLE %~nx0
    SETLOCAL ENABLEEXTENSIONS
    set t0=%TIME%, %DATE%
    cd ..\..
    call apps\set_idle.bat
    
    for %%F in (%*) do call :main %%F
    goto finalmessage
    
    :main
     SetLocal
      SET "newdir=g:\_encoders_test"
      if not exist "%newdir%" mkdir "%newdir%"
    
    :: NVEnc h264
      SET "encparams=-c h264 --cqp 12:14:17 --preset quality --aq-strength 10 --gop-len 12 --ref 1 --bframes 0 --lookahead 12 --qp-max 25 --aq --cabac --mv-precision q-pel --audio-codec aac --audio-bitrate 384 --avsync cfr"
      if /i "%~x1"==".avs" (SET "cm=") ELSE (SET "cm=--colorprim auto --transfer auto --colormatrix auto --colorrange auto")
      apps\nvencc\NVEncC64.exe -i "%~f1" -o "%newdir%\[nvenc].mp4" %encparams% %cm%
      echo LSMASHVideosource("%newdir%\[nvenc].mp4").Convertbits(8).ConvertToYV12() > "%newdir%\[nvenc].avs"
    
    :: x264
      apps\ffmpeg\ffmpeg.exe -hide_banner -y -i "%~f1" -pix_fmt yuv420p -c:v libx264 -crf 10.5 -preset veryfast -x264-params ref=1:bframes=0:keyint=12:min-keyint=1:qcomp=0.7:no-dct-decimate=1 -an "%newdir%\[x264].mov"
      echo LSMASHVideosource("%newdir%\[x264].mov").Convertbits(8).ConvertToYV12() > "%newdir%\[x264].avs"
    
    
    
    
     EndLocal
     goto :eof
    :finalmessage
    powershell write-host -fore cyan  ====================== Processing is FINISHED =======================
    echo ----------------------------
    echo Batch processing start time: %t0%
    echo Batch processing end time:   %TIME%, %DATE%
    echo ---------------------------- 
    pause
    Image Attached Thumbnails Click image for larger version

Name:	msu vqmt plot.jpg
Views:	8
Size:	128.1 KB
ID:	62950  

    Quote Quote  
  12. That last ssim plot looks "normal" to me . The peaks are I frames
    Quote Quote  
  13. Originally Posted by poisondeathray View Post
    You would expect remuxing to give the same result
    ...
    You can try lossless encoding, ... FFmpeg metrics are "objective"
    Well, in this very moment I just want the extracted mkv to have PSNR inf when compared to original m2ts.
    FFmpeg can manage to remove the filler data or whatever it is but cannot manage the PSNR measurement.

    Originally Posted by buzz1891 View Post
    For encoders comparison I use MSU VQMT. Lot of metrics, free for personal use, but has limitation - resolution must be below 720p
    I only work with 1080 or 4K. I like the way MSU VQMT looks but nothing more.
    Last edited by neveryman; 17th Jan 2022 at 19:07. Reason: 2 answers
    Quote Quote  
  14. Originally Posted by neveryman View Post

    Well, in this very moment I just want the extracted mkv to have PSNR inf when compared to original m2ts.
    FFmpeg can manage to remove the filler data or whatever it is but cannot manage the PSNR measurement.
    So that's a good start for "ripping" or remuxing. You verified the 1st step that is ok.

    But why are you getting those dips when encoding in post #4 ? It says all have drops in PSNR, SSIM, VMAF . But you only showed the VMAF chart. What is the magnitude in the PSNR (db) drops ? If you have like 10-20db drops, there is something else going on.

    Take it step by step. If you encode lossless, but it's not lossless, then there is problem with either the encoding step, or the measurement step.

    Then you perform more tests to rule out other problems. eg If you use frame accurate indexer e.g . vqmt accepts avs scripts (measure frame by frame, or decode to .yuv will work for vqmt), that should rule out any timestamp issues.

    You verify with other tools. Cross check results with say ffmpeg psnr/ssim. or vapoursynth . Sometimes the algorithm is slightly different, but the trend should be the same. And "lossless" encode should be lossless in all cases (except vmaf)
    Quote Quote  
  15. Very good piece of advise: step by step.
    step 1. m2ts compared to the extracted mkv should show PSNR INFinite. not -1db not -0.01dB. If not, my FFmpeg based tool is simply not good. Can't move to step 2.
    So, right now, I'm looking for a tool that loads the video stream from m2ts, compares it with the stream from the extracted mkv and shows PSNR INFinite, SSIM 1.
    Quote Quote  
  16. Originally Posted by neveryman View Post
    Very good piece of advise: step by step.
    step 1. m2ts compared to the extracted mkv should show PSNR INFinite. not -1db not -0.01dB. If not, my FFmpeg based tool is simply not good. Can't move to step 2.
    So, right now, I'm looking for a tool that loads the video stream from m2ts, compares it with the stream from the extracted mkv and shows PSNR INFinite, SSIM 1.


    ffmpeg is ok , but you have to normalize the container timebase for mkv and m2ts . Almost everything in ffmpeg runs on timestamps, so if there is a slight rounding difference, the calculation will be off.

    (*The example in the current documentation is wrong, settb entries should be 1/AVTB (NOT AVTB)


    eg. for ffmpeg PSNR

    Code:
    ffmpeg -i remux.mkv -i original.m2ts -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

    If you want to print a per frame metrics in a text file, you can specify stat_file which then can be used in other software (statistics, graphing, etc...)

    eg. for ffmpeg PSNR with per frame metrics

    Code:
    ffmpeg -i remux.mkv -i original.m2ts -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr="stats_file=ffmpeg_psnr.log"" -f null -
    Quote Quote