VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Results 1 to 11 of 11
Thread
  1. Member
    Join Date
    Jul 2018
    Location
    Germany
    Search PM
    Hello there,

    I've been having issues converting a 720p/yuv420p video into its frames as a png image.
    While at first everything seems fine, a closer inspection reveals a lot of really weird artifacts in the produced images.
    The command I've been using is:

    ffmpeg -i "in2.mp4" -r "1.00" -vf scale=out_color_matrix=bt709 "frames\f_%05d.png"

    Examples:


    I have no clue what is causing this, and after 2 nights of researching this I can simply not find an answer.
    Note that using bmp has no impact on the issue. Neither has taking out "-vf scale=out_color_matrix=bt709".

    Any help would be greatly appreciated.

    Additional ffmpeg output/info:
    PS D:\ffmpeg\bin> ffmpeg -i "in2.mp4" -r "1.00" -vf scale=out_color_matrix=bt709 "frames\f_%05d.png"
    ffmpeg version N-91399-g9f0077cc03 Copyright (c) 2000-2018 the FFmpeg developers
    built with gcc 7.3.0 (GCC)
    configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
    libavutil 56. 18.102 / 56. 18.102
    libavcodec 58. 20.104 / 58. 20.104
    libavformat 58. 17.101 / 58. 17.101
    libavdevice 58. 4.101 / 58. 4.101
    libavfilter 7. 25.100 / 7. 25.100
    libswscale 5. 2.100 / 5. 2.100
    libswresample 3. 2.100 / 3. 2.100
    libpostproc 55. 2.100 / 55. 2.100
    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in2.mp4':
    Metadata:
    major_brand : isom
    minor_version : 512
    compatible_brands: isomiso2mp41
    Duration: 00:03:23.63, start: 0.000000, bitrate: 1047 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 868 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
    handler_name : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 173 kb/s (default)
    Metadata:
    handler_name : SoundHandler
    Stream mapping:
    Stream #0:0 -> #0:0 (h264 (native) -> png (native))
    Press [q] to stop, [?] for help
    Output #0, image2, to 'frames\f_%05d.png':
    Metadata:
    major_brand : isom
    minor_version : 512
    compatible_brands: isomiso2mp41
    encoder : Lavf58.17.101
    Stream #0:0(und): Video: png, rgb24, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 1 fps, 1 tbn, 1 tbc (default)
    Metadata:
    handler_name : VideoHandler
    encoder : Lavc58.20.104 png
    frame= 205 fps= 19 q=-0.0 Lsize=N/A time=00:03:25.00 bitrate=N/A dup=0 drop=5903 speed=19.2x
    video:29757kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    Quote Quote  
  2. This looks like the effects of YV12 to me.
    Quote Quote  
  3. Member
    Join Date
    Jul 2018
    Location
    Germany
    Search PM
    Originally Posted by amaipaipai View Post
    This looks like the effects of YV12 to me.
    Thank you for your answer, but as I am quite unexperienced with ffmpeg I can't really apply this information to my situation.
    If the above is the case, is there any way to circumvent this behavior?

    Edit:
    Trying out this command:

    ffmpeg -i "in2.mp4" -r "1.00" -vf scale=out_color_matrix=bt709 -pix_fmt rgb24 "frames\f_%05d.png"

    to bypass any YV12 conversion which might happen by default produces the same issues as in the original post.
    Last edited by Blackcero; 7th Jul 2018 at 00:01.
    Quote Quote  
  4. I don't use ffmpeg for my stuff, but wait some more, maybe some one could help you out with ffmpeg options.
    From what I can see your stream is yuv420p, maybe if you could use yuv422p instead, please notice that some devices might not read or understand yuv422p, high422 for ffmpeg.

    With avisynth I always use at the end of my script:
    Code:
    ConvertToRGB32(matrix="Rec709")
    But, maybe this could help:
    https://trac.ffmpeg.org/wiki/Encode/H.264
    https://video.stackexchange.com/questions/13164/encoding-422-in-10-bit-with-libx264
    Quote Quote  
  5. Member
    Join Date
    Jul 2018
    Location
    Germany
    Search PM
    Originally Posted by amaipaipai View Post
    I don't use ffmpeg for my stuff, but wait some more, maybe some one could help you out with ffmpeg options.
    From what I can see your stream is yuv420p, maybe if you could use yuv422p instead, please notice that some devices might not read or understand yuv422p, high422 for ffmpeg.

    With avisynth I always use at the end of my script:
    Code:
    ConvertToRGB32(matrix="Rec709")
    But, maybe this could help:
    https://trac.ffmpeg.org/wiki/Encode/H.264
    https://video.stackexchange.com/questions/13164/encoding-422-in-10-bit-with-libx264
    Thank you very much for your help!
    I have converted the video to yuv422p and extracted the frames from that without any issues.
    However, I have one last side question, if it's not too much, since I did experiment around in avisynth a little bit.
    My script was quite simple, same yuv420p origin video:

    LSMASHVideoSource(source ="D:\ffmpeg\bin\in2.mp4")
    ConvertToRGB(matrix="Rec709")
    ImageWriter(file = "D:\ffmpeg\bin\f_%05d.bmp", type="bmp")

    this produced quite a nice result at first glance yet again, but zooming in I saw this:


    It seems like at the right and left borders of the orange the color brightness somehow got reversed.
    Do you have any idea what could be causing this?
    Quote Quote  
  6. Please test if you observing same issue when those option are used by scale:
    Code:
    scale=iw:ih:sws_flags=spline+accurate_rnd+full_chroma_int+full_chroma_inp:out_color_matrix=bt709
    i've struggle with similar issues on ffmpeg from long time - and i have mixed results - ffmpeg has nasty habit to call automatically libswscale in background without telling anything - check report thoroughly. Also it is good to explicitly control quantization range in ffmpeg (separately for input and output) frequently prefer to manually rector limited quantization range. If absolute processing accuracy is mandatory for you then Avisynth behave way better than ffmpeg.
    Quote Quote  
  7. How are you taking the "original" screenshot ? What software or algorithm ?

    It's not really the "original"; you're looking at a RGB converted representation of the original . (You started with YUV 420) . It's not necessarily "correct" either.

    There is a difference in chroma upsampling method. ie. How ffmpeg or avisynth is converting YUV 420 to RGB is different than the method you used for taking the "original" screenshot . If you have 1280x720 4:2:0, this means the U,V planes which contain the color information are only 640x360 in dimensions. How they get scaled to 1280x720 in RGB determines what you see

    There are dozens of different ways you can do this, even within avisynth or ffmpeg. The biggest difference will be in the chroma center interpretation, and which scaling algorithm you choose) . You can choose different algorithms eg. spline, bicubic, etc... Some are sharper but cause aliasing, some are softer, some cause ringing etc...

    (Although there shouldn't be a difference between PNG and BMP output for ffmpeg; there is one known difference - it writes a PNG gAMA tag, which causes some software to display PNG differently - but this is in terms of colors (R,G,B values) , not chroma upsampling which your screenshots show)
    Quote Quote  
  8. Member
    Join Date
    Jul 2018
    Location
    Germany
    Search PM
    Originally Posted by pandy View Post
    Please test if you observing same issue when those option are used by scale:
    Code:
    scale=iw:ih:sws_flags=spline+accurate_rnd+full_chroma_int+full_chroma_inp:out_color_matrix=bt709
    i've struggle with similar issues on ffmpeg from long time - and i have mixed results - ffmpeg has nasty habit to call automatically libswscale in background without telling anything - check report thoroughly. Also it is good to explicitly control quantization range in ffmpeg (separately for input and output) frequently prefer to manually rector limited quantization range. If absolute processing accuracy is mandatory for you then Avisynth behave way better than ffmpeg.
    Hmm, I've tried your suggestion and to me it seems like if I convert the video to yuv444p before splitting it into its frames I get better results, though slightly off-color which however seems to be fixed when turning the frames back into a video. It is quite hard to tell though.

    At this point I am quite interested in just using avisynth for this process, however in my previous post I have detailed
    another issue I'm encountering when using avisynth to extract the frames as bmp.

    Originally Posted by poisondeathray View Post
    How are you taking the "original" screenshot ? What software or algorithm ?

    It's not really the "original"; you're looking at a RGB converted representation of the original . (You started with YUV 420) . It's not necessarily "correct" either.

    There is a difference in chroma upsampling method. ie. How ffmpeg or avisynth is converting YUV 420 to RGB is different than the method you used for taking the "original" screenshot . If you have 1280x720 4:2:0, this means the U,V planes which contain the color information are only 640x360 in dimensions. How they get scaled to 1280x720 in RGB determines what you see

    There are dozens of different ways you can do this, even within avisynth or ffmpeg. The biggest difference will be in the chroma center interpretation, and which scaling algorithm you choose) . You can choose different algorithms eg. spline, bicubic, etc... Some are sharper but cause aliasing, some are softer, some cause ringing etc...

    (Although there shouldn't be a difference between PNG and BMP output for ffmpeg; there is one known difference - it writes a PNG gAMA tag, which causes some software to display PNG differently - but this is in terms of colors (R,G,B values) , not chroma upsampling which your screenshots show)

    I'm using MPC-HC with http://www.cccp-project.net/ to screenshot the frame over the File->Save Image option.

    Edit: Upon further testing, I guess the effect I'm seeing in AviSynth is most likely caused by the chroma upsampling method, same as when I'm using pandy's suggested scaling options.
    I guess this means there is basically no way to avoid this?
    Last edited by Blackcero; 7th Jul 2018 at 10:03.
    Quote Quote  
  9. You're starting with 1/2 width, 1/2 height color information. There is no way unless you start with 4:4:4. That's pixel perfect color , no subsampling. Which algorithm you choose determines how that gets converted to RGB (full color)

    You can get the same results as that MPCHC with CCCP is using if you choose the same and settings algorithm that it is using. It's chroma upsampling too. Notice the colors don't line up perfectly either. The point is what you "think" is "original", isn't really "original" either. It's just a representation
    Quote Quote  
  10. Member
    Join Date
    Jul 2018
    Location
    Germany
    Search PM
    Originally Posted by poisondeathray View Post
    You're starting with 1/2 width, 1/2 height color information. There is no way unless you start with 4:4:4. That's pixel perfect color , no subsampling. Which algorithm you choose determines how that gets converted to RGB (full color)

    You can get the same results as that MPCHC with CCCP is using if you choose the same and settings algorithm that it is using. It's chroma upsampling too. Notice the colors don't line up perfectly either. The point is what you "think" is "original", isn't really "original" either. It's just a representation
    Thank you for your help as well!

    It turns out that the weird brightness issues with AviSynth are simply a product of my video player after all.
    I consider the results to be as close to perfect as I could get them now, so no further replies are needed.

    Thank you everyone for your help, this community is great!
    Quote Quote  
  11. There is also other issue with 4:2:0 YCbCr used today - chroma samples location - in MPEG-1 chroma sample was located "between" luma samples (bilinear interpolation) later (MPEG-2 and up) chroma samples are located only on even luma samples (for interlace it is even more complex) as such some leading/trailing edge issues may occur https://en.wikipedia.org/wiki/Chroma_subsampling
    Quote Quote  



Similar Threads