VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Page 1 of 2
1 2 LastLast
Results 1 to 30 of 43
Thread
  1. Here is a little script which converts full-range 0 - 255 video to limited-range 16 - 235. It uses ffmpeg's eq filter, giving you finer control than simply re-flagging to limited range.

    I was experimenting with a camera that puts out 0 - 255 and DaVinci Resolve's "color match" feature with a DSC SMPTE One-Shot card. This script also adjusts the gamma. It's not perfect but close.

    Code:
    ffmpeg -y  -i inputVideo.mp4  -pix_fmt yuv420p  -c:v libx265 -vf eq=brightness=.06:contrast=0.84:gamma=0.6,scale=out_color_matrix=bt709:out_range=limited  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -c:a copy  outputVideo.mp4
    Quote Quote  
  2. Originally Posted by chris319 View Post
    Here is a little script which converts full-range 0 - 255 video to limited-range 16 - 235. It uses ffmpeg's eq filter, giving you finer control than simply re-flagging to limited range.

    I was experimenting with a camera that puts out 0 - 255 and DaVinci Resolve's "color match" feature with a DSC SMPTE One-Shot card. This script also adjusts the gamma. It's not perfect but close.

    Code:
    ffmpeg -y  -i inputVideo.mp4  -pix_fmt yuv420p  -c:v libx265 -vf eq=brightness=.06:contrast=0.84:gamma=0.6,scale=out_color_matrix=bt709:out_range=limited  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -c:a copy  outputVideo.mp4

    "re-flagging" doesn't change the actual range. Flags are just metadata or "labels" - and that might affect how some programs interpret the data - but the actual underlying YUV data is unchanged.

    eg. If you have actual full range data Y 0-255, flagged as full range, and then change the flag to "limited range" , it's still Y 0-255 full range data but flagged as limited



    The option for more control is generally a good thing, but those eq values convert to Y=220 max . ie. Y 0-255 to 16-220 . (Excessive range compression)
    Quote Quote  
  3. that might affect how some programs interpret the data - but the actual underlying YUV data is unchanged.
    Right, which is why I included the eq filter.

    those eq values convert to Y=220 max . ie. Y 0-255 to 16-220 . (Excessive range compression)
    Did you bother to test it or are you speaking theoretically? I tested it (it's a very time-consuming back-and-forth process) using the DSC card and it works for me. There is a lot of interaction among the settings, making it especially tedious to get the gamma right.

    The user is free to substitute different values if desired.
    Quote Quote  
  4. Originally Posted by chris319 View Post
    Did you bother to test it
    Yes

    Both synthetic , and real camera
    Quote Quote  
  5. To get ffmpeg cmd line into Vapoursynth, WITHOUT needing to encode it first and therefore use color pickers for value evaluation:
    Code:
    import vapoursynth as vs
    from vapoursynth import core
    import subprocess, ctypes, shlex
    
    source_path=r'C:\inputVideo.mp4'
    clip = core.lsmas.LibavSMASHSource(source_path)
    ffmpeg = r'H:\tools\ffmpeg.exe'
    
    #no source loading , no formats out
    ffmpeg_args ='-y -pix_fmt yuv420p -vf eq=brightness=.06:contrast=0.84:gamma=0.6,scale=out_color_matrix=bt709:out_range=limited  -color_primaries bt709  -color_trc bt709  -colorspace bt709'
    
    w = clip.width
    h = clip.height
    Ysize  = w * h             #for YUV420
    Usize = w * h//4           #for YUV420
    Vsize = w * h//4           #for YUV420
    
    
    #------------------------- END of USER input ------------------------
    
    ffmpeg_args = ffmpeg_args.strip()
    ffmpeg_args =  shlex.split(ffmpeg_args)
    command = [ffmpeg, '-i', source_path] + ffmpeg_args + [ '-f', 'rawvideo', '-']
    frame_len = Ysize + Usize + Vsize
    pipe = subprocess.Popen(command, stdout = subprocess.PIPE, bufsize=frame_len)
    
    def load_frame(n,f):
        try:
            vs_frame = f.copy()
            for i, size in enumerate([Ysize, Usize, Vsize]):
                ctypes.memmove(vs_frame.get_write_ptr(i), pipe.stdout.read(size),  size)
            pipe.stdout.flush()
        except Exception as e:
            raise ValueError(repr(e))    
        return vs_frame
    
    try:
        clip = core.std.ModifyFrame(clip, clip, load_frame)
    except ValueError as e:
        pipe.terminate()
        print(e)
    
    clip.set_output()
    I used to use it to follow Chris questions without necessarily encoding video for every change, so it might help others,
    vapoursyth reads from ffmpeg pipe and it is linear only during playback, only consequent frames can be viewed
    Last edited by _Al_; 9th Aug 2020 at 18:27.
    Quote Quote  
  6. Originally Posted by poisondeathray View Post
    Originally Posted by chris319 View Post
    Did you bother to test it
    Yes

    Both synthetic , and real camera
    And you got the exact same results for both?

    Did you test it with a DSC SMPTE One Shot card and Resolve's "color match" function?

    Again, the user is free to fudge the parameters as needed.
    Quote Quote  
  7. Originally Posted by chris319 View Post
    And you got the exact same results for both?
    Roughly +/- 2-3 , I used lossy compression for camera. But it's ~220 Max . Exactly 220 when testing synthetic


    Did you test it with a DSC SMPTE One Shot card and Resolve's "color match" function?
    No . I tested full range input. Applied the filter as specified. Measured the output with several tools to confirm

    Visually , results confirm what measurements and waveform show. Dynamic range is low, shadows compressed too.
    Quote Quote  
  8. It's been months and I've never gotten Vapoursynth to run without errors, so I gave up.

    At least ffmpeg runs and follows the commands.
    Quote Quote  
  9. Originally Posted by _Al_ View Post
    To get ffmpeg cmd line into Vapoursynth, WITHOUT needing to encode it first
    Another way is to pipe ffmpeg into ffplay for -vf waveform if you were looking at levels (e.g. some people can't get vapoursynth to run)

    eg.
    ffmpeg -i input.ext -vf <your filters here> -f yuv4mpegpipe - | ffplay -f yuv4mpegpipe - -vf waveform=g=green

    So you can adjust some filter parameters, run it again, adjust , rinse/repeat
    Quote Quote  
  10. Originally Posted by poisondeathray View Post
    Originally Posted by chris319 View Post
    And you got the exact same results for both?
    Roughly +/- 2-3 , I used lossy compression for camera. But it's ~220 Max . Exactly 220 when testing synthetic

    What kind of test chart? Does it have an 18% reflectance patch? Please post your output file.

    Did you test it with a DSC SMPTE One Shot card and Resolve's "color match" function?
    No . I tested full range input. Applied the filter as specified. Measured the output with several tools to confirm

    Visually , results confirm what measurements and waveform show. Dynamic range is low, shadows compressed too.
    So tweak the parameters! I'm not standing in your way.
    Quote Quote  
  11. Originally Posted by chris319 View Post
    So tweak the parameters! I'm not standing in your way.
    I wouldn't use it at all. If you're using other programs like Resolve, why not just do it there? Almost every program has a configurable levels filter that you can see in real time with a GUI. FFmpeg is not ideal for that


    I'm just pointing out it does not do what you claim . It's a heads up for people who don't check , they are going to get the wrong results
    Quote Quote  
  12. If you're using other programs like Resolve, why not just do it there?
    Tell me how. I have posted this question on the blackmagic forum and my post has yet to appear.

    Almost every program has a configurable levels filter that you can see in real time with a GUI.
    Think again. I've searched all over and can't find anything for Resolve.

    What did I say in my original post?

    It's not perfect but close.
    The blacks need to come up a bit but then I'd have to adjust the gamma. It's a time-consuming back-and-forth process. Later.

    Now answer my question: what kind of test chart are you using with your camera and does it have an 18% reflectance patch? How about posting a screen shot?

    Image
    [Attachment 54472 - Click to enlarge]
    Quote Quote  
  13. I'm just pointing out it does not do what you claim . It's a heads up for people who don't check , they are going to get the wrong results
    If video levels matter, they need to check them. If they don't matter (YouTube, private viewing) then don't worry about it.

    How many times must I repeat myself? If you don't get the levels you want then tweak the parameters!
    Quote Quote  
  14. Someone recommend this link, I think it was on this site, https://www.blackmagicdesign.com/uk/products/davinciresolve/training
    I just watched that link from "The Art of Color Grading". There are steps you should watch for, or maybe other videos are also that entertaining as that was.
    Quote Quote  
  15. Originally Posted by chris319 View Post
    I'm just pointing out it does not do what you claim . It's a heads up for people who don't check , they are going to get the wrong results
    If video levels matter, they need to check them. If they don't matter (YouTube, private viewing) then don't worry about it.

    How many times must I repeat myself? If you don't get the levels you want then tweak the parameters!

    You said it converts full range 0-255 to limited range 16-235
    Originally Posted by chris319 View Post
    Here is a little script which converts full-range 0 - 255 video to limited-range 16 - 235.

    I'm pointing out it does not.

    How many times do I have to repeat myself?

    It would have been better to post something that actually did , then fine tune as needed from that.
    Quote Quote  
  16. I wrote 16 - 235 to make clear what is meant by "limited range", not to set you up to take potshots at it.

    There were several hours of back-and-forth tweaking before getting that far.

    I'm pointing out it does not.
    I posted a darn screen shot. What more do you want?

    As usual, I don't see you posting an improved script, only criticizing.

    Once again, what kind of test chart are you using with your camera? And how about posting a screen shot of your test?

    No answer to my question and no screen shot. I'm starting to think B.S.
    Last edited by chris319; 9th Aug 2020 at 20:46.
    Quote Quote  
  17. Originally Posted by chris319 View Post
    I wrote 16 - 235 to make clear what is meant by "limited range", not to set you up to take potshots at it.
    Output Ymax is 220 when input Ymax is 255 . That's not right.

    People will get messed up results with that.

    If you think that works for your setup, great.

    But it won't for full range cameras . Please double check your work before posting

    How is it a "potshot" when it's a fact ? I'm only posting so people don't copy/paste and get incorrect results
    Quote Quote  
  18. But it won't for full range cameras .
    Wrong. That is a full-range camera output to a full-range file.

    Where's your improved version?

    Where are your answers to my questions?

    Where's your screen shot?

    All talk and no action.

    Don't say that I don't check my work because I do and the screen shot proves it. It works for me.

    Your lack of hands-on experience with professional video is showing.

    Enough arguing.
    Quote Quote  
  19. Originally Posted by chris319 View Post
    But it won't for full range cameras .
    Wrong. That is a full-range camera output to a full-range file.
    Look at the command line, it's not a full range file with yuv420p or out_range=limited


    Where's your improved version?
    I don't use -vf eq . I can try to figure it out, but I don't see any need to do it this way.

    A better starting point would be Y 0-255 to Y 16-235, instead of to Y16-220, don't you agree ?

    Don't say that I don't check my work because I do and the screen shot proves it. It works for me.
    If you think it works for your specific shot with your camera, great.

    It will not work for everyone else who has full range video 0-255

    Take any random full range video with full range values (Y 0-255) , or synthetic full range test video (Y 0-255) . Ymax is 220 after you apply that script . Are you saying this is wrong ? (I can make mistakes too)


    Maybe you're making a mistake or flawed assumption somewhere . It wouldn't be the first time...Think about it.

    The slope can be changed nonlinearly (e.g. curves) in most editors to hit a 18% target(or any % target) - but it's absolutely essential to nail down black and white. That eq setting gets Y=16 correct, but Y=220 is not white by any standard . Sure, you can adjust it afterwards, but why bother with all this to make a mistake and then have to correct that mistake later ?
    Quote Quote  
  20. I tested it on a window signal containing 255 and it puts 255 right at 235. See the first screen shot below.

    The black and gamma in the camera footage are different from the test signal. It's doing what I need it to do, which is to correct the camera footage.

    Below that is the output of Resolve color match and no ffmpeg script; whites to 255 and gamma way too high.

    Look at the command line, it's not a full range file with yuv420p or out_range=limited
    Wrong again. The command line has nothing to do with it. It is the output of Resolve. The script hasn't been applied yet.

    I don't use -vf eq . I can try to figure it out, but I don't see any need to do it this way.
    Fine. Let's see your way of accomplishing this even if it doesn't use -vf eq.

    You do have a test card with 18% reflectance, don't you? How many times have I asked and you haven't answered yet? And 90% reflectance for white?

    https://www.bhphotovideo.com/c/product/1311293-REG/dsc_labs_posp_pocket_oneshot_plus_test.html
    Image Attached Thumbnails Click image for larger version

Name:	OneShot_test.PNG
Views:	9
Size:	52.3 KB
ID:	54473  

    Click image for larger version

Name:	Original.PNG
Views:	8
Size:	510.2 KB
ID:	54474  

    Last edited by chris319; 10th Aug 2020 at 00:55.
    Quote Quote  
  21. Originally Posted by chris319 View Post
    I tested it on a window signal containing 255 and it puts 255 right at 235.
    To be clear, I'm talking about the 1st post, actual video files from the ffmpeg commandline. ie. The topic.

    We differ there, Y 255 becomes Y 220 in the output file after the ffmpeg commandline

    It's a problem when your brightest pixel is Y=220

    I think there is a problem somewhere with how you are measuring /testing or your file, or maybe some other programs doing something funky. I'm measuring the actual levels, verified with several programs


    The black and gamma in the camera footage are different from the test signal. It's doing what I need it to do, which is to correct the camera footage.

    Below is the original camera footage; whites to 255 and gamma way too high.
    I don't see how it can be - if original footage , original input file has Y=255, but the output file has Y=220



    Look at the command line, it's not a full range file with yuv420p or out_range=limited
    Wrong again. The command line has nothing to do with it. I clearly stated it is the direct output of the camera. The script hasn't been applied yet.
    You posted a Full Range to Limited Range video thread, using ffmpeg's eq filter in the first post. You said "That is a full-range camera output to a full-range file." Does the full-range file have nothing to do with it ?? The output full range file is the only logical conclusion I came to. But if you were referring something else, I'm misunderstanding what you're saying

    To be clear, all I'm talking about is full range input video, the ffmpeg command line in the 1st post, and the resulting output file .

    When the input file has Y=255, the output file has Y=220

    That's the only reason I'm posting, there is a problem with that commandline


    Fine. Let's see your way of accomplishing this even if it doesn't use -vf eq.
    You do it any decent GUI. Get your black and white points correct. Correct for the 18% grey card using curves (or other methods can be used too, but that' s the way most people use in resolve). This should adjust the levels and any possible color cast . This is a basic skill. There should be many tutorials, even free ones on youtube. Curves are generally used because usually non linear changes are required for different cameras
    Last edited by poisondeathray; 10th Aug 2020 at 01:07.
    Quote Quote  
  22. Below is the original camera footage; whites to 255 and gamma way too high.

    I don't see how it can be
    Correction: it is the output of Resolve color match, not the camera. My bad.

    You know how to read a scope. And you do know where 18% reflectance should sit on a BT.709 waveform at 2.2 gamma, right? The screen shot clearly shows the white patch at 255 and the 18% patch way above 43% (digital 111). That is the output of DaVinci color match.

    So where's the confusion?
    Quote Quote  
  23. The problem is primarily the white level, but also the compressed dynamic range

    I'm talking about the ffmpeg output file. When you use full range input Y 0-255, the output is Y 16-220 . A white Y=255 patch becomes Y=220. If you disagree, then we have to sort that out first



    If you import that into resolve, you will see the problem

    It has been converted to a normal range file , and if it was a "correct" conversion to normal range Y 16-235 you would see 0-1023 in resolve when the file is interpreted as video range (auto, since it's no longer flagged as full after conversion). Resolve retains data >1023 internally, you can bring those down (if you had them). But the Y=255 patch has levels well below 1023 (corresponding to Y=220 in 8bit). This corresponds with other tools as well

    You should see compressed shadows too because of the range compression (but that will partially depend on the camera)
    Quote Quote  
  24. If you disagree, then we have to sort that out first
    No. We're not going to. I stand behind my measurements.

    As I have said before, a measurement device or measurement software shouldn't alter the signal it's measuring. Think about it.
    Quote Quote  
  25. Originally Posted by chris319 View Post
    If you disagree, then we have to sort that out first
    No. We're not going to. I stand behind my measurements.

    As I have said before, a measurement device or measurement software shouldn't alter the signal it's measuring. Think about it.
    Yes, this suggest is wrong with your measurements or your video. If you stand by your measurements, then it could be your video. You posted an incorrect "window signal" video before, are you sure you're using a correct one ? Please upload your video


    You can check with ffmpeg/ffplay -vf waveform, or confirm with other like vapoursynth, avisynth, or elementary bitstream YUV analysis

    In the shotcut forum , Austin posted a correct Y 0,16,235,255 video, flagged as full range. I'll repost it here "FullValuesFullFlag.mp4"

    I copy/pasted the ffmpeg eq settings in the first post, the output is vf_eq_outputVideo.mp4

    Code:
    ffmpeg -y  -i FullValuesFullFlag.mp4  -pix_fmt yuv420p  -c:v libx265 -vf eq=brightness=.06:contrast=0.84:gamma=0.6,scale=out_color_matrix=bt709:out_range=limited  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -an  vf_eq_outputVideo.mp4
    Code:
    ffmpeg -i FullValuesFullFlag.mp4 -vf waveform=g=green:o=0.25 -frames:v 1 FullValuesFullFlag.png
    Image
    [Attachment 54476 - Click to enlarge]

    Code:
    ffmpeg -i vf_eq_outputVideo.mp4 -vf waveform=g=green:o=0.25 -frames:v 1 vf_eq_outputVideo.png
    Image
    [Attachment 54481 - Click to enlarge]


    I'll also include a Y 0-255 ramp, flagged as full range "greyscale_nd_fullrange_flagged.mp4" (and if you want I can upload one flagged as normal or not flagged . But if you recall , -vf waveform does not adjust it's digital reading according to presence of a full range flag , nor does it adjust when using IRE units) .

    Code:
    ffmpeg -y  -i greyscale_nd_fullrange_flagged.mp4  -pix_fmt yuv420p  -c:v libx265 -vf eq=brightness=.06:contrast=0.84:gamma=0.6,scale=out_color_matrix=bt709:out_range=limited  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -c:a copy  vf_eq_outputVideo_gradient.mp4
    Code:
    ffmpeg -i greyscale_nd_fullrange_flagged.mp4 -vf waveform=g=green -frames:v 1 greyscale_nd_fullrange_flagged.png
    Image
    [Attachment 54479 - Click to enlarge]

    Code:
    ffmpeg -i vf_eq_outputVideo_gradient.mp4 -vf waveform=g=green -frames:v 1 vf_eq_outputVideo_gradient.png
    Image
    [Attachment 54480 - Click to enlarge]
    Image Attached Files
    Quote Quote  
  26. Are we in agreement that it is not possible to set the color range (full/limited) in the ffplay scope?

    Just answer "yes" or "no" please.
    Quote Quote  
  27. Originally Posted by chris319 View Post
    Are we in agreement that it is not possible to set the color range (full/limited) in the ffplay scope?

    Just answer "yes" or "no" please.
    Yes

    (not directly)
    Quote Quote  
  28. Does this code look good to you for generating a full-level file?

    yuvj pixels intentionally omitted, so it will be flagged as "limited"

    What changes would you make if it looks wrong to you?

    Code:
    bin\ffmpeg  -y  -loop 1 -t 10  -i WindowSignalFull.bmp  -pix_fmt yuv420p    -c:v libx265  -vf scale=out_color_matrix=bt709:out_range=full  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -crf 0  -r 59.94  -an  Window_full.mp4
    Quote Quote  
  29. Originally Posted by chris319 View Post
    Does this code look good to you for generating a full-level file?

    yuvj pixels intentionally omitted, so it will be flagged as "limited"

    What changes would you make if it looks wrong to you?

    Code:
    bin\ffmpeg  -y  -loop 1 -t 10  -i WindowSignalFull.bmp  -pix_fmt yuv420p    -c:v libx265  -vf scale=out_color_matrix=bt709:out_range=full  -color_primaries bt709  -color_trc bt709  -colorspace bt709  -crf 0  -r 59.94  -an  Window_full.mp4

    That should give full range levels (Y 0-255), but flagged as "limited"
    Quote Quote  



Similar Threads