r103 only covers the topic of video signal tolerance. (Metadata flags are not part of the actual decoded bitstream data)
r103 does not cover other important aspects about video streams, eg. things like codec specific details, GOP makeup, reference frames, b-frames , containers, metadata, or dozens of other aspects that are important for submission
Go visit your workplace and see if any full range files exist, either correct flagged, or falsely flagged at any stage of the pipeline. Full range simply does not exist in end delivery formats (except in HDR scenarios). Full range acquisition formats get converted to standard range, they are never submitted as full range
Even if it was a correct file in terms of signal, why would someone purposely use an INCORRECT flag ? Why would you even risk it ? And if it was a correct flag, then your levels are wrong. There is no upside, it's all negatives.
And you can change flag metatdata. They are just VUI options. You can do it with codec specific options during encoding, or as a -bsf afterwards with -codec copy. ie. you can "fix" a wrongly flagged file
It was covered earlier. Full range equations in , RGB clipping, Full range equations out (that's your "unity" Y 0-255 <=> RGB 0-255) - does that ring any bells? Go back a few pages. You had it correct earlier. Right now you've gone back to limited range equations, so Y16-235 <=> RGB 0-255 .
As described earlier, converting to RGB works because YCbCr values that correspond to negative RGB values, and values >255 in 8bit are culled out by the conversion (but you can still generate new illegal value when converting back to YUV and subsampling)
Limited range equations for RGB conversion is a "more crude" of doing it because you cut off Y<16, Y>235 . If you've corrected it before ffmpeg, then it might be ok, otherwise you usually risk losing highlight and possibly shadow detail. Most cameras , even consumer cameras ,have usable detail in the Y235-255 range
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 301 to 324 of 324
Thread
-
-
Sure, I'll clarify - Vapoursynth editor (vsedit) is partially interactive GUI and you can visualize the results, hot pixel scans , display values, min/max, YCbCr, RGB pixel pickers, etc.. VSEdit2 has tabs to compare different versions of scripts (e.g. corrected , uncorrected)
These sorts of things should be possible in some GUI tools for libavfilter , but I did not see clearly working examples and code posted like the ones posted for vapoursynth earlier. Maybe when you have time to share the code you can post them
Is it more clear ? -
Vapoursynth is being loaded into a python script and is used as such. And python script can load other python module/script. Say a module that visualize a vapoursynth clip or whatever else, illegal values etc. You run a vapoursynth script and can get GUI or a visual feedback. Possibilities are endless. Mostly tapping into C language modules so it is very fast (like opencv to get a preview). So vapoursynth alone is not interactive but running it together with other modules it starts to be interesting. Vapoursynth is a part of python family and it gives you endless possibilities to load existing modules and have a fun with it.
-
And you can change flag metatdata. They are just VUI options. You can do it with codec specific options during encoding, or as a -bsf afterwards with -codec copy. ie. you can "fix" a wrongly flagged file
Last edited by chris319; 26th Jul 2021 at 18:23.
-
For ffmpeg libx264 during encoding
-x264opts fullrange=off
For ffmpeg libx265 during encoding
-x265-params range=limited
For an incorrectly full range flagged file, that you want to flag as "limited", use -codec copy with video_full_range_flag=0
https://ffmpeg.org/ffmpeg-bitstream-filters.html#h264_005fmetadata
eg.
ffmpeg -i avcinput.mp4 -codec copy -bsf:v h264_metadata=video_full_range_flag=0 avcoutput.mp4
https://ffmpeg.org/ffmpeg-bitstream-filters.html#hevc_005fmetadata
eg.
ffmpeg -i hevcinput.mp4 -codec copy -bsf:v hevc_metadata=video_full_range_flag=0 hevcoutput.mp4 -
Was your input file AVC/h.264 ?
Any error messages or clues in the console log ?
Works ok for me
before
Code:Codec ID : avc1 Codec ID/Info : Advanced Video Coding Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709
Code:Codec ID : avc1 Codec ID/Info : Advanced Video Coding Color range : Limited Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709
-
Before:
Code:Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 9 s 994 ms Bit rate : 26.9 kb/s Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 59.940 (59940/1000) FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.000 Stream size : 32.9 KiB (81%) Writing library : x264 core 164 r3065 ae03d92 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709
Code:Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 9 s 994 ms Bit rate : 26.9 kb/s Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 59.940 (59940/1000) FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.000 Stream size : 32.9 KiB (81%) Writing library : x264 core 164 r3065 ae03d92 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Color range : Full colour_range_Original : Limited Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709
-
ffmpeg method works if you demux to elementary avc stream first, then apply the bsf. This implies there is other info in the mp4 container, some atoms (maybe colr atom) for the original that the newest mediainfo reads . But then you have to remux video and audio
ffmpeg has experimental colr atom support, but there is no documentation . I'll try to experiment with it later
Code:mp4 muxer AVOptions: -movflags <flags> . . . write_colr E.......... Write colr atom even if the color info is unspecified (Experimental, may be renamed or changed, do not use from scripts) . .
mp4box -add input.mp4#video:colr=nclx,1,1,1,0 -new output.mp4
Color range : Limited
colour_range_Original : Full
But if you combine ffmpeg bitstream filter for the ES AVC stream, and then afterwards MP4box for the MP4 colr atoms , it also works, because those are the 2 locations where range information is kept
Color range : Limited
So those are your 2 options so far
1) demux to ES (this discards the colr atoms because no longer MP4 or MOV container), apply ffmpeg bitstream filter, remux
or
2) ffmpeg bitstream filter to "fix" video stream, then mp4box to "fix" mp4 colr atom
It would be nice if ffmpeg mp4/mov muxer option for -colr was documented or worked, that way ffmpeg could do it all in 1 step instead of 2 -
A lot of this fancy stuff is way over my head.
The consumer camcorders I use output 1080p natively. That's not an official broadcast format and is why I resize it to 720p. Next I will need to add interlace to the 1080 version. I've done that before. -
The last few posts is nothing "fancy" ; it's just changing the metadata or "labels" ( or what you see in mediainfo) in terms of full vs. limited range "flagging". Other programs look for this information too, including NLE's , QC software, software players, hardware decoder chips - It's important to get the correct "label". Although the underlying actual video data is not directly affected, the "flags" can govern the behaviour of some software and hardware - so effectively you can get different video appearance and levels
I guess it's nice that newer mediainfo could detect and alert you to both, because some programs look in one place, some in the other. If you take care of both, you cover all your bases -
This seems to work so far.
I need to test it a bit more thoroughly.
https://forum.doom9.org/showthread.php?t=152419Last edited by chris319; 27th Jul 2021 at 12:51.
-
This seems to work for "legalizing" levels from 0 - 255 to 16 - 235.
Setting out_range to limited introduces color shifts, so out_range is kept at full at this stage.
Code:ffmpeg -y -i "Bars.mp4" -c:v libx264 -vf format=rgb24,"lutrgb=r='clip(val,14,234)':g='clip(val,14,234)':b='clip(val,14,234)'",scale=w=1280:h=720:flags=spline:out_color_matrix=bt709:out_range=full,format=yuv420p -color_primaries bt709 -color_trc bt709 -colorspace bt709 -c:a pcm_s16be clipped.mov
https://sourceforge.net/projects/direct264/
Code:ffmpeg -y -i clipped.mov -vcodec copy -acodec copy -vbsf h264_changesps=limitedrange output.mov
Code:General Complete name : D:\Programs\ffmpeg\Batch_files\Retag as limited\output.mov Format : MPEG-4 Format profile : QuickTime Codec ID : qt 0000.02 (qt ) File size : 12.1 MiB Duration : 33 s 530 ms Overall bit rate : 3 026 kb/s Encoded date : UTC 1970-01-01 00:00:00 Tagged date : UTC 1970-01-01 00:00:00 Writing application : Lavf59.4.101 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L3.2 Format settings : CABAC / 4 Ref Frames Format settings, CABAC : Yes Format settings, Reference frames : 4 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 33 s 517 ms Bit rate : 1 480 kb/s Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 59.940 (60000/1001) FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.027 Stream size : 5.92 MiB (49%) Writing library : x264 core 164 r3065 ae03d92 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Language : English Encoded date : UTC 1970-01-01 00:00:00 Tagged date : UTC 1970-01-01 00:00:00 Color range : Limited Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : avcC Audio ID : 2 Format : PCM Format settings : Big / Signed Codec ID : twos Duration : 33 s 530 ms Bit rate mode : Constant Bit rate : 1 536 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Bit depth : 16 bits Stream size : 6.14 MiB (51%) Language : English Encoded date : UTC 1970-01-01 00:00:00 Tagged date : UTC 1970-01-01 00:00:00
Last edited by chris319; 27th Jul 2021 at 15:11.
-
The following works, too. Less conversion between pixel formats.
Code:ffmpeg -y -i "C0015.mp4" -c:v libx264 -vf format=yuv420p,"lutyuv=y='clip(val,34,215)'",scale=w=1280:h=720:flags=spline:out_color_matrix=bt709:out_range=full -color_primaries bt709 -color_trc bt709 -colorspace bt709 -c:a pcm_s16be clipped.mov
-
It still visibly changes the contrast, and shifts some channels +/- 10, resulting in incorrect colors and bars . Reference black is no longer Y=16, but Y=14 now on bars or a legal video
Too much contrast and color accuracy changes in channels . +/- 11 . Reference black is is now Y=21 instead of Y=16 on bars or a legal video -
There is no one-size-fits-all solution. It is fully expected that the user will need to adjust the values in the script to suit, depending on the characteristics of the input video. This is nothing new. In the days of analog we used hardware proc (processing) amps to adjust levels depending on the characteristics of the incoming video. This was done by adjusting knobs on the front panel.
These were revolutionary when they came out in the '70s. Only $20,000!
https://worldradiohistory.com/hd2/IDX-Site-Technical/Engineering-Broadcast/Archive-Bro...-Page-0013.pdf
The design goal is to bring consumer camcorder luminance levels into R 103 spec without introducing significant color shifts. It is not intended to adjust the luminance levels of color bars, although color bars can be used to check for color shifts.
My scope program can apply low-pass filtering per R 103. This filtering can also be bypassed.
Code:pipeIn$ = "-y -i " + filename$ + " -f image2pipe -s 1280x720 -vf unsharp=luma_msize_x=3:luma_msize_y=3:luma_amount=-0.25,scale=out_range=limited -color_primaries bt709 -color_trc bt709 -colorspace bt709 -pix_fmt yuv420p -vcodec rawvideo -"
For progressive signals
A quarter band filter applied horizontally and a half band filter applied vertically is recommended.
Horizontal Filter Coefficients: 1/16, 2/16, 3/16, 4/16, 3/16, 2/16, 1/16
Vertical Filter Coefficients: 1/4, 1/2, 1/4
Code:ffmpeg -y -i "Bars.mp4" -c:v libx264 -vf format=yuv420p,"lutyuv=y='clip(val,37,208)'",scale=w=1280:h=720:flags=spline:out_color_matrix=bt709:out_range=full -color_primaries bt709 -color_trc bt709 -colorspace bt709 -c:a pcm_s16be clipped.mov
I tested it on color bars and all of the color values are within tolerance, e.g.:
Bars before script is applied (input file):
Cyan:
R = 19
G = 179
B = 179
Bars after correction with the script (output file):
Cyan:
R = 20
G = 180
B = 180
As you can see, my script introduces one point of error between input and output on the cyan bar.
I don't know what you're doing to get such drastic color shifts, but I'm not seeing them. I'm satisfied that the script fulfills the design goal, provided the filters are properly adjusted.Last edited by chris319; 29th Jul 2021 at 14:27.
-
Maybe something wrong with your scope again ?
Yes, technically within limits and "compliant" , but the problem is color accuracy, white and black reference accuracy, contrast change and range compression. Y=16 => Y=24 . Y=235 => Y=224
Something like +/- 3 might be passable on colors, but some channels have a delta of 12. eg. CY is -12 in Cr.
And greyscale values should not change at all between the range Y16-235 . Should be +/- 0 . e.g 75% white is +11 in Y.
"Within spec" does not mean you should alter colors significantly and black and white reference levels
Reference colorbars are appended to submissions and are used for a reason.
Input YCbCr
75W 180,128,128
YL 168,44,136
CY 145,147,44
G 133,63,52
MG 63,193,204
R 51,109,212
B 28,212,120
40GY 104,128,128
Output YCbCr
75W 191,128,128
YL 177,32,137
CY 150,150,32
G 137,54,31
MG 55,202,215
R 41,106,224
B 24,224,119
40GY 102,128,128 -
Maybe something wrong with your scope again ?
To reiterate:
The design goal is to bring consumer camcorder luminance levels into R 103 spec
I'm not seeing the drastic color errors you complain of when YUV is converted to RGB using VLC player, just a point or two here and there.
Take a look at both scripts. What would you do to improve color fidelity? -
So you should get the same Y,Cb,Cr values as the previous post. The differences are too large
To reiterate:
The design goal is to bring consumer camcorder luminance levels into R 103 spec
I'm not seeing the drastic color errors you complain of when YUV is converted to RGB using VLC player, just a point or two here and there.
Take a look at both scripts. What would you do to improve color fidelity?
Normally people edit/grade their footage, then use some legalization method at the end .
Ideally , only illegal values should be adjusted. You usually don't want to adjust good values that don't need to be adjusted - you're just ruining the grade that was done
And if you're using this directly on video out of the camera to make it "legal" - it messes up properly exposed video, and alters the original color.
The "brighest" white is now Y=224, not Y=235. The "darkest" black is now Y=24, not Y=16 - so the dynamic range is compressed, image looks washed out, dull.
So it might be ok for some specific video, in some specific limited case scenario, but not for general use.
One of the earlier script versions was ok with the train, go back a few pages . -
One of the earlier script versions was ok with the train, go back a few pages .
Either that or propose modifications to the latest script version. -
Ideally , only illegal values should be adjusted.
I've asked you many times how you propose to do this. We've been through brightness and contrast and they introduced color shifts so no good. We've tried lutrgb and lutyuv and you weren't happy with those. What else is there? geq?
Once we've found a pixel that's outside the 235 limit, what do we do with it? We can hard clip it to 235. We can multiply it by 0.92 which puts 255 around 235.Last edited by chris319; 29th Jul 2021 at 23:49.
-
Here is what I am using for poor-man's legalization.
Code:ffmpeg -y -i "bars.mp4" -c:v rawvideo -vf scale=w=1280:h=720,"lutrgb=r='clip(val,16,235)':g='clip(val,16,235)':b='clip(val,16,235)'" -pix_fmt rgb24 -c:a pcm_s16be RGB.mov ffmpeg -y -i RGB.mov -vf "select=eq(n\,60)" -vframes 1 out.bmp
Note that the pixel format is rgb24. When subsampling is introduced, there are a few pixels which are 255 and 0. These values are reserved for sync data and should be avoided. If a distributor must have a 4:2:0 or 4:2:2 pixel, another solution will be neded.
Similar Threads
-
ffmpeg 4.1.4 question regarding "limited color range" output file
By bokeron2020 in forum Newbie / General discussionsReplies: 12Last Post: 1st Aug 2019, 18:28 -
Can I convert color profile with FFMPEG?
By PabstBlueRibbon in forum Video ConversionReplies: 0Last Post: 9th Sep 2017, 14:41 -
Color Range Question
By Akai-Shuichi in forum RestorationReplies: 4Last Post: 14th Feb 2017, 16:53 -
Change in color while reencoding with ffmpeg
By Epaminaidos in forum Video ConversionReplies: 24Last Post: 30th Sep 2016, 12:09 -
Can I alter h264 file's color range flags?
By bergqvistjl in forum Newbie / General discussionsReplies: 1Last Post: 17th Dec 2015, 13:00