Adjusting H.264 Levels from .mp4

[jagabo]

Really! I'm seeing zero on the HDMI output.

Are you looking at RGB or YUV? It's perfectly normal for YUV 16-235 to be expanded to RGB 0-255 for display.

How about this one?

Brights start out a bit bright (very common with camcorders), blacks are fine:



After about 20 frames the brights come down a bit, maybe 238 or so..

Note that the waveform graph is 256 lines tall -- one line for each Y value. If you really think "a scope without a graticule is pretty useless" you can used VideoScope() intead of Histogram(). Frame 40:



Again, the graph is 256 lines tall.

[chris319]

Well, for the live HDMI feed I'm using a program called escapi

http://sol.gfxile.net/escapi/

Technically, it's just a simple wrapper over Media Foundation, DirectMedia (or DirectShow)

It interfaces with the USB video device and gives me an RGB image which can be displayed on the screen. I read each pixel in this RGB image and apply the bt.709 coefficients to arrive at luma values which are plotted on the scope. To interface with the camcorder I use an Elgato Cam Link to go from the camera's HDMI output to USB, so it appears as a USB video device to escapi.

https://www.elgato.com/en/gaming/cam-link

[jagabo]

It interfaces with the USB video device and gives me an RGB image which can be displayed on the screen.

That RGB image is normally full range, 0-255.

I read each pixel in this RGB image and apply the bt.709 coefficients to arrive at luma values which are plotted on the scope.

You would normally use the limited range matrix for this to compress the range from RGB 0-255, to YUV 16-235. No out of range YUV values are possible.

http://www.equasys.de/colorconversion.html

[chris319]

So 16 - 235 is being recorded, but it's 0 - 255 when it gets to the screen via HDMI and USB. Do you concur? So I'll have to convert.

[poisondeathray]

Technically, it's just a simple wrapper over Media Foundation, DirectMedia (or DirectShow)

A clue might be your DirectShowSource() attempt from the doom9 thread - do you recall you couldn't use coloryuv , and it reported it wasn't YUV ? I hypothesized that your directshow filter chain converted to RGB somewhere

Check again with

DirectShowSource("video.mp4")
Info()

If you can adjust your directshow filter chain using preferred filter tweaker to use LAV filters, it should be sending YUV properly

It interfaces with the USB video device and gives me an RGB image which can be displayed on the screen

Is that the only option ? Can you read off YUV if YUV is sent ?

[chris319]

We can optimize for my machine, but what happens when Joe Doakes down the street tries to do all this stuff on his unoptimized computer? That's something to keep in mind.

I'm not trying to be argumentative, but we must consider all use cases.

[jagabo]

So 16 - 235 is being recorded, but it's 0 - 255 when it gets to the screen via HDMI and USB. Do you concur? So I'll have to convert.

What's viewed is RGB 0-255. What's stored in the video file is YUV 16-235. That is the industry standard (DVD, Blu-ray, streaming video, etc.).

Check again with

DirectShowSource("video.mp4")
Info()

Sometimes you need to force the color with pixel_type=...

DirectShowSource("video.mp4", pixel_type="YV12") # or YV24, YUY2, etc.
Info()

We can optimize for my machine, but what happens when Joe Doakes down the street tries to do all this stuff on his unoptimized computer? That's something to keep in mind.

You encode to the industry standards: RGB 0-235, YUV 16-235. What happens on the user's machine is his problem. It's not possible to make a video that plays properly on every mis-configured computer, player, TV...

[poisondeathray]

It interfaces with the USB video device and gives me an RGB image which can be displayed on the screen. I read each pixel in this RGB image and apply the bt.709 coefficients to arrive at luma values which are plotted on the scope. To interface with the camcorder I use an Elgato Cam Link to go from the camera's HDMI output to USB, so it appears as a USB video device to escapi.

So 16 - 235 is being recorded, but it's 0 - 255 when it gets to the screen via HDMI and USB. Do you concur? So I'll have to convert.

Ok I'm a bit confused. Are you saying you're getting 0-255 YUV from the scope plot, or did you mean 0-255 RGB?

If original is 16-235 Y , and you get 0-255 RGB , that is normal as jagabo explained. I'm just wondering why you think you have to "convert" ?

A HDMI connection from a consumer camcorder usually sends uncompressed 8bit 4:2:2 YUV . Typically, levels should match what your onboard recorder is recording, but the difference is it's uncompressed and 4:2:2. But this is an assumption, I do not know for sure if your model does.



Jagabo already explained it, but a normal PC conversion "maps" Y 16-235, CbCr 16-240 to RGB 0,0,0-255,255,255 . This is normal and expected for a computer. So if you were reading RGB zero that is expected, but Y zero is not

If you had a studio RGB setup, then YCbCr 0-255 gets "mapped" to RGB 0,0,0-255,255,255 . So if your original data was Y 16-235, RGB is also 16-235 . You mentioned studio swing earlier, but 99.999% of setups do not use this. If you had studio swing setup then you should not read zero anywhere

Unless you know exactly how your signal is being converted to RGB (is it being converted in directshow, or usb device, or somewhere else?), and what type of RGB you are using, I would forget about RGB. The ONLY important thing is Y 16-235 CbCr 16-240. That is legal in every setup. That's why a YUV waveform is critical.

Do you recall I said >90% of consumer cameras record Y 16-255 , and that most of them have usable overbrights? (ie. the 236-255 range is usable) . You showed the 16 scenario with the lens cap, but you didn't show the overexposure scenario, but I'm willing to bet your camera is in that 90% category. The top end is what usually needs to be adjusted not the bottom end

We can optimize for my machine, but what happens when Joe Doakes down the street tries to do all this stuff on his unoptimized computer? That's something to keep in mind.

I'm not trying to be argumentative, but we must consider all use cases.

Yes I know. That's why the ONLY thing important is Y 16-235 CbCr 16-240 . That's legal everywhere. Forget about all your RGB conversions . All that optimization stuff is so you can get some sort of working monitoring device working properly so you can read levels properly . You're trying to work backwards from some RGB conversions and it can be tricky

[poisondeathray]

Check again with

DirectShowSource("video.mp4")
Info()

Sometimes you need to force the color with pixel_type=...

DirectShowSource("video.mp4", pixel_type="YV12") # or YV24, YUY2, etc.
Info()

I've been burned by that before. If the DS filter chain outputs RGB, this can actually be a YV12=>RGB=>YV12 conversion

The reason I wanted to see Info() was to check what the default DirectShow configuration was sending. But I forgot to say that he should check both x86 and x64, because it he only checks x86, but x64 is used for the directshow path for his device wrapper, then it's meaninless

(But AVISource with UT Video Codec ALWAYS needs pixel_type now for YV12 in the newer versions. For some reason it's not automatically communicated and defaults to YUY2. But other codecs like Lagarith auto communicate the pixel type)

[poisondeathray]

ok here is another option , ffplay , from ffmpeg

ffplay "MVI_0003[1].MP4" -vf waveform=scale=ire:graticule=green:flags=numbers+dots

[Attachment 43205 - Click to enlarge]


There should be a way to stack the corresponding video image with the waveform

EDIT: maybe here
https://trac.ffmpeg.org/wiki/FancyFilteringExamples#waveform

And I checked it displays values <0, >100 IRE with full range video


When you have a chance I would try to get avisynth working , maybe with lsmash. Avisynth is a very handy tool, especially with avspmod

[chris319]

You guys are going way too far astray.

I didn't say anything about YUV. I don't know where you got that notion. You're confusing yourself.

The video delivered to my scope program is ultimately RGB, and apparently it's 0 - 255 and doesn't match what the camcorder is recording. What gets recorded internally in the camcorder is a black box to me. All I know is it's 16 - 235.

You can conjecture all you want about my particular DirectShow setup, etc. but it's one use case out of millions. We know that escapi interfaces to DirectShow, which AFAIK is a standard-stock Microsoft Windows 10 setup.

My scope program is read-only. It doesn't alter levels and doesn't write them to anything other than the screen display.

All I have to do is adjust my scope program to accomodate 16 - 235 and it will match what gets recorded.

When you have a chance I would try to get avisynth working

Cripes, I pissed away half of Saturday doing just that, with nothing to show for it. You're telling me to run Info() with no arguments and avisynth throws an error due to no arguments. I've been working with avisynth for all of two days and the documentation is a train wreck, so I wind up going down the garden path.

[jagabo]

avisynth[/url] working , maybe with lsmash.

Yes, use lsmash or even the old ffmpeg source filter. DirectShow is unpredictable and often not frame accruate.

http://avisynth.nl/index.php/LSMASHSource
http://avisynth.nl/index.php/FFmpegSource

For MPEG 2 video use DgMpegDec.

http://rationalqm.us/dgmpgdec/dgmpgdec.html

[poisondeathray]

What gets recorded internally in the camcorder is a black box to me. All I know is it's 16 - 235.

It's YUV. You uploaded the original video.

But your camcorder probably records Y 16-255, not Y 16-235. It's in the "90% category" I mentioned before. If you look at the waveform in jagabo's post #31, there are "illegal" levels in the >235 area. You can do some bright exposure tests and and see how much is actually usable

But the live feed through HDMI is usually YUV in most camera, but I never checked your model , so that is an assumption

Cripes, I pissed away half of Saturday doing just that, with nothing to show for it. You're telling me to run Info() with no arguments and avisynth throws an error due to no arguments. I've been working with avisynth for all of two days and the documentation is a train wreck, so I wind up going down the garden path.

Trust me, I know how you feel x10 . Go back a few years and read my complaints. I had to revisit it many many many times. There are a gazillion "gotchas" and little quirks. But it's well figuring it out eventually. Very useful tool

[chris319]

Black level is a little above Y=16. The lowest Y value I saw is 15, the highest 28. Most pixels are between 18 and 22.

I just looked at the file "lens cap raw.mp4" using the ffplay scope. Sorry, the video noise floor with a capped lens is roughly at digital 0. It is well below 16. This agrees with what my own scope program sees out of the HDMI output. I can vouch for my scope program because I wrote it and I know what the calculations are. So no, it is not putting out one set of levels for HDMI and recording another. It is not putting out broadcast-legal 16 - 235 which is par for the course for a consumer camcorder.

See for yourselves:

bin\ffplay "lens cap raw.mp4" -vf waveform=filter=lowpass:scale=ire:graticule=green:flags=numbers+dots

[poisondeathray]

Black level is a little above Y=16. The lowest Y value I saw is 15, the highest 28. Most pixels are between 18 and 22.

I just looked at the file "lens cap raw.mp4" using the ffplay scope. Sorry, the video noise floor with a capped lens is roughly at digital 0. It is well below 16. This agrees with what my own scope program sees out of the HDMI output. I can vouch for my scope program because I wrote it and I know what the calculations are. So no, it is not putting out one set of levels for HDMI and recording another. It is not putting out broadcast-legal 16 - 235 which is par for the course for a consumer camcorder.

See for yourselves:

bin\ffplay "lens cap raw.mp4" -vf waveform=filter=lowpass:scale=ire:graticule=green:flags=numbers+dots

Are you looking at the correct video ?

Your lens cap video. Notice everything is above IRE 0, or Y=16 . There might be some stray pixels slightly below, from compression artifacts (it's a lossy recording after all)

[Attachment 43212 - Click to enlarge]

This is a true full range with "well below digital 16". Notice it goes below IRE 0, and above IRE 100. So below Y=16 and above Y=235

[Attachment 43213 - Click to enlarge]

[jagabo]

I just looked at the file "lens cap raw.mp4" using the ffplay scope. Sorry, the video noise floor with a capped lens is roughly at digital 0.

No, it is at IRE 0. URE=0 is Y=16, IRE=100 is Y=235.

This agrees with what my own scope program sees out of the HDMI output. I can vouch for my scope program because I wrote it and I know what the calculations are.

You're using the wrong equation.

If you're such a video professional you need to learn what IRE and YUV are.

[chris319]

Ah yes, the scope is calibrated in IRE, not digital. My bad.

You're using the wrong equation.

If you're such a video professional you need to learn what IRE and YUV are.

If you're going to be rude and insulting then I must ask you to refrain from participating in this discussion.

[chris319]

Disregard.

[manono]

If you're going to be rude and insulting then I must ask you to refrain from participating in this discussion.

Hold on there. You've been discussing this with two of our most knowledgeable and polite forum members. You're the one that trumpeted your credentials but you're in our world now and you've wasted their time with your incorrect statements more than once and both have been very polite in setting you straight. Is it any wonder one of them finally scolded you mildly? As I see it, you had it coming. If there's an apology to be made, it should come from you.

[chris319]

I owe a debt of gratitude to poisondeathray who has been very helpful, kind and patient while I learn the ins and outs of ffmpeg.

So thank you, poisondeathray.