How to Fix Crushed Blacks?

[VideoFanatic]

When I play a standard definition video from my Sky satellite receiver on my TV or Elgato Game Capture HD via Scart,Composite or S-Video I think I get crushed blacks if that's the correct term. Normally the brightness looks fine but if someone's wearing dark colours then they look completely dark with no detail that I can see. The problem is even there on HDMI but it's not as bad. I can change the capture brightness so both S-video and HDMI give the same level of detail but they still seem to looked crushed.

Is there anything I can do to make the detail in dark areas visible?

I've uploaded a few identical clips.

• S-Video Default Brightness as Source.

• HDMI Sample Default Brightness as Source

• S-Video Changed Brightness to +30 on Elgato Capture

• S-Video Default Encoded to +40 Brightness

• Elgato Capture with Bright +30, Cont-20 (This does NOT have completely crushed blacks so you have something to work with

Here's the script I use on the S-Video source:

AssumeTFF() 
Tweak(Bright=40, Sat=0.7, Cont=1.0, Hue=0.0, Coring=False)
Yadif(mode=1, order=1)  
MergeChroma(awarpsharp2(depth=25))
Crop(16,2,-20,-8) 
Spline36Resize(720,576)
SeparateFields() SelectEvery(4,0,3) Weave()

[manono]

S-Video Default Brightness as Source.ts - Hopeless, blacks completely crushed
HDMI Sample Default Brightness as Source.ts - Not too bad. Needs brightness raised a little bit and contrast lowered a little bit.
S-Video Changed Brightness to +30 on Elgato.ts - Whites badly blown, blacks almost okay.
S-Video Changed Brightness to +30 on Elgato Encoded.m2ts - Contrast way too high (blown whites), blacks not too bad (maybe lowered a bit).
S-Video Default Encoded to 40 Brightness.m2ts - The worst of all because you took a ruined source and then wrecked it some more. The blacks are now grey and the whites badly blown.

You can't adjust both the brightness and contrast before capturing? You're using a histogram to show you the way? Try this as well and it'll help:

ColorYUV(Analyze=True).Limiter(Show="Luma")

And I have no idea what part, if any, the studio lights play. I don't think his white shirt is supposed to be blown out, even if it's okay for the lights themselves to be blown out.

Is your computer monitor calibrated decently? Even if it is you shouldn't go purely by your eyes on something like this, I don't think.

[VideoFanatic]

S-Video Changed Brightness to +30 on Elgato.ts - Whites badly blown, blacks almost okay.
S-Video Changed Brightness to +30 on Elgato Encoded.m2ts - Contrast way too high (blown whites), blacks not too bad (maybe lowered a bit).

Those clips are the same except for the encoded video where I just changed the saturation and stretched the picture to the correct proportions.

What is the purpose of that script you gave me? I don't know how to use the histogram.

I don't know how to calibrate my monitor but it's working correctly. I just adjust it until it looks right. I can watch and capture HD sources fine with no problems and I can watch VHS to DVD conversions with no problems. I watch VHS-to-DVD stuff on the "Natural" mode on my TV at default settings. I notice that some HD stuff and some SD digital satellite shows have crushed blacks so I have to use the "Dynamic" mode on my TV and then it plays correctly. I think this is because my cheap Hitachi TV is from 2007 and it doesn't support Deep Blacks (if that's the correct term?).

To see if the broadcast video is the problem or the Elgato, I watched the source show on my receiver on my TV in Dynamic mode. Looked fine. I think the whites are slightly crushed in the broadcast. I then tried to record composite without adjusting the brightness/contrast on the Elgato since I can record that directly into the Elgato without going through a SCART to S-Video conversion box (in case it was that box causing the issue). I then re-encoded with Avisynth and played on my TV and it's definitely more crushed compared to the satellite source when played on my TV. So it seems the Elgato Game Capture HD crushes blacks unless you change the brightness/contrast on the Elgato. Very annoying.

I did notice this problem several months ago but I used it to record some motorsport races and I simply added 20 brightness when re-encoding and it looked fine to me. I guess it looked fine because there were no dark colours on that.

I thought I'd be better off not adjusting the capture brightness/contrast during capture and instead fixing issues when I re-encode with Avisynth. I guess I was wrong! So I guess I'll be using +20 brightness, -30 Contrast on the Elgato in future. But is there anything I can do for the videos with crushed blacks as I can't re-record them as I deleted them off my receiver.

I've attached a clip I recorded at +20 Brightness and -30 Contrast. Could you let me know if it looks correct to you on your TV in the normal viewing mode you watch on. Do you watch in Natural Mode?

[manono]

S-Video Changed Brightness to +30 on Elgato.ts - Whites badly blown, blacks almost okay.
S-Video Changed Brightness to +30 on Elgato Encoded.m2ts - Contrast way too high (blown whites), blacks not too bad (maybe lowered a bit).

Those clips are the same...

And so were my comments nearly identical. I just had a look at those two at different times and didn't note one was the encoded version of the other until afterwards, and I didn't feel like changing my comments to make them identical.

What is the purpose of that script you gave me?

If you applied it to your samples, the 'purpose' would jump out at you. Based on your first post you were judging everything with your eyes.

I think I get crushed blacks

You think? You don't know? You don't know how to find out? This is why I mentioned that going only by your eyes isn't such a good idea.

I don't know how to use the histogram.

I'll assume you know how to add the line:

Histogram()

to your script. You're saying you don't know how to interpret what you see? Learn about it on the Histogram page. And read the Limiter page and the ColorYUV page to learn what those commands in that line I gave you mean.

But is there anything I can do for the videos with crushed blacks as I can't re-record them as I deleted them off my receiver.

That S-Video capture was pretty awful but yes, you can always fix as much as can be fixed as long as you know how to use the tools designed for that use.

I haven't had a chance to look at the new sample yet. Later, maybe. Others will come along, too, I expect.

Edit later: I've seen the Elgato Capture with Bright +20, Cont-30.m2ts now. I'd boost the brightness a little bit and lower the contrast a little more. And maybe add some YLevelsS to brighten up the whole thing. The guy on stage, but especially the audience, seems a little dark to me. But a lot of this is a judgement call.

[jagabo]

I've attached a clip I recorded at +20 Brightness and -30 Contrast. Could you let me know if it looks correct

The black level is about right. It's hard to say if the bright level is right because there are so few brights in the picture. And since it's way oversharpened the small bright peaks are overshoots, not reflective of the real bright level. But the picture does look dark, overall. Turn the sharpness setting down on your capture device or in the software you processed with.

[johnmeyer]

[QUOTE=VideoFanatic;2418050]

I don't know how to calibrate my monitor but it's working correctly.

Well ... it probably isn't.

"Working correctly" probably means, to you, that the picture is there, is not tilted or distorted, rolling etc. However, if the monitor is not calibrated, and if you in any way adjust levels or colors to "look right," you are going to create a HUGE mess because you are going to be correcting problems that actually aren't there, and instead are going to be compensating for the calibration errors in your monitor. Put another way, you are going to "bake in" your monitor's calibration errors, forever, into your video. Of course if you don't make any color or levels changes, then the monitor calibration will not matter.

To calibrate your monitor, there are simple software-only tools that will help you get the white a black points set more or less correctly.

software only monitor calibration

A good consumer-level colorimeter, like the Spyder will take you one step further and will also help get your colors correct.

The already-mentioned histogram is also an important tool, and it is easy to learn how to use, and you should spend the necessary five minutes (or less) to learn what it means and how to use it. Google it, and you'll find some tutorials. With the histogram, even with a badly calibrated monitor, you can see if your whites and blacks are indeed out of whack, or if everything is OK.

[johnmeyer]

P.S. This image shows a Waveform Monitor view of the levels for your "S-Video Default Brightness as Source.ts" sample. I actually find the waveform display more useful than a histogram for monitoring and correcting luma levels. Most software that can generate a histogram can also generate a waveform. As you can see, the "0" level is indicated by the vertical red arrow, and the other arrow points to the blacks in his shirt. Almost all the shirt levels are below the zero level and are therefore "crushed:" all detail has been lost, forever. You cannot increase the gain and recover detail.

The levels on most of your other samples do not show this, although there are various shifts, up and down in these other samples, some of which are probably not correct.

This display is easy to understand. The left to right corresponds with the left to right in the video itself. The vertical indicates the brightness (luma) value of every pixel in the vertical column of pixels at that location. Thus, to the left and right of center each column of round lights show in the display as big vertical humps with maximums around 25-30, with just a little bit of much brighter pixels (around 60) which come from the bright dot in the middle of each bulb. A brighter display at any location indicates that lots of pixels in that vertical column have that same luma value.



There is another, "next level" discussion about what levels of 0 and 100 mean, but you need to get started down this path before we start worrying about that.

[VideoFanatic]

OK I've done another capture at Bright+30 and Contrast -20. This seems to give the best detail with brightness at as close to correct levels as possible (that I can do in the capture). I've attached it to this thread. I'm none the wiser about how to use a Histogram or Waveform Monitor. I can't find any tutorials. I'm using Avisynth and AvsPmod to preview my script. Could somebody please tell me how I can improve the clip I uploaded and is there a way to bring the brightness of the whites down in his shirt without lowering the brightness/contrast of the rest of the video?

Would be helpful if you could give me a script please and info on how you came to find out that info. Perhaps then I can understand the Histogram/Waveform easier.

[johnmeyer]

Simply posting more clips really doesn't help: we've already seen some good and bad levels, and have tried to offer help.

The best advice I can give you at this point is to not alter the video levels during the capture process.

As for tutorials, you really need to do that research yourself. As the risk of sounding snarky and sarcastic, but actually trying to be helpful, did you try using Google and then reading some of the posts that result from that search? If you aren't sure how to use Google, here is a search you might try. Please click on this link:

video histogram waveform tutorial

[VideoFanatic]

P.S. This image shows a Waveform Monitor view of the levels for your "S-Video Default Brightness as Source.ts" sample. I actually find the waveform display more useful than a histogram for monitoring and correcting luma levels. Most software that can generate a histogram can also generate a waveform. As you can see, the "0" level is indicated by the vertical red arrow, and the other arrow points to the blacks in his shirt. Almost all the shirt levels are below the zero level and are therefore "crushed:" all detail has been lost, forever. You cannot increase the gain and recover detail.

The levels on most of your other samples do not show this, although there are various shifts, up and down in these other samples, some of which are probably not correct.

This display is easy to understand. The left to right corresponds with the left to right in the video itself. The vertical indicates the brightness (luma) value of every pixel in the vertical column of pixels at that location. Thus, to the left and right of center each column of round lights show in the display as big vertical humps with maximums around 25-30, with just a little bit of much brighter pixels (around 60) which come from the bright dot in the middle of each bulb. A brighter display at any location indicates that lots of pixels in that vertical column have that same luma value.



There is another, "next level" discussion about what levels of 0 and 100 mean, but you need to get started down this path before we start worrying about that.

I used histogram("mode=classic") in AvsPmod which adds a Waveform monitor. Here's a screenshot:


However I don't see the numbers on my graph so how do I know what levels are correct or not?

Is there anything I can do to lower the brightness of the whites on the shirt without effecting the rest of the clip?

[johnmeyer]

Your AVSPmod waveform display works in the vertical direction, rather than horizontal direction shown in my example. That's OK, and it would still be useful if only it had some calibration information. However, without a scale, it is pretty much useless, as I think you've already figured out. I did a quick scan of the documentation, and it appears this was added as an afterthought, after the developer received some requests, but he never really put his heart into it.

As for your specific request, to change just the luma values that fall within a given range, you typically want some sort of histogram curve tool. What such a tool does is to map each luma value to a new luma value. A diagonal line makes no change at all, which is what is happening to the dark through intermediate brightness pixels in the screenshot below (note that the lower left quadrant of the display is a perfect 45 degree diagonal line, so the brightness of the darker pixels are left completely unchanged). This snapshot shows the histogram feature working on your best capture, and reducing only the really bright pixels.



Another way to do this is to use a tool which lets you specify specific ranges of luma and chroma values, and then modify any pixels that satisfy these conditions. This is the tool that lets you, for instance, replace one color with another. Here is an example of what this looks like, using the tool in Sony Vegas Pro:

[LMotlow]

Blue is oversaturated.
Gamma is too low.
You can control high brights with Levels. Try Levels(16, 1.0, 255, 16, 245). SmoothLevels would be better.
Raise gamma.
Reduce blue saturation.
Raise red saturation.
Tweak in RGB.
Use full-color waveforms or parade histograms to check individuals colors.
Calibrate your monitor.
That suit in the image in post #10 looks like neon blue velour. And, yup, some dark blues are crushed.
Visit some free Photoshop tutorials and learn color and tone theory, or you'll go bonkers.

[manono]

Because his hair is bluish, I think he probably has a blue spotlight shining from above and his suit could really be black, for all I know. Or maybe not quite that 'purply' shade of blue. Trying to change the colors in that scene might throw off the colors in others. One would have to see the entire thing to be sure, I think.

[LMotlow]

The "real" colors area tough call. Color in all the captures looks pretty weird. Then add posterizing in the Elgato caps and even skin tones look screwy. It could just be the broadcast for all we know. The long captures that have views of the audience look weird, too, with that side shot of the audience applauding looking like orange neon. I think everything in the originals is over saturated besides being too contrasty.

Note in next to the last shot in HDMI Sample Default Brightness as Source.ts from post #1 that there are blue overhead backlights behind the guy onstage. Colors and lighting look, mmm, what we would call "tacky" down south where I come from. But a lot of copycat talk shows look that way.

[jagabo]

Your AVSPmod waveform display works in the vertical direction, rather than horizontal direction shown in my example. That's OK, and it would still be useful if only it had some calibration information. However, without a scale, it is pretty much useless

The brown areas show Y<16 and Y>235. You want to keep blacks and brights out of there (except occasional overshoots). If you want a more traditional horizontal waveform use:

TurnRight().Histogram().Turnleft().

That will give you a horizontal waveform graph above the video. If you want a horizontal histogram on the bottom use:

function HistogramOnBottom(clip vid)
{
  TurnRight(vid).Histogram().TurnLeft().Crop(0, 0, -0, 256).ColorYUV(off_y=-16).ColorYUV(gamma_y=100).ColorYUV(off_y=16)
  StackVertical(vid, last)
}

That also bumps up the gamma of the Histogram to make darker dots more visible.

[LMotlow]

And you could see saturation, overshoot, and nonlinear response problems with individual colors by using these histograms?

[jagabo]

You can check for out of bound U and V values with:

UtoY().Histogram()
VtoY().Histogram()

Histogram() has a true histogram mode:

Histogram(mode="levels")

And there are color vectorscope modes:

Historgram(mode="color")

Historgram(mode="color2")

Many combinations of YUV are illegal even if all three components are within their individual legal ranges. I wrote a quick/dirty function that shows those:

HighlightBadRGB()

https://forum.videohelp.com/threads/360935-Capturing-Correct-Chroma-and-Hue-Levels-From...=1#post2289672

That assumes rec.601. For rec.709 you'll have to change the ConvertToX() functions to use the rec.709 matrix. Earlier in that thread is a link to a post by Gavino at Doom9 which performs a more precise calculation but it's excruciatingly slow to start up. And also in the thread I posted an animation that shows all legal YUV values as slices along the Y axis.

Another useful filter is VideoScope(). I prefer that to Histogram(). but it's a little more complex, it doesn't highlight the illegal ranges (but is has tick marks), and you have to convert to YUY2() to use it. Some examples of VideoScope:

https://forum.videohelp.com/threads/360935-Capturing-Correct-Chroma-and-Hue-Levels-From...=1#post2289606

Of course, none of that tells you if the levels and colors are "right", just that they are within the legal ranges. You have to use your eyes and judgement and have a monitor that's reasonably well adjusted.

[LMotlow]

Played with the HDMI default capture this A.M. until I got sick of the damn thing. You can make the darks brighter if you want but they'll just look washed out with no detail gain. The miserly 6K bitrate didn't help. That crushed suit is pretty murky and noisy with the kind of discoloration usually seen in crushed colors (here, it was magenta blobs). I used SmoothUV and TemporalSoiften to clean it a bit. I don't know how the capture is set up, but AAC audio ain't valid for DVD/BluRay/AVCHD. Fixed levels in Avisynth with ColorYUV and SmoothLevels, then color and whatever else in RGB. You can make it look however you want. I didn't have the patience to cool line twitter. It's a mess.

[LMotlow]

Or if you want (WTH) you can tame RGB blue to nearly nothing and make the suit go black. But...other things look kinda kinky (like skin tones), the suit still has blue haze sometimes, seems to want to go green, has even less detail, and looks blotchy. I guess we'll never know.

[manono]

Hehe, and his hair's not blue any longer.

[vhelp]

Broadcasters screw up color levels all the time, and its not always possible to fix. Sometimes you get lucky and get close to what the levels should be. Anyway.

I have two good sources you can use as a reference for adjustments and comparisons:

Disney airs the Star Wars sneak peeks about twice an hour - This content has a lot of "black" in it.
Disney airs the Austin & Ally, episode** "Future Sounds and Festival Songs 2013" - This content has all "white" color level in it.
(both contents are great for measuring and fine-tuning your capture equipment)

** if you have both disney channels East and West, you can catch the next showing tonight around 12am, and you can capture both the Star Wars and A&A for testing.
The Star Wars is in the first commercial break, around first 10 minutes.

Since my directv dvr records in realtime, I get the first generation copy, thus recording it and capturing from the playback of the recording is always the same as the real-time airing of the show no matter how many times I may capture from it. That is what I love about this unit. I never have to worry about 2nd generation color and quality loss.

Anyway. I have captured these two contents multiple times with the Hauppauge HDPVR, the Pinnacle Studio AVDV, and the Elgato game capture box. If you can capture this two contents, maybe we can compare them. I will leave that up to you.

Oh, one more thing.. laptops are horrible at color reproduction. I would not recommend using it as a gauge in your analizing. Best to do that on a desktop with a real monitor. I have the HP Inspiron and its color is washed out, horrible for video work.

[johnmeyer]

Oh, one more thing.. laptops are horrible at color reproduction. I would not recommend using it as a gauge in your analizing. Best to do that on a desktop with a real monitor. I have the HP Inspiron and its color is washed out, horrible for video work.

I calibrate my laptops with my Spyder colorimeter. The look pretty good.