Artifact cleanup

[killerteengohan]

I have a source that I am trying to clean and not harm too much. I have tried using 3 different types of denoising. mctemporaldenoise, tnlmeans, and deen. They all help and combining 2 of them helps more, but no matter what combination I try, I cannot seem to clean this up well enough.


Here are a couple example videos.

No cleaning

https://mega.nz/file/l0piGK4Z#5BV0Cb66TXq2ATyBhTLYMsmTK0QV9jkMoSEhAgI74NM

crop(4,4,-4,0)
LanczosResize(848,480) # Lanczos (Sharp)
dehalo_alpha(rx=1.1, ry=1.1, darkstr=0, lowsens=0, highsens=100, ss=1.0)
mergechroma(awarpsharp2(depth=3, blur=0, type=1))
LimitedSharpenFaster(ss_x=1.00, ss_y=1.00, strength=2, overshoot=0, undershoot=0, soft=0, edgemode=0)

mctemporaldenoise cleaning (This one looks best to me overall, so far)

https://mega.nz/file/90QnXAZZ#NVZ-qkWDG1Y9uf75zmoFTEJ_A0pE-ufMCnknYLxzlBA

crop(4,4,-4,0)
LanczosResize(848,480) # Lanczos (Sharp)
dehalo_alpha(rx=1.1, ry=1.1, darkstr=0, lowsens=0, highsens=100, ss=1.0)
mctemporaldenoise(sigma=2, sharp=false, radius=1, ecrad=1, AGstr=0.6, GFthr=1.00)
mergechroma(awarpsharp2(depth=3, blur=0, type=1))
LimitedSharpenFaster(ss_x=1.00, ss_y=1.00, strength=4, overshoot=0, undershoot=0, soft=0, edgemode=0)

tnlmeans + mctemporaldenoise cleaning

https://mega.nz/file/MxI1EKga#kuQxKB8WQiFeiAXCD64wMVmCAy6WdeFnYNj0kS0ih5g

crop(4,4,-4,0)
LanczosResize(848,480) # Lanczos (Sharp)
dehalo_alpha(rx=1.1, ry=1.1, darkstr=0, lowsens=0, highsens=100, ss=1.0)
mctemporaldenoise(sigma=2, sharp=false, radius=1, ecrad=1, AGstr=0.6, GFthr=1.00)
source = last
emask = mt_edge(mode="cartoon", thY1=2, thY2=2, chroma="-128")
TNLMeans(ax=2, ay=2, az=1, sx=3, sy=3, h=1.1)
Overlay(last, source, mask=emask)
mergechroma(awarpsharp2(depth=3, blur=0, type=1))
LimitedSharpenFaster(ss_x=1.00, ss_y=1.00, strength=4, overshoot=0, undershoot=0, soft=0, edgemode=0)

No matter what I seem to try, I just cannot get rid of the leftover moving artifacts on the guys jacket. Even when using overkill like the tnlmeans + mctemporaldenoise you can still see little pieces moving around and to me that's more annoying than when they aren't cleaned and the entire jacket looks filthy.

Any suggestions for cleaning those little leftover pieces a bit better without taking much more detail out from the overkill of DNR? Maybe a better filter or filter settings I am unaware of perhaps?


Source video demuxed

https://mega.nz/file/E0ZkiA7A#DfQVwxE25MSR_akgmRhjwC3v7yEkP01eOexbf-FpK_I

[killerteengohan]

I just noticed, those little moving artifacts do not show up on the preview window when I preview it before it's encoded. Could this be from a codec setting I am using? The preview window looks much better to me than the encode.

Comparison Image (Preview Windows vs Encode)
https://slow.pics/c/vWtDpvN5


This is the script for the encode example and preview window

crop(4,4,-4,0)
LanczosResize(848,480) # Lanczos (Sharp)
dehalo_alpha(rx=1.1, ry=1.1, darkstr=0, lowsens=0, highsens=100, ss=1.0)
mctemporaldenoise(sigma=2, sharp=false, radius=1, ecrad=1, AGstr=0.6, GFthr=1.00)
mergechroma(awarpsharp2(depth=3, blur=0, type=1))
LimitedSharpenFaster(ss_x=1.00, ss_y=1.00, strength=4, overshoot=0, undershoot=0, soft=0, edgemode=0)

[jagabo]

You can crank up the radius for TMLMeans() But that slows it down and blurs other stuff more. If this is the only shot you care about you could use a mask to apply the extreme settings to just the grey suit.

[killerteengohan]

You can crank up the radius for TMLMeans() But that slows it down and blurs other stuff more. If this is the only shot you care about you could use a mask to apply the extreme settings to just the grey suit.

Any comment on my second post? Why would the preview and encode end up so different looking? Is it a codec setting perhaps causing this? Look at that example. I don't even have to worry about cleaning these artifacts if that can get answered.

The AVISynth preview window looks nice and since nothings been cleaned spotless. It's all nice and even grain, and the artifacts aren't so noticeable. The encode looks like it has tried to clean/smear some pixels spotless, and left other pixels alone, causing those moving specks/blocks to appear.
https://slow.pics/c/vWtDpvN5

I tried cranking up the bitrate to 6800kbps and they are less visible, but are still there in the finished encode, so I do not think this a low bitrate issue.

Here are the codec settings I am using.

https://forum.videohelp.com/images/imgfiles/d8k6zc6.png
https://forum.videohelp.com/images/imgfiles/xkEJXuW.png
https://forum.videohelp.com/images/imgfiles/P0V7sqD.png

[killerteengohan]

nvm, I think it might be a bitrate issue afterall. I tried a lossless CRF and again the artifacts were still noticeable, but it was way better and closer to the preview. They were almost non existant. I guess no matter what settings you choose, you gotta put up with a bit of degradation.

I cant use a high enough bitrate for the lossless because it will be too large, so I guess I'm out of luck here and will have to deal with it.

Oh well, thanks for the suggestion.

[jagabo]

Yes, posterization artifacts like that are what happens when there isn't sufficient bitrate to retain grain. 8 bit YUV isn't sufficient for perfectly smooth gradients. 10 bit encoding and playback may help.

[killerteengohan]

Yes, posterization artifacts like that are what happens when there isn't sufficient bitrate to retain grain. 8 bit YUV isn't sufficient for perfectly smooth gradients. 10 bit encoding and playback may help.

Is encoding an 8-bit source into 10-bit really going help that any? That sounds like trying to put detail that's not there to begin with. All I would really be doing is putting an 8 bit video source into a 10-bit encoded container, right? Wouldn't it still be 8-bit either way?

[jagabo]

It's well known that 10 bit encoding of 8 bit sources can deliver better quality. In part because deblocking of the 10 bit video can have smoother gradients.

[killerteengohan]

It's well known that 10 bit encoding of 8 bit sources can deliver better quality. In part because deblocking of the 10 bit video can have smoother gradients.

Nice to know. Here is something else I hope you can fill me in on.

You know how PC and TV levels are different? PC is full 0-255 and TV is generally 16-235/245. Most older TV's are 8-bit I believe. Can a new UHD 4K TV with 10-bit colors capability display full PC levels properly? If I hook my computer up to it and try to use it as a monitor, will I get the full 0-255 range of colors in my display, or will it still be limited because it's a television?

[jagabo]

8 bit YUV is normally limited range, 16-235, 16-240. On conversion to RGB for display, on TV or on a computer, it should become full range RGB (0-255). So there's shouldn't be any difference between watching on TV or on a computer if both are set up properly. 10 bit displays have the same issue -- limited range YUV is converted to full range RGB. But the granularity is better. Ie, the 10 bit display has 3 levels between each of the 8 bit levels.

[killerteengohan]

I understand it better now thanks to how you explained it.

Will it be able to display properly if the YUV is outside of 16-235? Say an encode is 4-249 YUV for example. I assume an older 8-bit TV would clamp/crush it to fit in legal RGB values. Would the new 10-Bit UHD TV be able to display it without clamping/crushing anything, or will it have to clamp it too? I'm just curious because supposedly 10-bit HDR TV's are supposed to have lots more colors it can display, and I'm trying to grasp its display capabilities better.

[jagabo]

I understand it better now thanks to how you explained it.

Will it be able to display properly if the YUV is outside of 16-235? Say an encode is 4-249 YUV for example. I assume an older 8-bit TV would clamp/crush it to fit in legal RGB values. Would the new 10-Bit UHD TV be able to display it without clamping/crushing anything, or will it have to clamp it too? I'm just curious because supposedly 10-bit HDR TV's are supposed to have lots more colors it can display, and I'm trying to grasp its display capabilities better.

The 10 bits don't get you a wider gamut of colors, just more in-between colors. For example, here's an RGB image with 6 bits of each primary (be sure to view the full size images):


[Attachment 52814 - Click to enlarge]

Compare that to a RGB image with 8 bits of each primary:


[Attachment 52815 - Click to enlarge]

As you can see, the overall range of colors is the same. But the 8 bit image has more in-between colors. The 6 bit image can only encode 64 different shades of each primary (and you can probably see every transition on your monitor), the 8 bit image can display 256 shades of each primary. A 10 bit image would cover the same range of colors but would have even more intermediates colors, a total of 1024 different shades of primary.

8 bit RGB represents about 16 million different colors and you can barely see the difference between adjacent colors. But the limits of 8 bit YUV mean it represents many fewer colors. YUV (more properly called YCbCr) is a rotation, translation, and scaling of the RGB colorspace:


from: https://software.intel.com/en-us/node/503873

The outer cube represents all 16 million possible YUV combinations. The inner cube represents the portion of that cube that lead to valid RGB colors. You can see that the inner cube occupies only about 1/6 of the YUV cube. So YUV can only represent about 1/6 as many colors, about 2.7 million different colors. This is why posterization is more of a problem with 8 bit YUV. Using 10 bit YUV, those 2.7 million different colors become about 21 million different colors. Those colors cover the same gamut but have better granularity -- just like the RGB images above cover the same gamut but with different granularity.

That said, 10 bit panels often support a wider gamut of colors too. But that's not because of the bit-ness, but rather because of other properties of the display.

[killerteengohan]

Well my understanding of how HDR 10-bit displays work has been completely changed now. Thanks for the detailed information.

[jagabo]

Note that HDR is a separate issue from bit-ness. HDR provides for a wider gamut.

[Cornucopia]

And with a wider gamut, you theoretically will need MORE bitdepth to properly cover them all.

Scott

[jagabo]

And with a wider gamut, you theoretically will need MORE bitdepth to properly cover them all.

Yes, they usually go together. But strictly speaking it's not necessary to use more bits. You could provide a wider gamut with 8 bits too. But then you would still have limited granularity.