Weird artifacts on video removal

[killerteengohan]

A source I am wanting to clean up has very weird artifacts on them. I am guessing its something to do with the Blu-Rays compression they used, I'm not 100% sure.

They resemble lines or cube shapes all over the place and sometimes move around rapidly. Very noticeable on greens.

Here is a screenshot. It can be kind of hard to see on a still image, but if you look, you will see darker lines in the colors.
https://forum.videohelp.com/images/imgfiles/pGxmSHW.png


Here is a unfiltered video sample. You can see what I am talking about much better using the video. If you pause, then play, then pause, then play and look for them, you cant miss them. Once you can see them, you cant unsee them.
https://mega.nz/#!g5p3RZ4Y!rac2OtA-GFeJAjhS5knXQ-zpqrCXt7khYy3z7ES2uiU


Is there anything I can do to remove or cover these up? I have tried really strong DNR and it has no real effect on them.

[killerteengohan]

I darkened the screenshot a lot and now they are very visible if you couldnt see them in the above screenshot.
https://forum.videohelp.com/images/imgfiles/dxSaJRv.png

[jagabo]

TNLMeans(ax=2, ay=2, az=1) will get rid of most of it without damaging the rest of the picture too much. KNLMeansCL() would be much faster but it doesn't run on my old computer.

[killerteengohan]

Thanks, I will give that a try tomorrow and see what I think of it. I never heard of it before.

I thought of maybe dithering it to see if that would help cover it. The default debanding in the mctemporaldenoise DNR filter not only reduced other artifacts, but the dithering it does for debanding (GFthr parameter) cleaned those annoying artifacts that were bothering me, pretty nicely without hurting detail barely at all. Have a look.

https://slow.pics/c/vB90GagQ

Its not 100% perfect, but it does do well enough to be acceptable.

[killerteengohan]

I gave the TNLMeans a try. It did indeed clean those artifacts better than the dithering method I had, but it smeared out quite a bit of the background and image detail. I been playing around with different parameter settings to see if I can protect the background detail better.

Would the KNLMeansCL be any better? or is it just a faster filter that is not as good?

[jagabo]

As I understand it, KNLMeansCL() is similar to TNLMeans() but it uses the GPU so it's much faster. I don't have a supported GPU so I've never used it.

You can try protecting other areas with an edge mask. Something like

emask = mt_edge(thY1=1, thY2=1, chroma="-128").BinomialBlur(3.0)
Overlay(filtered, unfiltered, mask=emask)

will protect almost all areas with real detail.

[killerteengohan]

It doesn't know what filtered means. Should I just change it to last?

[jagabo]

"filtered" and "unfiltered" are symbolic. Replace them with your two differently filtered versions of your video. For example, filtered would be the video before tnlmeans, filtered the video after tnlmeans.

[killerteengohan]

Is the binominal blur in there for a particular reason I am not spotting visually? The overall picture looks sharper without it, and aside from softening the picture, I'm not noticing any specific effect from it that helps the mask.

[jagabo]

The BinomialBlur is there to soften the edges of the mask. With some types of overlays a sharp edged masks can lead to visible demarcations between the two videos. Blurring the mask makes those demarcations less visible. Remove it if you want. Or use a smaller radius.

[killerteengohan]

I tinkered around with the settings for a few hours. It still is a bit more destructive than I would like, but I found a decent acceptable setting that cleans them as much as needed and doesn't go as overboard as the defaults with destruction. Thanks for the mask that helped some too.

source = last
emask = mt_edge(mode="cartoon", thY1=2, thY2=2, chroma="-128")
TNLMeans(ax=3, ay=3, az=1, sx=3, sy=3, h=1.1) # The ax, ay=2 you suggested wasn't enough to get the thicker ones so I raised. The sx, sy, and h were for more detail retention.
Overlay(last, source, mask=emask)

That TNLMeans certainly does clean those artifacts better than any of the other 5 denoisers I have tried. I have not been able to get rid of them by any other means I could think of so far.

A downside I am noticing though is it is extremely slow. It went from 3-4 days to finish, up to about 32 days to finish by using it. (0.08 FPS) That is officially the longest I will have ever waited for something to finish encoding lol. It breaks my top record of 8 days wait by quite a bit.

I will wait it out if I have to, but would there happen to be any other option that can be thought of to try?

[poisondeathray]

tnlmeans is slow and cpu limited, but you can get nice speedup using avs+mt x64. eg. for a normal quadcore expect ~3-3.5x faster

If you have decent GPU, KNLMeansCL is waaay faster, but results are quite different. It's not a drop in replacement

[killerteengohan]

You can see the results in these 3 short video samples if you want.
1. Original video artifacts
2. Attempt to cover with dithering
3. TNLMeans with mask

https://mega.nz/#!0oZRXIDJ!phu1yMSDGSL3Qkmf-Qx_IcRIDB1eGI6-J1Yq6SwcJtY


Are those really just a type of noise/grain artifact, and not from a compression method used by the studio that made the disc? I thought it was a type of compression the studio tried. It's not very often that I come across these kind. They arent very common in things I have encoded before.

[poisondeathray]

I don't know what it is; but that sample in the 1st post isn't cut from original blu-ray (bitrate too low, x264)

If it's a representative sample, it's definitely made worse from the 2nd encoding. How many/what kind of the artifacts are in the real original ?

[killerteengohan]

Its encode from the Blu-Ray. I cant give a 20GB M2TS file and I don't know how to give just a small select piece from M2TS files. I use DGIndex for DVD samples but it wont work with M2TS files for me.

[poisondeathray]

You can use tsmuxer or mkvmerge to cut segments from a BD transport stream

I suspect it's like that or very similar in the original too - but adding additional artifacts doesn't help you're trying to "fix" something

[killerteengohan]

My mkvmerge says its an unsupported container format. The tsmuxer seems to work fine however. Thanks!

[killerteengohan]

I don't know what it is; but that sample in the 1st post isn't cut from original blu-ray (bitrate too low, x264)

If it's a representative sample, it's definitely made worse from the 2nd encoding. How many/what kind of the artifacts are in the real original ?

Here is the real original demux (made from tsmuxer you suggested). You can see those artifacts are just as bad in the original.
https://mega.nz/#!VkJFFCLT!WKakIs__uxVtv7KmbbcMnU87XqtYC0fWx5Dy_F9FJGw