temporarily changing colorspace levels to denoise

[unclescoob]

I think I'm on to something, but I need additional information (I'm sure it's not a new idea, but anyway)...

I notice that the best denoisers still leave junk behind (or banding) on dark scenes. I know this is an ongoing problem for alot of people. And if I do manage to get rid of all of it...goodbye details and hello ghosts.

I think sanlyn mentioned something like this on another post: If I temporarily change the gamma, for instance, in dark scenes and make it really bright so that all the noise can become visible, then I apply my denoiser, and change it back to how it was, would the denoising be more effective with less side effects?

[jagabo]

I doubt it will help -- unless you're using some particular filter that's particularly sensitive to brightness. You may get less noise (because by crushing the gamma you've got less to work with) but you'll get more banding. Why don't you try it an report what you find.

[sanlyn]

I don't "temporarily" change gamma, levels, etc.. I open a file in YUV and use various histograms and 'scopes to check that darks aren't being crushed and highlights aren't being blown away. The clip's blacks are crushed when they're smashed against the left-hand (dark) side of a histogram, and brights are losing detail when they start climbing the right-hand walls or borders. I adjust those levels with ColorYUV, Tweak, or SmoothLevels, then denoise from there. I expect those adjustments levels to stay that way.

Lowering or raising gamma mostly affects midtones (between about RGB 64 and RGB 192).

[unclescoob]

So blacks are perceived as crushed when they're all smashed together on the dark side of the neighborhood, and whites begin to lose property when they start climbing walls (but on the right side, as the blacks hang out on the left).

I think we're getting closer and closer to a solution here

[sanlyn]

That's the idea. Not just "blacks" and "whites", but all dark or bright colors. For instance, shadows in clothing might be a really dark shade of the clothing, like blue or brown, etc., but detail will start to disappear if they're crushed (beyond the means of reproducing them properly); they blend into a blackish blob that will always look that way, even if you brighten the image later. It might turn a lighter gray or something, but it will just be a dark blob with no features. The same for bright colors, or faces too brightly lit.

Unfortunately you sometimes get your hands on some pr-processed video that already has lost detail in the extremes. Not much you can do about it. Once it gets badly distorted by previous mishandling, it's gone.

[unclescoob]

Well thank you. Now laugh at my racist joke. Go on, let me see an "LOL" in your next post.

[unclescoob]

BTW sanlyn, I posted the clips I promised on the "grid artifacts...which function is causing this?" thread. You asked me to get you the original source sample on that dark scene that I was having trouble with, so it's there for you to examine. Of course, the usual gang can check it out too. I really need help with this. You helped me on this clip a bit, so your advice is appreciated over there!!

Thanks!!

[themaster1]

Try the NLmeans GPU denoiser it uses a new algorithm i find it preserves shapes better compared to fft3d.It's best used with fft3d though according to my tests

NLMeansCL(A=4, S=2, B=1, aa=1.0, h=2.0,HC=3.5, plane=4,cpu=false,buffer=false,sse=true) # YUV
fft3dfilter(sigma=2.0,sigma2=2,sigma3=2,sigma4=2, bt=3, bw=64, bh=64, ow=16, oh=16,plane=3, sharpen=0.0,interlaced=false,dehalo=0.0) # UV

tested on real videos (not animes)

Before denoising there always is much work that can be done with color/levels correction: colormill, CCD, G.curves, Ylevels, Tweak mainly

[sanlyn]

rgemaster1, you should include specific and detailed instructions for loading and running anyhting with "CLMeans" or GPU requirements. Not everyone is using 128-bit gaming machines and 2-GB graphics cards.

[themaster1]

Well i run it with a modest mid range GT220 (1Gb)

If you want to try it download this zip and read the instructions: http://www.sendspace.com/file/akyzgt

[sanlyn]

Modest? What about an ATI X1300 256-MB? 1-GB isn't modest for the average consumer.

Does the plugin work better, or just faster?

[unclescoob]

Here we go again

[sanlyn]

For now, I think CLmeans, etc., will get us off track. 2Bdecided isn't wrong, you get faster processing and have a few more plugin choices available. But from what unclescoob has said earlier about the hardware being used, it might not be feasible.

[Cornucopia]

I think where a lot of those apps and plugins fall short and produce banding is in the mis- or non-application of dither after performing a calculation.

Unless one is leaving the video in FLOATING POINT format, EVERY calculation that produces a figure that can possibly be "between" digits" of the target/working bitdepth SHOULD be dithered.

Any resulting addition of "noise" incurred through the use of dither is minor compared to resulting banding (or worse) that would occur without it.*

Scott

*edit: Caveat: unless you have CRAP to begin with, then all bets are off...

[unclescoob]

Ok, before we continue. What is this CLMeans filter?? Is this an avisynth script like TemporalDegrain?

[sanlyn]

Filters programmed using Microsoft .NET instead of the usual C++/assembler for most Avisynth plugins. Only GeForce 8 series or higher graphics cards support the method used, which is called OpenCL. There are OpenGU/NLMeansCL versions of , I think, FFT3D, and some mothers I donb't recall (RemoveDirtMC ? ?). themaster1 has been using them, he can better inform which plugins use it.

[mammo1789]

Openc CL that is new one i hate cuda (nvidia ) only denoisers and encoders ( i have ati 5870 1gb and geforce 9800gt 1gb for cuda on the same pc )

Sanlyn i think it's time for you to change the computer and step up to the future

4-8gb ram and 1-2 gb graphic cards with dx10/dx11 and quad cores 3-4ghz are very common this days even for the average users.

Where is the download link master i cant seem to open it avira is blocking the site

[unclescoob]

Uh...thanks. Just the answer I was looking for.

[sanlyn]

Sanlyn i think it's time for you to change the computer and step up to the future

4-8gb ram and 1-2 gb graphic cards with dx10/dx11 and quad cores 3-4ghz are very common this days even for the average users.

The average user has 2GB RAM and 256MB graphics, usually built-in. Only overclocking hotrods and kids with terminal ADS get what you describe. It's overkill for video. Spending my $$$ now on building a new HTPC for HD recording off cable. I can edit those on the other three PC's I built for a/v work. Two of those guys have ATI All In Wonders, which I'm not giving up.

Where is the download link master i cant seem to open it avira is blocking the site

Avira ? ? Did those turbo cards cost so much you couldn't afford $40 for a real antivirus?

[unclescoob]

can the mod please block me from this thread?

[mammo1789]

The average user has 2GB RAM and 256MB graphics, usually built-in. Only overclocking hotrods and kids with terminal ADS get what you describe. It's overkill for video. Spending my $$$ now on building a new HTPC for HD recording off cable. I can edit those on the other three PC's I built for a/v work. Two of those guys have ATI All In Wonders, which I'm not giving up.

I was just joking off course I will not try to persuade you, but faster pc with more ram and better G card will do the same job much faster ( just think of denoisers and such and you will know what im talking about ). And the irony is that the newer stuff is also cheaper ( ddr3 is cheaper than ddr2 or even sdram lol ) for some parts. Also if you work with HD material try it with ati all in W and 1gb ram in adobe premiere or sony vegas and tell me how it goes . I don't underestimate the quality of older ati cards ( especially for capturing ) but really evolution is ruthless.
I am not a kid although cant agree that I'm not overclocking hotrod .
i use pc for tv, sat tv, work and video so I'm satisfied with it although have 2 laptops ( pretty powerful also) i like the big black box the most

Avira ? ? Did those turbo cards cost so much you couldn't afford $40 for a real antivirus?

it is avira internet security 2012 and costs around 42$ annually
but you and i know that money isn't everything, is avisynth or vdub worse than those all round suites ( i don't think so )

[sanlyn]

I realize this isn't all that serious, mammo1789. Yeah, the AIW's keep chugging away (four $85 125cfm cooling fans, two in each case with extra vents I made and the filter screens, helps keep up the lifespan -- even if each PC does sound like a turbine when in use. Those are basically for capture. The "main" desktop for most post-processing is a newer, modified job. A lot of $$$ for the IPS monitors, the EyeOne D2, ASUS 1942kb audio, all that. More $$$ for the Adobe Pro stuff and 3 copies of TMPGenc encoders, editors, authoring, for each machine. The upcoming custom home theater job will have some of that and some newer stuff, natch. But the old workhorses, well . . ..maybe I built them too well. It would be like giving up your pets, just to get a new one.

[themaster1]

I've looked at the nVidia website for GC comparisons their latest card has ~1500 cuda cores i can't imagine how fast you can encode/denoise with this beast (mine has ~48) cost an arm and a leg plus an ear but hey

I can't wait for the day they implement 2pass encoding

[mammo1789]

master give some working link for the gpu plugin you are talking earlier i cant enter that

I've looked at the nVidia website for GC comparisons their latest card has ~1500 cuda cores i can't imagine how fast you can encode/denoise with this beast (mine has ~48) cost an arm and a leg plus an ear but hey

You are talking about geforce 680 1,536 CUDA cores/shaders spread out over 8 SMXs ( 192 x8)to be exact

[unclescoob]

You goons never even answered my question.

[sanlyn]

You goons never even answered my question.

You waited 16 months to complain? .
Some have given partial answers.

Longer and better answer: http://forum.doom9.org/showthread.php?p=1386559

If your source has ghosts and banding (almost all of them do), you'll just have to work very carefully and patiently.

[unclescoob]

I admire and take heed to what you say, sanlyn. I don't care what the other monkeys in this site say about you when you're not on.

[jagabo]

You've had 16 months to try it for yourself. No luck? Maybe you could share your experience rather than expect everyone else to cater to you all the time.

[sanlyn]

I admire and take heed to what you say, sanlyn. I don't care what the other monkeys in this site say about you when you're not on.

I'll have to think about that one, but I do take heed from others when they're correct -- and they are, more often than not.

[unclescoob]

Ok jagabo, don't hit me! owwww, that hurt!