Recommended MC denoiser for interlaced and progressive content?

[fvisagie]

Hi All,

My video camera footage can do with some denoising, esp. that shot in low light. The challenge is that encoding is done via a Windows shell script, and some encodes retain interlaced input (when encoding for DVD), some deinterlace it, and others retain progressive input. Ideally I'd like to use a denoiser that does a good job with both interlaced and progressive input, preferably with the same calling syntax.

Is this asking for too much and should I rather get horses for courses, or can it be done?

What plugin/s would you recommend for AVS? VirtualDub would also be OK. Any other suggestions also welcome.

Many thanks,
Francois

[poisondeathray]

"good job" depends on what needs to be done specifically.

A general all purpose one would be MCTemporalDenoise , use interlaced=true to preserve interlaced content (it's set to false by default). You can start with the presets and/or tweak the dozens of settings to customize to your footage
http://avisynth.org/mediawiki/MCTemporalDenoise

[sanlyn]

Avisynth:
MCTemporalDenoise. Settings are variable. Default is interlaced=false, you can set it to interlaced=true if needed.
RemoveSpots() (a version of RemoveDirt, whose plugins you'll need). Use SeparateFields() then Weave() for interlaced, or deinterlace with QTGMC and re-interlace after processing.

VirtualDub:
NeatVideo
CamcorderDenoise
Both can be configured for interlaced or progressive.

For really ugly video, I often have to use both Avisynth and VirtualDub plugins at low to medium settings.

Unfortunately we haven't seen your video, so no one could say for certain.

[fvisagie]

Unfortunately we haven't seen your video, so no one could say for certain.

Here's a somewhat extreme example:
http://www.mediafire.com/file/3yq98m86d2e8q5o/Sony_DCR-TRV330_PAL_DV_Low_light.avi

Thanks for the suggestions so far.

[sanlyn]

Here's a somewhat extreme example:

Took a lot of work (and some very slow plugins!), but I've seen worse. Grain is the obvious problem, but also aliasing, block noise, dot crawl, etc. The settings could be tweaked almost infinitely, and some plugins could be changed, but I started with TemporalDegrain. Be prepared for some slow processing, even if you TemporalDeGrain is the only filter used. Major repairs take time:

Original:

[Attachment 13595 - Click to enlarge]

After:

[Attachment 13596 - Click to enlarge]

Step 1 (save as YV12)

ConvertToYV12(interlaced=true)
AssumeBFF().SeparateFields()
TemporalDegrain()
Weave()

Step 2:

MCTemporalDenoise(settings="medium",sigma=6,AA=true,useEEDI2=false,enhance=true,DeBlock=true,interlaced=true)
daa()
DeBlock_QED()
TTempSmooth()
LSFMod()

Did a little color work in VirtualDub (original looks overly warm). Could also have fired up NeatVideo instead of using some of the others, but didn't have time. TemporalDegrain did most of the prep work.

The possibilities are endless.

[fvisagie]

The possibilities are endless.

Thanks! You've given me a lot of homework, not least of which is learning what all those ...()s you call do!

[sanlyn]

Yeah, this clip is a toughie. Save that for later. Low light compression noise is a total nightmare. As for the plugins, all that I mentioned are in common use and can be found in Google by looking for "Avisynth" and the plugin name. You guessed correctly, the problem videos will take a lot of playing around. Once you find the right combo of plugins, the rest is easy (I use that term with reservations!). If you have problems finding them, let us know.

Here, the grain and block noise weren't that difficult. I kept playing with the clip and had to add some QTGMC (very slow) and some fairly strong NeatVideo (slow, but faster than the others) as a final step on that horrible grunge and black-level mottling every time something moved. Didn't get it 100%, but it "moves" OK in the attached sample. As with any temporal filtering, the first and last frame are still dirty (that's because the time-based filters have no previous or after-the-last frame to analyze).

[fvisagie]

that's because the time-based filters have no previous or after-the-last frame to analyze

That's certainly borne out in my limited experience too, but I find that surprising. Perhaps naively so, but I'm wondering whether it wouldn't be possible to use the motion vectors of the next and previous frames (respectively) to generate fake previous and subsequent frames to work with. In any case, I'm way out of my depth by now and will leave it there .

In this case, what was the main reason for using QTGMC()? I know it's primarily a deinterlacer that inherently does some denoising with additional user-controlled denoising, but what did you use it for here?

Many thanks again, and impressive work with that clip by the way!

[jagabo]

that's because the time-based filters have no previous or after-the-last frame to analyze

That's certainly borne out in my limited experience too, but I find that surprising. Perhaps naively so, but I'm wondering whether it wouldn't be possible to use the motion vectors of the next and previous frames (respectively) to generate fake previous and subsequent frames to work with.

The noise of the last frame would move with the motion vector. So you still don't have anything different to compare to.

[sanlyn]

that's because the time-based filters have no previous or after-the-last frame to analyze

That's certainly borne out in my limited experience too, but I find that surprising. Perhaps naively so, but I'm wondering whether it wouldn't be possible to use the motion vectors of the next and previous frames (respectively) to generate fake previous and subsequent frames to work with. In any case, I'm way out of my depth by now and will leave it there .

You can create another first or last frame with motion vectors from previous frames (it's just another step), but the created frames are too similar to the frames they "copy", so temporal filters won't see a difference. In practice, most video cuts include extra frames for that reason, if they exist.

In this case, what was the main reason for using QTGMC()? I know it's primarily a deinterlacer that inherently does some denoising with additional user-controlled denoising, but what did you use it for here?

QTGMC at "slow" was strictly for smoothing more of the compression noise. Tends to look worse than it really is on interlaced video on a PC monitor, less so when deinterlaced on TV. But I like to keep interlaced source as interlaced for compatibility and archival reasons. I reinterlaced, but one could take the even frames or odd frames (sometimes it looks better, sometimes not), which would make the video 25fps progressive. Didn't have time to try it both ways. NeatVideo is pretty good with such artifacts and did most of the cleanup as a last step -- great plugin for rotten video, but you can't just throw it at everything. For difficult sources, I'd say NV thru a great many projects has repaid its modest cost many times over.

Many thanks again, and impressive work with that clip by the way!

Appreciated. That was a toughie. These make great learning tools, so I'll keep it around. (<<- perpetual newbie, here.)

[fvisagie]

For this project's material QTGMC() and TemporalDegrain() did the trick! Thanks to all for the great help and many lessons learnt.


Edit: for the sake of posterity (and failing memory), here are some interesting results:

• Straight-forward strictest possible MDegrain3: very good denoising, result a tiny little more flat-looking than input
• FastDegrain (MDegrain3 settings modified to be equally strict): very slightly less denoising, too much sharpness initially but can be tuned down to level of input at settings of around 3
• TemporalDegrain (MDegrain3 settings modified to be equally strict): very slightly less denoising still, sharpness great, detail best of the 3. Called with degrain = 3 and SAD2 = 400 (MVTools2 default but TemporalDegrain defaults to 300 and incorrectly documents as 400).

Input footage was PAL DV AVI -> QTGMC() -> Deshaker.

At some later stage I'd like to compare denoising results for this kind of footage with MCTemporalDenoise() which I didn't have time for now.