Optimising Performance of NLMeans in FFmpeg

[Guanadon]

Hello people,

I wasn't sure where this post belonged so, as a newbie, I decided to drop it here.

As I'm sure anyone who's played with nlmeans in FFmpeg has already discovered, it's painfully slow when running default parameters and the output is often plagued with nasty artifacts. The latter is clearly not supposed to happen, and the former seems a little surprising considering that I'm running an i7-6700K CPU overclocked to 4.5GHz.

A little research revealed the original author Dirk Farin's default parameters, so I decided to play around with similar values until good things started to happen.


The accessible parameters in the current FFmpeg version of nlmeans are as follows...

s - Set denoising strength. Default is 1.0. Must be in range [1.0, 30.0].
p - Set patch size. Default is 7. Must be odd number in range [0, 99].
pc - Same as p but for chroma planes. The default value is 0 and means automatic.
r - Set research size. Default is 15. Must be odd number in range [0, 99].
rc - Same as r but for chroma planes. The default value is 0 and means automatic.


The shorthand method of calling nlmeans and applying these parameters takes the following form...

-vf nlmeans="s:p:pc:r:rc"

After extensive testing, I settled on the following values which are much closer to those suggested by the original author...

-vf nlmeans="1.0:7:5:3:3"

The useful range for strength (s) seems to be 1.0 to 1.8 for 720p video and 1.0 to 4.0 for 1080p.

The speed improvement from using these modified values is huge, and fine detail loss is close to zero whilst still providing significant noise reduction. Compressibility of noisy source material when using x264 in CRF mode can improve by a factor of 2 or more in some cases.

Hopefully, this information will be of help to someone else who's still as hooked on using command-line or batch file driven FFmpeg as I am.

TTFN.

[davexnet]

Thanks for posting your findings.
I've used it a few times in Handbrake through the GUI where you can select strength settings such as light,strong, etc.
I looked in the log and found this, corresponding to a setting of ultra-light. The implementation in Handbrake/Vidcoder is also quite fast

[Guanadon]

You're welcome, and thanks for the additional info.

I believe Handbrake is using a custom version of nlmeans with far more parameters available for fine tweaking for the brave.

My early attempts used matching values for the luma and chroma channels as in that preset, but I found it led to noticeable chroma blurring at higher strength settings, so I reduced the patch size for chroma a little to combat that. I guess this was happening because the chroma channel is at one-quarter the resolution of luma in h.264 source material, so the same number of chroma pixels covers 4 times the picture area compared to luma pixels.

Reducing chroma patch size also gave a small speed boost on top of the huge speed boost of lowering luma and chroma research size from the FFmpeg default of 15 to a more sane 3.

Here's a link to the original work by Dirk Farin. You may find it interesting if you're as geeky as me.

https://www.dirk-farin.net/projects/nlmeans/index.html

[davexnet]

You're welcome, and thanks for the additional info.

I believe Handbrake is using a custom version of nlmeans with far more parameters available for fine tweaking for the brave.

My early attempts used matching values for the luma and chroma channels as in that preset, but I found it led to noticeable chroma blurring at higher strength settings, so I reduced the patch size for chroma a little to combat that. I guess this was happening because the chroma channel is at one-quarter the resolution of luma in h.264 source material, so the same number of chroma pixels covers 4 times the picture area compared to luma pixels.

Reducing chroma patch size also gave a small speed boost on top of the huge speed boost of lowering luma patch size from the FFmpeg default of 15 to a more sane 7.

Here's a link to the original work by Dirk Farin. You may find it interesting if you're as geeky as me.

https://www.dirk-farin.net/projects/nlmeans/index.html

Thanks for the link. I see he makes reference to the Handbrake implementation and a link to the HB
forum where it was discussed:
https://forum.handbrake.fr/viewtopic.php?f=4&t=30135&sid=c2d2b10af30b495acb6138e8d5cf8622

[Selur]

As a side note: ffmpeg also offers nlmeans_opencl which usually is way faster.

[Guanadon]

As a side note: ffmpeg also offers nlmeans_opencl which usually is way faster.

Yes. I was considering going that route but, after downscaling and dithering to 720p with zscale (my preferred resolution for local storage), nlmeans with the settings suggested above only increases encoding times by around 30% on average with my hardware.

With very noisy sources (a few commercial Blu-rays fit into this category for me) and a high nlmeans strength setting, it can actually speed up encoding due to a large decrease in bitrate in CRF mode.

I will look into going OpenCL at some point though as it seems a waste of available horsepower to have an overclocked AMD Vega 56 sitting here doing almost nothing except when gaming or running madVR for video playback upscaling.