[Vapoursynth] How do I do replace noisy frames with freezeframing?

[hihflappy]

As the title says how do I do this? I want to replace this noisy frame with the previous one since it's the same one held without the noise. I have already tried denoising and deblocking with a mask but the latter doesn't do anything, and denoising requires way way too much sigma and doesn't get rid of it all.

This is the noisy one:

[Attachment 59126 - Click to enlarge]

It's frame 14243 and the clean one is 14242. I'm trying to do:

ff = core.std.FreezeFrames(src, first=[14243], last=[14243], replacement=[14242])

to replace 14243 but nothing's happening, it's not deleting the frame and replacing it with 14242. I read the doc too but I'm not sure if I'm getting it right http://www.vapoursynth.com/doc/functions/freezeframes.html

[manono]

There's nothing in that link that says anything about "ff =". Did you try leaving it out?

Or, simplify by returning to a more standard AviSynth:

FreezeFrame(14243,14243,14242)

[hihflappy]

The ff is just freezeframe I abbreviated to call it since you have to have something there and equals to call scripts and plugins for Vapoursynth. All my scripting work is in VSedit and it's what I'm just comfortable with. I haven't used Avisynth before and I'm still quite new to VS so I would prefer to try and solve the issue here rather than switching

[_Al_]

hihflappy, its working here.

you can try python slicing instead:
# 0 to 14242 + 14242 + 14244 to the end
ff = src[:14243] + src[14242] + src[14244:]

[hihflappy]

Thank you, I can't test it right now but when I do I will report back. Also just as a note, this is for an interlaced and telecined DVD. Not sure if the ordering matters in the script to do this before or after IVTC & Decimation

[_Al_]

after, do filtering, then find out what correct frame numbers are and then add at the end framefreeze or python slicing to replace a frame

[hihflappy]

It's still not removing the frame for some reason...

[jagabo]

Post your entire script. Did you do anything with ff after creating it? Something like ff.set_output()?

[hihflappy]

This is what it looks like right now:

import vapoursynth as vs
import lvsfunc as lvf
import kagefunc as kgf
import vsTAAmbk as taa
import fvsfunc as fvf
import mvsfunc as mvf
import havsfunc as haf
import functools
from vsutil import *
from nnedi3_rpow2 import *
core = vs.core

def postprocess(n, f, clip, deinterlaced):
   if f.props['_Combed'] > 0:
      return deinterlaced
   else:
      return clip

def decimate(src):
    src = core.vivtc.VFM(src, 1)
    return core.vivtc.VDecimate(src)


def detelecine(src):
    matched = core.vivtc.VFM(src, 1)
    src = core.nnedi3.nnedi3(src, field=1)
    return core.std.FrameEval(matched, functools.partial(postprocess, clip=matched, deinterlaced=src), prop_src=matched)

src = lvf.src(r'E:\Anime\VideoFile.m2v')
src = src[0:44075]
src = detelecine(src)
src = decimate(src)
src = fvf.Depth(src, 16)

ff = core.std.FreezeFrames(src, first=[14243], last=[14243], replacement=[14242])

crop = core.std.CropRel(src, 4, 0)
ef = core.edgefixer.ContinuityFixer(crop, [2,1,1], 0, [6,1,1], 0)
ef = fvf.Depth(ef, 16)

planes = split(ef)
planes[0] = nnedi3_rpow2(planes[0]).resize.Spline36(get_w(540, 4/3), 540)
planes[1], planes[2] = [core.resize.Bicubic(p, planes[0].width/2, planes[0].height/2) for p in planes[1:]]
scaled = fvf.Depth(join(planes), 16)

d_mask = kgf.adaptive_grain(scaled, show_mask=True, luma_scaling=8)
denoise_a = fvf.Depth(scaled, 16)
denoise_b = mvf.BM3D(scaled, sigma=[3,1.5])
denoise = core.std.MaskedMerge(denoise_a, denoise_b, d_mask)

aa = taa.TAAmbk(denoise, aatype='Nnedi3', cycle=1)
dehalo = haf.HQDeringmod(aa, darkthr=0, sharp=0, mthr=56)
darken = haf.FastLineDarkenMOD(dehalo, strength=24)

lineart_mask = kgf.retinex_edgemask(darken, 1).std.Maximum().std.Binarize()
deband_mask = core.std.Expr([d_mask, lineart_mask], expr='x y -', format=vs.GRAY)

deband_a = core.f3kdb.Deband(aa, range=13, y=16, cb=16, cr=16, grainy=12, grainc=0, output_depth=16)
deband_b = core.f3kdb.Deband(aa, range=15, y=12, cb=32, cr=32, grainy=24, grainc=0, output_depth=16)
deband = core.std.MaskedMerge(deband_a, deband_b, deband_mask)
grain = kgf.adaptive_grain(deband, 0.6, luma_scaling=4)


out = grain
final = fvf.Depth(out, 10)
final.set_output()

[hihflappy]

OH wait, I see the issue. I wasn't calling it to crop like crop = core.std.CropRel(ff, 4, 0). That's my huge dumb mistake.....

[hihflappy]

Just one thing though, can doing cause any stuttering or playback issues?

[jagabo]

There will be a jerk any time you duplicate a frame.

[hihflappy]

That's a shame but it's only for a few scenes anyway. Also just one more thing, do you know anything for Vapoursynth that can tackle Mosquito Noise like the above and here below?

[Attachment 59138 - Click to enlarge]

I tried deblocking but it needs to high of a value which hurts too much detail, the same same for denoising with BM3D

[jagabo]

I don't use Vapoursynth so I don't know what filters are available. In AviSynth I use Mpeg2Source's deringing filters to reduce those DCT ringing artifacts. Beyond that you can use MosquitoNR, or KNLMeansCL.

[hihflappy]

Thanks, KNLMeansCL is ported over to VS and seemed to do a good job. It couldn't get the few bad parts of the Opening though. I'm trying to further scenefilter it with:

###SCENEFILTERING
heavy_denoise = mvf.BM3D(scaled, sigma=[55,2])
scenefiltered = fvf.ReplaceFramesSimple(scaled, heavy_denoise, mappings="[987 996]")

but that doesn't seem to be working.

My whole script as of now:

import vapoursynth as vs
import lvsfunc as lvf
import kagefunc as kgf
import vsTAAmbk as taa
import fvsfunc as fvf
import mvsfunc as mvf
import havsfunc as haf
import functools
from vsutil import *
from nnedi3_rpow2 import *
core = vs.core

def postprocess(n, f, clip, deinterlaced):
   if f.props['_Combed'] > 0:
      return deinterlaced
   else:
      return clip

def decimate(src):
    src = core.vivtc.VFM(src, 1)
    return core.vivtc.VDecimate(src)


def detelecine(src):
    matched = core.vivtc.VFM(src, 1)
    src = core.nnedi3.nnedi3(src, field=1)
    return core.std.FrameEval(matched, functools.partial(postprocess, clip=matched, deinterlaced=src), prop_src=matched)


src = lvf.src(r'E:\Anime\VideoFile.m2v')
src = src[0:44075]
src = detelecine(src)
src = decimate(src)
src = fvf.Depth(src, 16)

deblock = core.dfttest.DFTTest(src, sigma=4, tbsize=3, opt=1)

replaceframes = [14078,14081,14094,14243]
replace = core.std.FreezeFrames(deblock, first=replaceframes, last=replaceframes, replacement=[14077,14080,14093,14242])


crop = core.std.CropRel(replace, 4, 0)
ef = core.edgefixer.ContinuityFixer(crop, [2,1,1], 0, [8,14,14], 0)
ef = fvf.Depth(ef, 16)

planes = split(ef)
planes[0] = nnedi3_rpow2(planes[0]).resize.Spline36(get_w(540, 4/3), 540)
planes[1], planes[2] = [core.resize.Bicubic(p, planes[0].width/2, planes[0].height/2) for p in planes[1:]]
scaled = fvf.Depth(join(planes), 16)

###SCENEFILTERING
heavy_denoise = mvf.BM3D(scaled, sigma=[15,2])
scenefiltered = fvf.ReplaceFramesSimple(scaled, heavy_denoise, mappings="[987 996]")

d_mask = kgf.adaptive_grain(scenefiltered, show_mask=True, luma_scaling=8)
weakdenoise = core.knlm.KNLMeansCL(scaled, d=2, a=2, s=4, h=1.1, channels="auto", device_type="gpu")
denoise_a = fvf.Depth(scaled, 16)
denoise_b = mvf.BM3D(scaled, sigma=[3,1.5])
denoise = core.std.MaskedMerge(denoise_a, denoise_b, d_mask)

aa = taa.TAAmbk(denoise, aatype='Nnedi3')
dehalo = haf.FineDehalo(aa, rx=2.0, ry=2, thmi=80, thma=128, thlimi=50, thlima=100, darkstr=0.4, brightstr=1.0, showmask=3, excl=True)
darken = haf.FastLineDarkenMOD(dehalo, strength=24)

lineart_mask = kgf.retinex_edgemask(darken, 1).std.Maximum().std.Binarize()
deband_mask = core.std.Expr([d_mask, lineart_mask], expr='x y -', format=vs.GRAY)


deband_a = core.f3kdb.Deband(aa, range=13, y=16, cb=16, cr=16, grainy=12, grainc=0, output_depth=16)
deband_b = core.f3kdb.Deband(aa, range=15, y=12, cb=32, cr=32, grainy=24, grainc=0, output_depth=16)
deband = core.std.MaskedMerge(deband_a, deband_b, deband_mask)
grain = kgf.adaptive_grain(deband, 3.6, luma_scaling=8)

out = grain
final = fvf.Depth(out, 10)
final.set_output()