DV restoration workflow: critique requested

[taigi]

Posting this workflow for a critique.
The process was developed almost entirely through the advice of this forum’s members; my own contribution was minimal.
Source material is DV footage from a single consumer camera with a vertical stripe sensor defect. The workflow is tailored to handle this, especially with the use of santiag.

Preprocessing in AviSynth+:

video = FFMPEGSource2("1.avi", atrack=1)
video = video.ConvertToYV16(interlaced=true)
video = video.QTGMC(Preset="Very Slow", SourceMatch=3, Lossless=2, Sharpness=0, Sbb=0, MatchPreset="Very Slow", NoiseProcess=1, NoiseRestore=0, NoisePreset="slow", StabilizeNoise=true, NoiseTR=1, NoiseDeint="true", Sigma=2.0)
video = video.FineDehalo(rx=2.7, ry=2.7, thmi=80, thma=128, thlimi=50, thlima=100, darkstr=2.0, brightstr=1.0, showmask=0, contra=0.0, excl=true)
video = video.FineDehalo2()
video = video.santiag(strv=3, type="nnedi3", nns=4, threads=0, nsize=4, halfres=false, scaler_post="LanczosResize")
video = video.FineDehalo(rx=2.7, ry=2.7, thmi=80, thma=128, thlimi=50, thlima=100, darkstr=1.0, brightstr=1.0, showmask=0, contra=0.0, excl=true)
# video = video.Levels(16, 1.0, 242, 16, 235, coring=false, dither=true)
video = video.Spline64Resize(1024,576)
video
Prefetch(2)

(FineDehalo is applied again after santiag, since santiag tends to generate new halos.
I would like to recover more detail using Levels, but this also reveals the same vertical stripes that santiag was used to suppress.
I use Spline64Resize because I cannot accurately assess results during the process - primarily due to Topaz’s processing.)

After AviSynth+:

NeatVideo denoising.
“Plastic” slowly starts.

Upscaling is executed using NNEDI3.
If Topaz VEAI, Artemis HQ is run without prior upscaling, it introduces excessive plasticity.

Upscaling with Topaz VEAI (Artemis HQ)
Commercial software, I'm not a pro.

Return to AviSynth+
Edges with mt_edge.

Frame Extraction and Sharpening
All frames extracted and processed with Topaz Sharpen AI, then recombined.

Restoration of noise to escape plastic and LUT processing are planned for a future.
Constructive criticism is requested.

[lordsmurf]

There's nothing commercial about Topaz. Pros don't use it.
Not free, but not commercial.

[taigi]

Pros don't use it.

I might sound boring, and I kind of know the answer already - but may I kindly ask, why? (Stubborn models, don’t feed the capitalists.)

[lordsmurf]

It really depends on the field, and the use.

- Law enforcement, for example, has special certified software to extract information. That's really forensics, not restoration, but close.
- Studios like Disney will do inpainting of frames, often using typical NLEs with specialized in-house plugins.
- Small and large operations alike often use Avisynth (or Vapoursynth, etc -- scripted), because of the power it offers.

Now, referring to sub-HD source specifically (like DV, a SD resolution)...

Topaz is quaint software for newbies. You're mostly paying for the GUI, on top of an inferior engine. It gets a lot of praise from low-end users, but lots of flaws are constantly showcased by higher-end video users.

The real question is always this: Do you really need to upscale the video? In most cases, probably not.

Especially since Topaz tends to "uncanny valley" it, leaving behind weird artifacts seen in motion. It odd to watch in casual viewing. You have to learn to ignore the "pixel peep" reviews (comparing stills, small clips), and instead focus on the overall viewing. It's not great.

Now, for HD sources, the upscale still has issues, but it's not a total mess.

Topaz cannot handle interlace well whatsoever, regardless of HD or SD.

[Alwyn]

That seems like a lot of work for DV.

I would have used the "AVISource" filter for DV. I think FFMS2 is more for the MPEG-type video codecs. I'm no expert on AVISynth though.

NeatVideo denoising.
“Plastic” slowly starts.

It doesn't need to. Wind back the denoising a bit. Have a play with the various sliders and you'll quickly find you can reduce the reduction to get a nice balance.

If you don't intend editing later it in an NLE, you could try denoising in AVISynth using TemporalDegrain2 rather than Neat.

[lordsmurf]

The work is due to camera defect, vertical striping.

Agree, AVISource for DV, not sure why ffms2 here.

NeatVideo is a meat cleaver, not a scalpel, when it comes to NR. It can be very effective for basic noise, but a mud-maker for anything even modestly noisy.

[Alwyn]

but a mud-maker for anything even modestly noisy.

Nonsense. It does even better when the video is otherwise almost unwatchable.

[taigi]

Wind back the denoising a bit

My idea was to remove all grain from the footage, then upscale it, and finally re-add the grain afterward. This way, the grain won’t be sharpened by any effects I apply later.

[lordsmurf]

but a mud-maker for anything even modestly noisy.

Nonsense. It does even better when the video is otherwise almost unwatchable.

You've shown us your samples before. No.

[_Al_]

AviSource plugin would load NTSC 4:1:1 video as 4:2:0. So there is chroma resize, maybe silent and not realized.
ffms2 can capture 4:1:1 and then using zimg resize or whatever else to resize to chroma used for filters, what is needed, so maybe 4:2:0 as well sure. Or it could be resized to 4:2:2 (QTGMC can use it) and then using other filters for that chroma.

[Alwyn]

Thanks for that info, AI.

[taigi]

Just to clarify for anyone working with PAL DV (as opposed to NTSC DV) - this issue with silent chroma conversion by AviSource is only relevant to NTSC DV. PAL DV natively uses 4:2:0, so when you load PAL DV AVIs with AviSource, you’re getting the original chroma.
So when you use FFMS2/LSMASHSource with PAL DV, you’re preserving the chroma format.