Help with guidance on Avisynth scripts

[Traderbam]

I've little experience in video restoration and have a ton of old motorsport tapes on Hi8. Capturing via Amarec with Huffyuv codec installed.

I'm using this basic script based on some research on here. I have around 50 of these tapes and don't have time to tweak much of a script for each tape but if I can pick up some tips that would useful. Target is for YouTube - 1080p would be nice to get the better algorithm/resources but 720p is probably easier in terms of time invested and it still looks ok on YT. Using Staxrip to handle the script for ease of use - target files are ProRes MOV (Normal quality).

Would someone be so kind as to help me understand 1) examples of problems in the source avis 2) suggestions to improve upon the basic script? Some of these events are in forests which are dark, dull among trees, others were better lit. This was shot on a fairly basic consumer spec Hi8 machine.

To me some of the captures in amarec do seem more blurry but couldn't get on well with VDub.

Here are the samples - https://1drv.ms/f/s!AoYVwYm0qHh9lsUhmXjWjjijO-lAOg?e=4II2Ae

#Audio was out of sync when using FFVideo - known problem with Huffyuv when there are inserted frames?
AviSource("%source_file%", audio=false)

AssumeTFF()
QTGMC(Preset="Fast")

#Crop overscan and add borders - fix right hand side issue on Sony camcorders with green bar
Crop(16,0,-32,-12)
AddBorders(16, 6, 16, 6)

#Resizing to 960x720 for YT
nnedi3_rpow2(rfactor=2, cshift="LanczosResize", fwidth=%target_width%, fheight=%target_height%)

#Sharpen 
aWarpSharp(depth=5)
Sharpen(0.2)

## Potential to include some noise reduction? e.g. QTGMC(preset="faster",EZDenoise=4,denoiser="dfttest",sharpness=0.8)

[lollo]

You capture is excellent. I recognize the usage of a Hauppauge USB-Live 2 here (its histogram is unique).

The video is 720x576, 4:2:2 YUV, interlaced, TFF.

The levels are fine and well distributed across the whole duration:


[Attachment 75029 - Click to enlarge]

There is no need for a deep restoration, and a basic processing with deinterlacing/denoising/sharpening is adequate, with an intermediate upscale to 1440x1080 for YouTube upload. Intermediate because I prefer to sharpen after upscale in this case, to reduce artifacts.
Before upscale the frame is rebuilt to 704x576 to be compliant with ITU-R BT.601-4 specification.
I stayed with lossless HuffYUV, including the output, which can be read by YouTube.

If you want to compress for distribution a simple command line is enough

ffmpeg.exe -i <input>.avs -c:v libx264 -crf 17 -preset slow -aspect 4:3 -c:a aac -b:a 128k <output>.mp4

My quick attempt on a portion of the source, without addressing the stabilization aspects:

video_org=AviSource("ScriptTest2.avi").trim(425,913)

# cropping 
	crop_left=12	# | removal of left, top and right black borders and bottom head switching noise	
	crop_top=2	# | 720-(12+12)x576-(2+12)=696x562
	crop_right=12
	crop_bottom=12
video_org_crop=video_org.crop(crop_left,crop_top,-crop_right,-crop_bottom)

# plugins directory
plugins_dir="<your plugin directory>"

	# QTGMC
Import(plugins_dir + "QTGMC.avsi")
	# Zs_RF_Shared
Import(plugins_dir + "Zs_RF_Shared.avsi")

	# TemporalDegrain2
Import(plugins_dir + "TemporalDegrain-v2.2.1_modGMa.avsi")
	# RgTools
loadPlugin(plugins_dir + "RgTools-v1.0\x86\RgTools.dll")
	# MaskTools2
loadPlugin(plugins_dir + "masktools2-v2.2.23\x86\masktools2.dll")
	# MVTools
loadPlugin(plugins_dir + "mvtools-2.7.41-with-depans20200430\x86\mvtools2.dll")
	# FFT3DFilter
loadPlugin(plugins_dir + "FFT3dFilter-v2.6\x86\fft3dfilter.dll")
	# FFTW
loadPlugin(plugins_dir + "LoadDll\LoadDll.dll")
loadDll(plugins_dir + "fftw-3.3.5-dll32\libfftw3f-3.dll")

	# LSFmod
Import(plugins_dir + "LSFmod.v1.9.avsi")
	# RgTools
#loadPlugin(plugins_dir + "RgTools-v1.0\x86\RgTools.dll")
	# MaskTools2
#loadPlugin(plugins_dir + "masktools2-v2.2.23\x86\masktools2.dll")

	# ColorMatrix
LoadPlugin(plugins_dir + "ColorMatrix\ColorMatrix.dll")

	# Nnedi3
loadPlugin(plugins_dir + "NNEDI3_v0_9_4_55\x86\Release_W7\nnedi3.dll")

### de-interlacing
deinterlaced=video_org_crop.AssumeTFF().QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

### convert to YV16
deinterlaced_yv16=deinterlaced.convertToYV16()

### denoising
denoised_yv16=deinterlaced_yv16.TemporalDegrain2(degrainTR=3)

### convert to YUY2
denoised=denoised_yv16.convertToYUY2()

### change color matrix
denoised_cm=denoised.Colormatrix("Rec.601->Rec.709")

### add borders
denoised_cm_borders=denoised_cm.addborders((crop_left+crop_right)/2-8,(crop_top+crop_bottom)/2,(crop_left+crop_right)/2-8,(crop_top+crop_bottom)/2)

### upscale
upscaled=denoised_cm_borders.nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize", fwidth=1440, fheight=1080)

### convert to YV12
upscaled_yv12=upscaled.convertToYV12()

### sharpening
sharpened_yv12=upscaled_yv12.LSFmod(defaults="slow")

### convert to YUY2 with chroma from YUY2 color space
sharpened=sharpened_yv12.convertToYUY2().MergeChroma(upscaled)

return(sharpened)

Comparison between original and restored (not fair because the original is interlaced, but the quality of the outcome is evident):


[Attachment 75034 - Click to enlarge]

Youtube upload on my channel: https://youtu.be/PQmB-R_4G0k

[lollo]

For completeness, a comparison between your proposed script (which I modified a little bit to introduce proper ColorSpace conversion and different upscale) and the previous script.
Small but evident differences, essentially because the denoise filtering I added.

Comparison side by side:

[Attachment 75035 - Click to enlarge]

Comparison with slider: https://imgsli.com/MjIyMDAy

[Traderbam]

Thanks so much for the insights, that's a significant improvement. A lot to dig into... I should have said that I'm exporting to ProRes so that I've a workable format for my editor. Resolve doesn't like Huffyuv avi files. Based on the graphs for this capture would you say that no colour tweaking would be needed in an editor, or can you not tell that from the levels alone?

QTGMC I'm familiar with when I tried different presets I didn't notice any discernable difference (not knowing what I am looking f0r). What impacts the choice here to go for slow with these parameters?

Denoising - is the convertToYV16() a pre-req to use the TemporalDegrain2 in the following line, and then the convertToYUY2() changes back to original value? I'm clueless on the ConvertTo formats - I think I am right in saying that the resulting file I get from my Staxrip process is a 4:2:2 ProRes so perhaps some of those conversions are happening outside of the core script. I think adding some sort of denoise to my script will have the most impact.

Resize - Is it fine going one jump to 1440x1080? I've seen recommendations to do this in 2 stages with sharpening after each. The challenge is the file sizes for 90min videos is quite substantial 1440x1080 prores files.

Will try LSFmod to see if it is available in Staxrip. Again, is the convert to YV12 a prereq for LSFMod, then the conversion back to YUY2 to complete the step?

Thanks again. I may need to look to switch away from Staxrip - the approach of assigning each step to a variable and using that on the next call looks useful.

[Alwyn]

with an intermediate upscale to 1440x1080 for YouTube upload.... I stayed with lossless HuffYUV, including the output, which can be read by YouTube.

2.7GB per minute of HUFF needs to be considered in relation to upload speeds. Probably impractical for videos over a few minutes long.

[lollo]

I should have said that I'm exporting to ProRes so that I've a workable format for my editor. Resolve doesn't like Huffyuv avi files.

Prores is ok, but an AviSynth frame server may help you if you prefer HuffYUV. I never experienced that with Resolve, others may help you on this subject better than me.

Based on the graphs for this capture would you say that no colour tweaking would be needed in an editor, or can you not tell that from the levels alone?

There is no need for a level change. Neither for a color adjustement imo, but that's your taste and preference if you want to do so.

QTGMC I'm familiar with when I tried different presets I didn't notice any discernable difference (not knowing what I am looking f0r). What impacts the choice here to go for slow with these parameters?

The impact of the settings of QTGMC few times can be checked with a fixed image (i.e. the overall look), but most of the time you need to analyze its behaviour through a motion segment, because in improving the look of consecutive frames is where QTGMC excels.
The "fast" preset looks ok for a fixed frame, but is not optimal for nice motion results; "slow" preset alone is better for motion aspects, but tends to introduce oversharpening and "plastic" look. I generally use "slow" preset and tune some parameters to retain "original" look. Experiment yourself, but again, do not use as test benches fixed images only.

Denoising - is the convertToYV16() a pre-req to use the TemporalDegrain2 in the following line, and then the convertToYUY2() changes back to original value?
I'm clueless on the ConvertTo formats - I think I am right in saying that the resulting file I get from my Staxrip process is a 4:2:2 ProRes so perhaps some of those conversions are happening outside of the core script. I think adding some sort of denoise to my script will have the most impact.

Yes, TemporalDegrain2 needs planar ColorSpace. I do not know Staxrip. In addition to ColorSpace conversion required by the filters, remember the Colormatrix("Rec.601->Rec.709") when upscaling to HD.

Resize - Is it fine going one jump to 1440x1080? I've seen recommendations to do this in 2 stages with sharpening after each. The challenge is the file sizes for 90min videos is quite substantial 1440x1080 prores files.

The 2 stages approach with intermediate sharpening applies very well to source originated by films (look at videofred's script in doom9 forums) and for animation content. For video originated from tapes, I prefer the single step approach. But, experiment yourself what you like more.

Will try LSFmod to see if it is available in Staxrip. Again, is the convert to YV12 a prereq for LSFMod, then the conversion back to YUY2 to complete the step?

Yes. With AviSynth+ poisondeathray pointed out a modded version of LSFmod not requiring YV12 convertion: https://forum.videohelp.com/threads/410299-QTGMC-%28deinterlace-problem%29?p=2713446&v...=1#post2713446

Thanks again. I may need to look to switch away from Staxrip - the approach of assigning each step to a variable and using that on the next call looks useful.

You do not need any useless GUI once you'll be able to manage the AviSynth scripting yourself, and you'll have the full control of any operation.

2.7GB per minute of HUFF needs to be considered in relation to upload speeds. Probably impractical for videos over a few minutes long.

For long video (thus large size) the .avs script can be used to generate an appropriate x264 encoding, using the code in my previous post.
The difference in term of quality is minimal, as I tested and reported in this old DigitaFaq post: https://www.digitalfaq.com/forum/video-capture/11510-parts-build-pc-2.html#post79818
(even here consider not only comparison with fixed I-frames, but also B- and P-frames and motion aspects).

edit: added last comment about mp4/HuffYUV comparison after YT encoding

[hello_hello]

lollo,
I thought after reading the DigitalFAQ thread you linked to I'd chime in with a bit of shameless self-promotion of my CropResize function (see my signature).
I'm starting with a 720x576 image for the examples, cropping and adding borders to simulate what you might find in the real world, and assuming the DAR is 15:11 rather than 4:3 (the active 4:3 picture area is 704x576)

Crop(14,6,-24,-4)
AddBorders(14,6,24,4)


[Attachment 75059 - Click to enlarge]

Adding the borders after resizing means the video can be resized so as not to require borders all round (assuming the source had them). It does mean that for 1080p resizing, along with the height, the width will probably need to be resized a little after nnedi3_rpow2. For the example below it's resized to 1440, cropped to 1364, then resized to 1420, but that way the height can be resized to 1080 without borders and the left and right borders are reduced in size. For 720p the width and height would both need to be resized anyway.

Resizing to the desired dimensions while cropping and adding borders as required. The cropping will be twice the cropping required before resizing with nnedi3_rpow2. Any extra cropping required to prevent aspect error will be applied by the function.

nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize")
CropResize(1440, 1080, 28,12,-48,-8, InDAR=15.0/11.0, Borders=true, Resizer="Resize8", ColorConvert="ColorMatrix", ColorMode="601-709", Info=true)


[Attachment 75058 - Click to enlarge]

Or if you prefer to crop first, you can use the appropriate SAR instead of a DAR, as cropping changes the DAR. In this case it'd be InSAR=12:11.

Crop(16,6,-24,-4) # Mod4 width for nnedi3_rpow2
nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize")
CropResize(1440, 1080, InSAR=12.0/11.0, Borders=true, Resizer="Resize8")

Of course if the cropping results in a DAR greater than 4:3, the width would be resized to 1440 and the borders would be added top and bottom instead.

Or you can resize the height to whatever you prefer while letting the script resize the width accordingly, but without adding any borders:
CropResize(0, 720, 28,12,-48,-8, InDAR=15.0/11.0, Resizer="Resize8")
If you need the same dimensions when cropping sections differently but don't want borders, the function will apply any extra cropping required to prevent aspect error while resizing to the specified dimensions.
CropResize(960, 720, 28,12,-48,-8, InDAR=15.0/11.0, Resizer="Resize8")

[lollo]

Thanks for your hint!

I already knew and experimented a little bit your tool

While being a nice piece of software, the reason I do not use it is the Resize portion of CropResize. Resize is a lossy operation and I avoid it for all my material that I keep in SD standards.

I am also not a big fan of upscale, and I only do it for YouTube upload, which is not my priority, because I only use it to share material showing specific aspects of the videos.
I agree, that in this case, because upscale is just a resizing to higher (standard) resolution, and because a lossy operation is performed anyhow, CropResize may have an advantage in building an upscaled final frame free of unnecessary black borders.
OTHH, I am very sensitive to respecting proportions and do not introduce any distortion, even minimal, so I have to "stress" your CropResize tool to check this aspect deeply before introducing it in my workflow. I will do in the near future

[hello_hello]

Info=true will always show you exactly what the script is doing, so if you happen to find any inaccuracies please let me know, but given the idea of the function is to resize without any aspect error, I'm hoping that'll never happen.

By the way, if you want to crop and replace any existing borders before upscaling, without the function resizing, you can use the CleanBorders argument. It'll crop the borders away according to the cropping you specify, then automatically add new borders for the original dimensions while centering the picture. To make it easier you can specify zero for the width and height. If you enable a cropping preview you can adjust the cropping without CropResize actually cropping anything, so you can get the cropping right before removing the CPreview argument.

CropResize(0,0, 14,6,-24,-4, InDAR=15.0/11.0, CleanBorders=true, CPreview=1)

CropResize now updates the sample aspect ratio in frame properties for newer versions of Avisynth+ (and will use it as the InSAR by default), although the SAR normally doesn't change when resizing is disabled (CleanBorders disables resizing). When specifying an InDAR/InSAR, CropResize will use it instead of any SAR in frame properties and it'll update frame properties accordingly, so when resizing is disabled, specifying an InDAR or InSAR is also an easy way to change the SAR in frame properties if need be.

[lollo]

Thanks for the additional details!

[Traderbam]

Hi all, I've another sample clip that I'd appreciate some help on. The footage here has been recorded from a camcorder onto VHS and looks very washed out for want of a better term. Is there anything that can be done to sharpen this? I don't think you can sharpen if detail not there so keen on how the experts would approach it.


[Attachment 76261 - Click to enlarge]

[Sharc]

Your link fails.
(Perhaps better upload & attach a short sample of a few seconds directly to your post here)

[Traderbam]

Does the download button work on the top left? OneDrive cannot playback the file but download works for me.

[Sharc]

Does the download button work on the top left?

No. Network error .....
How big is your file? Members might not always be willing to download large files when a few seconds would fullfill the purpose....

[Traderbam]

Strange, will try another platform - it's only 250Mb.

[Traderbam]

removed - link broken

[Sharc]

https://www.dropbox.com/s/5zjasnrcvzxe371/SamplefromVHS.avi?dl=0

Any better?

Yes ok. (But why not upload it directly here?)

[Sharc]

A quick comment only: You have some crushed/clipped brights (especially the Blue component). You can't do much in post processing with clipped signals.
I would suggest to lower the darks a few notches and try to sharpen the chroma.
You can't sharpen the motion blur though and you will probably have to live with the limited resolution of VHS, I think.

[Traderbam]

SamplefromVHS.avi

Uploaded here for reference

[Traderbam]

https://www.dropbox.com/s/5zjasnrcvzxe371/SamplefromVHS.avi?dl=0

Any better?

Yes ok. (But why not upload it directly here?)

Thanks, so the best way is a recapture with adjusted procamp settings on my Hauppauge card?

I can experiment with some of the options on my NV-HS860 around sharpening/dnr settings.

[Sharc]

Thanks, so the best way is a recapture with adjusted procamp settings on my Hauppauge card?

Maybe its exposed like this on the tape? Or the signal got clipped before it entered the USB Live-2? (The USB-live2 does not clip at Y=235)
In these cases fiddling with the digital proc amp of the USB-live2 does not really help. But you can try.
Anyway, the capture is not awfully bad. No need to 'overprocess' it.
Maybe someone else has a better proposal ....

[lollo]

That capture is not that bad, the overall quality depends on the tape content.

I do not see major whites clipping, worst case is frame around 755 (the sky in the background), but because the levels are inside the card capturable range, I suspect it is like that on tape.


[Attachment 76265 - Click to enlarge]


[Attachment 76266 - Click to enlarge]

Apart color and chroma correction, a basic processing like in post #8 should produce a pretty output. Try yourself!

[Traderbam]

thanks, have tested again but performance is incredibly slow.

Out of the denoise or the sharper, is there one in particular that is 1) much slower 2) offers ways to optimise results v performance? In other words, what compromises would be worth looking at in terms of configuration/settings?

[lollo]

To speed-up the script you can split it in several: the first does only the deinterlacing, you save its output in lossless format, i.e. HuffYUV. Then the second script reads this generated (deinterlaced) file and apply the denoise, and save its output in lossless format. Then the third script read this generated (deinterlaced and denoised) file and apply the rest (upscale and sharpening), generating the final lossless version, or can be used by the encoder (ffmpeg for instance) to generate the final compressed version.
In this way the speed is much much better, but you have to generate intermediate temporary files, so disk usage is higher. You can delete all intermediate files once done.
Let me know if this helps.

[Traderbam]

Thanks, will need to look into that. Will need to look more at avisynth manual scripted options.

I'm still using Staxrip which will generate a ProRes mov. I can't seem to get it to output Huffyuv lossless files.

For this file I'm processing now it must be running at least 18 hours - 720x576 to 960x720 with deinterlace (fast) denoise and sharpen. It's a reasonably specced laptop with SSD. When I used staxrip previously for only the deinterlace part it seemed much faster.

[jagabo]

I concentrated mostly on sharpening and smoothing edges. More work is needed but for what it's worth...