the worst avisynth+ script ever (and a truly awful Super8 source)

[hydra3333]

What's the current woke word for really evil ? Oh yes, "Problematic".

I have a Problematic 2-hour source which needs to be cleaned up "reasonably, anyway" and put onto DVD.
The author of the VHS tape source passed away a couple of weeks ago, he was 87.
So, it is non-reproducible and now something of a "historical family treasure of 4 generations from the 1950s/1960s" ... there is no alternative but to work with what's been given.

Characteristics of the source:
Super8 hand-held camera -> Super8 projector -> projector screen -> recording of projection by hand-held VHS camcorder with focus-in-and-out -> VHS copy -> VHS-DVD recorder -> DVD-Decrypter -> extracted mpeg2 file ...
The file is PAL SD 720x576i 25fps interlaced TFF 4:3, consisting of circa 100 "sub-clips" strung together by the author.
It has flickering, heavy blocking here and there, heavy dirt and spots, heavy noise, colours faded or reddish-shifted/yellowish-shifted here and there, blurry, constant in and out of focus ... all different on a clip by clip basis.
The attached extract is probably the "best" of it especially with colours.

An extract from the source is attached, 207 Mb.
The, ahem, "fixed" result is also attached, 164 Mb - the first half can only really usefully be single-stepped/dragged (eg with VirtualDub2 viewed at 200%) to readily see per-frame changes, while the second half can be played to see the resulting video.

So far ...
I despaired, searched a lot (and found videoFred's and John Meyer's lovely scripts to process super8 type stuff, you must choose to review those better ones),
sought useful advice and got it,
tinkered, became confused and foundered and despaired,
yielded that one set of parameters could not fit all clips,
experimented with despotting and denoising filters,
experimented with sharpening filters,
experimented with auto-tweaking and non-auto-tweaking gain/balance filters,
experimented with filters on separated and non-separated fields to see pragmatic results,
experimented with order of placement of the filters,
to see pragmatically what produced the most reasonable outcome.
I found I can't grade nor adjust colours to save my life so just went with "close enough, given the source".
I did not attempt to produce a motion-stabilised result for this source.

I ended up with a badly non-optimized, scattered fixed-parameters for this source only, product-of-evolution, script which is surely the worst avisynth+ script ever.
Really Problematic, in line with the source.

The result is perhaps only marginally less worse than the source; well, at least it is has a tad less noise and spots.
It speeds along at 3.3 fps on a 3900X.

I later found I could play with parameters (after cleaning up the Problematic script slightly with a tad more consistent parameterisation) and use it on other really bad VHS type sources.
Note: most of the sources I see posted in forums appear to be in MUCH better condition that the ones I need to fix - as I see those I ask myself tongue-in-cheek "what's your issue ?".
Perhaps I could provide a sample or two of before-after if someone is interested. I'd guess not, given the samples I see posted.
The scripts are of course not identical with the original Problematic script used above.

If someone wants to see "the thing" (an apt simile for that script) then fine.
You really don't, though. Have a "sick-bucket" handy if you do.
Other than objective/helpful observation/suggestion posts would hurt my feelings.

I'm hoping to hear about something to better soothe the constant luma up and down.

Cheers

[johnmeyer]

To make any improvements, you'll have to try to convert the video to "frame accurate." Someone in the doom9 forum put a front end on VideoFred's script and came up with this:

The power of Avisynth salvaging 'botched' transfers of old 8mm films to DVD

<edit> I just downloaded your clip and you should be able to make fantastic improvements to it. The problems are exactly what I'd expect: you have both blends and flickering as the result of photographing a projected image where the projector shutter is opening and closing. Every second or third frame is a blend of adjacent frames. Those need to be deleted. Some have two good frames, but part of a film frame is blended in the interlaced video with part of another frame. You'll need to separate the capture into fields, delete some of the fields, blend together two good matching fields into one frame, and do this over and over. That is what the script I linked to above does.

One you have a proper frame-accurate representation of the original film, you can apply a deflicker filter; you can apply a hot spot filter to reduce the projector bulb over-exposure at the center of the film, and then you can apply VideoFred's script (or my variation). The result should actually be pretty decent. As these sort of captures go, this one isn't all that bad.

[hydra3333]

Thank you !

[hydra3333]

For this to work you will first have to install SRestore (itself a script acting as a plugin, with the .avsi extension) and its required DLLs (MaskTools2 and TIVTC) : http://avisynth.org/mediawiki/Srestore

Are things better now ? You should see a lot LESS duplicate and combed frames... but maybe not NONE.

Remember the "Chaplin effect" ? Probably the film projector itself was not actually running at 18fps during the transfer... SRestore works its magic best when the "frate" parameter is closest to the actual value in the source.
But how do we find out ?

Sadly, there is no magic here, just trial-and-error and a lot of patience... Thanks to John Meyer for the Sherlock Holmes crash course !

What you do is count the frequency of duplicated/combed frames for a given frate; then add 0.5 or 1 fps, and start again.

Gradually both frequency counts should decrease. The nearer you get to the real value, the more frames you need to step through before seeing a duplicate or combed one ("several dozen" was the advice I got, possibly over a hundred). If the counts start going up again, you've overshot : a classic dichotomy search candidate.

So, clearly, your patience and persistence will define the level of precision you achieve. When you get to well below one rogue frame in a hundred, it will not be noticeable in normal viewing conditions, and speed of movement will feel natural. In my case that level was met at frate=19.75, so I left it at that.

In stepping though .1 by .1,
I'm having trouble identifying combed ones from the rest, a few every so often which are quite blurrier than the rest so it may be them I suppose.
There are what appear to close to duplicates but for example have ocean waves moving slightly, and then there are what appear to be exact duplicates which I guess is what is being asked to identify.
I'll persist with a less problematic clip of the video and see if it gets any easier to find the magic number.

[johnmeyer]

Where did that long quote come from?? It's not part of this thread. SRestore is not going to help you with this problem. The script I linked to, by contrast, will help, and might be able to give you a near perfect result.

The speed of the projector that did the transfer is not relevant to fixing the problem, although it will affect how many dups and blends you will get.

[hydra3333]

Oh, sorry.

I went to "The power of Avisynth salvaging 'botched' transfers of old 8mm films to DVD" link https://forum.doom9.org/showthread.php?p=1505175#post1505175 and read it and started on the script instructions at the 11th post.

I must be reading the wrong thing, I'll go back and try again to find the right place.

You'll need to separate the capture into fields, delete some of the fields, blend together two good matching fields into one frame, and do this over and over. That is what the script I linked to above does.

I'll also look into the 2 most recent version of your script I could find https://forum.doom9.org/showthread.php?p=1861734#post1861734 and https://forum.doom9.org/showthread.php?p=1877548#post1877548 in case I should be looking there.

[johnmeyer]

Make life simple, and reduce the script down to something that just recovers the original frames, and eliminates the shutter flicker. You can then import the output of that script into one of the film restoration scripts to do further work.

If that MPEG file you posted is the original, then you'll also need to apply some sort of DCT block reduction filter. Those DCT blockiness artifacts are pretty ugly. However, if you encoded this, then use a different encoder, because the MPEG is really ugly.

I'll give you a link to the improved version I created with this script when it finishes uploading (I have a VERY slow connection at present). You may still want to tune the SRestore parameters to help eliminate some residual artifacts (it has lots of additional parameters other than frame rate).

You will almost certainly want to reduce the "my_frame_rate" value until you sense that frames are being dropped.

# AVISynth script to salvage a botched 8mm film transfer to DVD, working from the VOB files
# Stripped down by John Meyer to just recovering botched frames
# Original discussion
# http://forum.doom9.org/showthread.php?t=144271
# http://forum.doom9.org/showthread.php?p=1505197

loadplugin("C:\Program Files\AviSynth 2.5\plugins\Film Restoration\Script_and_Plugins\RemoveDirtSSE2.dll")   #Version 0.9         5/05/2005    
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")  #Version 0.9         5/01/2005
LoadPlugin ("C:\Program Files\AviSynth 2.5\plugins\Film Restoration\Script_and_Plugins\mvtools2.dll")        #Version 2.5.11.9    2/24/2012
Import("C:\Program Files\AviSynth 2.5\plugins\Srestore\srestore.avs")

source = AVISource("e:\fs.avi").ConvertToYV12(interlaced=true, matrix="PC.601").assumeTFF()
Salvage (source)
#return Stackhorizontal(source.separatefields(),last.separatefields())
assumefps(16)

#-------------

function Salvage(clipname)  {

    # estimated projection rate at recording (point of least duplicated or combed frames)
    my_frame_rate = 25
    # remove combed frames by Dirée
    bob(clipname,-.2,.6)             # deinterlace MPEG source
    SRestore(frate=my_frame_rate)
    requestlinear()

    # remove flicker by Dirée
    rest=last
    calm = rest.temporalsoften(1,255,255,32,2).merge(rest,0.25)#.repair(rest,1).removegrain(11)
    calm = calm.temporalsoften(1,255,255,16,2).merge(calm,0.25)
    LOP = calm.bicubicresize(180,144).bicubicresize(720,576,1,0)
    HIP = rest.bicubicresize(180,144).bicubicresize(720,576,1,0)
    HIP = mt_makediff(rest,HIP)
    mix = LOP.mt_adddiff(HIP,U=2,V=2)
    return mix
}

[johnmeyer]

Here's the link to the output from the script I posted above:

https://www.mediafire.com/file/4jvnk85b40zy26j/Recovered_Frames.mp4/file

[hydra3333]

A thousand thanks for your generosity. Now I'm up up and away

The mpeg example was trimmed from the original via videoredo with it visibly using copy-frames and the original has the exact same blocking you saw.
A part of the awful script tried to deal with it via deblock_qed

[johnmeyer]

A thousand thanks for your generosity. Now I'm up up and away

The mpeg example was trimmed from the original via videoredo with it visibly using copy-frames and the original has the exact same blocking you saw.
A part of the awful script tried to deal with it via deblock_qed

Someone else can perhaps suggest a better function for dealing with MPEG/DCT macroblocks. The one you used is the one I would have suggested.

[themaster1]

to deblock an interlaced source with amDCT:

avisource()
assumetff()
par=getparity()
SeparateFields().PointResize(width,height)
# Default Settings: amDCT(clip, adapt=0, shift=3, quality=3, quant=1, matrix=1, expand=0, sharpWpos=5, sharpTpos=7, sharpWAmt=0, sharpTAmt=0, brightStart=255, brightAmt=0, qtype=1, ncpu=1)
amDCT(last, adapt=18, shift=3, quality=3, quant=1, matrix=1, expand=0, sharpWpos=5, sharpTpos=7, sharpWAmt=0, sharpTAmt=0, brightStart=255, brightAmt=5, qtype=3, ncpu=4)
AssumeFrameBased()
SeparateFields()
Merge(SelectEven(),SelectOdd())
par ? AssumeTFF() : AssumeBFF()
Weave()

to deblock a source with lots of blocking:

avisource()
QTGMC(InputType=1, Preset="Medium", EzDenoise=1.5)
gradfun3()

[hydra3333]

Thank you, themaster1 ! I will try those out !

johnmeyer, oh, I think I see. Maybe.

Because the captured video goes along faster than normal, eg the people walking, the super8 projector playback speed must have been set somewhat higher than standard.
So after determining that super8 projector playback speed, the script concept seems to be about "fixing" the difference between that speed and the 25fps VHS camera recording-off-projector-screen speed and then denoising etc. And then slowing down the result to a more normal looking playback speed.

Looking at the original mpeg2 source with 100 or so clips, it seems old Ted may have fiddled with the super8 projector playback speed almost on a clip by clip basis (or perhaps since there are multiple super8 cameras used together and years apart ?), so I'll need to look at each clip individually. Oh well, it's only time.

The resulting clips will have different framerates, or even if the same framerate they won't be standard PAL-DVD 25fps, so I suppose I'll need to turn them into 25fps using something like johnmeyer's script JohnFPS at https://forum.doom9.org/showthread.php?p=1847109#post1847109 to then burn to DVD.

If I interpret fpp's process correctly linked above as https://forum.doom9.org/showthread.php?p=1505175#post1505175 it seems to keep the framerate of the super8 projector onto the projection screen for final playback purposes (using pulldown from PAL 25fps) which isn't what I need since old Ted's clips are sped up and I need to slow them down and I don't think PAL DVD can do "odd" pulldowns.

edit: in starting to play in parallel with deflickering options, eg as in johnmeyer's script above, I notice that works and ReduceFlicker (sometimes chained) looks good if not maybe better however DeFlicker does not seem to work well with some of this source (especially when compared with ReduceFlicker), DeFlicker does something to near black parts and occasionally seems to put horizontal stripes it at least to my wonky old eyes).

[johnmeyer]

Because the captured video goes along faster than normal, eg the people walking, the super8 projector playback speed must have been set somewhat higher than standard. So after determining that super8 projector playback speed, the script concept seems to be about "fixing" the difference between that speed and the 25fps VHS camera recording-off-projector-screen speed and then denoising etc. And then slowing down the result to a more normal looking playback speed.

There are many ways that the playback can be too fast. Projector speed is just one of them.

Do not get hung up on projector speed. You can never know what it was, and therefore guessing gets you nowhere.

What you do need to know, so you have a starting point for your calculations, is that most 16mm and 8mm silent amateur film was shot at 16 fps (unless you go back to the early 1930s and before). Super 8 standardized on 18 fps and ALL Super 8 was shot at that speed.

Note that I am emphasizing what the video was shot at. If you can successfully recover the original frames, then no matter what the projector speed was set at when the film was transferred, you want to set the speed to 18 fps for Super 8. Period.

Looking at the original mpeg2 source with 100 or so clips, it seems old Ted may have fiddled with the super8 projector playback speed almost on a clip by clip basis (or perhaps since there are multiple super8 cameras used together and years apart ?), so I'll need to look at each clip individually. Oh well, it's only time.

It is true that if the projector speed was changed between reels during the transfer from film to video, then you will have to adapt the script, using the appropriate frate parameter for SRestore. You want to keep turning that down until you start to notice dropped frames. For the script I posted you probably want to try lower numbers than the 25 fps I used. Try something really low, like 15 fps, and transfer a minute of fast motion. Then, play that and see if you can find dropped frames (big jumps in the motion). Keep increasing it until you don't sense any dropped frames. The author's original 19 fps might end up being just about right for your clips as well.

The resulting clips will have different framerates, or even if the same framerate they won't be standard PAL-DVD 25fps, so I suppose I'll need to turn them into 25fps using something like johnmeyer's script JohnFPS at https://forum.doom9.org/showthread.php?p=1847109#post1847109 to then burn to DVD.

Use AssumeFPS(18) to set the playback rate to 18 fps. BTW, the aspect ratio looks a lot more like 8mm than Super 8 and therefore you might want to use 16 fps.

If I interpret fpp's process correctly linked above as https://forum.doom9.org/showthread.php?p=1505175#post1505175 it seems to keep the framerate of the super8 projector onto the projection screen for final playback purposes (using pulldown from PAL 25fps) which isn't what I need since old Ted's clips are sped up and I need to slow them down and I don't think PAL DVD can do "odd" pulldowns.

You can do odd pulldowns, but if you are not planning to make a DVD, then no pulldown is needed. I love DVDs, but the fact of the matter is that going forward, most video is going to be played directly off a storage device or streamed online. Neither of those requires pulldown.

You can use this simple script to add pulldown:

# Pulldown for ANY fps 
ChangeFPS(60000,1001) #Use (50,1) for PAL
AssumeBFF() # use TFF for PAL
SeparateFields()
SelectEvery(4, 0, 3)

in starting to play in parallel with deflickering options, eg as in johnmeyer's script above, I notice that works and ReduceFlicker (sometimes chained) looks good if not maybe better however DeFlicker does not seem to work well with some of this source (especially when compared with ReduceFlicker), DeFlicker does something to near black parts and occasionally seems to put horizontal stripes it at least to my wonky old eyes).[/I]

Deflicker will not work until you recover the original frames and eliminate all the ghosts, field blends, and field duplications.

[hydra3333]

Thank you for that amazing comprehensive explanation for a novice. Lots of things for me to correct and look into and work with, now.

The aspect ratio may be my fault although the mpeg2 example was cut straight from the capture, unless you mean the AR of the frame outlined within the main 4:3 viewable area seems more like 8mm than Super8. I'll see if old Ted's mrs knows what he and his brother-in-law used.

You can do odd pulldowns, but if you are not planning to make a DVD, then no pulldown is needed. I love DVDs, but the fact of the matter is that going forward, most video is going to be played directly off a storage device or streamed online. Neither of those requires pulldown.

OK ! I think I may just have changed tack based on that.
I guess the family could probably play it on TV from a USB stick.
I'll do some tests, plugging some interim clip results with non-25fps/50fps framerates into a few TVs to see how it goes, and into an old WDTV-Live which I know old Ted's mrs has without a useful internet. If I get desperate for a DVD, I'll possibly try/compare the pulldown script and the 25fps conversion script.

Deflicker will not work until you recover the original frames and eliminate all the ghosts, field blends, and field duplications.

OK. I'd seen in the srestore examples http://avisynth.nl/index.php/Srestore#Examples that ReduceFlicker was used to assist in preparing the srestore dclip parameter so was trying it out, and it looked reasonable at first glance.

[hydra3333]

Hello. A quick novice's query about denoising as shown in JohnMeyer's version of videoFred's script in this link https://forum.doom9.org/showthread.php?p=1861735#post1861735

#DENOISING
#...................................................................................................................................................................
input_to_removedirt=stab.crop(CLeft,CTop,-CRight,-CBottom)
stabcrop=RemoveDirtMC(input_to_removedirt,dirt_strength,false)

  prefiltered = RemoveGrain(stabcrop,2)
  superfilt =   MSuper(prefiltered, hpad=32, vpad=32,pel=2)
  super=        MSuper(stabcrop, hpad=32, vpad=32,pel=2)

  halfblksize= (block_size>4)   ? block_size/2 : 4
  halfoverlap= (block_over>2)   ? block_over/2 : 2

  bvec1 =  MAnalyse(superfilt, isb = true,  delta = 1, blksize=block_size, overlap=block_over,dct=0)
  bvec1 =  MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100) 

  fvec1 =  MAnalyse(super, isb = false, delta = 1, blksize=block_size, overlap=block_over,dct=0)
  fvec1 =  MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  bvec2 =  MAnalyse(super, isb = true,  delta = 2, blksize=block_size, overlap=block_over,dct=0)
  bvec2 =  MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  fvec2 =  MAnalyse(super, isb = false, delta = 2, blksize=block_size, overlap=block_over,dct=0)
  fvec2 =  MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)

I noticed that only the first of the MAnalyse used superfilt ... is there some reason it isn't used by all of the MAnalyse ?

Also, would it be too harmful if I popped thSAD up a bit, to perhaps 200 ? I'm not sure of its actual effect other than maybe to discard more noise in the vector calculation (perhaps I've got that wrong).

[johnmeyer]

hydra3333, nice catch. I wonder how long that error has been in my script? Correct version:

#DENOISING
#...................................................................................................................................................................
input_to_removedirt=stab.crop(CLeft,CTop,-CRight,-CBottom)
stabcrop=RemoveDirtMC(input_to_removedirt,dirt_strength,false)

  prefiltered = RemoveGrain(stabcrop,2)
  superfilt =   MSuper(prefiltered, hpad=32, vpad=32,pel=2)
  super=        MSuper(stabcrop, hpad=32, vpad=32,pel=2)

  halfblksize= (block_size>4)   ? block_size/2 : 4
  halfoverlap= (block_over>2)   ? block_over/2 : 2

  bvec1 =  MAnalyse(superfilt, isb = true,  delta = 1, blksize=block_size, overlap=block_over,dct=0)
  bvec1 =  MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100) 

  fvec1 =  MAnalyse(superfilt, isb = false, delta = 1, blksize=block_size, overlap=block_over,dct=0)
  fvec1 =  MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  bvec2 =  MAnalyse(superfilt, isb = true,  delta = 2, blksize=block_size, overlap=block_over,dct=0)
  bvec2 =  MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  fvec2 =  MAnalyse(superfilt, isb = false, delta = 2, blksize=block_size, overlap=block_over,dct=0)
  fvec2 =  MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)