VHS Tape restoration help

[Akuma786]

Happy New year.

Hi, I have this tape capture, that I like to restore.

https://www.swisstransfer.com/d/de389b5b-48e1-4fbe-8aa8-c09486bc2800

Looking to Deinterlace it, stabilise and improve picture quality and then output to 24fps please.

Below is a script I have started with, I would like help to get the best possible result.

AVISource("E:\BF VHS FILES\B FIST CAPTURE.avi")
ConvertToYV16(interlaced=true)
assumeTFF()
QTGMC( Preset="Slow", SourceMatch=3, Sharpness=0.4, SLMode=1, TR2=3, Lossless=2 )
SelectEven()   
crop(8,116,-16,-120)
Stab(range=1, dxmax=1, dymax=1, mirror=0)
SMDegrain(thsad=300, tr=3, PreFilter=4)
ConvertToYUY2()



Spline36Resize(720, 306) 

Prefetch(24)

Any advice and help would be appreciated.

[Akuma786]

Okay so i updated the drivers for the hauppage Live2 capture device and it seems im getting much better capture than previous capture. Before it was clipping the whites.Below is new capture:

https://www.swisstransfer.com/d/bd2f2b75-99e0-4728-949b-cd7cbf0f1ecc

[lollo]

Be careful with new drivers for USB Live 2 (6_0_119_39299), they feature an auto adjustement of high levels potentially causing AGC variations:
https://forum.videohelp.com/threads/418254-Info-Hauppauge-USB-Live-2-capturable-range
https://forum.videohelp.com/threads/418973-Info-IOData-GV-USB2-capturable-range

I would stay with old drivers (6_0_119_36075) and adjust the contrast to avoid clipping of the whites.

The source is PsF (not interlaced), so QTGMC + SelectEven is not necessary.

[Akuma786]

Thanks Lollo, I'll go back to old drivers and adjust contrast as suggested.

so if i remove QTGMC & selectEven, the picture looks interlaced. How do i make it look progressive ?

[lollo]

The defect is that in your source sometimes the fields are shifted.

For instance in frame 11344 the odd field is shifted up by 1 pixel.

You can correct it with a simple field shift operation: https://imgsli.com/NDM3OTM0

But is a difficult and boring processing, so yes, you were right, QTGMC + Stab may be an alternative.

[Akuma786]

I see what you mean. What will be the best settings for Stab to get this as best as possible.

[lollo]

I see what you mean. What will be the best settings for Stab to get this as best as possible.

No idea, experiment with the parameters.
I generally start with:

    range = 4
    dxmax = 4
    dymax = 8

[Akuma786]

Thanks Lollo, I'll experiment with them parameters.

Anyone got any suggestion, what filters to use for cleanup and sharpness?

[lollo]

Anyone got any suggestion, what filters to use for cleanup and sharpness?

Experiment

TemporalDegrain2(degrainTR=3)
LSFmod(defaults="slow")

For spatial denoising, you can try KNLMeansCL

For scratches and similar, you can try Spotless

[Akuma786]

The defect is that in your source sometimes the fields are shifted.

For instance in frame 11344 the odd field is shifted up by 1 pixel.

You can correct it with a simple field shift operation: https://imgsli.com/NDM3OTM0

But is a difficult and boring processing, so yes, you were right, QTGMC + Stab may be an alternative.

So I've noticed when that field shift happens the picture gets blurry and losses its sharpness for a moment. Is there anything that can be fix this ?

[lollo]

So I've noticed when that field shift happens the picture gets blurry and losses its sharpness for a moment. Is there anything that can be fix this ?

The ultimare solution is to fix the shift of the fields one by one, which is a tedious and very long operation, because you need to identify them and apply the appropriate "pixel(s) shift" up or down to even and/or odd fields.
If the shift persists for many consecutive frames it is an additional complication.

Otherwise, you can try to "mitigate" the problem with QTGMC, Stab and similar.

[Akuma786]

I've removed SMDegrain and added TemporalDegrain2(degrainTR=3) as suggested. Tried LSFmod(defaults="slow") but this made picture look too sharp and abit plastic so removed it.

Current script is :

AVISource("E:\BF VHS FILES\B FIST CAPTURE.avi")
ConvertToYV16(interlaced=true)
assumeTFF()
QTGMC( Preset="Slow", SourceMatch=3, Sharpness=0.4, SLMode=1, TR2=3, Lossless=2 )
SelectEven()   
crop(8,116,-16,-120)
Stab(range=4, dxmax=4, dymax=8, mirror=0)
TemporalDegrain2(degrainTR=3)
ConvertToYUY2()

Spline36Resize(720, 306) 

Prefetch(24)

Does it matter what order Stab and TemporalDegrain2 are placed on script ?
Also when I try to change fps to 24fps the audio changed to 46068 or something like that. I tried added SSRC to script to fix this but it didn't work I got an error instead. How do i resolve this ?

[lollo]

Does it matter what order Stab and TemporalDegrain2 are placed on script ?

Yes, denoise after stab

Also when I try to change fps to 24fps the audio changed to 46068 or something like that. I tried added SSRC to script to fix this but it didn't work I got an error instead. How do i resolve this ?

Why you want to change to 24 fps?

[Akuma786]

I have 2 audio tracks ( Cantonese & Mandarin) and English subtitles which are all 24fps. Easier to sync all files onto this video file if its 24fps.

[Akuma786]

Is there a filter which can help remove the issue in blue circled areas :


[Attachment 90588 - Click to enlarge]

[lollo]

AssumeFPS(24, sync_audio=true)

[lollo]

Difficult; try dehalo with large x depth or LGhost/GhostBuster

[johnmeyer]

I don't have time to look at the source, but I did write a script earlier this year which attempts to determine if a field is shifted and apply the fix only when a shift is detected. I've copied that below. Maybe it will help, or be a start for something which will save you from having to manually look at each frame.

#Script to shift fields, but only on frames where problem exists
#Hard-wired for shifting even fields
#John Meyer
#Written 2025

#These plugins are for denoising
Loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools\mvtools2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Cnr2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")

SetFilterMTMode("MDegrain2",     MT_MULTI_INSTANCE) 
SetFilterMTMode("MAnalyse",      MT_MULTI_INSTANCE) 
SetFilterMTMode("MRecalculate",  MT_MULTI_INSTANCE) 
SetFilterMTMode("MSuper",        MT_MULTI_INSTANCE) 
SetFilterMtMode("AVISource",     MT_SERIALIZED)

#Get input and then Bob.
input=AVISource("e:\fs.avi").killaudio().ConvertToYV12()
source=input.assumetff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = 2
even = selecteven(source)
odd  = selectodd (source)

even = crop(even,Hshift,Vshift,0,0)
even = addborders(even,0,0,Hshift,Vshift)

#This is the fixed, bobbed output. Reinterlace at the end of the script
output = interleave(even,odd)

/*
#Debugging: make sure YDifference metrics reliably detect field shift
script = """Subtitle("\nMetric = " + String(  \
            (YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
            / (YDifferenceFromPrevious(output) + YDifferenceToNext(output))), lsp=0)"""
Scriptclip(output, script)
*/

#Choose the original source if no field shift detected. Otherwise use the fixed version
fixed = output.scriptclip("""(YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
                         / (YDifferenceFromPrevious(output) + YDifferenceToNext(output)) > 1\
                          ? output : source""")


#Convert back to interlaced
evenbob = selecteven(fixed)
oddbob = selectodd(fixed)
EvenOrig = separatefields(evenbob).selecteven()
OddOrig =  separatefields(oddbob).selectodd()
interleave(EvenOrig,OddOrig)
fixed_field = weave()

#Enable next two lines for denoising
chroma=fixed_field.Cnr2("oxx",8,16,191,100,255,32,255,false) #VHS
fixed_field = MDegrain2i2(chroma,8,4,400,0)  

return fixed_field

#Alternate output to view before/after
#Interleave(input,fixed_field)

# Enable MT!
Prefetch(6)


function MDegrain2i2(clip source, int "blocksize", int "over", int "denoising_strength", int "dct")
{
  Vshift=0 # 2 lines per bobbed-field per tape generation (PAL); original=2; copy=4 etc
  Hshift=4 # determine experimentally 
  overlap=default(over,0) # overlap value (0 to 4 for blksize=8)
  dct=default(dct,0) # use dct=1 for clip with light flicker

  fields=source.SeparateFields() # separate by fields

  #This line gets rid of vertical chroma halo - use for VHS 2nd generation
  #fields=MergeChroma(fields,crop(fields,Hshift,Vshift,0,0).addborders(0,0,Hshift,Vshift))

  #This line will shift chroma down and to the right instead of up and to the left
  #fields=MergeChroma(fields,Crop(AddBorders(fields,Hshift,Vshift,0,0),0,0,-Hshift,-Vshift))

  prefiltered = RemoveGrain(fields,2)
  superfilt =   MSuper(prefiltered, hpad=32, vpad=32,pel=2)
  super=        MSuper(fields,      hpad=32, vpad=32,pel=2)

  halfblksize= (blocksize>4)   ? blocksize/2 : 4
  halfoverlap= (over>2)        ? over/2      : 2

  bvec1 =  MAnalyse(superfilt, isb = true,  delta = 2, blksize=blocksize, overlap=over,dct=dct)
  bvec1 =  MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100) 

  fvec1 =  MAnalyse(superfilt, isb = false, delta = 2, blksize=blocksize, overlap=over,dct=dct)
  fvec1 =  MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  bvec2 =  MAnalyse(superfilt, isb = true,  delta = 4, blksize=blocksize, overlap=over,dct=dct)
  bvec2 =  MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  fvec2 =  MAnalyse(superfilt, isb = false, delta = 4, blksize=blocksize, overlap=over,dct=dct)
  fvec2 =  MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  denoised = fields.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength)

  #Unsharpmask sharpening
  even=selecteven(denoised)
  odd= selectodd (denoised)
  denoised = interleave( unsharpmask(even,30,3,4), unsharpmask(odd,30,3,4) )

  Weave(denoised)
}

[johnmeyer]

I have another script which uses motion estimation to aid in detecting field shift. I can post that, if anyone wants.

In the meantime, here is another one I wrote twelve years ago. I forgot about this one before I wrote the one above.

# Script to both detect, and then fix, video in which one field has been shifted up by one line.
# June 4, 2013

#loadPlugin("c:\Program Files\AviSynth 2.5\plugins\MPEGDecoder.dll")
#loadPlugin("c:\Program Files\AviSynth 2.5\plugins\GRunT.dll")

#source=mpegsource("E:\film\1974 Rams 49ers 01.mpg")\
source=AVISource("e:\fs.avi").killaudio().assumetff().bob(0,1).ConvertToYV12()

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = 2
even = selecteven(source)
odd  = selectodd (source)

even = crop(even,Hshift,Vshift,0,0)
even = addborders(even,0,0,Hshift,Vshift)

#Alternate fix
#odd = crop(odd,Hshift,Vshift,0,0)
#odd = addborders(odd,0,0,Hshift,Vshift)

output = interleave(even,odd)

#The next two lines are used during debugging to make sure YDifference metrics
#really do detect field shift reliably
#script = """Subtitle("\nsource = " + String(YDifferenceFromPrevious(source)) + \
#         "\noutput = " + String(YDifferenceFromPrevious(output)), lsp=0)"""
#Scriptclip(output, script)

fixed = output.scriptclip("""YDifferenceFromPrevious(source) > \
        YDifferenceFromPrevious(output) ? output : source""")

#Convert back to interlaced
evenbob = selecteven(fixed)
oddbob = selectodd(fixed)
EvenOrig = separatefields(evenbob).selecteven()
OddOrig =  separatefields(oddbob).selectodd()
interleave(EvenOrig,OddOrig)
weave()

[Akuma786]

Difficult; try dehalo with large x depth or LGhost/GhostBuster

Thanks i give it a try.

[Akuma786]

I don't have time to look at the source, but I did write a script earlier this year which attempts to determine if a field is shifted and apply the fix only when a shift is detected. I've copied that below. Maybe it will help, or be a start for something which will save you from having to manually look at each frame.

#Script to shift fields, but only on frames where problem exists
#Hard-wired for shifting even fields
#John Meyer
#Written 2025

#These plugins are for denoising
Loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools\mvtools2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Cnr2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")

SetFilterMTMode("MDegrain2",     MT_MULTI_INSTANCE) 
SetFilterMTMode("MAnalyse",      MT_MULTI_INSTANCE) 
SetFilterMTMode("MRecalculate",  MT_MULTI_INSTANCE) 
SetFilterMTMode("MSuper",        MT_MULTI_INSTANCE) 
SetFilterMtMode("AVISource",     MT_SERIALIZED)

#Get input and then Bob.
input=AVISource("e:\fs.avi").killaudio().ConvertToYV12()
source=input.assumetff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = 2
even = selecteven(source)
odd  = selectodd (source)

even = crop(even,Hshift,Vshift,0,0)
even = addborders(even,0,0,Hshift,Vshift)

#This is the fixed, bobbed output. Reinterlace at the end of the script
output = interleave(even,odd)

/*
#Debugging: make sure YDifference metrics reliably detect field shift
script = """Subtitle("\nMetric = " + String(  \
            (YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
            / (YDifferenceFromPrevious(output) + YDifferenceToNext(output))), lsp=0)"""
Scriptclip(output, script)
*/

#Choose the original source if no field shift detected. Otherwise use the fixed version
fixed = output.scriptclip("""(YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
                         / (YDifferenceFromPrevious(output) + YDifferenceToNext(output)) > 1\
                          ? output : source""")


#Convert back to interlaced
evenbob = selecteven(fixed)
oddbob = selectodd(fixed)
EvenOrig = separatefields(evenbob).selecteven()
OddOrig =  separatefields(oddbob).selectodd()
interleave(EvenOrig,OddOrig)
fixed_field = weave()

#Enable next two lines for denoising
chroma=fixed_field.Cnr2("oxx",8,16,191,100,255,32,255,false) #VHS
fixed_field = MDegrain2i2(chroma,8,4,400,0)  

return fixed_field

#Alternate output to view before/after
#Interleave(input,fixed_field)

# Enable MT!
Prefetch(6)


function MDegrain2i2(clip source, int "blocksize", int "over", int "denoising_strength", int "dct")
{
  Vshift=0 # 2 lines per bobbed-field per tape generation (PAL); original=2; copy=4 etc
  Hshift=4 # determine experimentally 
  overlap=default(over,0) # overlap value (0 to 4 for blksize=8)
  dct=default(dct,0) # use dct=1 for clip with light flicker

  fields=source.SeparateFields() # separate by fields

  #This line gets rid of vertical chroma halo - use for VHS 2nd generation
  #fields=MergeChroma(fields,crop(fields,Hshift,Vshift,0,0).addborders(0,0,Hshift,Vshift))

  #This line will shift chroma down and to the right instead of up and to the left
  #fields=MergeChroma(fields,Crop(AddBorders(fields,Hshift,Vshift,0,0),0,0,-Hshift,-Vshift))

  prefiltered = RemoveGrain(fields,2)
  superfilt =   MSuper(prefiltered, hpad=32, vpad=32,pel=2)
  super=        MSuper(fields,      hpad=32, vpad=32,pel=2)

  halfblksize= (blocksize>4)   ? blocksize/2 : 4
  halfoverlap= (over>2)        ? over/2      : 2

  bvec1 =  MAnalyse(superfilt, isb = true,  delta = 2, blksize=blocksize, overlap=over,dct=dct)
  bvec1 =  MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100) 

  fvec1 =  MAnalyse(superfilt, isb = false, delta = 2, blksize=blocksize, overlap=over,dct=dct)
  fvec1 =  MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  bvec2 =  MAnalyse(superfilt, isb = true,  delta = 4, blksize=blocksize, overlap=over,dct=dct)
  bvec2 =  MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  fvec2 =  MAnalyse(superfilt, isb = false, delta = 4, blksize=blocksize, overlap=over,dct=dct)
  fvec2 =  MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  denoised = fields.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength)

  #Unsharpmask sharpening
  even=selecteven(denoised)
  odd= selectodd (denoised)
  denoised = interleave( unsharpmask(even,30,3,4), unsharpmask(odd,30,3,4) )

  Weave(denoised)
}

Thanks Johnmeyer, I use the 64bit avisynth, so im stuck on : "cannot load a 32 bit Dll in 64 bit avisynth - RemovegrainSSE2.dll issue. Is there anyway round this on 64bit ?

[lollo]

John, that was a nice exercise, but it only works for a shift of 2 pixels in the bottom direction of even fields. And it does not solve the problem of shifts for consecutive frames. The original topic is here: https://forum.doom9.org/showthread.php?t=186211

[johnmeyer]

Like I said, I didn't have time to look at the source, so if the shift is different, the script will need to be modified. However, the idea of separating into fields and then using Ydiff to decide whether a shift is needed (or not) should be applicable to other types of shifts.

As for the RemoveGrain error for 64 bit, just get rid of the two denoising lines and that should fix that problem. Denoising has nothing to do with the field shift and was included because I just copy/pasted some working code and didn't want to take any time to strip it down, which would have required testing.

As for consecutive frames, I'm pretty sure it actually does work for as many frames in a row as needed. I used this on some video from a sports video collector and the problem would come and go, sometimes in the middle of a play, but most definitely persisted, sometimes for many minutes, before the video went back to normal. The detection is not perfect, but it seemed to work about 98-99% of the time.

[Akuma786]

Thanks Johnmeyer

[lollo]

Like I said, I didn't have time to look at the source, so if the shift is different, the script will need to be modified. However, the idea of separating into fields and then using Ydiff to decide whether a shift is needed (or not) should be applicable to other types of shifts.

In the script there is no automatic detection of the type of shift (even or odd field, up or down shift) and of its quantity (1 pixel to X pixels).
So you have to run it dozens of times (and save the intermediate files) to hope to fix the stream, changing the parameters
And there will be both false positives/negatives, and no fix a group of consecutives fields shifted.

I am working on the ideas from real.finder in the thread I linked, but did not find yet a solid implementation.
Being this filelds shift a common problem to many captures, I will post about it if I find a general solution!

[Sharc]

Footnote: As an alternative it may be easier to substitute the glitchy frames or fields (even a few in a row) by motion interpolated frames of adjacent good frames. Using a good interpolator like RIFE the interpolated frames are hardly discernible from genuine frames and are unnoticed by casual viewers. Well, it's a possible workaround rather than a clean fix of the root cause.

[lollo]

Well, it's a possible workaround rather than a clean fix of the root cause.

The motion interpolation still needs the shifted fields to be identified to be applied, that's were the difficulty stands. (BTW I prefere the un-shift of the fields, because it does not alter the original content).

The root cause is the line base correction of the VCR (or the DVD Recorder in the case of OP) related to the condition of the tapes, so little can be done about that in my experience. I remember in some comparisons that a specific Pioneer DVD-Recorder was showing less occasional field vertical jitter than a VCR or a Panasonic DVD-R, but had less overall quality.

[Akuma786]

Would it make a difference to field shifts issue if I try recapture the VHS, but this time using a Panasonic DMR ES15 included in the process ?

[Selur]

How about:
a. separate the fields
b. apply Stab on even and odd fields
c. reinterlace
d. deinterlace
that seems to work fine here for the field shifting,...

[Akuma786]

Thanks Selur, my avisynth knowledge is weak, are you able to put that as a script for me to try on my source please.