Need advice on optimal AVS script for VHS captures (with ADVC-110)

[Snakeforhire]

Hello everyone !

I'm a n00b here (but was on doom9 15 years ago), and after a long lapse I'm coming back to capturing and encoding the rest of my VHS collection.
Sadly in the meantime all the settings I was using back then (satisfactorily) were lost in an HDD failure, and I have no recollection of what I used as scripts

My problem is this : I need to restore (very important) VHS captures before encoding them to MPEG2 (PAL Full D1, KDVD specs) for storage on Bluray discs.
My video capture setup is : JVC S-VHS VCR (composite output as the tapes are standard VHS. Forcing the output to S-video doesn't work ) -> Transcoder/TBC (Comworld KDV-500, essentially the same as a KDV-5000 but only has 12Mb frame memory. The tapes were recorded in SECAM-L) -> Canopus ADVC-110
The output is a Type 2 DV file, so "bottom field first".
I was wondering what Avisynth experts would recommend as a script to correct the slight bleeding in red/blue inherent to a conversion to DV (I understand that PAL DV captures should be reinterpolated to (/from ? :/ ) the YV12 4:2:0 colorspace but I'm clueless as to the settings I should use).
Also, the tapes were recorded from Canal+ (decrypted), which always gave me a "grainy" picture (because of a bad antenna and interference), with parasitic noise.
What would you suggest I use to smoothen this without deinterlacing (as I'm planning to encode interlaced as well), while still keeping the picture somewhat sharp ?

I'm including a clip of the problematic captures so you have an idea : https://ufile.io/22b6p5kn

Many thanks to whoever will answer !

[LMotlow]

If you intended to do some Avisynth cleanup and major restoration from VHS, why did you capture to lossy DV, which has never been recommended for repair work?


The DV crowd will be more than happy to lead you through the tiresome rigamarole required to clean up the mess from VHS to lossy DV captures. It won't be easy. But you might want to post a sample of the problem so others won't have to keep guessing what your video looks like. Don't post to Youtube (we don't want to solve YouTube re-encoding problems). If you have DV codecs on your 'puter you can make an edited sample in any NLE or in VirtualDub to post directly in the forum.

[Snakeforhire]

If you intended to do some Avisynth cleanup and major restoration from VHS, why did you capture to lossy DV, which has never been recommended for repair work?


The DV crowd will be more than happy to lead you through the tiresome rigamarole required to clean up the mess from VHS to lossy DV captures. It won't be easy. But you might want to post a sample of the problem so others won't have to keep guessing what your video looks like. Don't post to Youtube (we don't want to solve YouTube re-encoding problems). If you have DV codecs on your 'puter you can make an edited sample in any NLE or in VirtualDub to post directly in the forum.

Because that's what my equipment uses, I wasn't asked for an opinion...

And the address of the sample is at the bottom of the post, for those who can read... :/

[johnmeyer]

A few random thoughts that may help.

1. I don't understand why you have to "force" the output to S-video. Are you, perhaps, confusing S-VHS with S-video? They are two completely different things. If your VCR has S-Video output (a multi-pin connector) you should have video on that output no matter what you do. If you are talking about S-VHS, that is a higher-res format than regular VHS, but your video must be recorded in that format to begin with and you cannot "force" a standard VHS to become S-VHS. You definitely want to use the S-video output. Hopefully your ADVC-110 has an input for this. Using S-video should help reduce some of the color bleed problems, especially the reds.

2. I've not used the ADVC-110, but it should be adequate for transferring VHS. The nice thing about using a DV converter is that it produces output that is easier to edit than any other format ever created. If you are using a Windows computer, I always recommend using Scenalyzer as the capture application. It is bulletproof and has dozens of really neat features. It has been free for years ever since the author realized the demand was dwindling as people moved on to HD video.

3. There are dozens of AVISynth VHS cleanup scripts, both here and at doom9.org. I've posted many of mine over at doom9.org. You can use CNR for red color bleed, and use MDegrain2 for the dot noise. You can sometimes use a little sharpening, but I'm not a big fan of that.

4. Speaking of sharpening, one of the most important things you can do is to make sure ALL sharpening and "enhancements" are turned OFF in your VCR before you do the transfer. I think the JVC uses the term "edit mode" or something similar to that. Turning Edit mode ON turns sharpening and enhancments OFF, which is what you want. The picture will look a little less sharp, but you actually will capture more detail. That paradox is explained because, in order to make the lousy VHS picture look better, VCR designers added a simple peaking circuit that emphasized the high-contrast dark/light transitions by putting a halo around them. This gives the appearance of more sharpness but actually obscures detail. Modern software algorithms can do a far better job of improving the image without screwing up the detail. So, capture with those "improvements" turned off.

5. Some of the JVC decks have a simple form of time base correction. The one I use has this. If so, enable it and make sure it really improves the pictures. It almost always does, but can actually cause problems on some tapes. You'll just have to do your own experiments to see if it gets rid of flagging at the top of the screen or if it gets rid of herring bones on vertical objects.

6. Make sure you are capturing Hi-Fi audio, if your tapes have that track.

7. If you are archiving onto Blu-Ray, then you really don't need to use MPEG-2. There are better formats available today that will provide smaller file sizes while also providing fewer artifacts. h.264 in an MP4 wrapper would be a good choice.

8. I applaud your move to keep the video interlaced. Too many people think they need to deinterlace. Unless you are going to re-size the video or do some other operation which requires deinterlacing, you are much better not doing it. Deinterlacing always degrades the video. Modern TV sets have really good deinterlacers built in and it has been a decade since I've seen or heard any credible evidence that you are going to get noticeably better results using a software deinterlacer.

9. If you are transferring many tapes that were recorded in the EP (6-hour) or LP (4-hour) mode, pay attention to the sound. I am finding that as my VCRs age, one of the first things that begins to get flaky is the Hi-Fi playback in these slower modes. If you do have problems you can switch the deck to only play "normal" (non-HiFi) audio. The frequency response will be pretty bad, but this is often better than audio that has static from HiFi misalignment.

Finally, if and when you do get an AVISynth noise reduction script working, resist the temptation to remove too much noise. I've post this many times, but like the famous architect Mies van der Rohe said: "Less is more." I've seen way too many "improved" VHS video where there is absolutely no noise, but also absolutely no detail left, and all sorts of new artifacts have been introduced.

[themaster1]

To remove some chroma noise without artefacts:

ConverttoRGB32(matrix="rec601",interlaced=true)
separatefields()
LoadVirtualDubPlugin("C:\Program Files (x86)\virtualdubmod1.5\plugins\Camcorder_Color_Denoise_sse2.vdf", "CCD", 0)
CCD(10,1) #  0 to 100 / default =30 (strong)
weave()
converttoyv12(matrix="Rec601",interlaced=true)
#
# For Chroma bleeding (best to use on deinterlaced video):
chromavid = awarpsharp2(depth=24,chroma=3) # chroma=3 is better for real video imo
mergechroma(chromavid,1.0) # default= 0.5

[johnmeyer]

Here's a link to one of over a dozen VHS restoration scripts I've posted over at doom9.org. I'm sure you can find or create better, but this one has the advantage of being pretty simple; it works on interlaced video; it is aimed at reducing typical VHS artifacts, as opposed to noise found in other formats; and if you can get multi-threading to work, it is reasonably fast. The latter is important if you are going to be doing large projects. I've had people give me a huge box of several dozen tapes, often with at least two hours on each tape. When you have to transfer 20-40 hours of video, you don't want to use a 2 fps script.

Denoiser script for interlaced video using MDegrain2

[johnmeyer]

I looked at the video file. You have some issues with blown-out highlights. If those are on the original, you can't do anything about it, but if not, check to see if there any proc amp settings on your AVC-110. Most DV converters don't have such settings, but maybe yours does.

I am not at my main computer today, but when I get back to it, if I have a chance, I'll run your video through one of my scripts and see if they do much. That video is pretty bad, and only so much can be done with it.

[JVRaines]

My problem is this : I need to restore (very important) VHS captures before encoding them to MPEG2 (PAL Full D1, KDVD specs) for storage on Bluray discs.

If your tapes are very important, then you should purchase an inexpensive USB converter and capture uncompressed or losslessly-compressed video. The color decimation inherent in the DV format cannot be undone because the information is lost. You will also get noticeably better results if you use the s-video output, which, as johnmeyer observed, has nothing to do with tape format.

[manono]

The color decimation inherent in the DV format cannot be undone because the information is lost.

He's in PAL land so the color problem is not nearly as bad as their DV color subsampling is 4:2:0, not 4:1:1 as for NTSC DV.

[Snakeforhire]

Thanx for all your answers so far, keep'em coming plz !

As for the VCR, I have to make do with a HR-S5955MS, which does not have a dedicated s-video output, only SCART... And it has no inbuilt TBC
I can force s-video output in the settings, but this doesn't seem to work (the transcoder picks up a garbled signal even when setting it on s-video in), the only thing that works is "normal" output and composite input on the transcoder...
But I'm not going to go farther than that I think. I already had to shell out 100€ I couldn't spare for this one, and MS (SECAM compatible) models featuring a TBC are next to impossible to find and/or out of my financial reach...
The bitter irony is that I also possess a 7600 series VCR (with inbuilt TBC and s-video out), but it's not the MS version so it has no SECAM playback ability, only PAL/NTSC (even though it's equipped with a french wall plug. Go figure :/ )... But anyhow when I tried to power it up after a 10yr pause it didn't react, dunno if the fuse is blown or if it's more serious.

I have a USB converter also, but I'm not satisfied since it's a chinese POS which converts directly to MPEG2 and even can't record more than 4.3Gb of that (to fit on a DVD-R).
So far I've always been very pleased with the Canopus converter. The color bleeding doesn't look that bad to me, I'm not even able to tell the difference between a capture and normal playback on TV. And the "locked audio" feature is priceless to me
(I've bought a 1st-gen Radeon VIVO when it came out in 2000, but was very displeased with the Rage Theater chip : the luma was always shifting up-and-down even in a same scene, and with an Athlon 1GHz it was next to impossible to keep the audio synced for more than 5 minutes at a time. )
I'm more bothered by the horizontal noise and typical VHS color-bleed at the top of the frame, but since the tapes were recorded on an entry-level standard VHS VCR there's not much I can do about this except post-process.
But if someone has a suggestion for a good capture equipment that costs less than $100 and can do better than the ADVC110, I'm all ears...

My tapes are important to me (those are archives of a now-defunct very funny daily show), but not as important as family memories would be, so it's no biggie if some of the picture details are "lost in translation", the audio track is actually much more important...

[Snakeforhire]

I looked at the video file. You have some issues with blown-out highlights. If those are on the original, you can't do anything about it, but if not, check to see if there any proc amp settings on your AVC-110. Most DV converters don't have such settings, but maybe yours does.

I am not at my main computer today, but when I get back to it, if I have a chance, I'll run your video through one of my scripts and see if they do much. That video is pretty bad, and only so much can be done with it.

I've noticed this too, but it only seems to show when the background is all-white (that's the reason I picked this particular scene) and when played on a computer, when outputting to a TV the whites and blacks seem OK -as far as they can be for a VHS tape anyway... :/

[Snakeforhire]

I found this old script when backing up old HDDs to my FreeNAS server.
I probably adapted it from scripts found on doom9, but I have no recollection whatsoever...
Any thoughts on the filters and settings used ?


###### Modèle de script Avisynth pour captures PAL DV/VHS
# INPUT :

DirectShowSource("l:\rush&completed\51st.avi",25)

#########################
x1=12 # left
x2=46 # right
y1=128 # top
y2=128 # bottom
#start=292 # useful only for CCE
#end=131128
################################################## #

# Trim(start,end) # useful only for CCE


ConverttoYV12(interlaced=true) # common to most filters
# ColorYUV(autogain=true) # ,autowhite=true

SeparateFields() # Important !!

# script default : Convolution3D(1, 6, 10, 6, 8, 2.8, 0) for camcorder DV AVI
# preset="movieHQ" (good DVD source) (0, 3, 4, 3, 4, 2.8, 0)
# preset="movieLQ" (noisy DVD source) (0, 6, 10, 6, 8, 2.8, 0)
# preset="animeHQ" Anime Hi Quality (good DVD source) (0, 6, 12, 6, 8, 2.8, 0)
# preset="animeLQ" Anime Low Quality (noisy DVD source) (1, 8, 16, 8, 8, 2.8, 0)
# preset="vhsBQ" VHS capture Bad Quality (0, 32, 128, 16, 64, 10, 0)

odd=SelectOdd.Chromashift(C=0,U=0,V=0,L=0,R=0,G=0, B=0)
odd=SelectOdd.Undot().Deen()
odd=SelectOdd.DeSpot(p1=32,p2=16,pwidth=16,pheight =4,maxpts=64,minpts=1,color=true,mthres=32,interla ced=false,seg=0)
#odd=SelectOdd.Guavacomb(mode=PAL,recall=85,maxvar iation=40,activation=40)
odd=SelectOdd.Cnr2("xxx",4,5,255)
odd=SelectOdd.VagueDenoiser(threshold=1.5,method=1 ,nsteps=6,chromaT=0.8)
odd=SelectOdd.Convolution3D(preset="vhsBQ")
odd=SelectOdd.ConvertToYUY2().FixVHSOversharpL(30, 12,18).FixVHSOversharp(30,14,10).FixVHSOversharp(3 0,12,8)

evn=SelectEven.Chromashift(C=0,U=0,V=0,L=0,R=0,G=0 ,B=0)
evn=SelectEven.Undot().Deen()
evn=SelectEven.DeSpot(p1=32,p2=16,pwidth=16,pheigh t=4,maxpts=64,minpts=1,color=true,mthres=32,interl aced=false,seg=0)
#evn=SelectEven.Guavacomb(mode=PAL,recall=85,maxva riation=40,activation=40)
evn=SelectEven.Cnr2("xxx",4,5,255)
evn=SelectEven.VagueDenoiser(threshold=1.5,method= 1,nsteps=6,chromaT=0.8)
evn=SelectEven.Convolution3D(preset="vhsBQ")
evn=SelectEven.ConvertToYUY2().FixVHSOversharpL(30 ,12,18).FixVHSOversharp(30,14,10).FixVHSOversharp( 30,12,8)

# EXPERIMENTAL
# defreq debug : show=2, info=true
# defreq(fx=75,fy=0,fx2=50,fy2=0,fx3=57,fy3=-14,fx4=85.5,fy4=29)
# DeSpot(p1=32,p2=16,pwidth=16,pheight=4,maxpts=64,m inpts=1,color=true,mthres=32,interlaced=false,seg= 0)

#Greyscale()
#UtoY()
#VtoY()
#descratch debug : mark=true
#turnleft()
#descratch(mindif=6,maxgap=64,minlen=16,maxlen=700 ,maxangle=10,keep=75,modeY=3,modeU=3,modeV=3).turn right()

#####################################

Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd() # bottom field first !

Crop(x1,y1,-x2,-y2) # left top right bottom
AddBorders(x1,y1,x2,y2)

#####################################

# Blockbuster(detail_min=4,detail_max=7,method="nois e") # for MPEG/DIVX material

# LanczosResize(352,576) # for CCE

ConvertToRGB32(interlaced=true) # 4 TMPGEnc

#RGBAdjust(rb=16,gb=16,bb=8) # for CCE

#################################

[Snakeforhire]

Alright, I checked out the VCR's settings, I got the following :
B.E.S.T. (auto-correction according to tape quality detected) : auto or manual
Image control : auto / copy / net - / net +
(the manual says : "auto : use BEST system", "copy : minimizes image deterioration during an edit (record or play)", "net - : reduces bad image quality when viewing often-used tapes containing noise", "net + : offers sharper and brighter image when viewing images containing plane surfaces of uniform colors like anime")
Video stabilizer : manual / auto
("manual says "when on M eliminates automatically the vertical shaking when viewing unstable recordings from another VCR")

What would you suggest I use ?

[johnmeyer]

The script you found uses really old, outdated plugins. There are much better alternatives today.

B.E.S.T. sounds like the Edit switch I was talking about and you should use it. The key is to turn OFF all "enhancements," which "net" appears to be.

The stabilizer may, or may not, be a good thing. All I can say is that you simply need to try it and look at the result. Most of this isn't rocket science and especially with stabilization, if it looks better, and if you don't detect some massive loss in detail, then use it. The only thing that is sometimes a tough sell to people is to use the Edit switch because the image will at first appear to be noisier and less "sharp."

Oh yes, turn off all noise reduction, because like sharpening, it degrades the pictures. Let all that nasty dot noise come on through and then kill it with a modern de-noiser. Much better result.

[Snakeforhire]

OK TY. What filters would you suggest I use then ?

[jagabo]

Your source isn't really interlaced. It's really 25p (with blended fields). I'd start with something along the lines of:

AviSource("sample.avi") 
ColorYUV(gain_y=-16)
Crop(10, 8, -42, -8)
ConvertToYV12(interlaced=true)
QTGMC(Sharpness=0.5)
SRestore(frate=25)
TemporalDegrain(SAD1=200, SAD2=150, sigma=8)
MergeChroma(BilinearResize(160, height).aWarpSharp(depth=15).Spline36Resize(width,height))
ChromaShiftSP(x=3, y=1)
AddBorders(24,8,28,8) # if you need the original frame size

There's a severe discoloration to the right of the bright reds, but only in the V channel. I'd have to experiment a bit to address that.

[Snakeforhire]

I've browsed around and apparently the best entry-level USB capture device is this :
https://www.amazon.com/Diamond-VC500-Capture-Software-Convert/dp/B000VM60I8/ref=dp_ob_title_ce

Has anybody ever used this one ? Does it allow uncompressed capture ?

[manono]

You don't want uncompressed. You want lossless. And the answer to the question is 'yes'. Just don't use the included software for capturing.

[jagabo]

The Hauppauge USB Live 2 is decent too. http://www.hauppauge.com/pages/products/data_usblive2.html

[johnmeyer]

OK TY. What filters would you suggest I use then ?

I already gave you that answer in my post above.

[Snakeforhire]

Your source isn't really interlaced. It's really 25p (with blended fields).

Huh ? What makes you say this ? DV Type 2 is 25i, even GSpot says so when I run a sample in it. :S
Unless you're talking about the VHS tape as the "source" ?

[Cornucopia]

ALL DV (incl. DVC, DVCPro, DVCam, DVCPro50, DVCPro100/HD) is stored & flagged as interlaced. But that says nothing about whether the ORIGINAL material was shot/edited interlaced or not. That depends on whether the "timestamp" of each field is unique (aka comes from a different sample point in time).

BTW, Type 1 or Type 2 has nothing to do with it.

When talking about interlacing, it is always best to reference how it got created & stored on the source and the intervening steps.

If your material is all analog camcorder material which was saved on VHS, that's 100% interlaced.
If your vhs material has stuff from TV or other transfers (e.g. Film->PAL xfer with speedup 24 to 25fps and 2:2 pulldown), there is a very good chance of it being PsF or "Progressive-Stored in-Fields" (aka "Progressive-Segmented-Frames"). Which will look like interlaced to most programs including Gspot, but in fact is NOT. and if transfered and converted PROPERLY, you can restore it to its original progressive format WITHOUT ANY LOSS IN QUALITY. If not done correctly, you can most assuredly wreck it beyond fixing (by blending fields, etc).

Scott

[jagabo]

Your source isn't really interlaced. It's really 25p (with blended fields).

Huh ? What makes you say this ? DV Type 2 is 25i, even GSpot says so when I run a sample in it. :S
Unless you're talking about the VHS tape as the "source" ?

It's encoded interlaced but the underlying material was originally 25p. Look at 5 consecutive fields:


[Attachment 49268 - Click to enlarge]

Every other field is a blend of the field before and a field after, and the entire clip shows this pattern. So the original material was 25p and you can effectively restore that 25p as was shown in the script I gave -- QTGMC().SRestore(frate=25). Even simpler (and faster) would be QTGMC().SelectOdd() but it's possible that the pattern will change at some point and you'll need to SelectEven() rather than SelectOdd(). SRestore() will automatically adapt. You'll get better quality, better filtering, and better compression from 25p, rather than leaving it 25i.

[Snakeforhire]

I see. Thx for claryfying
My material was recorded direct-from-broadcast analog TV from a live show (no telecining or anything like that), the sample I posted is typical of it.
So based on what you're saying, I should use SRestore(), no matter what other filters I apply ?
Should this be applied before or after any denoising plugins by the way ?

[jagabo]

This particular type of field blending is very unusual. Even odder, only the luma is blended. The chroma is not. I've dealt with hundreds of PAL videos in the past and this is the first time I've seen it. Recording broadcast video onto VHS tape wouldn't cause it. Capturing your tapes with an ADVC 110 wouldn't cause it. So it must have been broadcast that way.

Things like this is why there is no single optimal script. You have to analyze each source and handle it appropriately. The script I gave was for this particular sample. You would not use the same processing for more "normal" PAL videos.

In general I like to make color/levels adjustments first, deinterlace or inverse telecine next, noise reduction after that, then final touchup if necessary.

[johnmeyer]

If the ADVC 110 has settings for frame rate, that might cause the problem (e.g., setting it for NTSC when the source was PAL).

[jagabo]

I played around with the video a bit more. First I tried removing the blending from the blended frames. Here's a blended field before and after deblending:


[Attachment 49279 - Click to enlarge]

The basic idea is that a blended field consists of a 50:50 blend of two fields, A and B: blend = (A+B)/2. Since we have a clear version of B we can reverse the process to restore A. Algebraically:

blend = (A + B) / 2
blend = A/2 + B/2
blend - B/2 = A/2
(blend - B/2) * 2 = A

And translated to AviSynth:

SeparateFields()
src=last
Overlay(last, last.Crop(0,1,-0,-0).AddBorders(0,0,0,1).Loop(2,0,0).ColorYUV(gain_y=-128, cont_u=-128, cont_v=-128), mode="Subtract")
ColorYUV(gain_y=256, cont_u=256, cont_v=256)
MergeChroma(src.Trim(1,0))
Interleave(last.SelectEven(), src.SelectOdd())
Weave()

This leaves you with 25p. Unfortunately, the blended fields have more noise than the unblended fields. And this method inherently increases that noise. And this left some aliasing -- so the restored field may not perfectly match the "missing" field.

Then I decided to try completely discarding the blended fields and resizing the remaining fields to full height with nnedi3(dh=true).

SeparateFields()
SelectOdd()
nnedi3(dh=true)

I followed that with QTGMC(InputType=1) to perform some edge cleanup and a strong TemporalDegrain(SAD1=400, SAD2=300, sigma=16) to reduce the hum bars in the original video. I changed the chroma sharpening and shifting a bit to match the new method. I thought this looked best. See the attachment...

[Snakeforhire]

Wow ! You're a true wizard, mate ! That is absolutely brilliant !
Could you post the complete script you used please ? I guess I'll have to tweak it a little bit but this would be a great start

[jagabo]

I've been playing with the script but I think this is close to what was used for the clip I uploaded:

AviSource("sample.avi") 
ColorYUV(gain_y=-16)  # pull brights down to legal levels
Crop(10, 8, -42, -8) # get rid of noisy borders

# turn odd fields (no blending) into full frames
SeparateFields()
SelectOdd()
nnedi3(dh=true)

ConvertToYV12()
#QTGMC(InputType=1) # clean horizontal edges
TemporalDegrain(SAD1=400, SAD2=300, sigma=16) # heavy noise reduction, temporal only
MergeChroma(BilinearResize(160, height).aWarpSharp(depth=15).Spline36Resize(width,height)) # sharpen colors
ChromaShiftSP(x=2) # shift the colors left a little
AddBorders(24,8,28,8) # restore the frame size, only if you need it

I usually don't like to use such strong settings with TemporalDegrain -- it can lead to smearing during motion, especially panning shots. But it was necessary to remove the hum bars. There's a filter called defreq() (I've never used it) that may reduce the hum bars, enough to let you use use weaker TD settings.

At some point I disabled QTGMC. I don't remember for sure if I did that before encoding the sample I uploaded. It's commented out here but you can restore it if you find you need more edge cleanup.

Also, the sequence for converting the unblended fields to frames assumes the odd fields are always the clean ones. If you find the video switches phase at times you'll have to switch from SelectOdd(), to SelectEven() for those sections.

[Snakeforhire]

[QUOTE=j Also, the sequence for converting the unblended fields to frames assumes the odd fields are always the clean ones. If you find the video switches phase at times you'll have to switch from SelectOdd(), to SelectEven() for those sections. [/QUOTE]

Uh-oh... So if I'm following right, I should apply this script BEFORE I edit the rushes with Virtualdub, otherwise I risk having 2 consecutive blended frames at some point, right ?