VHS capture (ati 600 usb , vdub, svideo, huffyuv) - needs cleaning

[TheCage]

Hello guys

I;m looking for help writing avisynth script to clean and encode this 1992 family archive vhs video...
I've been working with avisynth and VirtualDud/MeGui once in a while. I never improved my skills that much. Restoration of old VHS image video is probably the hardest thing in video filtering and requires highest level of skills in avisynth world that i dont have. I see many problems with the image but i cannot say even what is the name for them and what filter is required....
I've seen some unbelievable jobs ( https://vimeo.com/2823934 ) that proves nothing is impossible (of course if there is great capture source . Its hard to build similar experience and skills so im thankful for this place!

About the capture
I did my best with this one.
10GB , 16mins, Huffyuv, svideo, captured with Ati wonder 600 USB, VirtualDub 1.10. WIndows 7 x64.
VCR: Panasonic NV-FS200 (I used this settings -> check in the bottom

Few years ago i wanted to buy external TBC, but i realized that i have 6 tapes in the rack... so it wasnt enough to turn it in a hobby...

I'm able to capture the video again if necessary!
Planing to encode with Megui , x264 profiles. If you have other suggestions please let me know.

VIDEO SAMPLE LINK (12mb, 1sec long)

[killerteengohan]

The only avisynth filter I can think of would be use santiag(strv=0, nns=4, nsize=5) to reduce the visibility of all those jagged areas. Just that one avisynth filter in megui looks better to me.

[LMotlow]

Thanks for the sample. When it disappears from your off-forum site, much of this thread will be useless.

Good color and detail. The ATI card is cutting out at y=16, as usual, so dark shadows will look rather dense. Black levels are a little high and brights are clipped, the bright output exceeds y=255; since YUV expansion in RGB can't exceed 255, some bright details will be lost in display. y-luminance in YUV should not be higher than y=235, which in RGB expansion would hit 255 on the nose. Bright colors in the sample are clipped in RGB, saturation is a bit high.

One second of video isn't that much to work with, but it looks as if you have some kind of subtle RF noise in the video. The sample is too brief to really get a handle on it. I hesitate to post a script because 1 second won't prove much, and a longer clip of 8 to 10 seconds with some kind of motion would probably support a different filter set.

The attached .avi is in YV12 and lossless Lagarith for encoding, if you wish. The Lagarith codec will be very handy once you start working with Avisynth. You can get its handy-dandy no sweat installer at https://lags.leetcode.net/codec.html. You can read comments by its happy users at https://www.videohelp.com/software/Lagarith-Lossless-Video-Codec.

[TheCage]

Thanks LMotlow

yes i did some changes to brightness before capture... I usually prefer much better dark but in this video there are a few moments where the shadows were totally lost...So i thought is better to capture with higher brightness and then use filters.

I wasnt sure how big file size is allowed so i used wetrasnfer... I saw now 500mb file size is allowed here so i'll attach another sample and i'll try to put all important scenes.
The last scene - corridor - is where i was losing shadow details. if i lower the brightness the shadows were totally lost.

new sample - 20sec - 250mb

[LMotlow]

Thanks again for the sample.

I do see some mild herringbone noise, especially in the reds. I'll work up a script to post later and see what can be done.

[jagabo]

If your VHS deck has a sharpness control turn it down. It don't really sharpen the picture and the oversharpening halos it creates are hard to remove without damaging the picture. I tried to reduce them here without damaging the picture too much.

AviSource("1st-vdub-svideo-Huffyuv-sample2.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true) # eliminates Hanover bars, need YV12 anyway
Crop(12,0,-4,-0) # get rid of black bar on left, ITU frame size
ColorYUV(gain_y=-20, off_y=-2) # minor levels adjustment
BilinearResize(480, height) # downscale width to reduce herringbone/dot crawl noise
QTGMC(EZDenoise=1.0, DenoiseMC=true) # deinterlace, light noise reduction
Dehalo_alpha(rx=3.0, ry=1, BrightStr=1.2, DarkStr=1.2, LowSens=10, HighSens=10) # reduce oversharpening halos
MergeChroma(aWarpSharp(depth=10)) # sharpen chroma, not luma
ChromaShiftSP(x=-1, y=1) # shift the chroma
Spline36Resize(768, height) # 4:3 frame
Sharpen(0.4, 0.2) # light sharpening

[LMotlow]

@jagabo: another good job, as usual. Yep, halos are the pits.

I took a somewhat different route. I used denoising and resizing on the entire video, but cut it into 3 clips for levels and color. I ran it all in one script, but one could use three scripts and join the pieces later. It's very common to split VHS into parts because VHS varies minute to minute and scene to scene.

Below is image 14 in the original sample, with a YUV levels histogram and an Analyze readout (click for original image size). Note that the original borders have been cropped off to avoid affecting the histogram. You can see in the histogram (top band) that brights extend beyond y=235 at the right side of the graph. At the right side of the levels histogram note that there is a tall thin "spike" at the left, indicating that the ATI 600 is clipping blacks. It clips before VirtualDub gets the signal, so an external proc amp would be the only way around it. You can raise the blacks but, as you can see by the spike, you also raise the black clipping point. The numbers in the right-hand luma column show brights at 255 )which way over y=235) and the darkest blacks clipped at Y=22. Note that the original borders have been cropped off to avoid affecting the histogram.





Below, a 2x blowup of some of herringbone effects and clipped blacks. Clipped blacks have little or no detail.




below, frame 114 after levels adjust and partial denoising. At least the girls don't look as if they're wearing black wigs, and you can see the faces of the kids in the lower left corner.. Resizing and QTGMC.s EZDenoise was used for the herringbone. A ContrastMask filter was used to expand the available dark detail and brighten overall gamma.



Below, a 4x3 image from frame 440 in the original sample (left) -vs- levels fix on the right.


The script I used, splitting the original clip into three poarts and joining at the end. Thism is the script for the attached interlaced 576i mp4. For progressive video and resizing to 4x3 (640x480), remove the line "SeparateFields().SelectEvery(4,0,3).Weave()" near the end of the script and relpace it with whatever resizing you want.

AviSource("I:\forum5\VH\TheCage\B\1st-vdub-svideo-Huffyuv-sample2.avi")
AssumeTFF()
SeparateFields()
FixVHSOversharp(20,16,12)
FixVHSOversharp(20,8,4)
FixVHSOversharpL(20,12,8)
Weave()
ConvertToYV12(interlaced=true)
QTGMC(preset="very fast",EZDenoise=2,denoiser="dfttest",ChromaMotion=true,\
   border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse()
BicubicResize(width/2,height)
Spline36Resize(720,height)
DeHalo_Alpha()
Santiag(2,2)
FixChromaBleeding()
ChromaShift(V=2,L=-2)
MergeChroma(aWarpSharp2(Depth=20).aWarpSharp(depth=10))
Input=last

PART1=Input.Trim(0,412)
Part1
ContrastMask(enhance=8.0)
ColorYUV(off_y=-20)
Tweak(cont=1.05,sat=1.2,dither=true,coring=false)
Levels(8,1.2,255,16,245,dither=true,coring=false)
Part1A=last

PART2=Input.Trim(413,877)
Part2
ContrastMask(enhance=5.0)
ColorYUV(off_y=-20)
Tweak(cont=1.03,sat=1.2,dither=true,coring=false)
Levels(12,1.2,255,16,245,dither=true,coring=false)
Part2A=last

PART3=Input.Trim(878,0)
Part3
ContrastMask(enhance=10.0)
ColorYUV(off_y=-20)
Tweak(cont=1.03,sat=1.2,dither=true,coring=false)
Levels(8,1.2,255,16,245,dither=true,coring=false)
RemoveDirtMC(30,false)
Part3A=last

Out=PART1A+PART2A+PART3A
Out
grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(1.5,1.5)
Crop(16,2,-8,-14).AddBorders(12,8,12,8)
SeparateFields().SelectEvery(4,0,3).Weave()

return last

You don't have to go entirely by this script, this is just suggestions for various problems. ContrastMask(), ColorYUV(), Tweak() and Levels() are used to bring up shadows as much as they can, control highlight extension, and massage the overall image. Add those commands one at a time and observe the image and the YUV histogram while you tweak their settings. Also note that I didn't get into RGB filters, which would be used to tame the oversaturated brights. Unfortunately the dropouts (horizontal ripples) persist for too many frames for any filters that might be useful, and there is too much motion involved. One of those dropouts lasts for almost 80 fields!

[johnmeyer]

The levels seem very wrong to me: crushed blacks and blown out whites. You are better off capturing with the proc amp set to produce a slightly low-contrast picture and then correct in post. You won't be able to recover any detail in the blacks of the sample you created, and the blown-out highlights in the shirts are gone for good.

[LMotlow]

The levels seem very wrong to me: crushed blacks and blown out whites. You are better off capturing with the proc amp set to produce a slightly low-contrast picture and then correct in post. You won't be able to recover any detail in the blacks of the sample you created, and the blown-out highlights in the shirts are gone for good.

The ATI 600 USB clips blacks before the signal reaches VirtualDub. VDub's hook into the 600's proc amp won't fix it. You need an external proc amp to adjust black levels before they enter the 600. You can fix the highlights with the Levels() filter, much of the brights can be recovered. See the Levels() entries in the above scripts, which calm the highlights and contract output highs to y=235. It's too late for the blacks; they're crushed before the capture software sees them. It's a pain with the ATI 600 USB and the Hauppauge Live-2 USB.

[TheCage]

Thank you very much for your work!
LMotlow i totally love your sample from post #3!!! why is that? I prefer it more then the samples from post #7.
jagabo - your sample looks great too!

Now im sorry i didnt buy TBC-1000 from ebay 7 years go... a few auctions ended below 150$ used that time. i see them very expensive nowadays...

I did capture the video with 124-brightness
see what is the difference with lower levels... If you want i'll capture again with zero sharp and lower brightness!


https://files.videohelp.com/u/219510/brg108.png
https://files.videohelp.com/u/219510/brg112.png
https://files.videohelp.com/u/219510/brg118.png
https://files.videohelp.com/u/219510/brg124.png

[TheCage]

LMotlow can you please post the script from post #3 sample?

[LMotlow]

LMotlow can you please post the script from post #3 sample?

AviSource("J:\forum5\faq\TheCage\1st-vdub-svideo-Huffyuv-sample.avi")
AssumeTFF()
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
ColorYUV(off_y=-5,cont_u=-10,gain_u=-10,off_u=-2,cont_v=-10,off_v=-2)
Levels(16,1.0,255,16,235,dither=true,coring=false)

QTGMC(preset="medium",EZDenoise=4,denoiser="dfttest",ChromaMotion=true,\
   border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse()
Santiag(2,2)
MCTemporalDenoise(settings="low",enhance=true)
DeHalo_Alpha()
Crop(16,2,-8,-14).AddBorders(12,8,12,8)
SeparateFields().SelectEvery(4,0,3).Weave()   #<- reinterlace

LMotlow i totally love your sample from post #3!!! why is that?

Processing is different for each. I like #7 better, but....you are at liberty to make your videos look the way you want.

[TheCage]

thanks

do we see them in the same way? i'm using MPC-Enhanced Video Renderer

above #3 post
below #7 post 576p

[jagabo]

Here's one that uses FineDehalo() instead of Dehalo_alpha(). The former doesn't remove halos as much but it's also less damaging to the picture. Probably a better compromise.

LWLibavVideoSource("1st-vdub-svideo-Huffyuv-sample2.avi") 

AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(12,0,-4,-0)

ColorYUV(gain_y=24, off_y=-24) # adjust levels to make full range YUV
ColorYUV(gamma_y=50) # gamma works better with full range YUV
ColorYUV(levels="PC->TV") # convert to limited range YUV

BilinearResize(480, height)
QTGMC(EZDenoise=4.0, DenoiseMC=true)
FineDehalo(rx=2.0, ry=1.5, BrightStr=1.5, DarkStr=1.5)
MergeChroma(aWarpSharp(depth=10))
ChromaShiftSP(x=-1, y=1)
Spline36Resize(768, height)

[TheCage]

Thanks
I need some time to load the scripts... giving me errors all the time... i have to set up avisynth plugins...

[jagabo]

QTGMC is one of the hardest filters to get set up because it requires several other filters. You have to locate, download, and install them all. Be sure they are all the same bitness as the AviSynth you are using. And the editor/encoder must also be the same bitness. Ie, everything must be 32 bit, or everything must be 64 bit.

[LMotlow]

Thanks
I need some time to load the scripts... giving me errors all the time... i have to set up avisynth plugins...

The version of QTGMC that I use in forum posts is the "original" version 3.32, which is an .avsi file. The geeks keep spitting out updated versions, all of which have glitches until the update to the update comes out 15 minutes later, and so forth. So I end up with several subfolders in my Avisynth plugins folder, with names like QTGMC-2, QTGMC-2019, etc. Each subfolder has the files that match that version of QTGMC. So, in a script I just LoadPlugin the dll's and Import the .avsi and .avs scripts that I need for that version.

Attached is a .zip of the original QTGMC 3.32 .avsi, its support files, MFC runtimes that you might not have, and instructions, plus documentation for all the files. I picked this up a short while back on another forum. There are instructions telling you what goes where. Go into your Avisynth plugins and make a "QTGMC" subfolder for use with that version of QTGMC. Most of the support files are actually currently in use and should work with just about everything.

[TheCage]

Thanks... it is working!
LMotlow - can we fix the interlace in the last script? i see horizontal lines in this sample. in the samples from #7 there is deinterlace

jagabo - i'll encode your script too! It is ok to have 2 versions of the video!

What file size for this 16min video is reasonable for final video!? 500-600Mb?

I used to encode with MeGui + presets x264 years ago... then i found out that these are too old and not useful anymore...

What is the best way to encode these days?

Thanks
Alex

[jagabo]

You should learn what the tools do and pick the filters and settings that give you what you want. But as I mentioned before, if you player has a sharpness control turn it down. Fixing the oversharpening halos in software damages the video. You can sharpen the video in software after capturing and get better results than the play produces.

The samples I uploaded were encoded with x264 CLI at CRF=18 and the slow preset. In CRF mode you pick the quality you want and the encoder uses whatever bitrate (and hence whatever file size) is necessary to deliver that quality. At around 18 the decoded video looks about the same as the source at normal playback speed. But if you zoom in on still frames you'll see differences. Lower CRF values will deliver higher quality and larger files. Higher CRF values will deliver lower quality and smaller files.

If you're used to MeGui you can continue using it. Newer tools won't necessarily get you better quality unless you are looking to cram as much video as you can in a fixed space.

[TheCage]

I'll do another capture. When you talk about turning down the sharpen you mean on the VCR control and also on Levels in VirtualDub or only VCR and leave the defaults on Vdub levels? I guess turn them all?
Also when I capture again with no sharpen i'll have to remove oversharpen filters from the sripts right?

And you suggest to use Lagarith over Huffyuv to capture or only for workingwith AviSynth?

Is it normal every VHS to have some noise in the audio?

[jagabo]

I'll do another capture. When you talk about turning down the sharpen you mean on the VCR control and also on Levels in VirtualDub or only VCR and leave the defaults on Vdub levels? I guess turn them all?

Turn off the sharpen filter in the VCR and the capture devices proc amp controls. The former causes halos, the latter just increases noise in VHS caps.

Also when I capture again with no sharpen i'll have to remove oversharpen filters from the sripts right?

Yes. And other changes may need to be made too.

And you suggest to use Lagarith over Huffyuv to capture or only for workingwith AviSynth?

For capture, whatever works. They're both lossless if used correctly. Huffyuv is faster on a single threaded computer. Lagarith may be faster on a multithreaded computer.

Is it normal every VHS to have some noise in the audio?

Yes. Especially if you're using the linear audio track, not the Hi-Fi track.

[TheCage]

Hello again

I'm sorry for bringing back the thread but i'll need a little more help.

A few weeks after the last post here i bought Panasonic DMR-ES10. I did another capture of the tape using es10 and i think i got a better picture.

The problem is that the scripts that were written here by jagabo and LMotlow are not compatible with the new video in my opinion. I'm asking for a help to clean the image for one last time for this tape.



You can check the difference using es10 and without

Here is the latest capture
https://files.videohelp.com/u/219510/1st3-vdub-svideo-Huffyuv-sample_cut.avi

[jagabo]

The script I gave in post #14 works for the new video.

[Skiller]

Just a quick side note (I have a NV-FS 200 myself), as it appears in your photo in the first post you are using EDIT mode, which is good. The sharpness slider does not have any effect in this mode so do not bother with it – EDIT means all filters are off, including sharpness. You get what's on the tape.

[TheCage]

Ok thanks Skiller

jagabo i thought that because there is less noise in the picture it will need to remove some of the lines in the script or something like that..
if you say its good to go than ok!

also i still have some sympathy for LMotLow script from post #3 but with the new video it looks much darker.

[jagabo]

If you want less noise reduction change the EZDenoise value in QTGMC. Lower values will give less noise reduction (I usually stay below 2.0). You can also remove the ColorYUV calls if you want to keep the levels closer to the original. And you can remove the head switching noise at the bottom of the frame:

LWLibavVideoSource("1st3-vdub-svideo-Huffyuv-sample_cut.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(12,0,-4,-10).AddBorders(0,0,0,10)
BilinearResize(480, height)
QTGMC(EZDenoise=2.0, DenoiseMC=true)
FineDehalo(rx=2.0, ry=1.5, BrightStr=1.5, DarkStr=1.5)
MergeChroma(aWarpSharp(depth=10))
ChromaShiftSP(x=-1, y=1)
Spline36Resize(704, height)

[TheCage]

Оo that looks really good. Thanks

I havent use LWLibavVideoSource import before.

[jagabo]

One of your dark clips white balanced and gamma adjusted. Original on left, adjusted on right: