YAVHSR (Yet Another VHS Restoral): "LILI"

[jmac698]

I think lowest common denominator is the phrase, and yes, framerates are stored in avi as rather large numbers so there can be quite subtle differences.

[jagabo]

So Avisynth isn't looking at the "fps", it's using the numbers in the computation., e.g., "14988/625" ?

That's the only FPS there is an an AVI file -- the numerator/denominator pair.

[jagabo]

Oops, wrong thread.

[sanlyn]

jmac698 and jagabo, now I see. That certainly explains the message, and more.

[AlanHK]

I don't think that shows the actual values in the AVI file. For example, I have an AVI here with 23976/1000 but Info() reports "23.9760 (2997/125).

Ah, I see, they just reduced the values to the lowest -- uh, what's it called? -- whole number equivalents? Ie, 23976 / 8 = 2997, 1000 / 8 = 125. So yes, Info() will show you if there is a difference.

Yes, and even if there is no practical difference, (they're all 23.9760 to several decimal places) if they're not reported to be exactly the same Avisynth can't join them.

Another reason to prefer PAL. Not much variation seen on 25/1.

[manono]

Yes, and even if there is no practical difference, (they're all 23.9760 to several decimal places) if they're not reported to be exactly the same Avisynth can't join them.

But you can easily make them the same by adding an AssumeFPS(23.976) or whatever, to both clips. The difference is so minute it'll do nothing to make the audio go out of synch.

[jagabo]

Yes, the difference between 23.976 and 24000/1001 is only 3.6 ms after an hour.

[AlanHK]

Yes, and even if there is no practical difference, (they're all 23.9760 to several decimal places) if they're not reported to be exactly the same Avisynth can't join them.

But you can easily make them the same by adding an AssumeFPS(23.976) or whatever, to both clips. The difference is so minute it'll do nothing to make the audio go out of synch.

Yes, I said that in my previous post. Info tells you which ones you need to do that for, or if you have an fps that is really different -- like 24, or 25, or something else, and you may then have to worry about sync.

Also better I think to use AssumeFPS("ntsc_film") which is an alias for the exact integers 24000/1001 rather than the decimal approximation.

[sanlyn]

Yes, and even if there is no practical difference, (they're all 23.9760 to several decimal places) if they're not reported to be exactly the same Avisynth can't join them.

But you can easily make them the same by adding an AssumeFPS(23.976) or whatever, to both clips. The difference is so minute it'll do nothing to make the audio go out of synch.

@manono: that's what I did. Worked OK. Don't know what took so long to figure that out, it was right in front of me in Avisynth docs. I'm a sloooow learner.

[sanlyn]

One scene with really ugly color problems. From various discussions I learn that these effects are caused by (a) lousy mastering for the VHS, WNET's digital broadcast, and TCM's digital Broadcast, (b) Macrovision effects, and (c) VHS that's been played to death. The scene starts green, then red and green and blue "pump" for several seconds. None of the colors look correct anyway: the girl's clothing is supposed to be dull gray, the purse and suitcase black, lighting is harsh early morning sun -- no telling what the colors of the buildings are, they look different in every shot.

Here is Lagarith YUY2 AVI as captured (it has been ivtc'd, nothing else). Color pumping is obvious, especially in the brights:

http://dc595.4shared.com/download/GgXSSnOb/A01f_L3A_ivtc.avi (49 MB)

This is Lagarith YUY2 AVI of the histogram. Note bright pumping, especially blue:

http://dc589.4shared.com/download/ufWuGnI-/A01f_L3A_ivtcHIST.avi (2.4 MB)



After much filtering, despotting (exactly the same spots, cherries, grapes, etc., are in the digital broadcasts), AutoLevels, SmoothLevels, ColorYUV, MCTemporalDenoise, QTGMC, and lord knows what else. This is the first several seconds of the scene in MPEG2. You can see why most "auto" controls don't work here. They make levels and color balance worse whenever anything moves.

http://dc528.4shared.com/download/F1PbACnT/A01f_L3A_ivtc_RSYout.m2v (20 MB)

These effects are in all versions, from all sources. There are complaints about the godawful color in reviews on Amazon and elsewhere, and the problems aren't peculiar to VHS. The effects gradually fade; after 20 minutes they're less visisble. By 40 minutes, they're gone (thank goodness) and colors look normal, even on old tape. In WNET's versions the effect is partially masked by a heavy red color balance; TCM's is much too green and soft but it's still there.

My brilliant idea is to (a) live with grunge in this scene and move on, or (b) figure what I can do to try to stablilize red and blue layers separately, then rejoin in either YUV or RGB. Haven't found any ideas yet.

[sanlyn]

Tried for a couple of hours to create a negative grayscale mask to use for opening up shadow details and taming highlights. That would imvolve a lot of trial-and-error creating "lut" statements in masktools. So I thought I'd try a script from another thread that separated dark and light masks: https://forum.videohelp.com/threads/346098-applying-denoisers-to-different-color-spaces...=1#post2161883

But all I could get from this script:

AviSource(v"L3B1_ivtc_RS.avi")

umask=UtoY().BicubicResize(width,height).mt_lut("x 130 - abs").mt_lut("x 12 - 16 *")
vmask=VtoY().BicubicResize(width,height).mt_lut("x 130 - abs").mt_lut("x 12 - 16 *")
mask=Merge(umask, vmask).Invert()

mt_merge(ColorYUV(cont_y=-20), last, mask)

...is exactly the same thing I get from this script:

AviSource(v"L3B1_ivtc_RS.avi")
(ColorYUV(cont_y=-20)

After reading doom9 threads on masktools since 6:30 AM today (which was 15 hours ago), I'm putting it to bed for the night.

[robjv1]

Just wanted to thank you again for all of these posts -- this has been a fun project to watch develop and I've certainly learned a thing or two from it as well!

[jagabo]

But all I could get from this script:

AviSource(v"L3B1_ivtc_RS.avi")

umask=UtoY().BicubicResize(width,height).mt_lut("x 130 - abs").mt_lut("x 12 - 16 *")
vmask=VtoY().BicubicResize(width,height).mt_lut("x 130 - abs").mt_lut("x 12 - 16 *")
mask=Merge(umask, vmask).Invert()

mt_merge(ColorYUV(cont_y=-20), last, mask)

...is exactly the same thing I get from this script:

AviSource(v"L3B1_ivtc_RS.avi")
(ColorYUV(cont_y=-20)

So what you're trying to do is reduce the contrast of the most saturated portions of the picture? Change -20 to a much bigger value and it will be more obvious.

By the way, the occurrences of 130 should be 128. As was discussed later in that thread.

[poisondeathray]

For the "color pumping" , I think you can stabilize most of it with "auto" filters, then fine tune with whatever color adjustments after. I think if you reduce the pumping, you can at least minimize the work you have to do later in other programs

Something like this will reduce the "pumping", you should tweak settings of course, and you have to make adjustments after (since the colors will be skewed, but more stabilized temporally)

AVISource()
coloryuv(autowhite=true)
coloryuv(autowhite=true)
ttempsmooth(maxr=5, lthresh=8, cthresh=30, strength=6)
coloryuv(autowhite=true)

to create a negative grayscale mask to use for opening up shadow details and taming highlights.

For masks based on luminance (separation between certain Y' values) , I think the LumaMask from masks pack makes life a lot easier (there are also function for box and saturation masks)
http://doom10.org/index.php?topic=2195.0


So you might create a mask for shadows, and one for highlights and apply different selective filters accordingly though those masks

Since you have AE, it has plenty of options for creating masks. The luma key comes to mind. Very powerful mask options and feathering, matte choking/refinement, etc....

If you were only talking about levels (not applying selective filters), then AE has the shadow/highlight filter (it works almost the same as photoshop's shadow/highlight filter)

[sanlyn]

Just wanted to thank you again for all of these posts -- this has been a fun project to watch develop and I've certainly learned a thing or two from it as well!

Appreciated, robjv1. This began in 2005. Talk about a slow learner and accelerated hair loss!

Something like this will reduce the "pumping", you should tweak settings of course, and you have to make adjustments after (since the colors will be skewed, but more stabilized temporally)

AVISource()
coloryuv(autowhite=true)
coloryuv(autowhite=true)
ttempsmooth(maxr=5, lthresh=8, cthresh=30, strength=6)
coloryuv(autowhite=true)

I tried similar autocolors. But as the camera follows LILI down the sidewalk, color balance and gamma change with passing objects. The recessed alcove seen in the longer mpeg would enter the frame as dark cyan, turn blue, then deep red, then hot orange, then green, then exit as blue-gray. Passing door frames changed from dull red to magenta, windows brightened and darkened, etc. When a guy in a big white smock comes out of a shop doorway, the scene darkened and turned blue, and Lili's gray sweater looked lavender. Also some occasional mottling and posterization.

Strong smoothers destroyed detail in brickwork and totally blurred the shadowed alcove. Lili's ratty wool sweater looked like a smooth polyester windbreaker. Avisynth and VirtualDub versions behaved alike. My idea was this sequence: overall level correction, denoise, then pumping. I ended up with the following (actual values varied, didn't stop pumping but slowed it down):

ColorYUV (for levels)
RemoveSpots + DeVCR
QTGMC (InputType=1)
MergeChroma(MCTemporalDenoise)
AutoLevels + ColorYUV + SmoothLevels
VirtualDub AutoLevels + gradation curves
NeatVideo (chroma only + temporal turned on)

I tried all versions of deFlicker. They accomplished little, except to make the shadowed alcove and some doorways and windows pause and change direction a few times as the camera passed (!).

For masks based on luminance (separation between certain Y' values) , I think the LumaMask from masks pack makes life a lot easier (there are also function for box and saturation masks)

Yes, I think I'm over my head in doom9's masktools thread, so I'll try Mask Pack. I noted that autolevels affected mostly luma. So one of my masking ideas was to move U and V into Y and stabilize there. Obviously I'm not doing it correctly and I don't even know it will work.

The scene has really bright, harsh daylight. The pumping looks worse here because it's mostly at the bright end. You mentioned AfterEffects; I have it on another PC, still learning it, but I have a feeling AE might be next. This scene aside, I'm otherwise 25 minutes into the movie where the effect is so subtle it's practically gone.

Ed: Just re-captured early segments with a Panasonic PV-8664 I found on eBay last month. Has dynamorphous heads and apparently a scaled-down version of the DNR found on "AG" models. Calmed the pumping somewhat, not much but visibly improved.

[sanlyn]

So what you're trying to do is reduce the contrast of the most saturated portions of the picture? Change -20 to a much bigger value and it will be more obvious.

By the way, the occurrences of 130 should be 128. As was discussed later in that thread.

Right, that's the idea. And thanks, I found that correction and changed my script.

[sanlyn]

I always suspected that RemoveSpots() did more than despot. I know it's based on RemoveDirt, but I see that some element of its motion block restore code has come in mighty handy. Wish I knew how to tweak it. I took a short clip from LILI, removed about 50% of the frame (yes, that's why it looks so soft) and resized back to 640x480 -- but slowed down to 5 fps.

This is a damaged section of tape. In the first clip note how the buildings warp and jiggle, and nervous activity in the trees. RemoveSpots in the "RS" clip seems to steady things quite a bit, even if it didn't get all the spots. The scene needs much more work, but all I used here was TIVTC and Remove Spots, nothing else.

Don't suggest a JVC player with TBC. The damage you're looking at was caused by a JVC. And after $175 for a rebuild and servicing, at that.

[jagabo]

Keep in mind that when an edge moves by a pixel for a single frame, that one pixel (row or column) may be interpreted as a "spot" by RemoveSpots().

[sanlyn]

That's what I figured. Odd that plain RemoveDirt() doesn't work as well in this respect. I'm often surprised, too, that RemoveSpots often gets spots off some objects that are moving (but still misses a few). One of these days I have to learn to get into the innards of those two plugins and tweak some parameters. At this point the docs for those two still have my head spinning. But RS has come in very handy for some of these damaged scenes.

[vidsbook]

Why can't you IVTC the DVD?

yes

[sanlyn]

What DVD? The VHS capture has been ivtc'd. Has also been thru MCTemporalDenoise and QTGMC (InputType=1), among others.

If by "DVD' you mean any of the digital broadcasts I've recorded, it would make no difference. They have the same problems. Color is worse, the low-bitrate broadcasts have serious compression artifacts and a blurry image, noise in some segments looks worse, the audio is terrible, and there are TV logos all over the place.

[poisondeathray]

It's spam. Look at his link in the sig (Don't click on it) , and the username with 1 post.

[sanlyn]

Clever.

[sanlyn]

Making progress -- and finding more problems (why am I not surprised?)
I found a decent solution for those chroma pumping scenes. Not 100% OK, but maybe 80 ? ? Here's yet another damaged shot with bad red-green pumping at the start, not to mention a ton of noise. I was going non compos mentis over this shot and the one in Post #190. An MPEG version of the original AVI capture is attached below as B02b_original.mpg (13.3 MB). I applied 2 fixes. The pumping fix is highlighted in blue below. Then I tried some masking to help with burned highlights. Still some work to go there. Results of the script are attached as B02b_try1.mpg.

# -------- STEP 1  --------
# ----- input is YUY2 -----
Trim(250,743)
ChromaShift(C=-2)
Crop(0,0,-8,-4)
MergeChroma(ColorYUV(autowhite=true))
QTGMC(preset="medium",InputType=1)
AddBorders(4,2,4,2)
ConvertToYV12(interlaced=false)
Crop(4,2,-4,-2)
SmoothCurve(Vcurve="0-16;128-128;255-230")
SmoothCurve(Ucurve="0-16;128-128;255-230")
MergeChroma(fft3dfilter(sigma=.1, sigma2=.3, sigma3=.5, sigma4=40, plane=3))
ColorYUV(gamma_y=-25,gain_y=-10,off_y=8)
DeBlock_QED(quant1=24,quant2=16)
AddBorders(4,2,4,2)


# ---------------  STEP 2 -----------------
# ---- remove borders for LCE routine -----
# ---- (borders are affected by LCE)  -----
# ----         INPUT IS YV12          ----- 
Crop(4,2,-4,-2)
LCE(mode=0,brightness=80)
# -------- correct for LCE contrast -------
ColorYUV(off_y=4,gamma_y=-5)
mergechroma(aWarpSharp(depth=20.0, thresh=0.75, blurlevel=2, cm=1))
AddBorders(4,2,4,2)
# ----- RGB32 for VirtualDub/NeatVideo, etc. ---
ConvertToRGB32(matrix="Rec601",interlaced=false)


# -------- LCE ("Local Contrast Enhancement" via Doom9) ---------
function LCE(clip v, float "gblur", float "contrast", float "brightness",\
 int "mode", bool "correction")
{
brightness = default (brightness, 80)
correction = default (correction, false)
gblur = default (gblur,0.6)
mode = default (mode, 1)
contrast = (mode==1) ? default (contrast, 1.4) : default (contrast, 1.2)
v = (correction==true) ? v.autolevels() : v
brightness = (brightness<=255.0 && brightness>=0.0) ? float(-brightness) : -80
v2=v.Tweak(sat=0)
v2=v2.gaussianblur(gblur)
v2=v2.mt_edge(mode="min/max",thY1=0,thY2=255, Y=3, U=2, V=2)
v3 = (mode==1) ? v.Tweak(cont=contrast) : v.Tweak(cont=contrast,bright=brightness)
merged=mt_merge(v,v3,v2)
return merged
}

Still along way to go. Colors are still awful. I hate auto filters, but the combo of AutoWhite and FFT3D was the only way around the pumping. In the "fixed" clip you'll note how AutoWhite screwed up gamma in the latter part by trying to lighten darks and brighten whites at the same time: contrast is royally screwed as the scene ends. So I devised a 2-step process, the 2nd step being a higher-contrast version of step 1. I should be able to dissolve the two versions together somewhere around frame 360 onward.

After some noise is removed, etc., you start noticing other problems. The blown-up image below shows some reddish gunk on the girl's sweater, a blotch on the left-hand guy's tan jacket, and blue stain in the lower left (it's a blue peak around RGB 70 that screws up everything). Even with AutoWhite, the color balance has gone green, skin tones are weird, and the guy in the middle was wearing brownish red slacks that are going dark olive here.


The way the color balance shifts from too green/red at the start to too blue at the end is frustrating. An RGB histogram of the original AVI shows what colors look like at the early frames (left image), then how blue has shifted right and red left at the end (right image). Note in the right-hand histogram how midtones and brights have flattened out, with the scene looking wishy-washy. So I figure 2-step is the way to go here. Pain in the neck.


Anyone who wants to brave the original AVI capture of this scene can indulge from here (103 MB, Lagarith YUY2 640x480):
http://dc627.4shared.com/download/Y2w9nzD3/B02b_L3B_ivtc_cut.avi

Time for me to get back to learning After Effects for stuff like this. And MaskTools. And maybe MVTools for some of the shimmering objects you see in this clip (look at the brick wall). Don't just blame the tape or the player: the object shimmer is in the digital broadcasts, too.

[poisondeathray]

Time for me to get back to learning After Effects for stuff like this. And MaskTools. And maybe MVTools for some of the shimmering objects you see in this clip (look at the brick wall). Don't just blame the tape or the player: the object shimmer is in the digital broadcasts, too.

Looks much better

For shimmering objects and walls, you can use QTGMC in progressive mode (inputtype=1)

You can limit it to walls by motion tracking a mask, but "damage" to other objects won't be that noticable on low quality VHS sources

Is it me or there seems to be some "rumbling" in the try1 version? Like the wall is jittering at that end of the sequence that wasn't there in the original ?

EDIT: it is there in the original but seems worse in try1. It's frame warping primarily at the top 1/4 of the frame

[sanlyn]

The ripple is in the original, and in the tv versions -- where, in a few spots, it looks worse. I have 9 captures with 4 VCR's, it's exactly the same on all, same frames. RemoveSpots() actually calmed it down in a few shots, but didn't help here. Yes, you see it more clearly after QTGMC or even yadif, but it's definitely there under the noise as well. I also note the first two or so frames in a new scene change will often jiggle or is even a little tilted in the frame. And some frame-by-frame play shows a re-rendered, fuzzy first frame where there was apparently some sloppy lab editing and joining (?). You seldom notice it during play -- although there was one scene cut followed by 3 reddish, dark, ugly frames -- fortunately no audio there, so I just deleted them.

Looking at the books on AfterEffects now. 3 big tomes, 3 DVD tutorials. Gimme a couple of weeks. Thank heavens most of this poop video ends about 1/3 in. The worst of the pumping seems due to Macrovision (but it's in the tv versions, too) and dissipates halfway thru. I fear that masking in AE is the only way I can repair some of that color.

Thanks for the notes.

P.S.: that rippling, I did fix a couple of frames with MCReplaceFrames(), not shown in the script. Might tackle a few more, as they're usually just 1 frame, then 1 later, etc. Pain in the neck, but it did work.

[sanlyn]

So, 2 months later I fixed some more ripples and jitters. Contrast masking created a nightmare, notably because AutoWhite() created a gamma and contrast nightmare that changed every 150 frames -- but AutoWhite and FFT3DFilter cleaned up most of the chroma bounce (still some remnants in shadows, lower left). Made 2 versions of this scene, each with different levels/color fixes, then joined with a dissolve around frame 330. Thus, autowhite fixed one chroma problem and generated a dozen new ones.

The original clip and the earlier effort are in post #204. The scene immediately following this marks the start of fading Macrovision effects, waning noise, and decent color and detail to work with (saints be praised!).

[sanlyn]

Interlaced or not? So this capture from VHS is from the movie that is "supposed to be" progressive with 3:2 pulldown. The capture was first ivtc'd and should now be progressive, right? But I have one bad frame with a horizontal streak near the top of the frame that DeVCR and other filters didn't catch, likely because there's too much motion.

Original frame 382, with streak near top of frame:

[Attachment 15095 - Click to enlarge]

But if you SeparateFields() or use yadif (I used yadif here), you double the frame numbers and find that even frame 764 doesn't have the streak but "deinterlaced" frame 765 shows the streak near the top:

yadif frame 764 (Top), yadif frame 765 (bottom):

[Attachment 15096 - Click to enlarge]

[Attachment 15097 - Click to enlarge]

So I used the script below. It creates a copy of the clip ("a2") with Yadif, which deinterlaces the first 385 "progressive" frames. Then it adds 1 extra frame to a2. It then replaces the original progressive "bad" frame with the clean frame, then uses Select Even() to get a clip with the same frame numbers as the original clip. Finally it replaces the original bad frame 382 with the yadif'ed frame 382.

a1=AviSource("E:\LIL2\A\L3A_ivtc_RS.avi"
a2=a1.Trim(0,400).AssumeTFF().Yadif(mode=1,order=1)
a3=a2.DuplicateFrame(0)
a4=ReplaceFramesSimple(a2,a3,mappings="765").SelectEven()
Out1=ReplaceFramesSimple(a1,a4,mappings="382")
return Out1

The result looks like yadif'ed frame 764 (the middle image above, with no streak). I might have got a sharper frame with QTGMC, but I'm sick of looking at this damaged section of tape. Hope that fix is a valid one. It plays OK after more fixes and color work.

[jagabo]

What happened was one field was messed up on the tape (or when capture) so when that field was combined with the complimentary field the defects were retained. Yadif compares the two fields. Where there is no motion (or other mismatch) it uses data from the other field. Where there is motion (or other mismatch) it interpolates the missing line. So it saw the noise in the complimentary field, ignored it, and interpolated from the existing field.

Remember, analog video is transmitted as sequential fields. It's entire possible for one field to be messed up and the other to be just fine.

[sanlyn]

Yep. That aspect of VCR playback was evident when I saw a spot on two frames, but separatefields() gave me a spot on the odd field of frame 1 and the even field of frame 2. So I tell myself that effectively the tape is coming out of he VCR "as if" with some kind of funky interlace. The tape heads don't output exactly the same thing anyway -- which is probably why some real crappy sections of this ivtc'd "progressive" capture look so much cleaner with QTGMC InputType=1, even at its fastest presets.