luma error on VHS restoration

[dan59]

Hello
I have a problem when I'm restoring VHS.
VHS have certainly physically be dommaged and when I attempt to read them I get luma problem like in the attachement.
When I use the hitogram() function in avisynth, I see the first is verylow when picture is damaged.
Do you think it's really a luma problem ?
Do you thlink it's possible to store luma, to detect bad luma and replace it by the stored luma of the frame-1 ?
thanks for your help.

[jagabo]

You should post a video sample. Losslessly compressed or whatever your capture device outputs (not reencoded). I'm not sure what you're trying to show in those stills. The first has crushed blacks and brights, and is over saturated. You should adjust the video proc amp on your capture device to bring them more into line. If you can't do that then you'll have to resort to filtering in software.

[sanlyn]

I think the O.P., is saying that levels and saturation fluctuate because of tape damage. The bottom image has obvious layer damage and bad horizontal rips. A line tbc would help (such as via pass-thru, as I don 't think a high-end JVC machine would track this tape and I don't know what a Panasonic pro would do with it), along with a better tape player. I've seen effects like these on tape that was used with cheap auto winders. Apparently the damage is extensive, so playback won't be so wonderful no matter what is done.

[jagabo]

Obviously the luma levels are different during the good and bad fields. We can't tell much about the chroma because JPEG encoding has blended the chroma of the two fields together.

If the bad spots are only a few fields in a row you can use a function like ReplaceFramesMC() (aka RX()) to replace them with motion interpolation. Or, if there's very little motion, simply duplicate the good frame just before or after the bad sequence.

[dan59]

Hello

thanks for your help !

This is an extract of the original capture (mjpeg)
http://files.videohelp.com/u/217061/probleme%20lumiere.avi

This problem occur only in 2 VHS, the quality is godd for all the 20 vhs i have captured before.

I think it was a bad luma but perhaps I makje a mistake ?
On the "histogram()" of avisynth , the lume (first graph) doesn't appear to exist when the error occur

Thanks

[jagabo]

So what do you want to do? Replace the obviously bad frames/fields? I don't know what you mean by "the lume (first graph) doesn't appear to exist when the error occur". The luma exists, it's just brighter.

Even during the "normal" frames the video is blown out at the high end, too dark at the low end, and over saturated. When the tracking errors occur it gets brighter overall, and the chroma becomes less saturated.

Worst frames replaced with ReplaceFramesMC(), levels and saturation toned down a bit:

[dan59]

Hi,

you're result is good you're a king !
but how do you obtain it ?
In my mind, i need to detect when luma is bad and replace by the luma of the previous frame.
In my avisynth script I use :

Video = AVISource("20.avi")
Video=Video.SmoothDeinterlace(tff=true,doublerate= true,showlace=false)
#remove vhs border
Video= Video.Crop(14, 4, -10, -14)
#resize
Video= Video.Lanczos4Resize(720,576)
# I use a vid2 otherway
Vid1= Video.Crop(0,0,0,0)
#best detail
Vid1=Vid1.Asharp(2,1.5,2.3,true)
#remove temporal noise
Vid1=Vid1.FFT3DFilter(sigma=3.3,bt=5,beta=1.0,plan e=0,ncpu=2)
#make white white
Vid1= Vid1.ColorYUV(autogain=true, autowhite=true,analyze=false)

juste after the smoothDeinterlace, I want to adjust saturated frame but does I need to identify each frame manually ?
or can I use averageluma or something like this ?
How work "ReplaceFramesMC()" I 'just' found this post :
https://forum.videohelp.com/threads/346356-Interlaced-Telcine-Footage
thanks for your help, it's reallyupper to read people who know what they said

[jagabo]

Here's my current version of ReplaceFramesMC(), slightly improved from the earlier version:

function ReplaceFramesMC(clip Source, int N, int X)
{
 # N is number of the 1st frame in Source that needs replacing. 
 # X is total number of frames to replace
 #e.g. RX(101, 5) would replace 101,102,103,104,105 , by using 100 and 106 as reference points for mflowfps interpolation
 
 start=Source.trim(N-1,-1) #one good frame before, used for interpolation reference point
 end=Source.trim(N+X,-1) #one good frame after, used for interpolation reference point
 
 start+end
 AssumeFPS(1) #temporarily FPS=1 to use mflowfps
  
 super = MSuper(pel=2, hpad=0, vpad=0, rfilter=4)
 backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
 forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
 backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
 forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
 backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
 forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
 MBlockFps(super, backward_3, forward_3, num=X+1, den=1, mode=0)

 AssumeFPS(FrameRate(Source)) #return back to normal source framerate for joining
 Trim(1, framecount-1) #trim ends, leaving replacement frames
  
 Source.trim(0,-N) ++ last ++ Source.trim(N+X+1,0)
}

And the script with your source:

AviSource("probleme lumiere.avi") 
AssumeTFF()
ColorYUV(cont_y=-28, off_y=-4, cont_u=-80, cont_v=-80)

ReplaceFramesMC(4,3)
ReplaceFramesMC(12,1)
ReplaceFramesMC(28,4)
ReplaceFramesMC(35,3)
ReplaceFramesMC(39,1)
ReplaceFramesMC(40,4)
ReplaceFramesMC(45,1)
ReplaceFramesMC(47,4)
ReplaceFramesMC(57,1)
ReplaceFramesMC(62,1)
ReplaceFramesMC(64,3)

ReplaceFramesMC() should be used on progressive frames but your video didn't have much motion so I left it interlaced.

[sanlyn]

I see several common errors in dan59's script. First, you resized a noisy video (resizing and cropping was not necessary), and Lanczos4 over-sharpens noise and artifacts that weren't removed. Then you sharpened before denoising, which sharpens noise and defeats most of the work of denoising. Next, autowhite can often help with fluctuating luma and chroma, but it doesn't make white=white; I might have used autowhite for fluctations like this, but you need to make further corrections for color balance.

These are just suggestions. I wonder why people deinterlace PAL if the object is PAL DVD, which is usually interlaced for smoother playback with many playback methods. But that's what people do. You could use a better deinterlace than SmoothDeinterlace. Try QTGMC, which also effectively denoises.

[dan59]

thanks for your quick response

ok for the improved version
ok for the script but :
- I need to desinterlace since it's family movie and the is a soccer match between young and old with motion.
- I wish to obtain the 4,12,28,35... value (for ReplaceFramesMC )automaticaly do you think it's possible to have this detection (based in average luma or other native function ) ?
if it's impossible do you think it's possible to log in file the suspected bad frame (with the same function averageluma and other)

[dan59]

extra answer :
when I use ReplaceFramesMC(clip,101, 5)
I go to repair the problem between frame 101 to 106
but the new frame 102.103.104.105.106 is :
- copy of frame 101 ?
- average of frame 101 and 107 ?
- no new frame crated, just bad frame deleted and framerate modified for this 5 frame ?
-other ?
this ask is just for understanding how it work

[dan59]

Hello sanlyn
thanks for your help
effectively I'm beginner
I work on old family vhs and my 'test' build this script but I'm not locked on this.
I will test what you says but it's sometime very hard for beginner to understand hoiw to use correctly a function.
Can you explain more detailled my mystake please ?

[sanlyn]

I've looked at the sample video, but jagabo's script was a nice repair of the bad frames and general luma/chroma levels so I didn't go ahead with more experiments. The borders need some cleaning, but that would be one of the last steps: luma and chroma correction would change the "black" of the borders, so I'd save that for later. Rather than use resize to center the image and restore the frame, I'd fix the borders. Resizing analog source is not as clean as simply restoring borders. True, the image might be a bit smaller but it would look cleaner. Resizing before cleaning up the image also resizes noise and other problems, which should be cleared first.

I would correct levels first, repair frames, then deinterlace (with QTGMC), denoise, then repair borders without resizing. Then re-interlace -- the last, depending on what you want for final output. For standard PAL DVD, I'd reinterlace, but some people don't. My argument for reinterlace depends on whether you're permanently archiving your original capture: If you're archiving the original unprocessed capture, you can always return to that original later to take advantages of changing technology.

The sequence depends on how much noise you're dealing with. This is noisy video, not to mention damage. There is no sense to sharpening and resizing noise and damage, then trying to remove those problems. A better deinterlace such as QTGMC would do some denoising anyway without interpolating noise from frame to frame. You'll note that jagabo's script performed two tasks: correcting overall luma and chroma, then repairing frames. I'd start with that sequence, then follow with the other steps.

[jagabo]

extra answer :
when I use ReplaceFramesMC(clip,101, 5)
I go to repair the problem between frame 101 to 106
but the new frame 102.103.104.105.106 is :
- copy of frame 101 ?
- average of frame 101 and 107 ?
- no new frame crated, just bad frame deleted and framerate modified for this 5 frame ?
-other ?
this ask is just for understanding how it work

Frames 101 to 105 are replaced with motion interpolated between frames 100 and 106. Sometimes it works well, sometimes it doesn't. The longer the sequence the more likely it is to look bad. The more non-linear the motion the worse it will look. By the way, you need MvTools.

[dan59]

I've looked at the sample video, but jagabo's script was a nice repair of the bad frames and general luma/chroma levels
so I didn't go ahead with more experiments.

The borders need some cleaning, but that would be one of the last steps:
luma and chroma correction would change the "black" of the borders, so I'd save that for later.

->for me the border is just an indicator to confirm vertical is allign and black is black, i don't understand how you want I keep them in final ?

Rather than use resize to center the image and restore the frame, I'd fix the borders.
Resizing analog source is not as clean as simply restoring borders.

-> resizing is just to obtain an 'standard' output without badline colored (on the right, at the bottom, sometime at the top , etc)

True, the image might be a bit smaller but it would look cleaner.
Resizing before cleaning up the image also resizes noise and other problems, which should be cleared first.

I would correct levels first,
repair frames,
then deinterlace (with QTGMC),
denoise,
then repair borders without resizing.

-> ok but just for understand; what is the difference between correct level->repair frame->desinterlace and desinterlace->correct level->repair frame ?
since the denoise function is played after correct the frame what is the difference ? time to execute ?statistic ? mathematical function ?

Then re-interlace -- the last, depending on what you want for final output.
For standard PAL DVD, I'd reinterlace, but some people don't.
My argument for reinterlace depends on whether you're permanently archiving your original capture:
If you're archiving the original unprocessed capture, you can always return to that original later to take advantages of changing technology.

-> I not archived the original (who have bad frame and audio out of sync) but I archived the audio video resynchronised without compress(only mjpeg).

The sequence depends on how much noise you're dealing with.
This is noisy video, not to mention damage.
There is no sense to sharpening and resizing noise and damage, then trying to remove those problems.

-> this is non sense but I found it to increse detail, before blur colored area to in fine have a best result in x264 compression.I make a mistake ?

A better deinterlace such as QTGMC would do some denoising anyway without interpolating noise from frame to frame.

-> QTMC make a temporal denoising ? it possible to chose 1 or 2 frame before and after for reference ?

You'll note that jagabo's script performed two tasks:
correcting overall luma and chroma, then repairing frames.
I'd start with that sequence, then follow with the other steps.

I note some red discoloration along the right border, which is common with aged tape.
There are a few plugins that can try to clean it, but results are often imperfect.
You can try cleaning that red junk (which might or might not work properly),
keep it there, or crop it away.
I have used those cleaners with some (imperfect) success, so it depends on your tolerance for that noise.
Those plugins would be used while the video is deinterlaced.

-> i attempt some plugin for red junk or other 'genral' vhs problem, but it really slowly and the result is not really good, this i why i attempt to ajuts in script with basic and not to use all in one plugin.

[dan59]

to jagabo, do you see this ? you think it's impossible ?

thanks for your quick response

ok for the improved version
ok for the script but :
- I need to desinterlace since it's family movie and the is a soccer match between young and old with motion.
- I wish to obtain the 4,12,28,35... value (for ReplaceFramesMC )automaticaly do you think it's possible to have this detection (based in average luma or other native function ) ?
if it's impossible do you think it's possible to log in file the suspected bad frame (with the same function averageluma and other)

[jagabo]

- I wish to obtain the 4,12,28,35... value (for ReplaceFramesMC )automaticaly do you think it's possible to have this detection...?

It's probably possible. But I don't know how to do it.

[sanlyn]

No filter, except with some highly specialized and professional (and extremely expensive) gear and custom software programming can distinguish a "good" frame from a "bad" frame. The biggest problem with working with damaged video is the time involved in manual correction. Some clever people have devised ways of detecting, for example, totally black frames. That's an interesting concept, but in the final analysis the corrections are manual anyway, the programs make many errors, the results have to be manually inspected, verified, and debugged, and the time involved in devising the code to detect those frames is time-consuming in itself.

Working with damaged video is not a push-button exercise. With experimentation one can devise a few shortcuts and standard procedures that work in most situations. But every video has unique problems. This is why most hobbyists and professionals work with video in smaller sections to develop methods for handling those problems, then proceed with other parts of the video. If the problems were uniform and consistent, work would be much easier. But the nature of analog video and analog physical media is not that polite. If it were as precise as a good digital encode, most of the problems wouldn't exist.

I didn't address all of the questions, but in most cirumstances resizing should be one of the last steps, not one of the first. The final output will display as 4:3 anyway, not as 720x576.

[sanlyn]

- I need to desinterlace since it's family movie and the is a soccer match between young and old with motion.

It's my understanding that motion with 25 FPS PAL plays more smoothly on most TV's if its is interlaced, and more smoothly on a computer if it is progressive. Some members might have different opinions, but that's been my experience.

[sanlyn]

"Luma error" IMO is far from the worst problem , even if some dark shadows and many brights are wiped out forever and won't come back. The worst problem is MJPG compression artifacts and the kind of coarse, ugly skin shadows that compressed MJPG is noted for. Nopt to mention heavy grain that hides banding, so it sneaks up on you when grain and noise are removed.BAnding under the noise is almost visible below, and prominent along skin shadow edges, with shadow discoloration looking just like that in JPG photos. Odd, but even though Red is oversturated in the original, the oiverall color balance is too green (evident in the non-existent "whites" and "grays", while many "blacks" aren't black.

[Attachment 16615 - Click to enlarge]

The Red stain along the right wouldn't budge, so I removed 22 pixels from the right, 12 pixels of noise from the bottom, and fixed borders so the image could be resized to the proper 1.5:1 ratio in a 720x576 PAL frame. With much of the noise gone, in upper right you can see the return of the original video's color banding. It's most obvious when the cameras moves. Skin tones and shadows were a real challenge, and there's the temptation to make the priest's vestments "blazing" white, but that's not what vestments look like. The white hat of the lady in green, the white shirts, and shadows in white areas wrere guides for finding the white band gray balance, with black hair and a black bowtie the guide for RGB 16 black. How the original scene looked is anyone's guess, but at least the priest doesn't look plastic and ladies don't have cyan-tinged 5 o'clock shadow. Beyond that, more sharpeners and more denoisers can't overcome the losses from fuzzy MJPG compression.

[Attachment 16616 - Click to enlarge]

My script started with jabago's idea (I added 2 more frames to his list), then added:
SmoothLevels(for dithering to tv range), QTGMC ("medium"), DFTTEST, ChromaShift(c=-2), then DeHalo_Alpha + LSFMod() + GradFun2DBmod() defaults, and finished with AddGrainC(1.5, 1.5) to try to fend off banding. The attached "v5.mkv" is re-interlaced. The Resizer was Spline36Resize. The bulk of color and gamma corrections were in RGB with gradation curves for primary and secondary correction. There is no way to fix those whites and skin shadows in YUV. Usually people don't bother with color anyway but head straight for the sharpeners, so I'll just leave it at that.

[dan59]

waouuu !
I go to read that slowly to understanding all
thanks !!
I think I go to post complementary questions

[sanlyn]

The comments in my post (#20) can be visualized more clearly if you click on one of the images to view them in the forum's viewer. Then you can click "Prev" or "Next" buttons to switch quickly between the two images. Thanks to jagabo's script for leading the way with the basic luma/chroma fixes and frame replacements, which looks pretty good. All I did was add more cleanup and go farther into color.

[jagabo]

For the OP's benefit: If you're going to use QTGMC() it would be best to do it before ReplaceFramesMC(). Which means you'll have to change all the frame numbers in ReplaceFramesMC(). ReplaceFramesMC() shouldn't be used on interlaced video. I only did so as an example of how bad frames could be replaced. Since there was so little motion in the clip the problems weren't very noticeable. But with higher motion video you'll get more severe problems.

[sanlyn]

I thought about that, too, but time was slipping up on me. One problem I saw when I looked into it briefly was that many of the blips occur over more frames when deinterlaced, and others didn't. That kinda turned me off, but I think you're correct.

[dan59]

sorry do not respond quickly but i need to test all you say for understanding (i'm beginner on restoring)
thanks for your help, I post other reply here since i have tested.

[dan59]

Resize:
If I understood good, I not need to resize before apply filter but I need to mask default (like bottom border etc)
to do that i need to addborder at the start of script
and one of the last operation is to resize.

I understanding all rigth ?

[sanlyn]

If you look at the script I posted in post #22 above, the border is changed twice. Near the beginning of the script, the black borders were temporarily removed for one of the luma balancing filters (SmoothLevels) so that the relatively large black borders wouldn't affect the filter, but the same borders were immediately restored. You have to be careful with cropping before applying most denoisers and other plugins because they work only with frame dimensions and block sizes that are evenly divided by 8 or 16. The final crop/resizing should be done after cleaning.

[dan59]

Hello everybody
I'm come back
I download ,install and test all you say before
first : it's longggggggggg to process
I just test on few second but it's really slow (i understand it come from the temporal detection of noise)

for the colors/brigtness:
how to test the value witout encoding alla the video ?
I use SelectRangeEvery(2000, 500,2000) to obtain a sample video
but the good correction for the chruch is bad for the familly soccer
-> I understand i need to cut my video by geographical theme and ajust parameter for each theme

I have another problem :
sometime i have 15 or 20 second of bad/video, i can't use ReplaceFramesMC(4,3) for a big number of frame (especially in the soccer)
There is a way to 'correct' (just a little bit perhaps) this ?

Another question :
How to know if I need to denoise/degrain ?
visually I can't see grain or noise but the is freeware ? avi script ? that check the noise level for all the video ?

Another question :
My dark scene are blue... I think its du to my insufficient crfopping from the right, I'm rigth ?

[sanlyn]

I can't answer all of your questions in detail immediately, however......

it's longggggggggg to process
I just test on few second but it's really slow (i understand it come from the temporal detection of noise)

The fastest running part of the script is actually QTGMC. The slowest part is the denoising, resharpening, and resizing. The following script will run about 3X faster. It will not be quite as sharp and more noise remains, but it is still an improvment.

LoadCPlugin("[path to Avisynth plugins]\yadif.dll")  # <- required for QTGMC fast repair mode

ColorYUV(cont_y=-28, off_y=-4, cont_u=-80, cont_v=-80)
Crop(10,0,-12,-12)
SmoothLevels(6,1.0,235,14,220,chroma=200,limiter=2,tvrange=true)
AddBorders(10,0,12,12)
Tweak(Sat=0.9,hue=15)
ReplaceFramesMC(4,3)
ReplaceFramesMC(12,1)
ReplaceFramesMC(28,4)
ReplaceFramesMC(35,3)
ReplaceFramesMC(39,1)
ReplaceFramesMC(40,4)
ReplaceFramesMC(45,1)
ReplaceFramesMC(47,4)
ReplaceFramesMC(57,1)
ReplaceFramesMC(62,1)
ReplaceFramesMC(64,3)
ReplaceFramesMC(14,1)
ReplaceFramesMC(18,1)
ReplaceFramesMC(21,1)

AssumeTFF().QTGMC(preset="ultra fast")
RemoveSpots()
DeVCR(30)
ChromaShift(c=-2)
Dehalo_Alpha()
LSFMod(strength=120)
AddGrainC(1.5, 1.5)
Crop(2,0,-22,-12).AddBorders(0,0,10,0)
Spline36Resize(720,576) # restore frame size

# ---- Re-interlace for smoother PAL playback -----
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

# ---- to RGB for VirtualDub color correction -----
# ----     (if desired, else save as YV12)    -----
ConvertToRGB32(matrix="Rec601",interlaced=true)
return last

[dan59]

little question
it's possible to write in a file between each pulgin ?
something like thins
define file "time.txt"
writefile ('start YUV'+ getime())
ColorYUV(cont_y=-28, off_y=-4, cont_u=-80, cont_v=-80)
writefile ('stop YUV'+ getime())
....
what about my other previous question ?