VHS Family Wedding tape restoration

[2Bdecided]

Not sure what you mean by blacker-than-black is 'clipped-off' can you explain what you mean and where I would see that ?

On the last sample you posted, if you run this script...

avisource("motion sample.avi")
assumetff()
crop(16,16,-16,-16)
histogram(mode="levels")

...you can see from the top graph that luma is always within the 16-235 range (i.e. there are no values in the outer "shaded" areas of the top graph) which is fine, but in your video there's often a peak at 16 or 17 (the very left-hand side of the non-shaded area). This suggests that there were values lower (i.e. some detail in the black areas) but these have all been clipped to 16. (first picture attached).

That graph is plotting how many pixels there are of each possible luma value. In almost all videos, the luma/brightness distribution is fairly smooth.

Obviously if there is a large area of exactly one brightness level, you will get a single sharp peak in the graph corresponding to that value (because lots of pixels have that value), but large areas of absolutely identical colour are rare in real video - you're more likely to get areas of approximately the same colour - and this gives "rounded" peaks because the luma ranges over several values in that "almost the same colour" area.

In the luma histogram (top graph), where ever there are really sharp peaks, missing values, and/or the graph just seems to stop abruptly (like a cliff edge) rather than gently fall off (like a slope), it's probable that some processing caused this, and it's possible that the effect is visible.


Clipping everything below 16 and above 235 in the final encode isn't a terrible thing to do, because some TVs will do this anyway (most will clip everything below 16; most will preserve everything above 235 unless the contrast is set high - default/shop setting usually have very high contrast though). If, when I've got the picture looking just as I want, there's still some values below 16 or above 235, I try to avoid clipping them just because having them there may make the results look slightly better on displays that can cope with the extended range. Plus if I'm ever having to re-process from that version for something else, I might want that extra range. (The dynamic range and colour space of TVs is going to expand dramatically over the next decade). But really, it's not that important.


I think you hit a slightly worse problem though - still not hugely important, but worth knowing about: You exported to RGB, and then did further colour manipulations. This means some of the values you clipped off would have been brought back into the legal 16-235 range when you played around in Vegas - except those values had already been clipped off, so they've gone. I think that's happening over such a small range of values that it's barely visible in your case. I think you can see it at the top of the luma graph on attached picture 2.

While there's only a small range of luma (brightness) values that are out of range, I think some of the RGB values (i.e. colours) are probably being clipped slightly more. So while the original Y range might be, say, 10-240 (when values beyond 16-235 get lost), the YUV values are generating RGB values hitting, say, -15 to 270 (when values beyond 0-255 get lost). Maybe. It's really hard to tell. I don't think there's an easy 2D way of mapping this after it's been clipped. I think the function to check for potentially illegal RGB values before you convert to RGB is referenced earlier in this thread. I think it's from here... https://www.doom9.org/showthread.php?t=154731 ...but it might be another similar one.


I don't know the right lossless workflow from AVIsynth to Vegas to avoid this, and I don't know if it's worth it. Anyone nailed this?

Cheers,
David.

P.S. If it was me, I would be happy with what you already have.

[sanlyn]

I don't know the right lossless workflow from AVIsynth to Vegas to avoid this, and I don't know if it's worth it. Anyone nailed this?

Yes. And what about highlights?

Image below is frame 280 (original) with YUV and RGB histograms. Luma in the YUV "Levels" histogram is well within RGB 16-235, and U and V don't look especially spread out. Dark crush (cutoff) is seen by the peak at the left. The "Levels" histogram rarely shows chroma clipping (and according to doom9 posts, the chroma section isn't all that accurate anyway. The waveform monitor or other histogram mode is probably more useful for chroma). The RGB histogram shows how the display conversion YUV->RGB plays out: darks get expanded down to RGB 0 or near it at the left. At the right, red is clipped. Luma and the other two colors don't show RGB clipping here.


[Attachment 23649 - Click to enlarge]

In the above image, note the overexposed facial highlights. Skin tone is off (it's orange. If you reduce red contrast and increase blue contrast, the man will no longer look orange -- he'll look green). Bright details are clipped: there is little or no detail in the man's shirt or tie, and no detail at all in the flower. The flower is just a bright blob. In the other frames showing other people, some appear to be wearing a yellow flower. Here, there is neither detail nor color in the flower, except for a hint of red.

In the image below, frame 280 appears after using this code:

ColorYUV(cont_y=-40,gamma_y=-15)

[Attachment 23650 - Click to enlarge]

In this case contrast was reduced, which makes the darks brighter and the brights darker, and gamma lowers the midtones and brights a bit. Note that neither the YUV nor RGB histograms below show bright clipping, although the dark crush peak is still there. This illustrates that you can retrieve a little shadow detail by reducing contrast, but you won't retrieve much and it's mostly grainy noise. You can make the darks brighter, but you won't see any more "detail". The RGB histogram shows how darks and brights are expanded in YUV->RGB, and still no bright clipping -- except for a rather mild upward "toe" with red at the right side.

Brightening darks revealed a little more shadow detail, but what's seen here is about all you'll get. With the brights clipped earlier, lowering brights reveals no more bright detail than before. This would indicate to me that brights were clipped at some point, likely in the camera. Dynamic range in the bright end is severely limited due to clipping.

Autogain = Satan's work.

[poisondeathray]

Did you use a lossless YUV intermediate into vegas? The problem with that is vegas treats "lossless" YUV intermediates (like huffyuv, ut video, lagarith) as computer RGB. This means Y' 16-235 gets "mapped" to RGB 0,0,0-255,255,255 . Vegas would cause clipping in that case

Native camera formats like DV, AVCHD, HDV get "studio RGB" treatment. Y' 0-255 gets "mapped" to RGB 0,0,0-255,255,255. "superdarks" and "superbrights" are preserved . I think cineform gets studio rgb treatment as well (at least in some versions of vegas), that' s near lossless especially at "filmscan2".

Some uncompressed YUV intermediates get studio RGB, others get computer RGB. For example vdub's fourcc "YV12" gets computer RGB treatment but "IYUV" gets studio RGB treatment (Both YV12 and IYUV are uncompressed 8bit YUV 4:2:0, just different byte ordering and plane arrangements) . So the correct "lossless" YUV workflow would be to use uncompressed fourcc IYUV . It's not a big deal in this case, it's just information for "best practices"

[2Bdecided]

This would indicate to me that brights were clipped at some point, likely in the camera. Dynamic range in the bright end is severely limited due to clipping.

That's the point - it's either clipped in camera, or in the converttorgb(matrix="Rec601") near the end of the AVIsynth script. IMO, even if there was clipping in the camera, there's additional red clipping in that converttorgb step. Red is hitting maximum in the final encode.

In the image below, frame 280 appears after using this code:

ColorYUV(cont_y=-40,gamma_y=-15)

[Attachment 23650 - Click to enlarge]

In this case contrast was reduced, which makes the darks brighter and the brights darker, and gamma lowers the midtones and brights a bit. Note that neither the YUV nor RGB histograms show bright clipping, although the dark crush peak is still there. This illustrates that you can retrieve a little shadow detail by reducing contrast, but you won't retrieve much and it's mostly grainy noise. You can make the darks brighter, but you won't see any more "detail". The RGB histogram shows how darks and brights are expanded in YUV->RGB, and still no bright clipping -- except for a rather mild upward "toe" with red at the right side.

I think you're misreading this. What you've done hasn't helped at all. Because...

The workflow was:
1. YUV on tape
2. YUV DV capture
3. YUV in AVIsynth (hopefully - Tafflad should have used AVIsource but may have used DirectShowSource)
4. AVIsynth processing
5. RGB out of AVIsynth
6. RGB in Vegas
7. Vegas processing
8. YUV in the file we have (your latest tweak goes here)
9. RGB for display

Apart from rounding errors due to the processing in Vegas (which is what makes the Red peak in your last screen scan smaller and wider), you're not rescuing anything - just the rounding errors in Vegas. All the real out-of-range content as clipped at step 5 (or step 3 if DSS was used and returned RGB).

Hence I want to know if anyone has good advice on how to preserve out of range colours through AVIsynth and Vegas into the final encode. Of course it would be good to fix them and stop them being out of range (quite a serious battle with old camcorder videos - I may have mentioned this before! ) - but for the bits that creep through, there must be some way to avoid clipping them and make use of that YUV headroom.

EDIT: PDR describes the nightmare I found - different formats treated in different ways in Vegas. I didn't know about IYUV working properly though - I'll try that next time. Don't you also have to select something special in Vegas? 32-bit mode or something?! It's three years since I tried this, so I can't remember. btw, (and I didn't dare admit this before) last time I made it work by cheating: I used MPEG as an intermediate format which Vegas a) smart renders, and b) doesn't change the levels of. Though the preview and the final output looked completely different.

EDIT: there's something approaching an explanation of why it's so messed up in Vegas at the bottom of this page: http://www.designstudioschool.com/why-does-clamp-235-t230065-30.html

Cheers,
David.

[sanlyn]

The submitted sample was all I had to work with, and I assume it's not the original source.

As you say, the results shown aren't so terrible. To see what sort of clipping or other problems occurred, the source is the place to go, not a later sample. Offhand I would guess that pretty much the same filters and scripts were used for all scenes, but we know from earlier samples that levels/chroma varied widely from scene to scene. With tape, that comes with the territory. In any case the results as you stated earlier are a great improvement. For all we know, not having seen this shot before, a single setting won't work so well anyway because levels change considerably during the shot itself. I suspect that if you made the church door shot look better, it would make the other panning shots and views look worse. The camera's AGC going haywire during each pan is an obstacle in itself and likely can't be resolved without a great deal of effort.

For a newcomer to Avisynth working with a difficult video, I think tafflad did quite well.

[poisondeathray]

Hence I want to know if anyone has good advice on how to preserve out of range colours through AVIsynth and Vegas into the final encode. Of course it would be good to fix them and stop them being out of range (quite a serious battle with old camcorder videos - I may have mentioned this before! ) - but for the bits that creep through, there must be some way to avoid clipping them and make use of that YUV headroom.

Be specific about "out of range colours" . Did you mean out of gamut YUV=>RGB conversion ? ie Values that don't "map" to RGB color model in 8bit ? (That's not the same thing as 16-235 vs. 0-255). The only way in that case would be to alter values in YUV first, because vegas isn't capable of using a YUV timeline (other editors like Premiere, FCP can) , except for cuts only smart rendering projects

If you only meant preserving 0-255, you can do a full range conversion in avisynth to RGB first (PC matrix), or use a studio RGB matrix (which is the same thing that vegas uses when using native camera formats) . Trevlac wrote a studioRGB function for avisynth (which can be found at doom9)

EDIT: PDR describes the nightmare I found - different formats treated in different ways in Vegas. I didn't know about IYUV working properly though - I'll try that next time. Don't you also have to select something special in Vegas? 32-bit mode or something?! It's three years since I tried this, so I can't remember. btw, (and I didn't dare admit this before) last time I made it work by cheating: I used MPEG as an intermediate format which Vegas a) smart renders, and b) doesn't change the levels of. Though the preview and the final output looked completely different.

I think you were involved, we discussed this on a doom9 thread a few years ago. There were examples posted, many tests done. IYUV is treated as "studio RGB" (same as native camera formats, such as DV), in regular 8bit mode - there is nothing special to do, you just import it. The preview and final output handle exactly as you would expect if you just dropped some native camera format into vegas

There aren't any mathematically lossless codecs that I've found that get studio RGB treatment in vegas

[2Bdecided]

Thanks pdr.

I can't remember every thread I've ever posted to I remember the thread about studioRGB in AVIsynth though.

Many of the VHS camcorder tapes I have include colours far outside of RGB 0-255 even when Y is adjusted back to within 16-235. Luckily the digital camcorder tapes I have only exceed 16-235 in luma; the colours are mostly fine once the luma is corrected.

Cheers,
David.

[poisondeathray]

If there were a lot out of gamut and usable values, and if I was doing some project now or in the near future, I would probably wait awhile because of Rec2020 in the new 4K TV sets. Rec1361 (wide gamut displays) has been available for a few years, but Rec2020 is going to push the gamut dramatically

[Tafflad]

The workflow was:
1. YUV on tape
2. YUV DV capture
3. YUV in AVIsynth (hopefully - Tafflad should have used AVIsource but may have used DirectShowSource)
4. AVIsynth processing
5. RGB out of AVIsynth
6. RGB in Vegas
7. Vegas processing
8. YUV in the file we have (your latest tweak goes here)
9. RGB for display
.

I did use AVIsource

The output of AVIsynth (script 2 ) was Lagarith RGB
This was loaded into VD with the vcf config file
Output set as Lagarith YV12

This file then dropped into VEGAS for processing
Output of Vegas was Elementary files M2V & AC3 for DVD creation.

[poisondeathray]

lagarith YV12 into Vegas is a potential problem (again, not a huge problem in your case), because YUV isn't passed through in vegas, it gets converted to computer RGB causing clipping of superbrights/darks. But you shouldn't have any substantial superbrights/darks (they should already be corrected for earlier in avisynth), and/or already clipped by converting to YV12 from vdub

Lagarith RGB would probably be a better choice to export out of vdub if you've done some Vdub vcf processing . Also because you avoid YV12=>RGB=>YV12=>RGB=>YV12 , one fewer conversion

What kind of manipulations are being done in vegas ?

[Tafflad]

Did some white balance adjustments on some sections, basic edit, fades, etc.
Some restoration of audio using break out to soundforge.

[2Bdecided]

To see what sort of clipping or other problems occurred, the source is the place to go, not a later sample.

Do you have the original DV-AVI capture of this same section Tafflad? If you want, you can upload it, and we can see if the highlight details are there or not. If you do, it might help to post a reminder of the exact scripts you used too.

For a newcomer to Avisynth working with a difficult video, I think tafflad did quite well.

That's called damning with faint praise. Given the aliasing in the original source, I think the results are near miraculous.

FWIW I still don't have an answer to taming out-of-range colours - the basic problem I raised in 2008 still exists...
http://forum.doom9.org/showthread.php?t=135248
In the TV world I leave the illegal colours in there and (for now) watch the result on a TV that displays them just fine. The videos look just like they did in the 1980s/1990s. I know of course that you can do colour correction to remove the illegal colours - but the results don't look right to me. As recorded and watched in the 1980s/1990s via analogue, these are bright, saturated, unclipped pictures. Mapping into legal colours makes them look dark, desaturated, or both.

I know about BT 2020 (though all 4k TVs currently on sale only support Rec.709), but I can't see me upscaling VHS to 4k just to use that expanded colour space. There may be some support for BT 2020 at full HD resolutions, but it'll never be supported at SD resolutions. I guess it'll be an interesting thing to play with, and it'll be interesting to see if those out of range Rec.601 colours can be mapped in any useful way that gives a bright saturated unclipped picture. Another option might be xvYCC, and I guess I could try that today, if I knew how to author a BD5 with the appropriate flags for that colour space, and had a suitable BluRay player and TV.

(Sorry for going off topic).

Cheers,
David.

[Tafflad]

To see what sort of clipping or other problems occurred, the source is the place to go, not a later sample.

Do you have the original DV-AVI capture of this same section Tafflad?.

Yep ... I'll snip out the sample sections & post later today

[sanlyn]

For a newcomer to Avisynth working with a difficult video, I think tafflad did quite well.

That's called damning with faint praise. Given the aliasing in the original source, I think the results are near miraculous.

Just a very short time ago, tafflad didn't even know how to load a plugin into Avisynth, much less how to modify complicated scripts of which he had little or no knowledge. I'd say that's pretty darn good. How many newcomers would have the patience to recapture all that material, much less get involved with Avisynth and Virtualdub to begin with? I guess I understated matters the first time, so I'll say "good show", "well done", and "bravo". I believe those words mean the same thing in the UK that they mean over here.


And BTW, I and tafflad used and modified some of your own code from other threads. So from me and on behalf of the O.P., thanks for your help.

[Tafflad]

OK .. here is the same 2 clip sequence from the original capture.

Guys ... just to be clear here, I am very happy with the end result, and really pleased with support & patience I had from you ........ I only mentioned the 'motion' issues as I saw it in one or two places.
Just good to know if the restoration has exacerbated the problem .. and if so could it have been treated differently. (for my learning)

It looks far better 'played' than it does in frame by frame that is for sure

I have about 30 more tapes to work on ... but the material quality for bulk of them is much better, they were on my own Camcorder a Panasonic S VHS model , so have better starting point. Recorded of best quality tapes, good colours & no motion smearing, very low noise ... compared to the 'Wedding tape'

On this point is it typical to create a 'template' for treating vids from a common source ? ... i.e starting further along the curve so to speak by having a template for colour correction, levels, noise as a starting point ?

[2Bdecided]

Amazingly, I don't think you've clipped anything. Just, maybe, a tiny bit of shadow detail in the very darkest bits of the most "contrasty" frames, but no one would ever see it. The highlights are blown out on the original tape: not the capture - that's not clipped; not the processing - it doesn't make it any worse. Or, if it does, it's invisible on these clips to my eyes.

The following script will compare original vs processed, and it will avoid display-clipping by reducing the contrast dramatically. The picture will look weird and washed out because of the reduced contrast, but anything that's been lost during the processing will be present on the original, but not the "restored" version - and with reduced contrast, the shadows and highlights are very visible, and without clipping.

a=avisource("motion sample.avi").Trim(2,106)+avisource("motion sample.avi").Trim(113,307)
b=avisource("Motion orig capture.avi")

b
COlorYUV(cont_y=30,off_y=-7)
COlorYUV(cont_v=-120,off_v=1) #,Cont_u=-20)
ChromaShift(L=-4)
b=last


a=a.crop(16,16,-16,-16).assumetff().Bob().Subtitle("restored")
b=b.crop(24,16,-8,-16).assumebff().Bob().Subtitle("original")
stackhorizontal(b,a)

c=last

levels(0,1.0,255,50,200,coring=false)
mergechroma(c)

converttorgb(matrix="PC.601")

The darkest bits inside the church doorway are almost the same on both. The brightest bits on the man's face are the same on both.

Good job done IMO.

On this point is it typical to create a 'template' for treating vids from a common source ? ... i.e starting further along the curve so to speak by having a template for colour correction, levels, noise as a starting point ?

I do. I have two. One for natural light, and one for artificial light. The absolute video levels, colour balance, and level of noise were each very different between the two on my old camcorder. Some kinds of artificial light gave really weird results, so they can need something very different.

I tend to get it close-ish, do the edit, watch it on a TV, then fix the bits that annoy me - and only those. You can get carried away tweaking every bit on a PC. IMO. YMMV. etc.

I haven't got around to editing much yet! Maybe 15 or 20 3-10 minute sequences out of hundreds of hours of footage.

With loads of raw footage and a really "tight" edit (e.g. you use less than 10% of what you started with) I have cheated and edited the raw footage, then ran just the final edit through AVIsynth. Even using a couple of applyrange() statements to attack different parts of the final edit in different ways in one AVIsynth run is quicker (IME, YMMV) than going back to the original footage and processing it scene-by-scene.

Time has knocked any perfectionist tendencies out of me. (Except sticking with the horrendously slow TempGaussMC_alpha3 for deinterlacing when necessary.)

The other reason I've stopped being a perfectionist is because what most people care about more than anything is having just the best bits to watch rather than having to sit through everything you ever shot (unless you were very good and selective at "editing in camera", which I wasn't). The picture quality comes a distant second.

Cheers,
David.

[sanlyn]

Many people keep a subfolder of text files that contain snippets of scripts, functions, procedures, etc. You start a new script and copy snippets from those template files. It's a little difficult to re-use exactly the same settings for every video, but text for filters and filter chains are pasted into a new script and modified to suit the situation. Quite often you have similar pieces of video, but each piece might require something different in the filter settings; just copy the same script used for the first clip, then modify that copy for other similar clips.

For example, you can have a SmoothLevels statement saved somewhere that looks like this:
SmoothLevels(8, 0.9, 255, 15, 245, chroma=200,limiter=0,tvrange=true,dither=100,prote ct=6)
Copy that statement from your saved snippets, and then modify the parameters as required. Usually I modify the first 5 numbers to suit the video. The other parameters remain the same almost all the time.

You can save "AviSource" or DirectShowSource statements, and just change the path and filename to suit. The path statement for your Avisynth plugins folder can be saved and pasted into new scripts. Saves tons of time and typos.

[Tafflad]

Any other 'text snippets' you guys would like to share ... to help me start on my own collection ?

Dave .... great little compare script ... really helps you see difference

[sanlyn]

Original of "motion" capture: Yes, 2BDecided is right, there's some clipping in-camera. And a color balance shift between the two shots (the suits change hue, among other things). Tough shot.