Overexposed U-Matic Captures

[Winsordawson]

I am working with a company to transfer a collection of U-Matic tapes from the 1980s. Some of these tapes I captured myself before my deck died, but I decided to send them to a company for re-capture because I was not using a TBC and did not have the ability to bake tapes. All this the company can do.

Overall, they have done a great job. On certain segments of certain tapes that they converted, the scenes appear overexposed. However, on other scenes on the same tape there is normal exposure. But with the version that I captured previously, none of the scenes are overexposed. The only format difference is that I converted to .dv and they convert to Pro Res.

I have attached one example, my personal capture originally from .dv (I just ran it through ffmpeg to copy to .avi), the company's capture before Premiere Pro and Avisynth, and the company's capture after these tools where I tried to lower the whites. As you can see, the forehead detail is visible in my personal capture while it is lost on the professional capture, even after trying to lower the exposure.

They claim that because ten years passed between my capture and their's, that the luminance could have degraded during that time, and that they have no ability to set a different white point on their capture system. My guess is that possibly because I was capturing in .dv 4:1:1, that automatically prevented the overexposure compared to their Pro Res 4:2:2.

Is there anything I can do about this or advise to them for their set-up to prevent this for future tapes I send to them?

Thank you.

[Sharc]

The 'professional before' has heavily clipped brights. All bright details are lost. You cannot recover any bright details by post processing. What is lost is lost, unfortunately.

[Winsordawson]

Thank you. Do you know why my personal capture not suffer the same issue? I was not using a TBC. I was using back then an Canopus ADVC 110.

[Sharc]

Your personal capture is MUCH better in this respect. See the luma histogram

[Sharc]

Thank you. Do you know why my personal capture not suffer the same issue? I was not using a TBC. I was using back then an Canopus ADVC 110.

Because you (or your ADVC110) set the analog video levels (luma) right, and the professionals messed it up. They did a bad, diletantic job IMO.
This has nothing to do with TBC, converting to ProRes or DV.

Edit: I would really consider post-processing your capture (in Avisynth for example, deinterlacing, denoising ...). It doesn't suffer much from missing TBC.

[Winsordawson]

Thanks--in the end I used my capture and put it through Avisynth for denoising and de-interlacing. Is it normal that they said that they cannot adjust the luma levels before capture? Or is it true that some devices do not allow for that?

[Sharc]

Thanks--in the end I used my capture and put it through Avisynth for denoising and de-interlacing. Is it normal that they said that they cannot adjust the luma levels before capture? Or is it true that some devices do not allow for that?

Sounds more like an excuse to me. Your capture proves it. Worst case they could have inserted an attenuator in the chain. Or their tape player was defective and clipped the signal.

[Winsordawson]

Thanks--I'll mention the attenuator to them and report back. To be fair, the rest of the video, both before and after, was not overexposed, and contained multiple segments. So is it possible that maybe this segment was more exposed but the Canopus had some built-in attenuator because it was meant for consumers? Or, for the same reason, it set the luma point lower?

[Sharc]

What tape player model did you use? Maybe their U-MATIC player clipped the signal and has no analog video level adjustment, so an external attenuator wouldn't help either. Once it's clipped the damage is there - irrecoverable.
Or their TBC messed with the levels. Then an attenuator before the signal goes into the TBC device might help - if the TBC has no level adjustment.
Your capture proves that the tape is not overexposed. The flaw happens somewhere along the playback process.

[Winsordawson]

I was using two decks at the time: JVC CR-600U and CR-850U. I only stopped using them because they broke and I didn't have the best set-up anyway.

That sounds reasonable. I will report back their response.

[Sharc]

Sidenote: The 'professional before' is interlaced BFF (which is perfectly ok for the capture), but the stream is flagged progressive. It should be flagged correctly as interlaced to prevent issues with playback or future processing.

[Alwyn]

That "Professional Before" is a shocker. Sounds pretty dodgy they can't adjust the levels for capture.

The "after" has the wrong display aspect ratio; it's 3:2 instead of 4:3. You can tell the heads are too wide. The 10 bit messed with me too!

I just learnt a new word... "diletantic". Good one, Sharc!

[Winsordawson]

Sharc: Thanks--I still import their captures to Premiere Pro as BFF and treat it as such in Avisynth with AssumeBFF. Perhaps it happened when I quickly downsized the file in ffmpeg, because according to Mediainfo the original Pro Res lists it as interlaced BFF.

Alwyn: The "after" is a snippet of corrections I tried to make to the professional capture with Premiere Pro and Avisynth. According to the restoration sub forum here,

Some NTSC capture devices capture 720x486 to include the entire 704x485 of the active NTSC frame. The extra width is captured in case the signal is slightly off center. The extra height to get the entire half line at the top and half line at the bottom. This ensures you get things like the line 21 closed captions.

Since the standard in NTSC land is 720 x 480, I cut off some of the black lines around the footage.

[Sharc]

... Since the standard in NTSC land is 720 x 480, I cut off some of the black lines around the footage.

Sure, but the display aspect ratio flag is nevertheless wrong (or missing, defaults to 3:2). The player needs to force the playback as 4:3. But that's something for later, perhaps ...

[Winsordawson]

In that case, is the solution to add -aspect 4:3 to the ffmpeg export?

[Sharc]

In that case, is the solution to add -aspect 4:3 to the ffmpeg export?

yes, and crop to 704x480 to be more exact, because for standard analog captures the 4:3 picture is represented by 704x480 pixels of the captured frame for NTSC, in very good approximation.

[Alwyn]

Since the standard in NTSC land is 720 x 480, I cut off some of the black lines around the footage.

Yes, the standard in PAL land is 720:576, also not 4:3. Both the standards are intrinsically 4:3 video but the raw resolutions on capture don't match that. That's why you have to encode them, so they display at 4:3 as opposed to 720x480 or 720x576. You export using either the appropriate SAR or square pixels ie 720x540.

Out of interest, is "professional_before" the file they gave you or has that been re-encoded?

[Winsordawson]

Thank you both. I know about the 704 cropping, but because I also submit a version of this video to public cable access for television airing, I have read that cropping to 704 might throw off their system and mark the video as non-compilant, since 720 is the NTSC standard. For YT, yes I could crop.

The "professional_before" is just something I did quickly in ffmpeg by converting Pro Res to an mp4 file to save space. Their actual Pro Res is interlaced BFF. The "professional_after" is after running their file through Premiere and Avisynth.

[aramkolt]

Interesting thread! I have several U-Matic decks and I'm starting to dabble in doing transfers for others as a service using a BVU-950 (via composite with internal TBC option) or a VO-9850 modified for S-Video output, or Via DUB transcoding) to ProRes422.

I am surprised you were given an excuse that the tape somehow changed for luma levels to be blown out now. It is possible for tape to lose magnetic retentiveness, but it'll do so uniformly and the gain circuits will usually auto-adjust to get sync tip to be -40IRE, or they'll be universally low, requiring some luma and chroma boosting with a proc amp.

It seems clear that they didn't use a proc amp before the digitization stage to prevent clipping and they weren't monitoring for clipping with a waveform monitor/vectorscope. Some Frame TBCs will clip incoming luma levels if over a certain point and then adjusting them down lowers the output level, but you'll still see it is "clipped" by having a very flat peak on the waveform monitor.

All professional capture setups should really have a vectorscope and waveform monitor in the chain. I like SDI because you can monitor what the levels are after they've been converted to digital so you'll know how the recording device will store those values without the possibility of clipping. Histogram monitoring can represent the same information in a different way, but in this case, they clearly weren't using that either.

If you want high quality U-Matic captures using the best Sony equipment made, I can do it for $25 a tape, feel free to PM me. My guess is that's quite a bit less than the service you used. Most services I see charge double or more and use inferior equipment.

My YouTube channel has various mostly U-Matic captures that I've done, here's a more recent capture I did that is an example of what a good U-Matic capture can look like:
https://www.youtube.com/watch?v=TghOWybScrY

[Sharc]

For comparison here the OP's original personal_capture with a bit of post processing.

BSSource("personal_capture.avi")
assumeBFF()
ConverttoYV16()
colorYUV(gain_Y=-5,cont_u=-10,off_V=5)  # tweak colors to taste
crop(16,0,-16,-12)
QTGMC(lossless=2)
temporaldegrain2(degrainTR=3)
KNLmeansCL(d=2,h=2)
FineDehalo(rx=4.0, ry=4.0)
histogram("levels") #remove for real

Now encode x264 with --sar 10/11

[Winsordawson]

Thanks Sharc--could you share code you used if in Avisynth? I usually use a small amount of AddGrain at the end for dithering.

aramkolt--do you have the ability to bake tapes?

[Sharc]

Thanks Sharc--could you share code you used if in Avisynth?

Nothing extraordinary or optimized, just playing a bit to show that you can probably do better yourself than what the shop did. I added the script to post#20.
Sidenote: Colors are still oversaturated. Note that the background is totally out of gamut. It doesn't matter much as it is artificial anyway.

[aramkolt]

Thanks Sharc--could you share code you used if in Avisynth? I usually use a small amount of AddGrain at the end for dithering.

aramkolt--do you have the ability to bake tapes?

Yes, I do have the ability to bake tapes and physically clean them if necessary. I do not charge anything extra for doing those things. Mot tapes I'll bake for at least 24 hours, but I'll only physically clean them if there's obvious mold or if they still aren't playing well after baking.

[Winsordawson]

aramkolt: Thanks. I think I will see which tapes they have problems with and send those to you. I will PM you at that time.

Sharc: I spoke with the technician and he said that their U-Matic decks do not have video level adjustments on them, so perhaps this is the source of the problem. He said he was confused because according to him, an attenuator affects the RF signal, not the video signal, and it increases exposure, not decreases it, so he did not know if there was another term that I meant instead. What say you?

[Sharc]

Sharc: I spoke with the technician and he said that their U-Matic decks do not have video level adjustments on them, so perhaps this is the source of the problem. He said he was confused because according to him, an attenuator affects the RF signal, not the video signal, and it increases exposure, not decreases it, so he did not know if there was another term that I meant instead. What say you?

Well, I don't know their equipment and setup, and their "RF" argument confuses me.
- What U-MATIC player model are they using?
- What digitizer model are they using?
- How do they connect the player (U-MATIC) to the digitizer?
- How do they explain that your earlier personal captures were much better?

Maybe look for another shop and run a test with them, or maybe a user here will help ....

[Winsordawson]

If I recall the guy said they were using a some Sony BVU model and a Blackmagic device. They were using an S-Video/Dub connection. They really did not have an answer for why my personal capture was better, other than possibly 1)that deck had better video adjustments 2)the RF signal degraded in the ten years between captures. But I think it's probably because for the former.

Looking online, it seems that the deck I was using, a JVC CR-850, has a few extra features than some decks, like a Y-Freq Response and Video AGC (automatic gain control) dials. So maybe that could also explain it.

[timtape]

A good transfer tech would at least notice by eye that the highlights are clipping and check if their own gear is the problem. This is basic.