Color/hue problem. Any way to recover?

[fbreve]

Attached are the first 3 minutes of this video. Most frames have major color problems, but some of them are right.

The original tape was probably in NTSC, but my copy is in PAL-M since most of our earlier VCR would only record in PAL-M. They would also transcode NTSC tapes to PAL-M on playback. Maybe something went wrong on the copy.

The intro doesn't seem to suffer as much as the main video, so maybe the problem was already on the original tape.

If I pause the VCR, the colors keep oscilating like on playback.

Unfortunately I don't have another VCR to try. I'm using a 7-head Gradiente GSV-860HF. It uses JVC technology, as Gradiente and JVC had a joint-venture back then. All of my other tapes (both PAL-M and NTSC) play just fine on it, with all correct colors.

I tried setting hue to -70 ~ -75 with the Virtualdub filter. This makes the colors better on most frames, but then it spoil the few fames that were actually right. I couldn't find a "auto hue" filter that would only correct the bad frames.

[themaster1]

Try this with avisynth:
converttoyv12()
input=last
U = UtoY()
V = VtoY().Invert()
YtoUV(U, V, last)

[jagabo]

The main problem is that the V channel is inverted most of the time:



It's easy to invert the V channel, but as you noted, sometimes it flips back to normal:



Note how the green waveform at the bottom is inverted compared to the first image.

It might be possible to flip only the frames with the inverted V channel but there will still be some frames where the V channel is flat:



Since this is mostly low motion material you probably copy the previous frame's V channel over the flat ones.

The only hope of doing this in an automated fashion is with advanced filtering in AviSynth.

[fbreve]

Thank you. It is much better now.

However, the few good frames became bad. Is there any way to fix the bad ones and keep the good ones?

[fbreve]

The main problem is that the V channel is inverted most of the time

Thank you. I didn't see your answer before my first reply. I never used Avisynth before, so I'll try a few tutorials to see how to do that.

[jagabo]

The command to invert the V channel in AviSynth: ColorYUV(cont_v=-512). Selecting the right frames is much more complicated.

[jagabo]

Here's a script that inverts the V channel only when necessary:

LWLibavVideoSource("arruda-sample.mp4") 
norm = last
inverted = ColorYUV(cont_v=-512)
ConditionalFilter(norm, norm, inverted, "AverageChromaV()", "greaterthan", "128")

That leaves you with the frames with a flat V channel still do deal with. And, of course, there's still all the other problems, like frames that are only partially discolored.

But I'd look into trying a different VHS deck and capturing again.

[jagabo]

I came up with a crude way of replacing flat V frames with the chroma from the previous frame. Since there are sometimes two flat frames in a row the procedure is run twice.

function AbsV(clip vid)
{
  V = vid.VtoY()
  Overlay(v.ColorYUV(off_y=-128), v.Invert().ColorYUV(off_y=-128), mode="add")
}

LWLibavVideoSource("arruda-sample.mp4") 

norm = last
inverted = ColorYUV(cont_v=-512)

# replace inverted V frames
ConditionalFilter(norm, norm, inverted, "AverageChromaV()", "greaterthan", "128")

# substitute previous frame's chroma if flatV
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

# substitute previous frame's chroma if flatV , catches two flatV in a row
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

This code won't work in the general case because other videos may not have these same properties. But it works for the live video portion of this video (I didn't bother with the graphics in the first half).

From here you would follow up with more typical color corrections, white balance, saturation, etc.

Small sample attached.

[fbreve]

Thank you very much.

It is much better now.

There are frames outside the sample that are still green, like this one:



If I raise the threshold from 128 to something like 136~140 it will fix this, but than other frames will become green.

Now, starting from your script, I'm trying to figure out some other clue that V is inverted other than the AverageV. Maybe some relation between AverageV and Luma or ChromaU?

[johnmeyer]

If you can get 90% of the way there with jagabo's amazing solutions, you could simply then put the corrected and the original versions into your NLE and cut between them as necessary. In my NLE (Vegas), I can scrub the timeline at some multiple of normal speed, so it doesn't take too long to get through a lot of video. Since it sounds like your remaining problems are still many frames long (i.e., not one single "flash" frame of green) you should be able to spot the places where you need to cut back to the original, even when scrubbing at 4x or 8x.

[jagabo]

There are frames outside the sample that are still green, like this one:


[Attachment 38324 - Click to enlarge]

If I raise the threshold from 128 to something like 136~140 it will fix this, but than other frames will become green.

Now, starting from your script, I'm trying to figure out some other clue that V is inverted other than the AverageV. Maybe some relation between AverageV and Luma or ChromaU?

You'll probably want to work in sections. For a shot like that, if it's not moving too much, you can crop the frame down to just his face and use that as the test clip in ConditionalFilter().

ImageSource("attachment.png") 
ConvertToYV12()

norm = last
inverted = ColorYUV(cont_v=-512)
test = norm.Crop(240,0,-240,-0)

# replace inverted V frames
ConditionalFilter(test, norm, inverted, "AverageChromaV()", "greaterthan", "128")

This technique only works when the picture in question has more red/puple than green/blue.

Some tools you should be aware of:

ColorYUV(analyze=true) will show you average and extents of Y, U, and V.

VideoScope("both", true, "U", "V", "UV") shows waveform graphs of U and V. You can see Y to by changing the arguments.

Histogram() also has some modes that show the chroma channels.

You can see the chroma channels as greyscale with UtoY() an VtoY().

[jagabo]

Some other things you should try:

Capture the same segment of video multiple times. Are the discolored frames in the same location every time? If not you can use a median (of 3, or of 5) filter to get rid of the outliers.

https://forum.videohelp.com/threads/362361-Median%28%29-plugin-for-Avisynth?highlight=median

Try using an old Panasonic DVD recorder in passthrough mode as a line TBC. It may clean the signal up enough to get rid of the discolored frames. Do you see the same discolored picture when watching on a TV? Capture cards are generally more susceptible to problems like this. I'm not sure what the implications of Brazilian video standards are on this.

https://forum.videohelp.com/threads/319420-Who-uses-a-DVD-recorder-as-a-line-TBC-and-wh...light=recorder

[fbreve]

I believe the median may work. The colored frames are the exception and it seems that they don't appear in the same place every time I play.

I'm already using a Panasonic DMR-ES10 as a line TBC. The color issue is the same with or without it.

I see the same discolored picture when watching on TV.

[fbreve]

Median is working

Median of 3 captures got rid of most color issues, but there are still a few green frames. I'll add two more captures to see if it improves any further.

This is my current script. Please let me know if something is wrong:

clip1 = AVISource("arruda.avi")
clip2 = AVISource("arruda2.avi")
clip3 = AVISource("arruda3.avi")

clip1 = trim(clip1,507,222520)
clip2 = trim(clip2,327,222341)
clip3 = trim(clip3,360,222374)

Median(clip1,clip2,clip3)

ColorYUV(cont_v=-512)

[jagabo]

The script looks right. You can always use modifications of the earlier techniques to detect inverted V frames, after using Median().

[fbreve]

Yes, I did include it again after I added two more captures. The video is now good enough to go through the other steps. It will still have a few green frames and some partially green frames, but that's much better than what I started with. I'll keep the VHS well stored just in case I find a VCR that can play it correctly.

Thank you very much for all your help.

This is my current (and probably final) script:

function AbsV(clip vid)
{
  V = vid.VtoY()
  Overlay(v.ColorYUV(off_y=-128), v.Invert().ColorYUV(off_y=-128), mode="add")
}

clip1 = AVISource("arruda.avi")
clip2 = AVISource("arruda2.avi")
clip3 = AVISource("arruda3.avi")
clip4 = AVISource("arruda4.avi")
clip5 = AVISource("arruda5.avi")

clip1 = trim(clip1,507,222520)
clip2 = trim(clip2,327,222341)
clip3 = trim(clip3,360,222374)
clip4 = trim(clip4,321,222322)
clip5 = trim(clip5,358,222359)

Median(clip1,clip2,clip3,clip4,clip5,sync=15)

converttoyv12()

norm = last
inverted = ColorYUV(cont_v=-512)

# replace inverted V frames
ConditionalFilter(norm, norm, inverted, "AverageChromaV()", "greaterthan", "160")

# substitute previous frame's chroma if flatV
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

# substitute previous frame's chroma if flatV , catches two flatV in a row
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

ColorYUV(autowhite=true, autogain=true)

I'll use Virtualdub to get rid of those VHS garbage lines on the bottom and to apply some noise reduction. I'll post a sample of the result later.

[jagabo]

The video is now good enough to go through the other steps. It will still have a few green frames and some partially green frames

For discolored bars that appear in only one frame you can use this:

MergeChroma(last, Median(last.Trim(1,0), last, last.Loop(2,0,0)))

Basically, that's picks the median color of three consecutive frames. Since this is mostly low motion video it works out ok. It gets rid of most of those defects. Before fix on left, after fix on right:

I'll use Virtualdub to get rid of those VHS garbage lines on the bottom

Why not do it in your AviSynth script?

Crop(0,0,-0,-10).AddBorders(0,0,0,10)

While you're at it you can clean up the left and right black borders too.

and to apply some noise reduction.

If you don't mind a 60p video you can use QTGMC(EZDenoise=3.0). Or there are many other noise reduction filters in AviSynth: MCTemporalDenoise(), TemporalDegrain(), etc.

[fbreve]

For discolored bars that appear in only one frame you can use this:

MergeChroma(last, Median(last.Trim(1,0), last, last.Loop(2,0,0)))

Basically, that's picks the median color of three consecutive frames. Since this is mostly low motion video it works out ok.

Thanks. Just added it to the script.

Why not do it in your AviSynth script?

Crop(0,0,-0,-10).AddBorders(0,0,0,10)

While you're at it you can clean up the left and right black borders too.

Indeed, it is easier. Just added it too. In my previous videos I didn't care much about other borders because they are already black and Handbrake autocrop would cut all black borders anyway. But this video is also going to Blu-ray, so you are right, I'd better clean them too and make them easier to encode.

If you don't mind a 60p video you can use QTGMC(EZDenoise=3.0). Or there are many other noise reduction filters in AviSynth: MCTemporalDenoise(), TemporalDegrain(), etc.

I'm keeping interlaced content interlaced on all my videos. I prefer to leave this task to the player because in the past I used to deinterlace and now there are better algorithms and I can't use them since the details lost in deinterlacing are lost forever. So I prefer to keep them. But I might rethink that. While software like VLC Player does a great job with Yadif 2X, my new Panasonic TV does not deinterlace content from media servers like Sony TVs does. Maybe future hardware will lose the ability to deinterlace.

How MCTemporalDenoise and TemporalDegrain compare to MSU Denoiser Virtualdub filter? I was using it in all my videos because I thought it was the state-of-the-art in video denoise nowadays. But if there is something better I'll be glad to move on.

The current script is now:

function AbsV(clip vid)
{
  V = vid.VtoY()
  Overlay(v.ColorYUV(off_y=-128), v.Invert().ColorYUV(off_y=-128), mode="add")
}

clip1 = AVISource("arruda.avi")
clip2 = AVISource("arruda2.avi")
clip3 = AVISource("arruda3.avi")
clip4 = AVISource("arruda4.avi")
clip5 = AVISource("arruda5.avi")

clip1 = trim(clip1,507,222520)
clip2 = trim(clip2,327,222341)
clip3 = trim(clip3,360,222374)
clip4 = trim(clip4,321,222322)
clip5 = trim(clip5,358,222359)

Median(clip1,clip2,clip3,clip4,clip5,sync=15)

converttoyv12()

norm = last
inverted = ColorYUV(cont_v=-512)

# replace inverted V frames
ConditionalFilter(norm, norm, inverted, "AverageChromaV()", "greaterthan", "160")

# substitute previous frame's chroma if flatV
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

# substitute previous frame's chroma if flatV , catches two flatV in a row
prev = MergeChroma(last, last.Loop(2,0,0))
ConditionalFilter(AbsV(), last, prev, "AverageLuma()", "greaterthan", "5")

MergeChroma(last, Median(last.Trim(1,0), last, last.Loop(2,0,0)))

ColorYUV(autowhite=true, autogain=true)

Crop(2,0,-4,-10).AddBorders(2,0,4,10)

[fbreve]

Here is a sample of the result.

[jagabo]

It's looking much better.

There's one other thing you can look into: replacing frames with discoloring with the colors motion interpolated from the frames before and after. This can fix the colors of glitches that last more than a single frame. This is manually intensive since you need to find the frame numbers of the discolored frames and the number of frames over which the discoloration lasts. Then call:

MergeChroma(last, ReplaceFramesMC(4936,4))

That replaces the colors of frame 4936, 4937, 4938, and 4939, with colors motion interpolated from frames 4937 and 4940. Note that this is best done with progressive frames but with low motion video you can get away without deinterlacing it.

https://forum.videohelp.com/threads/352741-Frame-interpolation?p=2226119&viewfull=1#post2226119