Horizontal noise bars - what are they called, and how to remove?

[johnmeyer]

I have been asked to fix this video, but have not yet received the DVD. It is of a 1945 American football game. I don't have access to the original film, but I was told that the original film to video transfer was done professionally. Whatever created these horizontal bars happened after that time.

I have been told that they are static (i.e., they do not crawl up or down the screen as the video is playing).

Question 1: does anyone know what this interference pattern might be called (so I can Google it)?

Question 2: does anyone have some general ideas of what they would use to fix it? If they truly are static, then I might just be able to get away using a mask of some sort and then adjust the exposure and color saturation. If they move, I'll have to devise another approach.

Question 3: Does anyone know what might have caused this?

I ask the last question because the person who purchased this disc may be able to find a source that is closer to the original transfer if we can pinpoint what caused the problem, and by knowing that, figure out how to go around the person involved.

Thanks!

Here's an unedited clip from the DVD:

Noise Bar Clip

[jagabo]

Hum bars. The can be reduced by a lot in AviSynth:


[Attachment 58299 - Click to enlarge]

ImageSource("nUPErnm.png", start=0, end=23, fps=23.976) 
src=last
ConvertToYV12() # image has already been deinterlaced?
BilinearResize(720,480)  # assuming the original was 720x480, NTSC DVD.

TurnRight()
defreq(fx=16.6, fy=0.0, dx=2, dy=2, sharp=0, info=false, show=0)
defreq(fx=49.4, fy=0, dx=2, dy=1, sharp=0, info=false, show=0)
TurnLeft()
aWarpSharp(depth=5).Sharpen(0.3)
Spline36Resize(src.width, src.height)

You can probably get better results with the original DVD video. And if the bars are really static and appear over the entire video all the time they can probably be removed almost completely by simply subtracting them.

[kelvin]

Hum bars. The can be reduced by a lot in AviSynth:


[Attachment 58299 - Click to enlarge]

ImageSource("nUPErnm.png", start=0, end=23, fps=23.976) 
src=last
ConvertToYV12() # image has already been deinterlaced?
BilinearResize(720,480)  # assuming the original was 720x480, NTSC DVD.

TurnRight()
defreq(fx=16.6, fy=0.0, dx=2, dy=2, sharp=0, info=false, show=0)
defreq(fx=49.4, fy=0, dx=2, dy=1, sharp=0, info=false, show=0)
TurnLeft()
aWarpSharp(depth=5).Sharpen(0.3)
Spline36Resize(src.width, src.height)

You can probably get better results with the original DVD video. And if the bars are really static and appear over the entire video all the time they can probably be removed almost completely by simply subtracting them.

you are very smart. You have an incredible gift.

If I lived in your city and knew how to speak English, I would pay private tuition to learn part of what you know

[jagabo]

johnmeyer, when you get the DVD upload a sample and I'll update the script.

[johnmeyer]

jagabo,

The image is (or will be by the time I do IVTC) progressive.

You got a great result. Your approach is similar to what I was going to initially try, given that I've used Defreq before.

I have two questions.

1. I've never resized to the same resolution. You then perform a second resize, also to the same resolution, as the last step. I'm sure you did this for a reason. What does it do?

2. Since you can specify the angle and spacing in Defreq, why the Turnright/Turnleft?

I would have just done the Defreq and gone home.

P.S. When you say "subtracting them," are you talking about a mask? I was thinking of doing a blur and using the result for a mask.

[jagabo]

1. I've never resized to the same resolution. You then perform a second resize, also to the same resolution, as the last step. I'm sure you did this for a reason. What does it do?

Since you said you would be starting with a DVD I resized to a DVD frame size. I thought that would be more applicable to the source. Of course, it will likely be interlaced so you'll have to separate fields, defreq at half the frequencies, then weave. Or QTGMC before defreq. I upscaled back to the original image size so the output would be easier to compare to the input.

2. Since you can specify the angle and spacing in Defreq, why the Turnright/Turnleft?

In my experience defreq doesn't work well with horizontal lines. You get much better results with:

TurnRight()
defreq(fx=16.6, fy=0.0, dx=2, dy=2, sharp=0, info=false, show=0)
defreq(fx=49.4, fy=0, dx=2, dy=1, sharp=0, info=false, show=0)
TurnLeft()

than with:

defreq(fx=0.0, fy=16.6, dx=2, dy=2, sharp=0, info=false, show=0)
defreq(fx=0.0, fy=49.4, dx=2, dy=1, sharp=0, info=false, show=0)

When you say "subtracting them," are you talking about a mask?

No, I was thinking of something like Overlay(source, bar_image, mode="subtract"). Where bar_image is an image of just the bars on a black background (built somehow from the actual video).

[johnmeyer]

Since you said you would be starting with a DVD I resized to a DVD frame size.

OK, got it.

Of course, it will likely be interlaced so you'll have to separate fields, defreq at half the frequencies, then weave.

As I said, the original content is film, so it's progressive. It will either be 23.976 on the DVD, with a pulldown flag, in which case it will be progressive, or it will be hard-telecined, in which case I'll do IVTC. I won't need to deinterlace.

I now understand the resize and I probably won't need to do that either, since the results of my effort will go right back onto another DVD (that's the deliverable).

2. Since you can specify the angle and spacing in Defreq, why the Turnright/Turnleft?

In my experience defreq doesn't work well with horizontal lines.

OK, I get that. It's a good thing to remember, because I've used Defreq before on horizontal lines, and it left more residual than what you showed with your work.

When you say "subtracting them," are you talking about a mask?

No, I was thinking of something like Overlay(source, bar_image, mode="subtract"). Where bar_image is an image of just the bars on a black background (built somehow from the actual video).

That is what I thought. Maybe my use of the term "mask" wasn't correct, but I was thinking of constructing some sort of grayscale image of the noise bar and then using that to apply gamma and color correction of my own. If I go that route I should be able to construct a video mask (as opposed to a single, static image mask), so that if the bars on the original video shift location, the "mask" will shift as well.

[killerteengohan]

santiag(strv=0, nns=4) will probably help clean out the leftovers. I see what looks like really tiny lines left over that look like leftover interlacing artifacts, even if its not interlaced. Santiag usually masks or removes those pretty well most of the time.

[johnmeyer]

santiag(strv=0, nns=4) will probably help clean out the leftovers. I see what looks like really tiny lines left over that look like leftover interlacing artifacts, even if its not interlaced. Santiag usually masks or removes those pretty well most of the time.

The still shot came from the client and, since the video is from a DVD, it is most likely telecined, but could also be interlaced. Either way, when most people do a screen shot of either telecined or interlaced content, they are going to end up with fields from two moments in time, and that will certainly cause "teeth" or lines.

I can guarantee that any temporal issues between fields will be gone before I start doing anything (I know how to IVTC or deinterlace), and so I will be working on progressive film frames.

[johnmeyer]

Here's an unedited clip from the DVD:

Noise Bar Clip

I found that the last two minutes of the DVD is a blank screen (the film had run out) which included the noise bars. It provides a hint as to what caused the problem. I note that the bars are not horizontal and that they jitter up/down slightly. I've attached a snapshot of the bars.

I'll post my results, after I try a few of the suggestions.

[jagabo]

Yes, I noticed the bars weren't perfectly horizontal too. Earlier I had taken an image of the bars at the far right of the frame, stretched it to the full image width, then subtracted it. Here's what I got:


[Attachment 58341 - Click to enlarge]

You can see that it worked pretty well at the right half of the frame but failed at the left. You're bar image shows why.

[johnmeyer]

Yes, I'm working on that now. Using the bars as a mask is close, but no cigar.

Also, the clip must be IVTC'd before you can do anything. It's just standard 3:2 pulldown.

[johnmeyer]

I used jagabo's Defreq script (without the resize, since that's not needed) and it's pretty darn good. I'll try seeing if I can tune it a bit, using the secondary Defreq parameters. Once I eliminate all the white flashes at scene changes (all movie cameras do this), run it through my film restoration script, and adjust the exposure, it should look pretty good.

[edit]The one problem is that there Defreq seems to fail fairly often on the top 10% of the frame. There is also some residual from the bars up and down the center of the frame, but I think that is due to the bar's discontinuity at that point. This will probably be good enough for this project, unless someone has a few tweaks to suggest. I'll post a before/after in a few days when I'm finished.

[themaster1]

the filter destripe was made to take care of these stripes http://avisynth.nl/index.php/DeStripe

something like:
avisource() # ivtc'd
TurnLeft()
DeStripe(2,3,16)
DeStripe(2,2,16)
DeStripe(2,1,12)
TurnRight()

edit: nevermind defreq works better it seems

[killerteengohan]

santiag(strv=0, nns=4) will probably help clean out the leftovers. I see what looks like really tiny lines left over that look like leftover interlacing artifacts, even if its not interlaced. Santiag usually masks or removes those pretty well most of the time.

The still shot came from the client and, since the video is from a DVD, it is most likely telecined, but could also be interlaced. Either way, when most people do a screen shot of either telecined or interlaced content, they are going to end up with fields from two moments in time, and that will certainly cause "teeth" or lines.

I can guarantee that any temporal issues between fields will be gone before I start doing anything (I know how to IVTC or deinterlace), and so I will be working on progressive film frames.

I was not saying anything about you doing IVTC or deinterlace incorrectly. Those leftover small horizontal white bar artifacts that resemble leftover interlacing that I was referring to, are also in Jagabo's good ending result. Sometimes even doing IVTC correctly, you can get little leftovers that look like those. It's not IVTC or Interlacing causing the issue you want fixed, nor was my suggestion meant to be a 100% fix for it. I was giving a suggestion that may clean, or reduce visibility of the leftover bar artifacts I can still see all over the grass, even after Jagabo's nice idea does its work. If they are not in your final output, then don't worry about it.

[jagabo]

Yes, one often gets those slight comb-like artifacts after IVTC of hard telecined material. They occur because compression artifacts cause small differences between the matching fields, especially when the two fields come from different video frames. I've always used vInverse() to eliminate them but your santiag(strv=0, nns=4) seems to work better in a few quick tests I ran.

[johnmeyer]

I finished the project. Thanks jagabo for your Defreq settings. I tried to improve on them but, once again, failed to figure out how to use the "show" information to tune the parameters. Once in awhile, when I've used Defreq in the past, I hit the right numbers, but it isn't just as easy as looking at the on-screen numbers and then putting them into the appropriate Defreq settings.

[edit]I was going to upload my "after" but it was essentially what jagabo already posted, although with some additional fixes, like gamma correction, stabilization, and a little dirt removal. Since I ended up using jababo's code, without alteration, there is nothing new to show regarding the removal of the bars.

[johnmeyer]

Since I did do a few other things, here is the "before/after."

[Alwyn]

Excellent, John!

[killerteengohan]

I still think using santiag(strv=0, nns=4) as well would benefit it with the leftover horizontal lines if you haven't already used it.