Interlace artifacts in progressive scan video

[SupraGSX]

As the title states I have a clip that suffers from either a poor deinterlace job or something to do with field delay. The video is from the internet best known source Nascars official website. It has been given the black bar 4:3 to 16:9 treatment and is now 1280x720 progessive 29.97fps. Originally this would have been an ntsc 29.97i broadcast I'm assuming in 720x480. Unfortunately instead of leaving it interlaced or properly deinterlacing to 59.94fps they decided to do this. Fast motion is what triggers the artifacts they aren't present in static shots but it being a racing video there's a lot of fast motion. Bill Elliots hand movement shows what I mean. Can this be fixed or not? I've seen people say after bad interlacing is baked in its done and impossible to repair, and I've seen instances where using things like debicubic and qtgmc have been reported to fix it. I'm not sure what makes one fixable and one not so I figured I'd ask for help and maybe learn something as this seems to be the most common issue I see in the types of media I enjoy. Any AviSynth scripts to fix this?

[jagabo]

I don't think you'll be able to get back the smooth motion of 59.94 fps. For 29.97 fps try

BicubicResize(852,480)
QTGMC(InputType=2)

Better analysis could be done with a medium speed horizontal panning shot. Especially one with a vertical bar that runs top to bottom.

[SupraGSX]

Jag, I tried to find a clip that suited your request I think you can use the light poles as vertical markers and this pace lap is about as medium speed pan as I could find

[jagabo]

Here's a start:

LWLibavVideoSource("Horizontal Pan.mp4") 
AssumeTFF()
DeBilinearResizeMT(852,480)
BWDIF(field=0)

[SupraGSX]

Here's a start:

LWLibavVideoSource("Horizontal Pan.mp4") 
AssumeTFF()
DeBilinearResizeMT(852,480)
BWDIF(field=0)

Definitely helps it might run a smdegrain pass as well. For my own better understanding can you walk me through how you arrived at the 852,480 number and why bilinear vs bicubic resize. Is there a method or is it more trial and error?

[jagabo]

Yes, following up with a good motion compensated temporal noise reduction filter will help. I tried

Santiag()
QTGMC(InputType=2, EZDenoise=2.0, DenoiseMC=true)

and it looked pretty good. Santiag reduces some of the aliased edges, Since I was using QTGMC to reduce the buzzing edges I added the NR there.

As to why I resized to 480 lines -- to deinterlace properly the video needs to be restored to the original height. And you want to use the inverse algorithm that was used to upscale it. DeBilinearResizeMT left fewer artifacts than DeBicubicResizeMT or any of the others. I determined the 480 lines by first guessing that it was upscaled from a standard interlaced NTSC cap. I tried a few other sizes but that appeared to work best. And the width was set to 852 to keep the 16:9 AR.

You can upscale back to the original's 1280x720 if you want. I'd use nnedi3_rpow2, maybe followed by an aWarpSharp2 and Sharpen.

[SupraGSX]

Yes, following up with a good motion compensated temporal noise reduction filter will help. I tried

Santiag()
QTGMC(InputType=2, EZDenoise=2.0, DenoiseMC=true)

and it looked pretty good. Santiag reduces some of the aliased edges, Since I was using QTGMC to reduce the buzzing edges I added the NR there.

As to why I resized to 480 lines -- to deinterlace properly the video needs to be restored to the original height. And you want to use the inverse algorithm that was used to upscale it. DeBilinearResizeMT left fewer artifacts than DeBicubicResizeMT or any of the others. I determined the 480 lines by first guessing that it was upscaled from a standard interlaced NTSC cap. I tried a few other sizes but that appeared to work best. And the width was set to 852 to keep the 16:9 AR.

You can upscale back to the original's 1280x720 if you want. I'd use nnedi3_rpow2, maybe followed by an aWarpSharp2 and Sharpen.

Thanks for the explanation I appreciate it. I think it looks as good as it can almost all of the interlace artifacts have been taken care of. The only outstanding question I have is if you know of any interpolation that can handle fast motion like racing. I've tried both InterFrame and FrameRateConverter and they both were not able to handle it with any settings I can find. It would be very obvious when the car passed text on the wall or they went under light poles. The car and text or pole would blend together instead of being detected as separate objects. I'd really love to get this back to 59.94 fps as racing footage at 29.97 can get jittery. I'll attach a clip that's been rendered using the previously discussed settings for experimenting if you think you may know of a combination that might work. Thanks again.

[jagabo]

RIFE currently gives the best frame rate interpolation for most material. It's still not perfect though.

LWLibavVideoSource("C:\Users\John\Desktop\Interpol ation.mp4", cache=false, prefer_hw=2)

LWLibavVideoSource("Interpolation.mp4") 

z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
Rife(gpu_thread=1, model=9, fps_num=60000, fps_den=1001)
z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l")

[edit]

Oops, I forgot to enable scene change detection. See sc and sc_threshold in the RIFE docs.

[/edit]

[rgr]

Scene detection works fine in RIFE now? RIFE at default settings works quite well (aside from the glitches it tends to have), but completely ruins scene transitions.

In my experience -- it works better than the previous version of Topaz, but worse than the current one.

[jagabo]

Scene detection works fine in RIFE now?

It still has problems. I find sometimes it works but sometimes it makes scene changes worse.

[SupraGSX]

I found rife worked really well for the cars moving against things like the wall or fence but if there was a graphic of the current position of drivers that was static in a corner it didn't handle that. In topaz chronos is useless but Apollo was very close to rife it handled the static graphic very well and only would mess up cars against poles or text on walls every so often. Neither is perfect but Apollo for me in this situation was better.

[jagabo]

I found rife worked really well for the cars moving against things like the wall or fence but if there was a graphic of the current position of drivers that was static in a corner it didn't handle that. In topaz chronos is useless but Apollo was very close to rife it handled the static graphic very well and only would mess up cars against poles or text on walls every so often. Neither is perfect but Apollo for me in this situation was better.

It would be a lot more work but you can overlay different parts of the different interpolations to get the best of each.

[SupraGSX]

It would be a lot more work but you can overlay different parts of the different interpolations to get the best of each.

I thought about doing that the box where driver position is almost always the same square. I could encode two versions one Apollo and one rife and mask the Apollo just over that square. Its an extra step but an option. It seems topaz releases a model then never updates any iterations which is the point of ai, train models to improve, it's frustrating but rife is getting regular updates hopefully rife v5 will release and just work. Overall I'm thrilled with how everything looks from a strict fixing interlace artifacts on progressive video approach it's fixed. I export the 852x480 video into topaz and use the beta nyx compression revert/denoise model to crop to 640x480 then upscale back to 1280x720 and it looks great nyx doesn't distort text or apply blurring/sharpening like the other models it just cleans.

[jagabo]

It may be possible to automatically overlay a different interpolation whenever the graphic appears. For example, if the background color is distinctive you could switch on that. Use RIFE() for the main video, Interframe() or FrameRateConverter() for the Graphic. Can you supply s sample with the graphic?

[SupraGSX]

Can you supply s sample with the graphic?

Sure, here you go and this is my current code I'm using so feel free to tweak if you can get a better result. The motion interpolation failing is noticeable by stepping through frames and observing the laps to go counter, it isn't rendering as desired. I'm using model 22 which is supposed to be the latest 4.6 version of RIFE.

LoadPlugin("C:\Users\Documents\AvisynthRepository\AVSPLUS373_x64\plugins\RIFE\RIFE.dll")
SetFilterMTMode("QTGMCp", 3)
FFmpegSource2("C:\Users\Videos\Static Graphic.mp4", atrack=1)
AssumeTFF()
DeBilinearResizeMT(852,480)
t = QTGMCp( Preset="Placebo", InputType=2, SourceMatch=3, Lossless=2, NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.15, Sigma=1.8, NoiseDeint="Generate", StabilizeNoise=true )
b = QTGMCp( Preset="Placebo", InputType=3, SourceMatch=3, Lossless=2, NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.15, Sigma=1.8, NoiseDeint="Generate", StabilizeNoise=true, PrevGlobals="Reuse" )
Repair( t, b, 1 )
BWDIF(field=0)
SantiagMod()
TurnRight().SantiagMod().TurnLeft()
aWarpSharp2()
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
Rife(gpu_thread=4, model=22, fps_num=60000, fps_den=1001, sc = true, sc_threshold = 1.0)
z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l")
Prefetch(2)

[jagabo]

Are you talking about the "BATTLE FOR 2ND" graphic or the "5 TO GO" graphic?

[SupraGSX]

5 to Go is where I see it the most.

[jagabo]

You can just preserve that area

cleartext = Crop(178,110,78,16).ConvertFPS(framerate*2) # preserve the N TO GO text

(double frame rate with RIFE here)

Overlay(last, cleartext, x=178, y=110)#, mask=fmask) # restore the N TO GO text

You still get some distortions in the area around the box but the text in the box remains readable. You may want to apply the sequence only when the "N TO GO" text appears. It may be hard to detect that automatically given how small the text is and how it's devoid of other details that you could use to identify it.

[SupraGSX]

It may be hard to detect that automatically given how small the text is and how it's devoid of other details that you could use to identify it.

What methods are available for automatic detection? I ask because it may not work here but I have several nascar videos that I want to give similar treatment too and I may be able to apply it there.

[jagabo]

What methods are available for automatic detection?

Unfortunately there's no simple "find this" function for this. You have to find something identifiable then figure out how to detect that. Maybe comparing edges, subtracting one image from another, looking for a certain color, etc.

[davexnet]

What methods are available for automatic detection?

Unfortunately there's no simple "find this" function for this. You have to find something identifiable then figure out how to detect that. Maybe comparing edges, subtracting one image from another, looking for a certain color, etc.

oops wrong thread ...

[Selur]

I threw: Resize(852x480)+Santiag+BasicVSR++(model=6) at it, seems a bit smooth,...

[SupraGSX]

I've never heard of basic vsr+ but I definitely need to check it out. Good results better than anything else I've tried on it including topaz. Hopefully videogigagan isn't all hype and videos like this might actually have a chance of being brought into this millennium.

[Charles-Roberts]

I threw: Resize(852x480)+Santiag+BasicVSR++(model=6) at it, seems a bit smooth,...

This is new to me, I googled VideoGigaGAN and saved web page.

What do I need to try this?
Got any info and links to install?

[Selur]

VideoGigaGAN <- never heard of it
I used Hybrid and it's torch-addon.

[Charles-Roberts]

Thanks @Selur

I have a few pages to read.
About 11 gigs of downloads!

Bit big ATM, but I will get back to this down the track.