Drop, comets...

[rgr]

Is there a filter that can remove such comets? They occur on the top field, relatively small on the bottom field. I tried the DePulse filter, but managed to achieve about a 50% reduction.

[jagabo]

One possible workaround is to use only the good field:

LWLibavVideoSource("out.m2ts")
QTGMC(FPSDivisor=2) # lossless and sourcematch might help here

You lose half the motion but you can try restoring it with:

LWLibavVideoSource("out.m2ts") 
QTGMC(FPSDivisor=2) # lossless and sourcematch might help here
InterFrame(GPU=false, cores=4)

and add depulse to remove the remaining light comets:

LWLibavVideoSource("out.m2ts") 
QTGMC(FPSDivisor=2) # lossless and sourcematch might help here
DePulse(whatever)
InterFrame(cores=4)

[rgr]

InterFrame:

Script error: There is no function named 'SVSuper'.
(C:IProgram Files (x86)/AviSynth+/plugins64+/lnterFrame2.avsi, line 152)
(CzlProgram Files (x86)/AviSynth+/plugins64+/lnterFrame2.avsi, line 58)


Anyway... I checked the whole video. It won't be easy though. Sometimes comets are in the top field and sometimes in the bottom field depending on the scene, so they are probably caused by the source camera.

[jagabo]

SVSuper comes with the SVPFlow package.

https://us.cnn.com/videos/business/2023/05/18/guy-fieri-celebrity-chef-food-politics-c...-cprog-vpx.cnn

There might be a way to automatically detect which field is bad and switch accordingly. If you can provide a clip that switches a few times I'll take a look.

[rgr]

SVSuper comes with the SVPFlow package.

https://us.cnn.com/videos/business/2023/05/18/guy-fieri-celebrity-chef-food-politics-c...-cprog-vpx.cnn

There might be a way to automatically detect which field is bad and switch accordingly. If you can provide a clip that switches a few times I'll take a look.

Attached. The only way I can see is to write down the frames in which the change occurs and switch fields on that basis.

[jagabo]

Here's my first attempt. It works most of the time but there are a few shots where it's not so good.

LWLibavVideoSource("out3.mpg") 

QTGMC(preset="fast", lossless=1)
even = SelectEven()
odd = SelectOdd()
ConditionalFilter(Subtract(even, odd), even, odd, "AverageLuma", "greaterthan", "126.0", show=false) # use show=true to see stats

# follow up with DePulse and motion interpolation
#StackHorizontal(even, odd, last) # enable this line if you want to see the two candidates and the chosen frame side by side (even, odd, chosen)

I'll see if I can come up with something better tomorrow.

[rgr]

Good idea. I've done some tweaking, but I still don't know how to make the ConditionalFilter pick a brighter frame.

#LWLibavVideoSource("out3.mpg") 
ffms2("out3.mpg")
QTGMC(preset="fast", lossless=1)
even = SelectEven()
odd = SelectOdd()

#my
even1 = crop(even,8,0,20,576)
odd1 = crop(odd,8,0,20,576)
even1 = coloryuv(even1,gamma_y=2000)
odd1 = coloryuv(odd1,gamma_y=2000)

ConditionalFilter(Subtract(even1, odd1), even, odd, "AverageLuma", "greaterthan", "126.0", show=true) # use show=true to see stats
#ConditionalFilter(Subtract(even, odd), even, odd, "AverageLuma", "greaterthan", "126.0", show=true) # use show=true to see stats

StackHorizontal(even1, odd1, last) # enable this line if you want to see the two candidates and the chosen frame side by side (even, odd, chosen)
#StackHorizontal(even, odd, last) # enable this line if you want to see the two candidates and the chosen frame side by side (even, odd, chosen)

[jagabo]

Here's something that works better:

LWLibavVideoSource("out3.mpg") 

t_even = SeparateFields().SelectEven().mt_edge(mode="0 -1 0   0 2 0   0 -1 0", thy1=25, thy2=25, chroma="-128")
t_odd = SeparateFields().SelectOdd().mt_edge(mode="0 -1 0   0 2 0   0 -1 0", thy1=25, thy2=25, chroma="-128")

#Return(StackHorizontal(t_even.ScriptClip("Subtitle(String(AverageLuma))"), t_odd.ScriptClip("Subtitle(String(AverageLuma))"))) # for analysis

QTGMC(preset="fast", lossless=1)
even = SelectEven()
odd = SelectOdd()
ConditionalFilter(Subtract(t_even, t_odd), even, odd, "AverageLuma", "lessthan", "126.0", show=false)

#StackHorizontal(even, odd, last) # for analysis

t_even and t_odd are maps of horizontal edges found within the fields. The comets show up as edges. I only saw a few sporadic bad frames starting at frame 2551. The "bad" frames there are overall sharper -- so they generate more edges.

[rgr]

SVSuper comes with the SVPFlow package.

I started testing, added SVPFlow and:

Script error: SVSmootts does not have a named argument "url"
(C:lProgram Files (x86)lAviSynth+/pluginsG4+I|nterFrame2.avsi, line 156)
(CzlProgram Files (x86)lAviSynth+/plugin564+1|nterFram§2.avsi, line 58)
(bad OCR)

[jagabo]

SVSmoothFps no longer requires the url string. In Interframe2.avsi (version 2.8.2) on line 156 remove the url argument from the call to SVSmoothFps. Ie change:

	smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, url="www.svp-team.com", mt=Cores)

to

	smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, mt=Cores)

[rgr]

Thanks.

I tested it, but the result is quite poor compared to VEAI.
https://imgsli.com/MTgxMjc4

[jagabo]

Interframe defaults to blending when it doesn't find a lot of motion vectors. I would have expected better from a simple panning shot like that. From what I've seen (I've never used it, only seen samples) VEAI is generally better.

[poisondeathray]

RIFE is probably the best overall for interpolation. There is an avisynth version that runs on NCNN/Vulkan .
http://avisynth.nl/index.php/RIFE
https://github.com/Asd-g/AviSynthPlus-RIFE/releases/

If you have Nvidia card, vapoursynth tensor or cuda version recommended, it can be 1-4x faster under Tensor

SVP uses RIFE too now. AVS version of SVP is using older non RIFE version
https://www.svp-team.com/wiki/RIFE_AI_interpolation

[rgr]

RIFE is probably the best overall for interpolation. There is an avisynth version that runs on NCNN/Vulkan .
http://avisynth.nl/index.php/RIFE
https://github.com/Asd-g/AviSynthPlus-RIFE/releases/

Looks like too complicated tool.

"RIFE: failed to load model"

[poisondeathray]

RIFE is probably the best overall for interpolation. There is an avisynth version that runs on NCNN/Vulkan .
http://avisynth.nl/index.php/RIFE
https://github.com/Asd-g/AviSynthPlus-RIFE/releases/

Looks like too complicated a tool.

"RIFE: failed to load model"

Where did you put the models ?

`models` must be located in the same folder as RIFE.dll.

If you autoload RIFE.dll in the plugins folder, then there must be a "models" folder in the same plugins folder. In the models folder will be the folders of each model

[jagabo]

Put the models folder inside the plugins folder where you have rife.dll:

[Attachment 71235 - Click to enlarge]

Inside that folder there should be many other folders that contain the model data files.

After setting up the models folder... RIFE in AviSynth requires planar RGB32. So call it with:

ConvertToPlanarRGB().ConvertBits(32).Rife(gpu_thread=1).ConvertBits(8).ConvertToYV12()

[rgr]

Which one should I download?

https://github.com/Asd-g/AviSynthPlus-RIFE/tree/main/models

Is it the same as Flowframes?

[jagabo]

I have all of them in my models folder. The default is rife_v2.3. By the way, this is the first time I've used it.

[Selur]

https://github.com/Asd-g/AviSynthPlus-RIFE/releases/tag/1.0.0 comes with all the models, so downloading v1.0.0 and taking the models from there might be more convenient than downloading them one by one.
-> okay forget that, Asd-g seems to want users to download them from the folder,..

[rgr]

Any trick to download them all?

[Selur]

Download the whole project as a zip. (https://github.com/Asd-g/AviSynthPlus-RIFE/tree/main -> Code -> Download as zip)

[poisondeathray]

When converting to RGB float, avsresize (zimg library) is about 2x faster in benchmarks if you have AVX2 . Internal avisynth ConvertToXX /Convertbits functions are not as optimized. (Most vpy ported machine learning projects to avisynth need RGB float)

blankclip(10000,1920,1080, "YV12")
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f") #fastest 
#ConvertToPlanarRGB(matrix="rec709").ConvertBits(32) #slower
#ConvertBits(32).ConvertToPlanarRGB(matrix="rec709") #slowest

[jagabo]

When converting to RGB float, avsresize (zimg library) is about 2x faster in benchmarks if you have AVX2 . Internal avisynth ConvertToXX /Convertbits functions are not as optimized. (Most vpy ported machine learning projects to avisynth need RGB float)

blankclip(10000,1920,1080, "YV12")
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f") #fastest 
#ConvertToPlanarRGB(matrix="rec709").ConvertBits(32) #slower
#ConvertBits(32).ConvertToPlanarRGB(matrix="rec709") #slowest

Given how slow RIFE is (at least on my computer) it won't make any difference.

[rgr]

I put the rife-v2.3 folder in the models folder (in plugin64+). No longer 'no models error' but: 'System Exception -- Access Violation'

ffms2("10000000_924752528810545_132584178015633738 9_n.mp4")
convertbits(bits=32)
ConvertToPlanarRGBA(matrix="rec709")
RIFE()

[poisondeathray]

I put the rife-v2.3 folder in the models folder (in plugin64+). No longer 'no models error' but: 'System Exception -- Access Violation'

ffms2("10000000_924752528810545_132584178015633738 9_n.mp4")
convertbits(bits=32)
ConvertToPlanarRGBA(matrix="rec709")
RIFE()

Did you fulfill the requirements list ? Especially the Microsoft VisualC++ Redistributable Package 2022 ?
http://avisynth.nl/index.php/RIFE#Requirements

[rgr]

Did you fulfill the requirements list ? Especially the Microsoft VisualC++ Redistributable Package 2022 ?
http://avisynth.nl/index.php/RIFE#Requirements

Yes.
"Microsoft Visual C++ 2015-2022 Redistributable (x64) - 14.31.31103"
(+x86)

[rgr]

I put the rife-v2.3 folder in the models folder (in plugin64+). No longer 'no models error' but: 'System Exception -- Access Violation'

ffms2("10000000_924752528810545_132584178015633738 9_n.mp4")
convertbits(bits=32)
ConvertToPlanarRGBA(matrix="rec709")
RIFE()

Did you fulfill the requirements list ? Especially the Microsoft VisualC++ Redistributable Package 2022 ?
http://avisynth.nl/index.php/RIFE#Requirements

Upgrading to the latest test version of Avisynth helped.