Avisynth herringbone removal?

[lordsmurf]

Several filters appears to be outdated, rely on Avisynth 2.5 and old versions of plugins. For example, FanFilter doesn't work anymore (not that I'm sure it ever did).

What can be done now?

Sample attached.

[jagabo]

DeFreq() can be good at removing noise like that. I'm not very good with it but here's an example:

LWLibavVideoSource("herringbone-sample.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)
SeparateFields()
#Defreq(info=true) # enable this to get numbers for the next line
Defreq(fx=9.7,fy=-10.0, sharp=71.9)
Weave()
QTGMC()

Original on the left, defreq on the right (QTGMC for both):

[Attachment 54815 - Click to enlarge]

Unfortunately it didn't work as well on other frames. I don't know how to fix that.

[johnmeyer]

I suggested Defreq earlier today, but my post has gone AWOL. I've used it before, and it is far from perfect, and you do have to do a lot of tuning to get the angle and space between the interference lines tuned just right.

[edit]I found it in another thread (oops):

https://forum.videohelp.com/threads/398729-Video-editing-program-with-multi-audio-tracks#post2594524

I guess I shouldn't post early in the morning.

[Sharc]

Several filters appears to be outdated, rely on Avisynth 2.5 and old versions of plugins. For example, FanFilter doesn't work anymore (not that I'm sure it ever did).

What can be done now?

Sample attached.

FanFilter works here under avisynth+ 3.6.1 32 bit without problems.

[lordsmurf]

Several filters appears to be outdated, rely on Avisynth 2.5 and old versions of plugins. For example, FanFilter doesn't work anymore (not that I'm sure it ever did).

What can be done now?

Sample attached.

FanFilter works here under avisynth+ 3.6.1 32 bit without problems.

I guess that's something.
But are you sure it even works well with this clip?
Does the filter work well, period? I've never used it, didn't find much about it, rarely saw where anybody was using it.

BTW:

I have Avisynth 2.6 MT, and then Avisynth+ x64
My experience and observation has always been that the +x86 had issues compared to official, so it's not installed. Furthermore, anything that +x86 did better than 2.6 was done even better in +x64. I could never get 2.6/MT to play nice with +x86, and +x86 made stuff not work. At one point, for a few weeks, I had no 32-bit Avisynth, because the + installer fubar'd everything. Admittedly, it's been a while since I last tried, but I don't want to go through that mess again.

In hindsight, I think that the forking of Avisynth+ was bad form, and should have been a side-by-side installation, not a replacement attempt. Because all that's done is make complications and messes.

I really wish there was a way to easily "portable-ize" Avisynth+, but unsure of how. It seems selur is doing with with Hybrid?

I don't know enough about Avisynth DLL writing to know why filters like FanFilter can't simply be updated to 2.6 standards. Is a full rewrite required? Or is it a few tweaks of commands and syntax, and done?

EDIT:

Wow... I wrote about FanFilter 15 years ago.
It never worked correctly.
https://forum.videohelp.com/threads/212496-cannot-understand-avisynth-filter-help%21

I think FanFilter is a non-starter.

[Sharc]

I guess that's something.
But are you sure it even works well with this clip?
Does the filter work well, period? I've never used it, didn't find much about it, rarely saw where anybody was using it.

I just downloaded it out of curiosity and found that it "works". Didn't really try to fix your clip, I just noticed that playing with the params had some effect.

BTW:

I have Avisynth 2.6 MT, and then Avisynth+ x64
My experience and observation has always been that the +x86 had issues compared to official, so it's not installed. Furthermore, anything that +x86 did better than 2.6 was done even better in +x64. I could never get 2.6/MT to play nice with +x86, and +x86 made stuff not work. At one point, for a few weeks, I had no 32-bit Avisynth, because the + installer fubar'd everything. Admittedly, it's been a while since I last tried, but I don't want to go through that mess again.

In hindsight, I think that the forking of Avisynth+ was bad form, and should have been a side-by-side installation, not a replacement attempt. Because all that's done is make complications and messes.

You may find Groucho's Universal Avisynth Installer useful for easy switching between avisynth versions.

I really wish there was a way to easily "portable-ize" Avisynth+, but unsure of how. It seems selur is doing with with Hybrid?

Yes, Selur has recently put a lot of efforts into providing a reliable Avisynth xyz localization in Hybrid. You may want to PM him. I don't have the expertise.

[jagabo]

You can use the 32 bit Defreq (I haven't found a 64 bit version) in 64 bit AviSynth+ using the MP_Pipeline() filter:

MP_Pipeline("""

### platform: win64
LWLibavVideoSource("C:\Users\John\Downloads\herringbone-sample.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)
SeparateFields()
### ###

### platform: win32
Defreq(fx=9.7, fy=-10.0, sharp=71.9)
### ###

### platform: win64
Weave()
QTGMC()
### ###

""")

[themaster1]

F1quiver is good at removing hbones:

F1Quiver(last,"custom", 25, 100, 50, 35, 70,0, 80,90, 120,30,test =true,row = 155, erow=300,degree = 8,gamma =0.2 )

[lordsmurf]

F1quiver is good at removing hbones:

F1Quiver(last,"custom", 25, 100, 50, 35, 70,0, 80,90, 120,30,test =true,row = 155, erow=300,degree = 8,gamma =0.2 )

I don't know what's wrong.

You may find Groucho's Universal Avisynth Installer useful for easy switching between avisynth versions.

I know he puts a lot of work into it, but still read about conflicts and problems. It's not something I can look into right now, can't risk my currently in-regular-use setup.

I may try Avisynth+ on an Ubuntu system, sometime, as I do have one of those free. Not powerful, just an i3, but still far beyond what we were all using 15 years ago (and Avisynth worked then).

Yes, Selur has recently put a lot of efforts into providing a reliable Avisynth xyz localization in Hybrid.

Yes, he really has.

You can use the 32 bit Defreq (I haven't found a 64 bit version) in 64 bit AviSynth+ using the MP_Pipeline() filter:

"There is no function named 'MP_Pipeline'"

I ran the script against the full video, in AvsPmod, and it overall had little effect. I tried changing some of the settings, but that didn't have any effect either. I vaguely remember having this same issue some years ago, Defraq() did nothing.

[jagabo]

"There is no function named 'MP_Pipeline'"

http://avisynth.nl/index.php/MP_Pipeline

[lordsmurf]

"There is no function named 'MP_Pipeline'"

http://avisynth.nl/index.php/MP_Pipeline

I'm tired.
For some reason, I thought that was a base/included filter of Avisynth+
Doh!
I'll install that tomorrow, try again,

[johnmeyer]

I finally had a moment to try Defreq. Here is my initial (five minute) try:

loadPlugin("C:\Program Files\AviSynth 2.5\plugins\defreq.dll")
source = AVISource("E:\fs.avi").KillAudio().converttoYV12(interlaced=true).AssumeTFF()
#DeFreq(show=2,info=true,fx=10,fy=-4,dx=1.5,dy=2.0)

output = DeFreq(source,fx=10,fy=-4,dx=1.5,dy=2.0)
#output = DeFreq(show=2,info=true)

return output

#interleave (source,output)

I've attached the video that results from this script. I'd say that it gets about 80% of the herringbone, and in the center, close to 95%.

The interleave statement lets you step through your clip and see the original frame, and then when you go to the next frame, it shows that same frame, but with Defreq applied. To actually use that, remove the comment from the "interleave" line and comment out the return statement.

I'm going to see if I can understand the secondary parameters and improve this quick and dirty result. Right now, while I understand FFT, and have seen similar parameters in the FFT3D noise filter, I'm not quite sure I understand how these are to be used or tuned. I assume they adjust how rapidly the filter diminishes from the center of the noise bar.

[johnmeyer]

Minor tweak.

I realized after I posted my code that the video is interlaced so theoretically the Defreq operation should be performed on each field separately, and then recombined. Here's the code which does that:

loadPlugin("C:\Program Files\AviSynth 2.5\plugins\defreq.dll")
source = AVISource("E:\fs.avi").KillAudio().converttoYV12(interlaced=true).AssumeTFF()
even=separatefields(source).selecteven()
odd=separatefields(source).selectodd()
outputeven = DeFreq(even,fx=10,fy=-8,dx=1.5,dy=2.0)
outputodd = DeFreq(odd,fx=10,fy=-8,dx=1.5,dy=2.0)
interleave(outputeven,outputodd)
output = weave()
return output
#interleave (source,output)

Since the video is half height after the separatefields() operation, the fy parameter has to be changed.

The difference is undetectable, but not doing it properly might introduce problems, so it is probably worth the extra seven lines of code to do it right.

[lordsmurf]

Thanks for the attempts, but none of these are really doing anything meaningful. On the full video, all you really get is momentary blips of half-clarity, and that's it. This is just an ugly error, and there's no way to remove it easily.

We just need to beat it to death with NR.

Most of the noise actually disappears with a strong TTempSmooth.

TTempSmooth(maxr=6, strength=4, lthresh=20, cthresh=25, lmdiff=10, cmdiff=15)

What NR can be used for the motion parts of the image? I'm just not finding anything, with any settings, that will zap the herringbone when objects and people move.

"There is no function named 'MP_Pipeline'"

http://avisynth.nl/index.php/MP_Pipeline

This still doesn't work.
I did this: "Copy files from x64 (or x64_xp) folder to the plugins directory for 64 bits plugins"

The install/troubleshooting/general documentation is terrible -- or rather, it doesn't exist. That 7zip file contains all kinds of stuff. What is all of that? x64 vs. x64_xp? Windows XP? I see there's a few brief script usage notes, but I can even get that far.

I get frustrated by this sort of stuff, all because some coder can't write a damned document explaining WTF his tool entails, does, how to install, etc.

I was actually hoping this would allow FanFilter to run, since that plugin can supposedly work in +x86.

[poisondeathray]

"There is no function named 'MP_Pipeline'"

http://avisynth.nl/index.php/MP_Pipeline

This still doesn't work.
I did this: "Copy files from x64 (or x64_xp) folder to the plugins directory for 64 bits plugins"

The documentation is terrible -- or rather, it doesn't exist.
That 7zip file contains all kinds of stuff. What is it?
x64 vs. x64_xp? Windows XP?

Yes, XP. Believe it or not, there are significant numbers of XP users that want (demand) support. Ignore that folder

Copy the contents of the x64 folder into the x64 plugins directory

Almost all x86 filters can be run inside a x64 avs+ host script with mp_pipeline

[jagabo]

Note the MP_Pipeline files need to be in AviSynth+'s Plugins64 folder, not the Plugins64+ folder.

[oln]

F1quiver is good at removing hbones:

F1Quiver(last,"custom", 25, 100, 50, 35, 70,0, 80,90, 120,30,test =true,row = 155, erow=300,degree = 8,gamma =0.2 )

For anyone trying this, this line is for the older version (listed here as 2.5.8 avisynth version) of f1quiver, the newer(current) one has different parameters.

[lordsmurf]

F1quiver is good at removing hbones:

F1Quiver(last,"custom", 25, 100, 50, 35, 70,0, 80,90, 120,30,test =true,row = 155, erow=300,degree = 8,gamma =0.2 )

For anyone trying this, this line is for the older version (listed here as 2.5.8 avisynth version) of f1quiver, the newer(current) one has different parameters.

Thanks for that.
All this script does is add heavy ghosting, and some blurring, to cover up the herringbone -- not remove it.
I think this a dead end.

Again, I'll just hit it over the head it with strong NR, and call it a day. Still not great, but at least maybe better.

Note the MP_Pipeline files need to be in AviSynth+'s Plugins64 folder, not the Plugins64+ folder.

Did that.

Yes, XP. Believe it or not, there are significant numbers of XP users that want (demand) support.

They're nuts. XP is a capture OS, maybe DVD authoring, but that's it.

[johnmeyer]

Thanks for the attempts, but none of these are really doing anything meaningful. .

So you are saying that the improvement between these two images isn't worth doing??





I wish all restoration involved getting absolutely perfect results, but gosh, this is a LOT of reduction, and it is done with no bad side effects, unlike many types of noise reduction which often make the video look weird.

What's more, with a little tuning, it could be made far better. Finally, you can get even better results if you take a second pass. I'm not sure where you're coming from on this. I spent a lot of time getting Defreq to work on this, and if it were my project, I'd be eager to use it.

If other parts of the clip aren't being improved as much, you may just have to use slightly different settings for those sections. This is because the interference may change width and/or angle and therefore the script will need to be tuned differently for those sections. This is often the case with other noise filters, and is why Neat Video, as one example, requires a noise pattern so that it can understand what the artifact looks like.

[lordsmurf]

Thanks for the attempts, but none of these are really doing anything meaningful. .

So you are saying that the improvement between these two images isn't worth doing??

That sample looks good, but most of the rest of video does not.

you can get even better results if you take a second pass.

Interesting.

and if it were my project, I'd be eager to use it.

I was eager ... until I saw it just was not working.

the script will need to be tuned differently for those sections.

Nothing that I was doing was having any effect.

[johnmeyer]

Nothing that I was doing was having any effect.

Find a section of the video where the parameters in my script are failing. Add:

Show=1,Info=True

inside the Defreq call. Scrub through a dozen frames in the section where you are not getting good results and look at the suggested fx and fy values. Remove the show/info, plug in those fx and fy values and try the revised script on the problem section.

I obviously don't have the rest of the video, but with these beat patterns what happens is that the pattern often changes because the conditions which caused the interference were not constant over time. The angle, the size, and the width of the pattern therefore changes, and the script must be re-tuned for the different conditions.