Bad toon tape here, chroma is destroyed.
How much can be fixed?
What Avisynth filter can do an aggressive temporal NR that only attacks the chroma?
Don't worry about audio, tearing -- just the color issue.
TL;DR update: Final repair attached here:
+ Reply to Thread
Results 1 to 30 of 34
As you doubtless know, the chroma problems are quite different in each field. I was able to get some improvement by separating (via lossless bob), merging the chroma of one field with the other, applying CNR2 and then recombining. However, no miracles. I would say the result was about half as bad. I was going to post the result but had to quit to finish some work.
If the bogus colors are not used elsewhere in the frame, you might be able to use RT_Stats to detect the horizontal chroma stripes and then replace the chroma with the pixels immediately above and below, or pixels from the other field.
I was able to detect horizontal stripes using the code discussed in this thread:
Bad 1950s Kinescope
This shows how I was able to remove a moving horizontal stripe:
So, the idea would be to
1. do stripe detection on each separated field:
2. compare stripes of one field with another
3. when stripes don't match, merge the chroma from the "good" field to fill in the stripe in the bad field
The tough part will be determining which field has the good color, and also what to do when the stripes in both fields BOTH need to be removed.
You may also be able to use the fact that animation sometimes has several frames which repeat. You can do a duplicate detection, using the B&W version of the frames, perhaps with some blur applied, and then do averaging across those duplicate frames.
As you can tell, I don't quite have a good solution to this.
You can force any filter to affect chroma only - by applying to U and V directly (and on separated even/odd grouped fields if you want to preserve interlace) .
Or, another way is to filter using whatever, then merge back the original unfiltered Y with mergeluma()
I wouldn't have high expectations given the condition of the source. So I don't think it matters much what you use - just use something heavy like neat video, mctd, ttempsmooth , ccd , with strong settings , etc...
In this example, I used some vertical and general blurring to start breaking up the horizontal bars and ttempsmooth for temporal smoothing , some heavy knlmeanscl for general denoising on the separate u,v planes of each separate odd/even field. To smooth things over more, I ran it through neat video, then merged back the luma .
The color bars are pretty much gone, but there is still some tiny color flicker, so I probably wasn't aggressive enough with the settings. You're going to get chroma ghosting with strong ttempsmooth settings, especially on shots where there is more motion. This sample wasn't a good one to demonstrate that problem but you're going to have to make trade offs
[Attachment 43371 - Click to enlarge]
SetFilterMTMode("DEFAULT_MT_MODE", 2) AVISource("BadToonSample.avi") ConvertToYV16(interlaced=true) orig=last ev=orig.assumetff().separatefields().selecteven() od=orig.assumetff().separatefields().selectodd() ev ue_chroma = UToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=6,lthresh=150, strength=6).KNLMeansCL(d=3, a=8, h=6, device_type = "GPU", device_id = 1, channels="Y") ve_chroma = VToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=6,lthresh=150, strength=6).KNLMeansCL(d=3, a=8, h=6, device_type = "GPU", device_id = 1, channels="Y") YToUV(ue_chroma, ve_chroma) MergeLuma(ev) ev_filtered=last od uo_chroma = UToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=6,lthresh=150, strength=6).KNLMeansCL(d=3, a=8, h=6, device_type = "GPU", device_id = 1, channels="Y") vo_chroma = VToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=6,lthresh=150, strength=6).KNLMeansCL(d=3, a=8, h=6, device_type = "GPU", device_id = 1, channels="Y") YToUV(uo_chroma, vo_chroma) MergeLuma(od) od_filtered=last interleave(ev_filtered,od_filtered) assumefieldbased().assumetff().weave() Prefetch(6)
Input to neat video , then Merge back original luma
orig=AVISource("BadToonSample.avi") AVISource("NV_output.avi") #output from NV will be RGB ConvertToYUY2(interlaced=true) MergeLuma(orig)
Attached is my attempt. I was just seeing if anybody had anything better.
Mine is a bit aggressive. If you go frame-by-frame, chroma blending is obvious. But in full motion, when watching it (the intention after all, right?), you rarely see any major distractions. The timebase/timing on this tape was blah, but not horrid. The worst could be cropped/masked, and was intermittent full-frame. The tracking was terrible, but my stacked stabmod() got most of it, with occasional false fixes.
poison was typing at the same time I was! Hmm... I may like your method better than my own. I need to test it on the full clip.
You can improve chroma filtering by stabilizing first - it will reduce the temporal smoothing errors because x,y translations/ rotation fluctuations will be less severe .
That example demonstrated pure chroma filtering only, but you wouldn't do that in real life, because some operations like stabilization and at least some Y filtering will be in common, so you'd probably combine at least some things to make it faster
Maybe less so on "toons", but another way to get back details from overfiltering is to use a weighted merge(a,b, weight=x) to bring back some texture. So if your heavy duty filter chain affets Y,U,V and kills the Y too much you can bring back some of the original signal
I'm getting an error on that script.
KNLMeansCL: no compatible opencl platforms available! line 10
I'm also curious what your NeatVideo settings were.
And I can provide another clip, more motion.
use device_id=0 for iCPU , or just omit the device_id parameter . Typically "0" is iCPU, "1" is 1st discrete GPU, "2" is 2nd discrete GPU etc.... that filter can't use multiple GPU's (yet)
Neat video was just strong temporal , strong everything especially cb, cr . Detail preservation / blurring didn't really matter for this example because the luma was being discarded anyway
I set device_id=0m, and it at least loads.
SetFilterMTMode("DEFAULT_MT_MODE", 2) Prefetch(6)
Do you have a suggestion for regular Avisynth?
SetMemoryMax(2048) SetMTMode(5,8) AviSource SetMTMode(4,8)
Honestly, I don't remember why I have those values, or what it means anymore. The wiki reads like "blah blah blah".
Right now, without MT, I'm getting 0.5 frames per second -- on 16gb RAM i7-6700K!!! At that speed, it's a proof-of-concept filter only, not anything real-world usable.
What speed are you getting from this?
this Toon video also need to have the chroma upped/moved to the right i believe:
Yes avisynth+ x64
There is no question will get faster results with x64 or avisynth+ or MT, but the large bottleneck is the GPU filter, so you can replace that with something else faster, or just increase the NV settings a bit more.
There is nothing "special" about this - you can replace any of the filters with a similar filter. For example ttempsmooth could be replaced by temporal smoother in vdub. You could use any variation of preblurring (but you probably want vertical blurring more than horizontal blurring) . You're just denoising chroma selectively as per original topic, and given that it's a cartoon, and the poor condition of source - it's not as if you're trying to preserve massive amounts of chroma detail so just use some strong filters
Damn PoisonDeathRay, you sure have a lot of experience with crazy unusual fix ups. I wouldn't have even began to know where to start to fix that and I been working with animation for years. I didn't think it was going to be possible to clean it up as well as your example looks. I expected at least some of those color artifacts to be impossible to remove, but you sure impressed me. I really need to get better at chroma and luma stuff.
Can Avisynth+ and standard 2.6 be installed concurrently?
Is it as simple as letting both install in their native folders?
I don't have time to mess with a borked system, if there will be known conflicts.
Other people have completely moved over to avisynth+ , 32bit/64bit / MT without any regrets - I would say that's the way the majority of people migrating set it up. You can still load original 2.6 plugins in 32bit avisynth+, so in theory you shouldn't need to set it up the way I have
Is it as simple as letting both install in their native folders?
I don't have time to mess with a borked system, if there will be known conflicts.
In the end it's worth it. More stable, no memory issues, significantly faster (even in single threaded vs. single threaded) . But it takes some fiddling around to get working, along with x64 plugins , updated version etc...It's like "dll hell" all over, or learning to walk again.
It's been a while since I last installed Avisynth, so I'd forgotten that no program is actually in the program folder, just the plugins.
I'm actually using 2.6 MT (had forgotten that fact), and simply renamed the default avisynth.dll to something else, and replaced with the 2.6MT. I downloaded the binary zip from the + site, and again renamed the old 2.6MT DLL, and copied the + DLL 32-bit to the syswow64 folder. But it's not working. Getting script errors in AvsPmod ("no function named setfiltermtmode").
Saved the existing plugins folder (rar backup), overwrote plugins for +
Where'd I go wrong?
Guessing devil.dll in syswow64 also needs replacing.
I see a quote by Groucho:
Don't use the system32/syswow64 directories for the avisynth/devil DLLs, use a directory to which your PATH environment variable points.
1) You need the full original avisynth+ install first, before you overwrite with the newer MT updates . The updates don't include everything. So it's a very old version you have to install first
I just used the Groucho2004 batch file to set the directory path(s). Open it up in notepad and you can edit the paths. It has simple explanations of what is going on. You can set custom paths if you want things placed differently
So my old 32bit vanilla install is untouched. I just installed avisynth+ x64 on top of it, with it's own separate directory and plugin paths. For example mine was on a different drive
I installed the new Avisynth+, and now Avisynth just hangs as soon as videos preview in either AvsPmod or VirtualDub. Sigh. Now what?
I did find one older script that still loaded a Huffyuv file, but it's skewed, like a codec error. Outside Avisynth, Huffyuv is fine. It hangs after a short time.
there is no function named SetFilterMTMode
Now I can't use anything.
Is it as simple as the avisynth.dll in the syswow64 folder? Where do I get that file?
Edit: Nope. Found it in binary, did nothing.
Removed it, put back 2.6 MT DLL, now loading scripts fine (at least those tested so far) using Avisynth+. Not what I want, but at least working now.
How can I get your SetFilterMTMode script to work?
or just increase the NV settings a bit more.
Edit: The plot thickens.
64-bit VirtualDub Filtermod with SetFilterMTMode script works, but now filters fail. I founud the 64-bit KNLMeansCL, but not binomialblur. Are you using 64-bit?
32-bit is still not loading when SetFilterMTMode present.
Huffyuv requres 64bit version install if you use it in 64bit script
64bit avisynth.dll would reside in System32 folder . 32bit avisynth.dll would reside in SysWow64 folder . It might be a good idea to use the utility by Groucho2004 mentioned earlier. You can edit the ini for custom options
It sounds like you're mixing up arguments and versions . SetFilterMTMode is for avisynth+ MT . SetMTMode is for avisynth(-) MT . You cannot have both avisynth+ 32bit and avisynth- 32bit installed concurrently and both active. But you can swap them with groucho2004's utility
binomialblur is part of VariableBlur.dll . If you are using 64bit, you need the 64bit version . A handy list of avisynth+ 64bit filters
I use 64bit for avisynth+MT , 32bit for vanilla /MT for avisynth- , but you can set it up however you like . That script should work in either 32bit or 64bit. But 32bit will probably have memory issues and will definitely be slower . It should work for "old" MT as well, just change the SetFilterMTMode to old SetMTMode arguments with SetMtMode(2) for other filters and erase the prefetch line
I already know most of that, and am not mixing anything up. C'mon, you know I no rank newbie, just new to x64 Avisynth.
I have both Huffyuv 32 and 64 installed. One is via ffdshow, though I forget which is which. MainConcept TotalCode is 64, doesn't see Huffy32 -- and Vdub32 doesn't see Huffy64. Both cannot install at same time, but ffdshow was a cheat! Anyway, no real issue with that one now.
Win7x64 here, so correct, all 64 in system32 and 32 in syswow64 (a horrible naming convention by MS!)
Avisynth dll in sys32 is 64, dll in syswow is 32.
I'm guessing Vdub32 works for Avisynth32, Vdub64 for 64. Sound correct?
SetFilterMTMode did not work in 32+ with supposed dll. As I understand, + is based off MT, as the MT dll still worked. No SetFilterMTMode, but (unstable!) SetMTMode did. Even when 32, it does help. If + is better, should help more.
The old Avisynth folders are now gone, replaced by +, and +64 works, so 2.5 can't still be installed. It has to be dll issue.
So I manually reinstalled all the dlls, and now it seems to be working!
On 32-bit MT dll, not even +32, getting 0.5fps in Vdub
With 64-bit in Vdub64, with the + SetFilterMTMode(2), now getting 0.45 -- what!
(1) is getting 0.55
I have i7 with 16gb RAM, and 4 of 8 threads have minimal use in task manager monitors. I want to make that kittie purr. What can I do to feed the script more power?
Getting random fatal error warnings from KNLmeansCL in MT dll and 32+. That's nasty filter. Powerful but dirty.
Found the AvsPmodx64 build from March 2017 on Doom9, put it in Program Files, left 32 in PF(x86). Both seem to work just fine. So at least there's something. I read many others have issues there.
AVSmeter would also barf. I know, WTF right?
Yeah, GPU. I'm looking to borrow one later this week from a gamer, his older card.
I read where it could make encodes 80x faster.
Is there no way to tap into the GPU of the Intel HD 550 (i7-6700k onboard graphics GPU on same chip).
Any ideas on a cheap good card to buy for this? I'd only plug it in when needed for encoding. I've never gotten into GPU encoding, do to lousy support.
I just switched to AVISynth+ and so have gone through all the teething pains. The SetFilterMTMode is quite different from the previous MT mode. You need to explicitly call it for each filter you are using. Also, as I found out, you cannot have a "return" call be the last call in your script before the Prefetch call, or MT won't work.
Decode, filter, encode. I get that. But usage would show in Taskmgr. RAM/SSD isn't bottleneck, so it must be CPU? What else is there? I altered the script you posted to not use GPU as you mention.
I may stick to using the MT dll, even if + is installed, as it seems to be working as usual. I'm not liking it.
The x64 is the main draw, though limited.
My mood has been getting more sour by the hour, so probably time I leave the system for the day. At the current rate, that encode will take 125 hours, and BS on that. I'll have to just set it aside until some future tech solves it. That's what I did in the 2000s on some projects, and they're now doable. Same for stuff from 90s saved to 2000s. I'l try GPU, but doubting it'd help. Nice proof of concept, but nothing else for now.
Poison, what was your fps on that? You never told me that. Great script, amazing, but it's just too danged slow right now.
You said " 4 of 8 threads have minimal use in task manager monitors" . This implies low CPU usage and idle resources . This implies bottleneck somewhere.
Do it systematically. Erase KNLMeansCL completely (don't replace it for now), run avsmeter, compare. Replace it with another filter, run avsmeter, compare. If that's not the bottleneck, do the same for other filters etc... Adjust prefetch value, rinse, repeat, . When testing systematically with avsmeter, cpu-z, gpu-z, that will tell you a lost of info about script speed optimization . Yes you have to do multiple test, there is no way around it
Poison, what was your fps on that? You never told me that. Great script, amazing, but it's just too danged slow right now
As one example, I don't know of any program in the world that would have let me do this:
I had to look at the pixels in each frame and, using various comparisons, identify the location of the moving white noise bars. I then had to apply an exposure mask to those bars and decrease the gamma so that the overall exposure matched the surrounding video.
I am not aware that anything else has ever done this particular restoration, even though the problem is quite common.
I do understand the frustration in dealing with the "open source" nature of the AVISynth project, where each plugin is of different quality and documentation, if available at all, is not always helpful.
I also know that you are not a fan of doom9.org, the main support forum for AVISynth, but I actually have found that forum to have become much less obnoxious over the past three years, and I generally get really good help there now.
Other examples of things you can't do without AVISynth:
Last edited by johnmeyer; 6th Dec 2017 at 18:51. Reason: added two more links; later, typos
I changed the KNL from "GPU" to "CPU" and saw CPU peg at 100% -- and with 0.19fps encoding. Yikes! I think it actually was using the GPU embedded on the i7-6700 chip after all.
Such a shame. You wrote such a nice script.
Wait another decade?
My own script is not as good as yours, but still giving decent results -- and at about 10fps or better. So I'll just stick with it for now.
Appreciate your input on this thread.
Yeah, great "program" (pseudo scripting language, sort of), no disagreements here -- just awful documentation.
I have nothing against Doom9, not sure where you got that. I use it from time to time. Some of the members were horses butts about a decade ago, but those folks are long gone. Same for some bad cdfreaks/myce users.