Video filter for old animation movie

[askiplop]

No problem

What do you mean by:

Still thinking about the field change business.

[askiplop]

(Double post sorry lol)

[sanlyn]

Trying to make out that .txt file is a kind of CIA decoding exercise, but basically it means that the TFF video went BFF. That's a problem for the telecined frames when using Force Film. The DGIndex manual discusses this, though briefly. I see in Doom9 where one would elect to not fix the transition section. You can try going there and saying "no" insetad of "yes" when asked for a fix, and see what you get (likely a field order problem that's a headache to fix, but not impossible). Or...you could try playing the "bad" m2v file (or is it just a "bad" d2v instead of an m2v?). If it's m2v you can open directly in VirtualDub and see what's there.

[manono]

It worked on the short sample. I can understand where it might not work on the full movie. Your D2V is way too short and only has that 9% of the film. In that case I'd demux the MKV back to M2V and AC3 and use DGindex again. Either that or switch to FFVideoSource (FFMS2). Me, I avoid DirectShowSource like the plague. Maybe that works with no problems, I don't know as I don't use it.

[sanlyn]

If all that was output is a "bad.d2v", you can open that with MPEG2Source and read it that way.

[sanlyn]

Yes, manono, I read where DirectShowSource has problems. With me, it has either worked OK or not at all. Anyway, there's bound to be a way to retrieve all of the mkv intact.

[askiplop]

Humm. No, there's no m2v file, only the d2v.bad file.
Annnd nop, clicking on "No" do nothing... except bringing frustration :P

I tried something else though. I used MKVmerge to demux the video from the mkv, giving me a mpg file.
Then in DGIndex, I used this mpg file as the input, and it output a demuxed.m2v file, exactly the same size as the input, so I guess it worked?

The CIA called me actually, they were thinking using the file as a missile launch password, but I refused.

What is TFF and BFF? Looking on the internet for TFF and BFF: Too Freaking Funny and Best Friend Forever are not related to what we're talking about here...

I know DVDs are TFF. I dont know why I know that and if it will help me one day but hey! I know it.

[poisondeathray]

What is TFF and BFF? Looking on the internet for TFF and BFF: Too Freaking Funny and Best Friend Forever are not related to what we're talking about here...

field order
TFF = top field first
BFF = bottom field first

I know DVDs are TFF. I dont know why I know that and if it will help me one day but hey! I know it.

Not necessarily . DVD can be TFF or BFF

But HD content is 99.9% TFF

[manono]

Humm. No, there's no m2v file, only the d2v.bad file.

In that case I'd demux the MKV back to M2V and AC3 and use DGindex again.

But I guess you figured it out anyway.

[askiplop]

poisondeathray: Thanks for the info
manono: Yess it's working!! I've imported the movie in Vdub with your Avisynth script:

LoadPlugin("C:\Videos\AviSynth 2.5\plugins\DGDecode.dll")
MPEG2Source("C:\Videos\vid_onceupon\once_upon_a_fo rest.d2v")

But since sanlyn wrote an avisynth script just for me ( I'm lucky ), I'm adapting this one.

But I'm missing two required plugins: QTGMC-3.32 and daa3.

I''ve found QTGMC-3.32.avsi, can I jsut rename it QTGMC-3.32.avs?
If anyone can upload the daa3 plugin, it would be great, cause this one, I can't find it. (I'm not used to search for avisynth plugin, any good website?)

Thanks in advance!

[askiplop]

Find out that I mistaken for the QTGMC-3.32.avsi file, it's already set this way in the script. :S

[manono]

I''ve found QTGMC-3.32.avsi, can I jsut rename it QTGMC-3.32.avs?

Ordinarily you'd leave it as an AVSI which means it's supposed to autoload. Then you add a line in your script somewhere:

QTGMC()# maybe with presets or various parameters added

If the AVSI doesn't autoload (VDub complains), then import it into the script like so:

Import("C:\Path\To\QTGMC-3.32.avsi")#adjust for your name and path

[sanlyn]

QTGMC-3.32.avsi would be the autoload ver4sion, while "avs has to be imported manually as in the script shown earlier. It can be used either way, but I don't like dozens of "autoload" avsi's being scanned every time I run an avs script.

The script I posted can be run with any means of opening that video. However...every video has its problems. The same script could be overkill for some clips and underkill for others. Often, RGB corrections aren't needed either. Videos can vary from minute to minute. But you seem OK with some Avisynth and VirtualDub conventions, so it should be less of a hassle for you than for total noobs (like me, not too long ago). We're all still learning this stuff. I've learned most of this from more advanced members like manono.

Meanwhile, you're just getting this video started. Relaxing fun, huh?

Here's daa3.avs :

#Anti-aliasing with contra-sharpening by Didée, via Doom9
# Uses MaskTools v2.0 or later, nnedi3, RemoveGrain, and Repair.
# For interlaced video, use before deinterlacinfg or before IVTC.
FUNCTION daa3(clip c) {
nn    = c.nnedi3(field=-2)
dbl       = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD     = mt_makediff(c,dbl,U=3,V=3)
shrpD     = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
DD     = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3) }

[askiplop]

LOL yep! I'm so close of getting this video opened in VDub! (Yes, only that!)

Thanks for the daa3 script.

Now when I open the script # 1 (See previous page of this threat), I get an error:

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
LoadPlugin: unable to load "C:\Videos\AviSynth 2.5\plugins\RemoveGrainT.dll"
(C:\Videos\AviSynth 2.5\plugins\RemoveSpots.avs, line 6)
(C:\Videos\vid_onceupon\Script1.avs, line 20)
---------------------------

And, for fun, I changed RemoveGrainT.dll to RemoveGrainT.dll.bak and I got:

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Script error: there is no function named "mt_makediff"
(C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi, line 776)
(C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi, line 386)
(C:\Videos\vid_onceupon\Script1.avs, line 23)
---------------------------

Script1 code:

vidpath=""
ppath="C:\Videos\AviSynth 2.5\"

Import("C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi")
Import("C:\Videos\AviSynth 2.5\plugins\RemoveSpots.avs")
Import("C:\Videos\AviSynth 2.5\plugins\daa3.avs")
Import("C:\Videos\AviSynth 2.5\plugins\FastLineDarken 1.3.avs")
LoadPlugin("C:\Videos\AviSynth 2.5\plugins\MaskTools.dll")

# ----------------------------------------------
# .d2v project file created with DGIndex using
# "Ignore pulldown flags" switch for 100% 
# progressive 23.97 fps video (telecine removed).
#              (Input is YV12 MPEG2) 
# ----------------------------------------------
MPEG2Source("C:\Videos\vid_onceupon\once_upon_a_forest.d2v",CPU=5)

Any ideas? :P

[poisondeathray]

You need masktools2. You have masktools1 (which is Masktool.dll)

are you using avisynth 2.5.x or 2.6.x ?

if you don't know, type version() at the end of a script and preview

mt_masktools_26.dll is for avisynth 2.6.x => this is the one you have in the screenshot (but it's in the wrong folder)

mt_masktools_25.dll is for avisynth 2.5.x

[askiplop]

AviSynth v2.58 (It's the first script ever than I ran on AviSynth :P)
I've downloaded mt_masktools_25.dll here: http://www.mediafire.com/?vp9xpjg8p0whsdi which is 1246 KB.

Thank you for the info.

Unfortunatly I still get the same "loop".

EDIT:
I re-read a post form sanlyn:

FastLineDarken is an older screipt (I like it better, fewer problems), requires old MaskTools 1.5.8, attached below.

And he attached the file. So normally I shouldn't need the mt_masktools_25.dll file, right?

EDIT 2:
Do I need some sort of codec for YU12? I know it's not a "real" codec, but I read somewhere that I may need a special library so Windows can handle it.

[sanlyn]

You need masktools-v2. There are several versions of v2 (don'tcha luv it?), but v2.0a45 is the more useful, especially since QTGMC can use it for both YV12 and YUY2. v4.0a45 is a version for Avisynth 2.5.x, available here: http://manao4.free.fr/masktools-v2.0a45.zip. You can have the old Masktools.dll and the newer Masktools-25 dll in the same plugins folder, because the function names are different in each; however, some older plugins use the old v1, so you'll see scripts loading v1 manually.

You have version 2.6 for Masktools in your plugins folder. This works for Avisynth 2.5x and 2.6x, but not all 2.5x plugins will work with the 2.6 dll. (Oh, man!). Some plugins for Avisynth 2.5 will use it, others won't. You'll note in the 2nd avs script that Dither.dll uses Masktools 2.6, not Masktools 2.5. The solution? Put the 2.5 Masktools dll in your plugins folder. Then create another plugins folder and call it plugins2. Put the Masktools 2.6 dll in plugins2, and load it explicitly in a script when you need that specific version. For now, MaskTools 2.5 is what most people are using most of the time.

Later, when you've collected 250 Avisynth plugins and scripts and 17 versions of half of them, you'll need tricks like this.

[askiplop]

LOL You're very funny to read, you know dat?

I've downloaded the version you suggested and replaced the one I had.
I also created a folder inside plugins called otherver.

But I still get the same &?%&$?%# error (A). [< it's an angel, for those who doesn't know Msn emoticons]

Any idea, sanlyn to fix this? I'm like,... 2 steps away to finish (I hope)

EDIT:
No wait!The error still:

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
LoadPlugin: unable to load "C:\Videos\AviSynth 2.5\plugins\RemoveGrainT.dll"
(C:\Videos\AviSynth 2.5\plugins\RemoveSpots.avs, line 6)
(C:\Videos\vid_onceupon\Script1.avs, line 20)
---------------------------

But *suspense sound like in old movies* If I remove the RemoveGrainT.dll file now I got:

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Script error: there is no function named "MSuper"
(C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi, line 402)
(C:\Videos\vid_onceupon\Script1.avs, line 23)
---------------------------

Does this give you any hints?

Maybe I don't have the right RemoveGrainT.dll file and QTGMC needs it?

[sanlyn]

Yes, and you've been remarkably patient. The good news is that the stuff you're loading up now are used by a whole slew of other plugins, so once you have these you're set for a while. Let's see if we can backtrack here...

My version of RemoveGrainT.dll is a zero-length file from the v0.9 pre-release, so I'll attach the dll version I've downloaded from others who recommended it.

Here's a link to everything used by QTGMC and all its various presets and options: http://www.spirton.com/uploads/QTGMC/QTGMC%2032-bit%20Plugins.zip . It's a big zip, but all you need are the dll's that you don't already have. It includes some html and source code, etc., that you don't want in your plugins. It also contains a deinterlacer (yadif.dll) that QTGMC sometimes uses. Yadif.dll goes in your plugins folder. If you ever need it, yadif is a non-standard plugin loaded with this statement (which, for the time being at least, you don't need yet):

LoadCPlugin("path-to-plugins\yadif.dll")

Once you get all the goodies in that QTGMC package, you'll be set for lots of other dandy filters, like MCTemporalDenoise, TemporalDegrain, FFT3D, and a bunch of other important guys.

[sanlyn]

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Script error: there is no function named "MSuper"
(C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi, line 402)
(C:\Videos\vid_onceupon\Script1.avs, line 23)
---------------------------

"Msuper" is a function in MVTools v2: file:///D:/DVD/AviSynth%202.5/fizick/mvTools25113/Documentation/mvtools-v2.5.11.3.zip
. I see you have mvtools2.dll in another folder(?) with dither's stuff. It should be in your main AviSynth plugins folder. Called by a great many plugins.

[askiplop]

You were very patient with me too

Thx for the file, I've putted it in my plugins directory.

A couple of hours ago, I've downloaded the same exact file from the same exact location for the QTGMC plugin, ... but I didn't put all the files, just the one specified in the script... So I took all DLLs from AviSynth 32-bit Plugins... and I copied 2 DLLs to WIN/sys32 (fftw3.dll and libfftw3f-3.dll) as requested by the readme.txt

We've made another step. But seems that VirtualDub still miss a friend:

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Script error: there is no function named "DeBlock_QED"
(C:\Videos\vid_onceupon\Script1.avs, line 24)
---------------------------

DeBlock_QED plugin... Found here: AviSynth Wiki the AVS file and it's requirements. It needs DeBlock.dll and DCTFilter. Downloaded, copied to plugins folder (only the DLLs, I think the source code is included XD) But I still get the same "Script error: there is no function named "DeBlock_QED" message.

And yes, I hope I won't need to add anymore plugins for a while.
-----
EDIT
Yes thank you, it was included with QTGMC, so I deleted the one in Dither folder and kept the newly extracted one in the main plugins folder.

[manono]

DeBlock_QED plugin... Found here: AviSynth Wiki the AVS file and it's requirements. It needs DeBlock.dll and DCTFilter. Downloaded, copied to plugins folder (only the DLLs, I think the source code is included XD) But I still get the same "Script error: there is no function named "DeBlock_QED" message.

Once again, only the AVSIs will autoload. If it's an AVS you have to import it into the script. Try renaming it as an AVSI and see if it autoloads. If it doesn't, then import it as I described earlier.

[sanlyn]

Wow, you guys have been busy while I was doing chores. Herewith, a ZIP of the last 3 plugins in the script. Hope DeBlock_QED was OK, I had just started looking for its parts when I got back to this thread.

The Zip has files for:
- GradFun2dbMOD (which uses GrandFun2DB, included here)
- TTempSmooth
- aWarpSharp
One of these uses AddGrainC.dll, but that's already included with previous support files.
Note that GradFun2dbMOD is an "avsi" that should load automatically. The others are dll's.

[askiplop]

Oops sorry, I didn't see, I've included the script, in the wrong file.

Ok and I've downloaded :
aWarpSharp.dll
gradfun2db.dll
TTempSmooth.dll
GradFun2DBmod.v1.5.avsi

I opened the d2v file in VirtualDub.
I can see the image, but it has a green overlay. When I press Play, the overlay opacity vary, let set from 25% to 100% but I can't get a clear image. And sometimes its purple.

When I press Play (Play 1 not Play 2), 1 second is like 1.75 seconds. It plays in slow mo...tiooon. I've attached screenshots of 5 images between frame 1 and 79.

I decided to comment each line to see which one cause this problem and it seems to be FastLineDarken().
I was able to see the actual image after disabling that line.

I've opened the original d2v file and the script file and compare each videos, the one generated with avisynth have a little green bar on the right, the colors are fade and there is a quality loss.

I think it's because I didn't convert the file to a lossless codec, is it possible?
(Screenshots below)


Again, here's the full script:

vidpath=""
ppath="C:\Videos\AviSynth 2.5\"

Import("C:\Videos\AviSynth 2.5\plugins\QTGMC-3.32.avsi")
Import("C:\Videos\AviSynth 2.5\plugins\RemoveSpots.avs")
Import("C:\Videos\AviSynth 2.5\plugins\DeBlock_QED.avs")
Import("C:\Videos\AviSynth 2.5\plugins\daa3.avs")
Import("C:\Videos\AviSynth 2.5\plugins\FastLineDarken 1.3.avs")
LoadPlugin("C:\Videos\AviSynth 2.5\plugins\MaskTools.dll")

# ----------------------------------------------
# .d2v project file created with DGIndex using
# "Ignore pulldown flags" switch for 100%
# progressive 23.97 fps video (telecine removed).
# (Input is YV12 MPEG2)
# ----------------------------------------------
MPEG2Source("C:\Videos\vid_onceupon\once_upon_a_fo rest.d2v",CPU=5)

ColorYUV(off_y=12,cont_y=15)
ColorYUV(off_u=4)
RemoveSpots()
#daa3()
FastLineDarken()
QTGMC(InputType=1,preset="medium")
DeBlock_QED()
GradFun2DBmod()
TTempSmooth()
mergechroma(aWarpSharp(depth=20.0, thresh=0.75, blurlevel=2, cm=1))
AddGrainC(var=1.5,uvar=1.5)

# ---- to RGB for VirtualDub work ------
ConvertToRGB32(matrix="rec601",interlaced=false)
Crop(2,4,-2,-8).AddBorders(2,6,2,6)

[sanlyn]

Hm.Well, darnit, you made it with no error messages! Now, that's really amazing for one day's work. When I first started with those plugins it took me 2 months to get the parts together.

I think I know where the problem might be. Be back shortly, lemme look up something.

Meanwhile...."vidpath=" is just a string variable that I created to cut down some typing. You don't need it, it's not even an Avisynth term. I simply used vidpath just for the folder path to the video clip or d2v.

The same goes for ppath, which just stands for "plugin path". Apparently your plugins are in C:\Videos\AviSynth 2.5\plugins. So you would use that ppath user-created variable to store the location of your plugins folder, with a "\" at the end of the string. The "+" in the statements below means "join the ppath text value with the following text." You would end up with:

ppath="C:\Videos\AviSynth 2.5\plugins\"

Import(ppath + "QTGMC-3.32.avsi")
Import(ppath + "RemoveSpots.avs")
Import(ppath + "DeBlock_QED.avs")
Import(ppath + "daa3.avs")
Import(ppath + "FastLineDarken 1.3.avs")

But you don't have to do it that way. Your version worked just fine, so you don't need vidpath or ppath.

[askiplop]

Hehe, I wouldn't made it without you and manono :P

Yeah I was jsut too lazy to put variables, but now that I'm close to the end, I'm going to add some

Thanks for taking a look into it sanlyn! Take your time, I'm going to bed in 3-4 hours maybe :P I'm so excited!! I want this finished (not converted but...) before I go to bed, so tomorrow it will be on my desktop, waiting for me (But I'm sure I'll find something not perfect enought and will re-encode it again)

MUHAHAHA


As I said earlier, I'm using the d2v/m2v file, compressed... maybe this is the problem, or one plugin is incompatible with Windows 8 x64 (even if i'm using AviSynth and VDub x86.

I tried for fun to export it into AVI with File > Save as AVI... 5 FPS seconds... 4 hours.. even with my I7-2600K .... but hey, it's better than nothing at all!

EDIT:

Maybe I have a corrupted plugin? or mismatching version or something?

[sanlyn]

I decided to comment each line to see which one cause this problem and it seems to be FastLineDarken().
I was able to see the actual image after disabling that line.

Yep, I checked that. The v1.3 I posted was just downloaded today from the 'net, and it contains the same errors that were in that script over a year ago. The correction was in doom9, but apparently no one corrected the download. I found the corrected version, attached below. Just replace your entire copy of the avs with the attached file. My fault, folks, I made an unwarranted assumption.

The portion of the avs corrected (in red):

function FastLineDarken( clip c, int "strength", int "luma_cap", int "threshold", int "thinning") 
{
str = string(default(strength, 48) /128.)
lum = string(default(luma_cap, 191))
thr = string(default(threshold, 4))
thinning = default(thinning,24)
thn = string(thinning /16.)
exin=c.expand().inpand()
diff = yv12lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? 127 +",uexpr="x",vexpr="x",u=2, v=2)
linemask = yv12lut(diff.inpand(),"x 127 - "+thn+" * 255 +")\
.yv12convolution("1 1 1","1 1 1",y=3,u=0,v=0)
thick = yv12lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +",uexpr="x",vexpr="x",u=2, v=2)
thin = yv12lutxy(c.expand(),diff,yexpr="x y 127 - "+str+" 1 + * +",u=2, v=2)
return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)
}

Some of these plugins are a little pokey running (QTGMC especially). There are faster versions, but don't worry about that now. The processing tends to pick up speed later. Even NeatVideo does that.

[askiplop]

It's okay sanlyn Thanks for the corrections, testing it right now.

I don't care if it's a little bit pokey, as long as it does the job

EDIT:

Its... WORKING!

But it still looks like a Jpeg file with high compression, on the left window... is it normal?
And if I want it back to MKV after (to put all audio tracks and subs... I just need to go in File> Save as AVI (without any kind of configuration for codec) and then use mkvmerge ?

[sanlyn]

EDIT:
Maybe I have a corrupted plugin? or mismatching version or something?

With the first avs script, you're not running any VirtualDub filters. When you open the script in VirtualDub to check it out, check these settings before letting it run:
1. Go to Video -> Color depth..., and set the colorspace to RGB24.
2. Go to Video -> Compression, and Select Lagarith, set it for RGB24 (default).
3. Go to Video -> and check "Fast recompress". Don't use the other execution options like "full processing mode", etc. Fast recompress is what you want..

You can then click "Save AVI...". You'll note that the "output video pane" doesn't change, but VDub is processing the script.

Remember that this is the first step in your restoration. Later when you open that AVI and run the VirtualDub filters you'll see improvements. IF it looks really bad at this first stage, let us know. I wouldn't worry about audio until later.

[askiplop]

Oops that'S true I forgot . Thanks though for the options settings...

But just to make sure, I attached a printscreen showing green pixels around the borders.
And check the quality of the image on the left, its like a jpeg lol.

EDIT:

For the compression, its RGB24 32 and I cant change it.

The log tab on the work status shows that the format being used is RGB32. (See attached image for details)

For the audio, no problem, I'm sure you'll give me a hand at that point

Edit 2:

Okay frame 4552 out of 101 744 frames (4.47%)
1 500Mb... ( * 100 / 4.47) Output file : 33 557 MB = > 34 GB... I dont know where I'm going to put that!
LOL