AviSynth 2.5x and Convolution3D guide for AVI captures

[FulciLives]

How to use AviSynth with the Convolution3D "video noise" filter including editing with VirtualDubMod

A mini-guide by John "FulciLives" Coleman

PLEASE NOTE that this guide is based on using AVI files from captures such as DV AVI or PICVideo MJPEG or HuffyUV etc.

You will need:

1.) VirtualDubMod (newest version always of course)
2.) AviSynth 2.5x (I think 2.54 is the current newest version)
3.) Convolution3D for AviSynth 2.5x (the YUY2 version not the YV12 version)

Download and install AviSynth 2.5x ... the newest version is 2.54 I think. You can get it in the TOOLS section. Very easy to install nothing fancy there.

Now go to this website:

http://www.avisynth.org/warpenterprises/

Download Convolution3D for AviSynth 2.5x ... notice that there are 2 downloads listed under AviSynth 2.5x ... you want the first one. This works with YUY2 video sources such as DV AVI and PICVideo MJPEG etc. The other second version only works with YV12 sources.

The Convolution3D download is a ZIP file that contains only 2 files. One is a TXT file explaining how to use Convolution3D and the other is the actual filter file which is called Convolution3D.dll

Now copy the Convolution3D.dll file to your WINDOWS/SYSTEM32 folder (please note it may be the SYSTEM folder instead of the SYSTEM32 folder ... I think I put the files in both folders to "be safe"). This way when you "call" on it from your AviSynth script you don't need to input a path location. Alternatively you could put it elsewhere but then you will need a dir path info in your script.

Open up notepad and copy and paste this into it:

LoadPlugin("Convolution3d.dll") 
avisource("D:\CAPTURE\capture.avi") 
crop(8,4,-8,-4) 
SeparateFields() 
odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
Interleave(evn,odd) 
Weave() 
DoubleWeave.SelectOdd()
AddBorders(8,4,8,4) 
LanczosResize(352,480)
ConvertToRGB(interlaced=true)

Now save this in NOTEPAD but make sure you select all files instead of *.txt as file type. When you type in the filename be sure to add avs to the end so it looks like this: filename.avs

Now let's break this down one line at a time so you understand what's going on:

*LoadPlugin("Convolution3d.dll")
This loads your Convolution3d.dll file. This is all you need if you put it in the WINDOWS/SYSTEM32 folder. If you put it in another folder then you will have to point towards it. For example if you put it on your C drive in a folder called FILTERS then you would change it to look like this:
LoadPlugin("C:\FILTERS\Convolution3d.dll")

*avisource("D:\CAPTURE\capture.avi")
This loads your captured file. Of course you will change the DIR PATH and FILENAME to match your own.

*crop(8,4,-8,-4)
This is where you crop your file. If you don't want to do any cropping then delete this line. In my example I have cropped 8 from the left and right and 4 from the top and bottom. The "syntax" for this command is: crop(left, top, right, bottom) and left/bottom must be a negative value.

*SeparateFields()
*odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
*evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
*Interleave(evn,odd)
*Weave()
*DoubleWeave.SelectOdd()
This is a little bit of "magic" needed to properly apply a video filter to interlaced video. It looks confusing I know but basically interlaced video has frames and each frame is made up of 2 fields. We need to separate the frames into fields ... apply the filter to each field individually ... then combine the fields back into frames. This is what does all that. Also this is optimized for video that is bottom field first as is DV AVI format. If you have video that is top field first (such as a PICVideo MJPEG capture or HuffyUV capture) then delete the last line here (the DoubleWeave.SelectOdd() line).

Now there are 6 Convolution3D settings and in my example I've used the settings known as "movieLQ".

You can change to any of the other settings based on your source.

For instance:

Movie Hi Quality (good DVD source) then use:
Convolution3D (0, 3, 4, 3, 4, 2.8, 0)

Movie Low Quality (noisy DVD source or most captures) then use:
Convolution3D (0, 6, 10, 6, 8, 2.8, 0)

VHS capture Bad Quality (some captures) then use:
Convolution3D (0, 32, 128, 16, 64, 10, 0)

These settings and more are explained in the Convolution3D text document you get when you download the Convolution ZIP file.

*AddBorders(8,4,8,4)
Remember that in this example I cropped 8 from the left and right and 4 from the top and bottom. This adds a "black" border (or space) to "fill up" when I cropped or cut off so that the original size of the capture is retained. The "syntax" for this command is: AddBorders(left, top, right, bottom) and all values should be positive.

*LanczosResize(352,480)
This resizes the video to Half D1 resolution. If you don't want to do that then delete this line.

*ConvertToRGB(interlaced=true)
This last line is for the benefit of TMPGEnc. You might find that you don't need this but I suggest trying it. This will convert your YUY2 colorspace to RGB which is what TMPGEnc uses. Since we have interlaced video we add that inside the () otherwsie you would just have () with nothing inside of them. Please note that if you use CCE (as I do) then this line should change to: ConvertToYUY2(interlaced=true)

Now ... once you have your script with your capture file name in it and your preferred setting for Convolution3D ... we want to open it with VirtualDubMod.

So open your filename.avs script and your video will load.

Now do your editing.

When you are done do this (you will LOVE this part it's so kewl).

On the top of VirtualDubMod go to TOOLS and under that go to SCRIPT EDITOR

The VirtualDubMod script editor window pops up and you will see your script!

Now go to the line just after the one that load your avi file and hit enter so you have a space there.

So it looks like this:

LoadPlugin("Convolution3d.dll") 
avisource("D:\CAPTURE\capture.avi") 

crop(8,4,-8,-4) 
SeparateFields() 
odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
Interleave(evn,odd) 
Weave() 
DoubleWeave.SelectOdd()
AddBorders(8,4,8,4) 
LanczosResize(352,576) 
ConvertToRGB(interlaced=true)

Now with your cursor on that blank line go to the top of the script editing window and select EDIT and under that select IMPORT FRAMESET AS TRIMS

This will now insert your edits into your script. No need to break your file down into a bunch of small files using DIRECT STREAM COPY. In fact if you have one giant capture of several episodes you can copy and paste your script calling it episode1.avs episode2.avs episode3.avs etc. with each AviSynth avs script having the trim values that isolate THAT particular episode.

Talk about saving time!

Now be sure to save your script.

It will now look something like this:

LoadPlugin("Convolution3d.dll") 
avisource("D:\CAPTURE\capture.avi") 
Trim(195,42620) 
crop(8,0,-8,-0) 
SeparateFields() 
odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
Interleave(evn,odd) 
Weave() 
DoubleWeave.SelectOdd() 
AddBorders(8,4,8,4)
LanczosResize(352,576) 
ConvertToRGB(interlaced=true)

The Trim command shows your starting edit point and ending edit point. If you have a capture of a TV show for instance and want to cut out the commercials then the Trim line might be very long and look something like this:

Trim(344,22665) ++ Trim(32677,47725) ++ Trim(51051,77018) ++ Trim(85055,101699) ++ Trim(107309,154994)

Anyways at this point you simply load your AviSynth AVS script into TMPGEnc (or CCE) just as if you were loading the original AVI file and go ahead and do your encoding.

THE END

- John "FulciLives" Coleman

[cd090580]

Nice guide,
I wanted to try it with an analog capture from TV but I noticed that Convolution3D needs SSE to work

[FulciLives]

Nice guide,
I wanted to try it with an analog capture from TV but I noticed that Convolution3D needs SSE to work

Really?

It works for me on my pathetic Pentium 3 :P

- John "FulciLives" Coleman

[jimmalenko]

Good work Fulci...looks awfully like a PM I got from you a couple of weeks ago

[FulciLives]

Good work Fulci...looks awfully like a PM I got from you a couple of weeks ago

What? ... you think I typed ALL that up just for your lazy ass!

OK actually I did

Then I did a copy and paste of it in another thread (with a few revisions).

After that I figured SCREW THAT I'll just make it guide ... this way I don't have to keep typing up the same crap all the time!

I hope this helps some people. This is such a better method IMHO than using VirtualDub filters and frameserving but don't let SatStorm hear that or he will go on a rampage about how AviSynth uses are really clueless and blah blah blah

- John "FulciLives" Coleman

[hrlslcbr]

Great work Fulci!!!
Good guide, it explains everything I wanted to know about Convolution3D and a few other things
btw, there's a little error in the explanation of the crop command. The right syntax would be crop(left, top, right, bottom) (the same goes for addborders)
Thanks for this guide.

[FulciLives]

Great work Fulci!!!
Good guide, it explains everything I wanted to know about Convolution3D and a few other things
btw, there's a little error in the explanation of the crop command. The right syntax would be crop(left, top, right, bottom) (the same goes for addborders)
Thanks for this guide.

Oh well no guide is "perfect" right?

Thanks for correcting me. I have edited the guide to reflect the correct syntax for those two commands.

- John "FulciLives" Coleman

[hrlslcbr]

Yep, no guide is perfect...
It's good to help correcting those little 'imperfections' of your guide

[SatStorm]

And just because I like this guide, I discard one of your cards!

[FulciLives]

And just because I like this guide, I discard one of your cards!

Well maybe not ALL VirtualDub users are bad afterall!

Thank You!

- John "FulciLives" Coleman

P.S.
I guess I got 3 more guides I need to write now 8)

[LNielsen76]

Ok
I am obviously an idiot because this guide does not work for me.

Upon load I receive an error in line 5 telling me that Convolution3D only accepts YUY2 video.

Anyway, I just find the trim points and write them on my own, as I suspect the only reason we are loading this in Virtualdub is to find the trim points.

So, I have my final filename.avs file with all scripts includes as shown in the guide.

I load the avs into TMPGEnc as the video file and when I go to encode, it does not call on the scripts but instead encodes the file itself!!

I am obviously not in the loop on this one as I am without a clue as to why I get an error code and why my DV AVI is not called upon in the encoder when loading the avs.

Any ideas what my naivety might be?

[FulciLives]

I don't normally work with DV AVI format so maybe it isn't YUY2 but I thought it was.

There are two versions of Convolution3D for AviSynth 2.5x and one version works with YUY2 and the other works with YV12.

So I guess try the other version to see if it works.

The AviSynth website was down for days and days but I just checked it and it's back up again.

- John "FulciLives" Coleman

P.S.
Here is the website to download Convolution3D
http://www.avisynth.org/warpenterprises/

[LNielsen76]

Will try it. BTW, I am capturing via ADVC-100 Firewire with ScenalyzerLive. I will try the YV12 version of Convolution3D.

[BSR]

If you are using the Panasonic DV codec, it only outputs RGB.
Then you will need to add a colorspace conversion in your script before convolution3d like converttoyuy2.
What DV Codec are you using?

[LNielsen76]

I am using the Panasonic DV Codec. I tried the YV12 driver and it did not work. My DV AVI is in neither the YUY2 or YV12 format.

What is the entire script for conversion to YUY2. I am pretty sure this is the problem. Is it simply converttoyuy2?

[BSR]

Your DV avi is in the YUY colorspace, but the Panasonic Codec decodes it to the RGB colorspace only. Limitation of the codec, but very good codec.

Converttoyuy2(interlaced=true)

Add this line right after AVISOURCE line.

[LNielsen76]

I will try it. Thanks for the insight. I will update you with my findings.

[jimmalenko]

Will try it. BTW, I am capturing via ADVC-100 Firewire with ScenalyzerLive. I will try the YV12 version of Convolution3D.

I was also using an ADVC-100 with ScenalyzerLive and the YUY2 worked. I was using the Canopus DV codec though.

[LNielsen76]

Awesome! The converttoYUY2(interlaced=true) after the avisource command worked!! I replaced the Panasonic DV Codec with the Canopus DV codec for future use.

I am able to load the avs into TMPGEnc XPRess and TMPGEnc DVD Source Creator but not TMPGEnc 2.5. In TMPGEnc 2.5 the encode last the duration of the film with audio in the final product but the video is black screen fro the duration of the film. I don't know why it isn't producing the video. I guess as long as it works in XPress, Source Creator, and CCE then I am alright.


Thanks for the great guide and all the help in finding my problem!

[csaag]

I have the same problem with my output, namely a black screen.
My input was a 22GB avi file of a 45 minute VHS capture in 720x480 using IuVcr (with Huffy). The avi plays fine.

I used the following script:

LoadPlugin("C:\Progra~1\Capture\extra\Convolution3d.dll")
avisource("D:\VidCapture\andrew_play.avi")
crop(8,4,-8,-4)
SeparateFields()
odd=SelectOdd.Convolution3D (0, 32, 128, 16, 64, 10, 0)
evn=SelectEven.Convolution3D (0, 32, 128, 16, 64, 10, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()
AddBorders(8,4,8,4)
ConvertToRGB(interlaced=true)

TmpgEnc Plus (2.521) ran for about 90 minutes (I had ES video checked using motion search est & the Tmpgenc noise filter checked) The avs script was my input file. After 90 minutes, the m2v output was a black screen. Is this a Tmpg Plus problem or what did I do wrong?

[piano632]

Can someone tell me how to load DV-AVI files made by Pinnacle card into Virtualdub. Every time I try I get this message:

Couldn't locate decompressor for format 'dvsd' (unknown).
VirtualDub requires a Video for Windows (VFW) compatible codec to decompress video. DirectShow codecs, such as those used by Windows Media Player, are not suitable. Only 'Direct stream copy' is available for this video.


Somehow I get the feeling from this message that it either isn't going to work or I need to install a codec. Help would be appreciated as I have lots of video tapes with chroma noise, etc. that could use a good clean-up.

[FulciLives]

I have the same problem with my output, namely a black screen.
My input was a 22GB avi file of a 45 minute VHS capture in 720x480 using IuVcr (with Huffy). The avi plays fine.

I used the following script:

LoadPlugin("C:\Progra~1\Capture\extra\Convolution3d.dll")
avisource("D:\VidCapture\andrew_play.avi")
crop(8,4,-8,-4)
SeparateFields()
odd=SelectOdd.Convolution3D (0, 32, 128, 16, 64, 10, 0)
evn=SelectEven.Convolution3D (0, 32, 128, 16, 64, 10, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()
AddBorders(8,4,8,4)
ConvertToRGB(interlaced=true)

TmpgEnc Plus (2.521) ran for about 90 minutes (I had ES video checked using motion search est & the Tmpgenc noise filter checked) The avs script was my input file. After 90 minutes, the m2v output was a black screen. Is this a Tmpg Plus problem or what did I do wrong?

People are not reading my guide very well

If you have TOP FIELD video then you need to delete this line from the script:

DoubleWeave.SelectOdd()

As stated in the guide HuffyUV captures should be TOP FIELD video. The only real video codec for capture that is BOTTOM FIELD is the DV format. Pretty much all other codes like HuffyUV and PICVideo MJPEG are TOP FIELD video.

Also the CROP and ADDBORDERS commands were put there for illustrative purposes. Some of you may not want to use those lines at all or use some thought if you do. For instance a VHS usually needs at least 8 to 12 lines cut from the bottom to remove that "video junk" that you get from VHS tapes on capture. For instance with a VHS video I often do this:

crop(8,4,-8,-12)
AddBorders(8,8,8,8)

This cuts 8 from either side but 12 from the bottom and only 4 from the top. Then you are adding 8 to all sides. This "shifts" the image down 4 pixels but is not noticeable and I like equal amounts of black all around myself since you never know how the TV overscan will be. Some TV's show more of the bottom than other TV's.

Moving along ...

I have no idea why so many people are having trouble with TMPGEnc as I've never had this trouble but I barely use TMPGEnc anymore ... I use Cinema Craft Encoder aka CCE

But I've tested TMPGEnc with the sample script using my PICVideo MJPEG captures and I have no troubles. I don't get the dreaded BLACK screen.

I think that if you are having trouble then the problem could be that your source is something other than YUY2 or YV12. It could also be because there are two versions of Convolution3D ... one for YUY2 sources and one for YV12 sources ... it does not work with RGB sources. So make sure your source is either YUY2 or YV12 and make sure you are using the correct version of Convolution3D for your source.

If your source is anything other than YUY2 or YV12 then you can modify the script this way:

LoadPlugin("Convolution3d.dll") 
avisource("D:\CAPTURE\capture.avi")
ConvertToYUY2(interlaced=true) 
Trim(195,42620) 
crop(8,0,-8,-0) 
SeparateFields() 
odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0) 
Interleave(evn,odd) 
Weave() 
DoubleWeave.SelectOdd() 
AddBorders(8,4,8,4) 
LanczosResize(352,576) 
ConvertToRGB(interlaced=true)

This is the same as the last SAMPLE script in my guide with one small difference ... I have added ConvertToYUY2(interlaced=true) right after the AviSource line of the script. This converts the video to YUY2 and if you are using the YUY2 version of Convolution3D then there you go!

Please note that the last line ConvertToRGB(interlaced=true) is only for using the script with TMPGEnc. If you are using it with CCE then you can delete that line and you do NOT need to put ConvertToYUY2(interlaced=true) at the end since it is there earlier in the script. Again this is a "work around" if you source isn't already either YUY2 or YV12.

Whenver possible though captures should be YUY2 and not RGB or anything else.

- John "FulciLives" Coleman

P.S.
This is for CSAAG ...

Modify the script you posted to this:

LoadPlugin("C:\Progra~1\Capture\extra\Convolution3d.dll") 
avisource("D:\VidCapture\andrew_play.avi")
ConvertToYUY2(interlaced=true) 
crop(8,4,-8,-12) 
SeparateFields() 
odd=SelectOdd.Convolution3D (0, 32, 128, 16, 64, 10, 0) 
evn=SelectEven.Convolution3D (0, 32, 128, 16, 64, 10, 0) 
Interleave(evn,odd) 
Weave() 
AddBorders(8,8,8,8) 
ConvertToRGB(interlaced=true)

If you use the YUY2 version of Convolution3D then this script should work just fine with TMPGEnc

[FulciLives]

Can someone tell me how to load DV-AVI files made by Pinnacle card into Virtualdub. Every time I try I get this message:

Couldn't locate decompressor for format 'dvsd' (unknown).
VirtualDub requires a Video for Windows (VFW) compatible codec to decompress video. DirectShow codecs, such as those used by Windows Media Player, are not suitable. Only 'Direct stream copy' is available for this video.


Somehow I get the feeling from this message that it either isn't going to work or I need to install a codec. Help would be appreciated as I have lots of video tapes with chroma noise, etc. that could use a good clean-up.

I'm no DV AVI expert but it sounds to me that you don't have a DV codec on your computer. I've heard that the Panasonic codec is a favorite and it is free to download.

- John "FulciLives" Coleman

[fmctm1sw]

Download Convolution3D for AviSynth 2.5x ... notice that there are 2 downloads listed under AviSynth 2.5x ... you want the first one. This works with YUY2 video sources such as DV AVI and PICVideo MJPEG etc. The other second version only works with YV12 sources.

This may explain why I have never been able to get this to work for me....

[mgh]

Thanx a ton-solved problems for my TV captures.
Filter is fast compared to my earlier convoluted attempts(pun intended)!!

[csaag]

Fulci,
thanks for the info.
My bad on missing parts of the guide.

I had to tweak the priorities a bit in Tmpgenc under Env Options to be able to get started.

[kenmo]

Awesome guide. I can't wait to try it...

Many thanks from a 'dumb VirtualDub user"....

[vhelp]

Morning guys :P

Fulci wrote:

I hope this helps some people. This is such a better method IMHO than using VirtualDub filters and frameserving but don't let SatStorm hear that or he will go on a rampage about how AviSynth uses are really clueless and blah blah blah

Actually, I prefer the virtualdub route myself too
hehe..
.

But, to each their own.., right pal ?? :P

Anyways. I appreciate the guide you put up (and taken the time to piece together)
I'll try it out. But, it looks like its for Interlaced sources (such as DV home
footage etc) but that's probably a script technique (that only looks like its
for pure Interlaced )

@ Fulci,

Question.. is there a similar vdub filter that does the same steps as the
Convultion3D filter for AVS scripts ??
.
Not all the other filter functions, just the Convolution3D filter itself ??

Thanks pal.., your friend,
-vhelp

[FulciLives]

@vhelp

There is a VirtualDub filter called general convolution that works similar to how Convolution3D works but the VirtualDub filter comes with no documentation that I have been able to find so I never did ever figure out exactly how it works.

As for VirtualDub filters I prefer to work in the YUY2 colorspace since that is what most captures are to begin with. Thus my use of AviSynth, Convolution3D and Cinema Craft Encoder.

VirtualDub does an RGB conversion as does TMPGEnc

- John "FulciLives" Coleman

[axolotls]

I appreciate this guide John. Very helpful. I've used this sorta thing before but without the interlacing separation.

I am trying it on some good quality high8 captures now