restore old film with avisynth

[flash30]

hi i am new to avisynth and do the basic script but i am trying to clean up old 8mm film to get rid of dirt and scratches. i have seen what avisynth can do but for me is very hard..i can get so far then i hit a wall.

thanks

[videofred]

Hi Flash,

RemovedirtMC() is an Avisynth function. Therefore it must be loaded first.
It also needs some plugins (dll) and these must be loaded too.
Try this first:

Copy all dll files and functions to the special Avisynth 'plugins' folder.
This folder can be found in 'program files (x86)/avisynth'
Everything in this folder will be auto-loaded at the start of an Avisynth session.
Your script should work now.

Fred.

[LMotlow]

videofred got here first, but here are more details:

.avs" plugins don't load automatically when a script is run. They must be explicitly loaded with the Import() function:

AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveDirtMC.avs")
RemoveDirtMC()

-or the more verbose way is to include the text of the .avs filter at the bottom of your avs script:

AviSource("derby.avi")
RemoveDirtMC()

function RemoveDirtMC(clip,int limit, bool "_grey")
{
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
  backw = MFlow(clip,i,bvec)
  forw  = MFlow(clip,i,fvec)
  clp=interleave(backw,clip,forw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

-or:

If you want an .avs file to load automatically, change its name from "RemoveDirtMC.avs" to "RemoveDirtMC.avsi". avsi and .dll filters load automatically.

Oh, I forgot:

Copy all dll files and functions to the special Avisynth 'plugins' folder.
This folder can be found in 'program files (x86)/avisynth'

The O.P.'s profile sez he's using Win7 32-bit.

[flash30]

thanks fred and motlow
i have changed RemoveDirtMC.avs to RemoveDirtMC.avsi

and i have done this script

AviSource("derby.avi")
RemoveDirtMC(40)

the great thing is there is no error message but i am not seeing no changes. is there anything else i am doing wrong ?

i am seeing on virturaldub (dub in progress) does that mean it is working ?

[LMotlow]

VirtualDub displays what the video looks like after it comes from Avisynth processing. "dub in Progress" means that Avisynth's output is being written to a new file.

What kind of "dirt" are you trying to remove? Comment-out the RemoveDirt line (put a "#" as the first character in the line) to disable the filter and take a look at the video without the filter. Then remove the "#" and run it again.

[flash30]

just normal dirt and pops you find in 8mm films, i will give this a shot thanks

[flash30]

just tried that and i am seeing no changes...i have tried 20, 40 and even 100 and still no change

[LMotlow]

RemoveDirtMC does indeed do some cleaning, but it doesn't work much on spots and pops. Try RemoveSpotsMC, which is another version of RemoveDirt. Don't change any parameters, Just use "RemoveSpotsMC()".

[flash30]

thanks motlow i will try this...once i get the whole thing working with a basic script of AviSource("derby.avi") i am getting a error message yet it worked yesterday

[flash30]

i got it all working now...but still no change. i put all the script in unchanged:
function RemoveSpotsMC(clip, int "limit", bool "_grey")
{
_grey = default(_grey, false)
limit = default(limit, 6)
i=MSuper(clip,pel=2)
bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
backw = MFlow(clip,i,bvec)
forw = MFlow(clip,i,fvec)
clp=interleave(backw,clip,forw)
clp=clp.RemoveSpots()
clp=clp.SelectEvery(3,1)
return clp
}

function RemoveSpotsMC2x(clip, int "limit", bool "_grey")
{
_grey = default(_grey, false)
limit = default(limit, 6)
i=MSuper(clip,pel=2)
bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
backw = MFlow(clip,i,bvec)
forw = MFlow(clip,i,fvec)
clp=interleave(backw,clip,forw)
clp=clp.RemoveSpots().RemoveSpots()
clp=clp.SelectEvery(3,1)
return clp
}


function RemoveSpots(clip clp, int "limit", bool "_grey")
{
_grey = Default(_grey, false)
limit = Default(limit, 16)

clmode = 17

clensed = Clense(clp, grey=_grey, cache=4)
sbegin = ForwardClense(clp, grey=_grey, cache=-1)
send = BackwardClense(clp, grey=_grey, cache=-1)
alt = Repair(SCSelect(clp, sbegin, send, clensed, debug=true), clp, mode=limit, modeU = _grey ? -1 : limit)
restore = Repair(clensed, clp, mode=limit, modeU = _grey ? -1 : limit)
clp = RestoreMotionBlocks(clensed, restore, neighbour=clp, alternative=alt, gmthreshold=70, dist=1, \
dmode=2, debug=false, noise=10, noisy=12, grey=_grey)

return clp
}

and still the same..i have even changed to 30 where is "clp=clp.RemoveSpots()" and still no change at all


i have just tried a part of it on its own:

function RemoveSpotsMC(clip, int "limit", bool "_grey")
{
_grey = default(_grey, false)
limit = default(limit, 6)
i=MSuper(clip,pel=2)
bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
backw = MFlow(clip,i,bvec)
forw = MFlow(clip,i,fvec)
clp=interleave(backw,clip,forw)
clp=clp.RemoveSpots()
clp=clp.SelectEvery(3,1)
return clp
}
and still no change.

[flash30]

i have even tried a copy and paste full and parts of freds scripts and nothing...i am starting to give up now i don't think i will ever get this working..everything i try i either get no change or error message..i can add filter and change things just in virtualdub but with Avisynth just errors or no changes.

[videofred]

If you want an .avs file to load automatically, change its name from "RemoveDirtMC.avs" to "RemoveDirtMC.avsi". avsi and .dll filters load automatically.

Of cource.. my mistake, thank you for correcting this.

@Flash30: don't give up. Please upload an example clip so we can have a look at it.

And please show us your script. Perhaps you did not apply the filter correctly on the source file.

Fred.

[LMotlow]

I agree with videofred. Show us the entire script you used. Also, saying you get "an error" doesn't help much -- thousands of errors are possible, so we need to know what the error says specifically. Both RemoveDirt and RemoveSpots and hundreds of other Avisynth plugins are used every day all over the world in very simple scripts with no problem. So rather than just give up, let us know all the details.

Be careful about the changing the text of Avisynth plugins that are coded as avs or avsi scripts. The plugin's designer put those instructions together carefully. I think you can see that the plugin text itself includes three different functions, each with its own purpose and logic.

I suspect the problem might be that even if you insert the full text of the RemoveSpots avs file into your script, your code has to have a line that actually executes that function. It's easy to change the name of a plugin from avs to avsi so that it imports automatically into the processing stream. But in this particular case, the frequent advice about changing the names of .avs plugins to avsi can sometimes lead to glitches because many of those avs plugins call functions that are duplicated inside some of those other renamed avsi scripts. That leads to confusion during processing, so you see errors. In this particular case, I'd leave just about every avs plugin named as they were originally designed. If the plugin comes to you as an .avs, just leave it as-is and use Import() to prevent problems. Eventually you'll be more familiar with reading up on that type of plugin and can spot conflicts more easily.

Leave RemoveSpotsMC.avs and RemoveDirt.avs with their original names. Then try the script this way:

AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")
RemoveSpotsMC()

There are only 3 lines in the above script. Line #1 opens the avi file. Line #2 imports the filter. Line #3 executes the filter. You need all three of those statements. If you don't have line #3, the filter won't be executed.

Again, you probably see in the RemoveSpotsMC script that it has 3 functions in it. In fact, each of those is a filter. One of those functions calls either of the other two filters internally. This is one case in which changing an avs to avsi can trip things up. When you run a script, every avsi in your plugins folder is checked for the coded functions your script wants to use. This can cause some duplications of code that give errors.

Yes, it's extra work to write an "Import" line when you need it, but why slow down your script by making Avisynth look all over the plugins folder for what it wants? Let's say you wanted to run RemoveDirt and RemoveSpots in the same script (I don't why you would, but the world is full of strange folks). The script below would run both of those avs plugins:

AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveDirtMC.avs")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")
RemoveDirtMC(40,false)
RemoveSpotsMC()

Just importing a function or including it in a script doesn't actually run the function. You have to have a line that explicitly executes it. BTW, lots of people seem to have trouble with RemoveDirtMC. Like many, I did too, so I don't use it. I just use the original RemoveDirt. RemoveDirt itself has a dozen different versions -- fortunately, most of 'em have slightly different names(!). Aas a matter of fact, folks, RemoveSpots is really a different version of the original RemoveDirt.

It helps to show a complete script if you have problems with it, and describe specific errors. It would also be a good idea to load a few seconds of video from your sample. Many of these filters don't work so well with interlaced video. If you don't know how to create and load a short sample into the forum, just ask.

[flash30]

avisynth open failure:
script error: there is no function named "repair"
("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs,line44)
("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs,line13)


this is for this code:
AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")
RemoveSpotsMC()

and this
avisynth open failure:
import: couldn't open "C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs"
(c:\user\chris\desktop\derby\new avisynth script.avs. line 2)


for this code:
AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveDirtMC.avs")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")
RemoveDirtMC(40,false)
RemoveSpotsMC()

and i have both removespotsMC.avs and RemoveDirtMC.avs

so when i thought i was getting somewhere..another wall

has mentioned i am running windows 7 on 32bit, my memory is 4gb, amd athlon x2 260 processor 3.2 ghz, server pack 1 and my graphics is ATI radeon HD 5400

this is the video i want to work on..its only a rough version till i return to uk and do a better print with manual focus so this is why i want to practice now plus i have a family film i newly found and would love to do that and it lasts 25min
https://www.dropbox.com/s/f30sbf5ozq7tc5n/derby.avi?dl=0

[flash30]

the scripts i have tried are from freds 20)Film_Restoring_vs_06_2012 plus others i found here and there that i would hope to work...also the ones you have posted here. either nothing i can see change or i get an error message.

[LMotlow]

"Repair" is a function in Repair.dll. It comes with the RemoveGrain plugin package. If you look at your original RemoveGrain download, you'll see Repair.dll included. Copy that dll into your plugins folder.

[flash30]

thanks i have solved the repair.dll error but still no changes..the first script is loading but again nothing different. and still have issues with RemoveSpotsMC.avs yet its in the plug in folder. looks like it is not meant to be.

[flash30]

i have just tried this from another site:
function RemoveSpotsMCTest(clip clp, int "limit", bool "_grey", int "Vectors")
{
clp # Assign last for FrameEvaluate


_grey = Default(_grey, false)
limit = Default(limit, 16)
Vectors = Default(Vectors, 1)
Vectors = (Vectors < 1) ? 1 : (Vectors > 3) ? 3 : Vectors
FrameEvaluate("""RT_Debug("STEP_1")""",After_frame =true)
super = MSuper(clp, pel=2, sharp=2)
FrameEvaluate("""RT_Debug("STEP_2")""",After_frame =true)
bvec3 = (Vectors == 3) ? MAnalyse(super, isb=true, delta=3, blksize=8, overlap=4, truemotion=true) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_3")""",After_frame =true)
bvec2 = (Vectors >= 2) ? MAnalyse(super, isb=true, delta=2, blksize=8, overlap=4, truemotion=true) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_4")""",After_frame =true)
bvec1 = MAnalyse(super, isb=true, delta=1, blksize=8, overlap=4, truemotion=true)
FrameEvaluate("""RT_Debug("STEP_5")""",After_frame =true)
fvec1 = MAnalyse(super, isb=false, delta=1, blksize=8, overlap=4, truemotion=true)
FrameEvaluate("""RT_Debug("STEP_6")""",After_frame =true)
fvec2 = (Vectors >= 2) ? MAnalyse(super, isb=false, delta=2, blksize=8, overlap=4, truemotion=true) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_7")""",After_frame =true)
fvec3 = (Vectors == 3) ? MAnalyse(super, isb=false, delta=3, blksize=8, overlap=4, truemotion=true) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_8")""",After_frame =true)
backw3 = (Vectors == 3) ? MFlow(clp, super, bvec3) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_9")""",After_frame =true)
backw2 = (Vectors >= 2) ? MFlow(clp, super, bvec2) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_10")""",After_fram e=true)
backw1 = MFlow(clp, super, bvec1)
FrameEvaluate("""RT_Debug("STEP_11")""",After_fram e=true)
forw1 = MFlow(clp, super, fvec1)
FrameEvaluate("""RT_Debug("STEP_12")""",After_fram e=true)
forw2 = (Vectors >= 2) ? MFlow(clp, super, fvec2) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_13")""",After_fram e=true)
forw3 = (Vectors == 3) ? MFlow(clp, super, fvec3) : NullClip(clp)
FrameEvaluate("""RT_Debug("STEP_14")""",After_fram e=true)

clp = (Vectors == 1) ? interleave(backw1, clp, forw1) : \
(Vectors == 2) ? interleave(backw2, backw1, clp, forw1, forw2) : \
interleave(backw3, backw2, backw1, clp, forw1, forw2, forw3)
FrameEvaluate("""RT_Debug("STEP_15")""",After_fram e=true)
clp = RemoveSpots(clp, limit=limit, _grey=_grey)
FrameEvaluate("""RT_Debug("STEP_16")""",After_fram e=true)
clp = (Vectors == 1) ? SelectEvery(clp, 3, 1) : \
(Vectors == 2) ? SelectEvery(clp, 5, 2) : \
SelectEvery(clp, 7, 3)
FrameEvaluate("""RT_Debug("STEP_17")""",After_fram e=true)

return StackHorizontal(clp,Last).Crop(0,0,width,-0) # Ensure FrameEvaluate()'s are processed
}

again it worked but no changes the output is the same as the input..so i can't see how this is ever going to work

[LMotlow]

Not sure what that last script is doing, even though I recall seeing it a while back.

None of the de-spotters and de-scratchers work well with a lot of motion involved. They're only partially effective with motion. Anotehr problem your video has is that some of those spots and hairs hang around for multiple frames. If a piece of noise hangs on in the same spot for multiple frames, most filters see it like a "permanent fixture" instead of noise.

I borrowed a trick (OK,I stole it) from a former member. Trick is to filter alternating frames separately, then put 'em back together. That often breaks up the noise, making it look as if it's only in one frame or changes a lot. It removes a lot of the blips. In scenes with less motion, it cleans up more. But you have to accept a little detail loss, as always with strong filters. I got these before/after results from early frames in your sample where things are in constant motion:

Frame 13, before/after:



Frame 22 before/after:



Didn't have time to make a clip from this, needs tweaking and sharpening, and you should add back some fine film-grain with something like AddGrainC. The script below creates 3 clips from one by separating every 3rd frame. Also lowers contrast a bit because the levels here are way out of range for broadcast video and even for PC. You'll have to change the "path" statements to match locations in your system:

AviSource("drive:\path\to\your\video\derby.avi")
av=last
av1=av.SelectEvery(3,0).RemoveSpotsMC()
av2=av.SelectEvery(3,1).RemoveSpotsMC()
av3=av.SelectEvery(3,2).RemoveSpotsMC()
Interleave(av1,av2,av3)
RemoveSpotsMC()
ColorYUV(cont_y=-20,off_y=-7)
return last

Frankly, I always thought that overcleaning old movie film loses a lot. Everyone knows this is old movies so personally I'd leave some of those defects there. Scrubbing it too squeaky clean loses a lot of its on-the-spot authenticity. Anyway, that's IMO.

[manono]

...but no changes the output is the same as the input..so i can't see how this is ever going to work

How are you determining it's not working? How are you comparing input and output? If with the two VDub screens it's already been pointed out to you that they're supposed to be the same. The filtering has already been done before the video hits VDub. In addition to what LMotlow suggested, you could open the script without the filtering in VDub and then open another instance of the script with the filtering applied in a second VDub, go to the same frame in both, and then compare. Then the changes should be obvious.

[LMotlow]

I was about ask the same thing. manono beat me to it. I'll add, though, that there was so much camera movement and object motion it gives despotters a rough time unless you play some tricks with it.

Meanwhile I suspect flash30 doesn't grasp that the two VirtualDub viewing panes are, in this case, not supposed to be different. Matter of fact, if he does see motion in the results pain while the file is being saved, the clip is running in full processing mode and has likely gone through an RGB conversion.

[flash30]

thanks motlow, i am very new to this so am like a fish out of water...with vitrualdub on its own and the filters for that i have played around but its avisynth i need to learn about.
it looks great motlow i have tried the script but again i am getting a error message about RemoveSpotsMC yet i have got an error message again. and i have it in my plugins folder

[manono]

i have tried the script but again i am getting a error message about RemoveSpotsMC yet i have got an error message again. and i have it in my plugins folder

You have the AVS in your plugins folder. So, to be able to use it you have to import it into your script as well. Did you do that? For it to load automatically you have to rename it as an AVSI. You've already been told that so pay attention.

.avs" plugins don't load automatically when a script is run. They must be explicitly loaded with the Import() function:

AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveDirtMC.avs")
RemoveDirtMC()

If you want an .avs file to load automatically, change its name from "RemoveDirtMC.avs" to "RemoveDirtMC.avsi". avsi and .dll filters load automatically.

[LMotlow]

@manono,-- yep, flash30 should use one method or the other. Either keep the original .avs name and use Import(), or change the .avs to .avsi. Back in post #13 I suggested that RemoveDirt and RemoveSpots .avs files should just remain as avs files and imported as needed, because there are so many versions of those two plugins that duplicate those functions internally.

In any case, I ran that RemoveSpotsMC script using RemoveSpotsMC.avs, not the "avsi" name. So to recap that script, if RemoveSpotsMC.avs is what flash30 has in his plugins folder, then the entire script would be this:

Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")

AviSource("drive:\path\to\your\video\derby.avi")
av=last
av1=av.SelectEvery(3,0).RemoveSpotsMC()
av2=av.SelectEvery(3,1).RemoveSpotsMC()
av3=av.SelectEvery(3,2).RemoveSpotsMC()
Interleave(av1,av2,av3)
RemoveSpotsMC()
ColorYUV(cont_y=-20,off_y=-7)
return last

Something else I just noticed about flash30's image of his plugins folder: the plugins contain multiple versions of the RemoveGrain support files. In particular, SSE3 versions contain bugs. This is the part of the earlier image that shows the plugins folder:



RemoveGrain documentation and discussion threads discuss how the multiple versions often conflict with one another. I'd suggest using only the original versions of those dll's.

@flash30, You have these RemoveGrain versions in your plugins folder:
RemoveGrain.dll
RemoveGrainS.dll
RemoveGrainSSE2.dll
RemoveGrainSSE3.dll

Keep RemoveGrain.dll. Remove the other "RemoveGrain" dll's.

You have these Repair versions in your plugins:
Repair.dll
RepairS.dll
RepairSSE2.dll
RepairSSE3.dll

Keep Repair.dll. Remove the other "Repair" dll's.

You don't need RSharpen. But if you have to keep one version, just use the original "Rsharpen.dll".

Most Avisynth plugins aren't this troublesome. It just so happens that you're working with a couple of specimens that went through a lot of development and alteration, and a lot of smart people came up with all kinds of "improvements". Unfortunately they weren't interested in naming these versions more conveniently for us trusting users.

The RemoveSpotsMC script removes most of the pops and hairs, etc. You'll never get rid of 100% of them without going frame by frame through thousands of frames and applying special techniques. As I said, this is old movie film, so people won't be surprised by a few remaining old spots. Try the script on your short sample. If you want to know how to tweak the results, or how to sharpen it and make it look less "filtered", we can pursue that later.

[flash30]

i am giving up..it won't work i have RemoveSpotsMC what i downloaded from you and its giving me an error could not open RemoveSpotsMC..i will try a changing to avsi but i don't think it will work maybe its my pc i do not know..but i have tried all that you tell me and nothing works

[LMotlow]

Hm. The image you posted shows that you have RemoveSpotsMC.avs in your plugins folder located at C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs . You had it running earlier with no errors. So something in your script or your plugins folder has changed since that earlier post. Now you get a different error message that your script "cannot open" that avs with the import function. At this point you really need to post the entire text of the script you used. Seriously. The way you typed the contents of the error message in the above post is stated as "could not open RemoveSpotsMC." . That error message text ends with a period ? ? ? It should say "could not open RemoveSpotsMC.avs", which should include the full name of the file it couldn't open, not just the front part of the name without the "avs" file ending.

[flash30]

i was tired at the time and yes i noticed i had lost removespotsMC but i re-download it again and still don't work i will have last try today if no luck i will call it a day

[flash30]

has you can see with just using this basic script...i have removedirtMC.avs yet it still will not load it...somehow i have lost repair.dll and i can not find on the web to download it. but i do not think it would make a difference because i will still get errors no matter what i do..even if i have all the .dll's .avs and right script i will still get an error message.
AviSource("derby.avi")
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveDirtMC.avs")
RemoveDirtMC()

[poisondeathray]

RE: "Invalid arguments....blah blah"

It's not clear which version you are using, but several versions of RemoveDirtMC (there are dozens of different variants) require the strength value, and false

eg.

AVISource("derby.avi")
RemoveDirtMC(30,false)

[flash30]

finally its working, with:
Import("C:\Program Files\Avisynth 2.5\plugins\RemoveSpotsMC.avs")

AviSource("derby.avi")
av=last
av1=av.SelectEvery(3,0).RemoveSpotsMC()
av2=av.SelectEvery(3,1).RemoveSpotsMC()
av3=av.SelectEvery(3,2).RemoveSpotsMC()
Interleave(av1,av2,av3)
RemoveSpotsMC()
ColorYUV(cont_y=-20,off_y=-7)
return last

i downloaded repair.dll what i will do save the filters i have so i know this is what i need when i am back in uk....and i can fall back on these if i lose them
when i return to uk i will be doing a much better print then this of this film has the original is quite good for its age...but while i have the free time i thought i was use this time to try and learn how to use this damn thing.