8mm Film Restoration

[minimax]

Thanks jagabo, but what should I do with it?
And what about MSVC runtimes version 7, 8, 9 or 10?
Could you elaborate a little on both issues?

[jagabo]

sse2tools.dll is an AviSynth dll. So you do the same thing with it as all the other AviSynth dlls. Put it in your AviSynth plugins folder or import it manually.

Attached is a collection of the MSVC runtime libraries if you need them. They go in \Windows\System32 (32 bit Windows) or \Windows\SysWOW64 (64 bit Windows). I recommend you only use the ones you need. AviSynth will tell you when you try to use a filter that requires them.

[johnmeyer]

I'd suggest downloading and running the AVSInfoTool:


AVSInfoTool127.zip

[johnmeyer]

Here are selected plugin entries I got when I just ran the AVISynth Info Tool:

[General Info - Avisynth.dll]
Version String:  AviSynth 2.60, build:Mar  9 2013 [13:28:27]
File Version:    2.6.0.3
Directory:       C:\WINDOWS\system32
Timestamp:       March 09, 2013, 00:28:48
MT Support:      Yes

[Plugin Directories]
C:\Program Files\AviSynth 2.5\plugins

[Supported Color Spaces]
YUY2, Y8, YV12, YV16, YV24, YV411, RGB24, RGB32

C:\Program Files\AviSynth 2.5\plugins\RemoveDirt.dll
  Timestamp:         May 07, 2005, 12:26:34
  MD5 Hash:          6857D2FD3F72B93CAD2F2CD8E2F02506
  DLL Dependencies:  KERNEL32.dll, MSVCR71.dll
  Plugin version:    AVS 2.5 Plugin
  No VersionInfo available


C:\Program Files\AviSynth 2.5\plugins\RemoveDirtSSE2.dll
  Timestamp:         May 05, 2005, 21:16:08
  MD5 Hash:          5BD6940938EA67A827F81560443EB3A1
  DLL Dependencies:  KERNEL32.dll, MSVCR71.dll
  Plugin version:    AVS 2.5 Plugin
  No VersionInfo available

[minimax]

Re: #149 Thank you poisondeathray, I'll give it a try, it may come in handy.

Re: #150 & #152 jagabo
Yes, I saw it is a dll and I put it in the plugin folder together with the johnmeyer dll's mentioned in post #148.
However, it is not represented in johnmeyer's script, so how does it work and what does it do?

MSVC runtime libraries
I have a Windows 7 64bit computer and Avisynth2.6 32bit.
In the zipfile I found 8 different dll's.
1. Avisynth has not reported anything in this respect, so how do I know whether I need them at all?
2. If so, how do I know which ones I need?
3. What should I do with them? Where to put them?
Could you please elaborate a little? Would be most grateful, since I cannot quite follow due to my still limited knowledge.


Re: #153 & #154 Thanks John,
There are two downloads:
1) AVSInfoTool127.zip and
2) AVSInfoToolSetup127.exe
I ran the exe file (not the SetUp file) and this looks like a portable application. Is that correct?
And is it the right choice?

It gave me only this:




Plus a list of 114 External functions.
It seems there is a lot missing in comparison with the code you posted. What should I do next?

[johnmeyer]

Download the zip and run the EXE in that zip. Then, look at the menu options. You'll find additional displays in those menus.

[minimax]

Re: #156 Thanks John. The ''main screen" is exactly the same as what I pasted in #155.
However, under Tools, Plugin info, I found the full report which is attached to this post.
Could you please let me know what I should do next?

[johnmeyer]

Sorry, I don't know what to suggest.

[minimax]

Re: #158 johnmeyer
Hmm, but why did you suggest to run the program in the first place?

[Groucho2004]

Could you please let me know what I should do next?

Strangely, you didn't mention the plugin load errors you got when you opened the pugin info dialog. Look at the bottom of the log, all the errors are there. You are clearly missing MS*71.dlls.

I would however suggest that you use RGTools.

[minimax]

Re: #160 Groucho2004
Sorry, I think I did mention the LoadPlugin error I got, when trying to play the script.
I remember also seeing a number of error warnings after starting up AVSInfoTool, but I could not reproduce those. Is that what you mean?
Indeed I saw the errors at the bottom of the log, but I don't know what to do with them.
Would you know what I should do next in order to get the johnmeyer script running?

By MS*71.dlls I assume you mean MSVCP71.dll, MSVCR71.dll and the likes?
If I don't have these things on my PC, which apparently should be there:
- where can I get them ?
- how should I use/install them? and
- in which order?

May be I am asking to much, but I really am in need of full instructions.
Please consider that I have been working to get Avisynth running since October last year.
I am an old man trying to rescue some 30 rolls of super-8 for his (grand) children.. Have mercy

[johnmeyer]

Re: #158 johnmeyer
Hmm, but why did you suggest to run the program in the first place?

Because I thought it would reveal that you were using either an incompatible version of AVISynth, or a plugin version other than the ones that work with the version of AVISynth you are using.

[johnmeyer]

I am an old man trying to rescue some 30 rolls of super-8 for his (grand) children.. Have mercy

I too am north of 60.

Here is a zip file that contains everything in my AVISynth folder. You can create a new folder on your computer and unzip the contents of this file, making sure to do the unzip so that all the folders are preserved. Then, take the AVISynth DLL that is in the root folder of the zip, and put that into the Windows\System32 folder, or wherever it is supposed to be on your computer. I am running Windows XP Pro 32-bit, so on my computer it is in C:\Windows\System32.

This zip also contains the msvcr71.dll that is required by RemoveDirt. This should be put in the same location as the AVISynth DLL.

I recommend you read the documentation for ALL of these plugins, or at least read the introductions where the author always provides information on that plugin's dependencies. If you read the documentation for RemoveDirt, what you'll find is this sentence:

If it doesn't work, this is probably due to missing msvcr71.dll library. Either you install this library in C:\windows\system32 or you delete RemoveDirt.dll and replace it by RemoveDirtS.dll.

There are many plugins that require other DLLs or scripts in order for them to work.

So, in conclusion, the point of supplying this zip is that it gives you everything I have, in exactly the same locations where they reside on my computer. If you still can't get it running, then you are either missing some other DLLs in the System32 folder, or you have something else that is interfering.

Meyer AVISynth.zip

[LMotlow]

By MS*71.dlls I assume you mean MSVCP71.dll, MSVCR71.dll and the likes?
If I don't have these things on my PC, which apparently should be there:
- where can I get them ?
- how should I use/install them? and
- in which order?

The msvclib dlls were attached to post 152 by jagabo 2 days ago. You need the files with '71' in the file name. They are 32-bit dll's, so copy them into C:\Windows\SYSWOW64. You might not need to do any more than just copy them. If you get the same dependency erros in the AVS Tool checker, let us know. They don't belong in any particular order.

The '71' runtime libs are not installed with Windows 7. They would be installed if you had .NET Framework 1.1, but .NET 1.1 can't be installed in Windows 7. All you need here are the '71' dll's.

The RemoveDirt functions require RemoveDirtSSE2.dll. You appear to have multiple versions of RemoveDirt dll's in your plugins. You only need the "SSE2 version of the dll. If you have any other "RemoveDirt" dll's in your plugins folder, remove them. If you have RemoveDirt avs or avsi files, leave them as-is.

RemoveDirt also requires the RemoveGrain plugins, or else the RgTools plugin mentioned earlier, which is another version of the RemoveGrain functions. If you don't have the RGTools version, then you need RemoveGrain v1.0b . Do not download the RemoveGrain RAR package to your Avisynth plugins folder. Instead, download it to another subfolder somewhere else, because the RAR package contains some files that you won't need. Decompress that RemoveGrain package into the download location. The only RemoveGrain dll's that you should copy into your Avisynth plugins folder are:
- RemoveGrainSSE2.dll
- RepairSSE2.dll

[jagabo]

By MS*71.dlls I assume you mean MSVCP71.dll, MSVCR71.dll and the likes?
If I don't have these things on my PC, which apparently should be there:
- where can I get them ?
- how should I use/install them? and
- in which order?

I already gave you a ZIP file with all those dlls and told you where they go. The order doesn't matter.

[minimax]

[ Because I thought it would reveal that you were using either an incompatible version of AVISynth, or a plugin version other than the ones that work with the version of AVISynth you are using.

Right, may I assume that both are OK?

[minimax]

Re: #163 johnmeyer
Good to know you're in my league, although at just 'south' of 80 I don't pick up new things as quickly as 10 years ago..
Many thanks for your understanding though, which is clearly reflected in your extensive advice!
I shall start doing my homework now and hopefully revert later today.

Meanwhile I found that Windows 7 Pro is riddled wich all sorts of safety measures. Purely by accident I lately discovered by rightclicking a dll, that it was blocked by Windows "because it emanated from a different computer".
So I had to de-block manually all imported dll's, which I did. I am afraid W7 may contain more of this nonsense.

Be happy you are still using XP! I do have a Virtual XP version on my computer but it will probably be 'protected' in the same way..

#164 Lmotlow & #165 jagabo
Thank you both, things are slowly dawning on me and I am getting to work right now.

[minimax]

Re: #163 johnmeyer
Thanks for the zip file. This is what I did, thus far:

1. I unzipped your file and copied both AVISynth DLL and msvcr71.dll to C:\Windows\System32 i.e. the same in Win7. (Both were indeed missing there).
Result: the script remains stuck at RemoveDirtSSE2.dll

2. Then I realized that, apart from the plugin-dll's, also these could be blocked by Win7. And yes, they were.
This is what it looks like after rightclicking the dll >> Properties:



Translation:
"Security: This item emanates from another computer and is possibly blocked in order to better protect this computer."
The red arrow points at a button called: "Unlock"
After clicking the button, Apply & OK I checked whether this had done the trick. Sadly not.

Now, your file contains 169 items. This would mean that after transfer to my plugin folder, I would have to unlock them one by one…
Do you or DOES ANYONE KNOW A SOLUTION TO THIS WINDOWS-7 PROBLEM ?

3. Otherwise, could I limit copying dll's to those mentioned in your script?

4. Your file contains 7 folders called CNR, Decomb etc. …..Srestore. Should I copy those too?

5. I see that you have Avisynth 2.5 installed. I run 2.6.
Would it be wise for me to uninstall 2.6 and install 2.5 instead?
If so, how can I make sure I get the 32-birt version? The Sourceforge site is not clear about this.

I shall revert later to the remainder of your most interesting comments!

[jagabo]

1. I unzipped your file and copied both AVISynth DLL and msvcr71.dll to C:\Windows\System32 i.e. the same in Win7. (Both were indeed missing there).
Result: the script remains stuck at RemoveDirtSSE2.dll

Once again, 32 bit dlls go in C:\Windows\SysWOW64, not C:\Windows\System32 (yes, this naming convention is confusing). You should not be running 64 bit AviSynth -- most third part filters are not available in 64 bit versions. If you have 64 bit AviSynth installed remove it to reduce possible confusion.

Translation:
"Security: This item emanates from another computer and is possibly blocked in order to better protect this computer."

I'm running 64 bit Windows 7 Pro and I don't have this problem. Are you running from an account with Admin privileges? Maybe you can change the attributes or security settings of the entire AviSynth folder tree?

[johnmeyer]

+1 Jagabo.

[minimax]

Gentlemen, today I did the following.

I spent most of the day trying to find a solution to the 'locked dll' problem, but did not succeed. Unless I would start fooling around in the registry, which I considered unwise. Therefore I choose to install the dll's John supplied so graciously, as they will be required…

In C:\Windows\SysWOW64 I replaced the existing avisynth.dll (391kB) by the 1.703kB version I got from him as well. Furthermore I added msvcr71.dll.

Subsequently I unpacked jagabo's mslibs.zip and copied (and unlocked!) 6 out of 8 of them to SysWOW64. Two exceptions:
msvcr71.dll of course was already there
msvcr100.dll (809kB) was not used since an older 756 kb version was already present. Was that a wise decision? Or should I better replace it by the newer(?) and larger one?

Apparently I made some progress since the 1986-test 1e JMscript-3.avs script now gets stuck at this error:




I copied the fftw3.dll from John's file to my plugin directory, but I am still getting the same message.
Any clue? I shall pursue the other points you raised earlier, tomorrow.
Meanwhile I shall be pleased with any comment you may care to make!

[jagabo]

fftw3.dll goes in your SysWOW64 folder.

Regarding unlocking dlls, can you select multiple dll files and unlock them all at once?

[minimax]

fftw3.dll goes in your SysWOW64 folder.

Regarding unlocking dlls, can you select multiple dll files and unlock them all at once?

Re: 172 jagabo

1. Oops! This is confusing. The more so since johnmeyer sent me this one in a large folder (169 items + 7 folders, see my post #168) together with filters like RemoveDirt and the like, which apparently is his plugin folder.
So, how could one tell where a dll should go? Are there any more in johnmeyer's list that should go into SysWOW64?

2. Unlocking dlls: this is the first thing I tried. However, as soon as you select more than one dll you lose the security tab in Properties. Hence, I have to unlock the dlls one by one. It seems to be the only way, apart from fiddling with the registry..

3. After copying fftw3.dll into SysWOW64: GENTLEMEN, IT WORKS !!.
I now have a reproduction that equals the one in johnmeyers post #109
FANTASTIC! Many thanks to him and to all of you!

4. Now that the script works I would like to use it, of course, on my 30 rolls of Super-8 and again I have some questions, if I still may..

a. How to adapt the script to single display?
b. How to cancel the subtitle?
c. It seems that some cropping has been done. Could that be undone?
d. Which are the parameters I should vary in order to adapt
- Dirt Removal
- Scratch Removal
- Color
to this specific film stock?

The gain in sharpness is far better than I had expected and, probably, I better leave it as it is.

[jagabo]

a. How to adapt the script to single display?

Change the line that reads:

result="resultS4"                                 #specify the wanted output here

All the "S" results show the original and the filtered video side by side. For example, to see just result4 change that line to:

result="result4"                                 #specify the wanted output here

b. How to cancel the subtitle?

The above change also removes the captions.

c. It seems that some cropping has been done. Could that be undone?

Change CLeft, CRight, CTop and CBottom. Note that you may be left with junk at the edges of the frame if you change them to 0.

d. Which are the parameters I should vary in order to adapt
- Dirt Removal
- Scratch Removal
- Color
to this specific film stock?

Play with the "strength" parameters in the DENOISEING PARAMETERS section.

[johnmeyer]

Are you using my script or Fred's script?

You are going to lose some material around the borders because of the motion stabilization. If you don't want that to happen, you need to reduce the cropping parameters AND reduce the amount of stabilization.

All the parameters you need to change are at the start of the script, and are describe by the comments in the script. Read those comments.

There is no scratch removal.

Color compensation is best done in your video editing program. The "autocolor" gets it wrong too often, so I normally never use it. If you want to see the "before/after" when the script does autolevels and autocolor correction, choose S1 for result, and change it to 1 when you are ready to create your video and don't need to see the "before" video.

[minimax]

Re: #174 jagabo & #175 johnmeyer

Thank you both for your instructions!

1. Before I forget, and in view of future cases, I really would like to have your comments still to my question in #173, i.e. how could one tell whether a dll has to go into 'plugins' or into 'SysWOW64' ?

2. Yes, I am using johnmeyer's great script! Leaving out the 'before'-screen was no problem. When I subsequently try saving the avs script as an avi however, VirtualDub crashes every time. Nevertheless, the resulting avi's are OK, it seems. Any clue to this strange behaviour?

3. I do understand why for stabilization an adequate amount of cropping is required. In John's script this is done (I think) in lines 22 to 25.
a. In most of my movies I do not need stabilization, so there I would prefer to see the full image.
Question: in order to achieve this, which values would I have to change and to what extent?
b. In other parts, the movie is quite unstable (when someone else held the camera..) and I could use a much higher amount of stabilization.
Since I have no idea of the scale (e.g. is it 1-20 or 1-200?) I would highly appreciate your guidance here.

4. John, in your demo Polavision clips I saw that the dirt has completely disappeared. They really look great. Although in my Dixons material, the script removes a very substantial part of the stains, I would like to make further improvements of course and I hope I may tap some more from your vast experience.
So, could you please indicate which lines (in your script) are concerned and what would be the maximum / minimum limits starting from the values in the present script. Again, I have no idea of the scale.

[johnmeyer]

1. Before I forget, and in view of future cases, I really would like to have your comments still to my question in #173, i.e. how could one tell whether a dll has to go into 'plugins' or into 'SysWOW64' ?

This answer isn't meant to be snippy, but you have to read the documentation for each DLL. There really isn't any generic answer or general guideline. However, 98% of them go into the "plugins" folder.

When I subsequently try saving the avs script as an avi however, VirtualDub crashes every time. Nevertheless, the resulting avi's are OK, it seems. Any clue to this strange behaviour?

I have posted many times in the doom9.org forum that my script can be unstable when you use multi-threading (the "SetMTMode" statements enable this). This is because many of the plugins used (especially RemoveDirt) were created long before the multi-threading kludge was added to AVISynth, and they don't have the requisite code that helps them "play nice" with multi-threading.

Sometimes you can change the stability by changing the SetMemoryMax amount. Try changing from SetMemoryMax(1000) to SetMemoryMax(768) or SetMemoryMax(512). You can also decrease the number of threads. Your computer has to be multi-core (most modern computers are), but each computer has a different number of cores. The Performance tab in the Windows Task Manager will show you how many you have. Try decreasing the number of cores from 6 to some smaller number. Try 3 for starters. This may slow down the script, but it may improve stability.

I do understand why for stabilization an adequate amount of cropping is required.

Any software stabilization program works by moving each frame so that it "lines up" with the adjacent frames. This involves moving the frame up, down, left, and right, as well as rotating and zooming the image. The problem is, when you move the image, say to the left, this leaves nothing on the right side of the frame. All you will have is a black border. Not only that, but as the motion stabilization algorithms move each frame around, you will see a black border moving around. This is horribly distracting and completely unacceptable. There are two solutions. The simply solution is to zoom in (i.e., crop and then re-size) which is what this script does. This always works. A more ambitious way to fix the problem is to steal pixels from adjacent frames and fill in the borders with those pixels. This is a little bit like the video you see on TV where they are showing old 4:3 video in a 16:9 format, and they duplicate and blur the video at the left and right edges.

Stabilization is generally a very good thing to do with film, even when the camera was on a tripod. This is because film never comes to rest in the camera or projector gate exactly the same way as the adjacent frames. Therefore, even a static shot of an unmoving building will bounce around. The motion stabilization can completely eliminate that "gate weave," and the result is much, much easier to view. Night and day, actually.

If you want to reduce the stabilization and cropping to the minimum, where that step in the script only corrects the subtle gate weave motion then set these parameters as follows:

maxstabH=4
maxstabV=4

CLeft=4 CTop=4 CRight=4 CBottom=4

If you start seeing black borders, change the four crop parameters to 8. The crop parameters must be a multiple of four.

I describe how stabilization works in a monograph I wrote almost ten years ago, and which can be found in dozens of locations. Here is one version:

http://dvformat.digitalmedianet.com/article/A-Guide-to-Using-DeShaker-28849

Since I have no idea of the scale (e.g. is it 1-20 or 1-200?) I would highly appreciate your guidance here.

As the comments in the script say, the stabilization parameters are in pixels. 20 is a pretty large number of pixels to move the video, and I wouldn't go much higher than that.

John, in your demo Polavision clips I saw that the dirt has completely disappeared. They really look great. Although in my Dixons material, the script removes a very substantial part of the stains, I would like to make further improvements of course and I hope I may tap some more from your vast experience.

I have a heavily-revised version of the script that I developed specifically for Polavision. I have applied it to a few other clips that had massive chemical staining (large blotches). However, the script does considerable violence to details. There is a recent thread in this forum where I posted examples and several people immeidately noticed that a few details had disappeared. Polavision is such crummy stuff, that the massive improvement in watchability far outweighs the loss of a few details. Here is an example:

[jagabo]

1. Before I forget, and in view of future cases, I really would like to have your comments still to my question in #173, i.e. how could one tell whether a dll has to go into 'plugins' or into 'SysWOW64' ?

AviSynth dlls that you want to load automatically go in AviSynth's plugins folder. System dlls (those that are designed to be used by any program that wants to use them) go in the SysWOW64 folder. The instructions that come with the AviSynth plugins usually tell you if you need to put a dll in the System folder. For example FFT3DFilter:

http://avisynth.org.ru/fft3dfilter/fft3dfilter.html

You MUST put FFTW3.DLL file from this package (not fft3dfilter.dll) to some directory in path (for example, C:\WINNT\SYSTEM32). Filter will NOT work without it!

2. Yes, I am using johnmeyer's great script! Leaving out the 'before'-screen was no problem. When I subsequently try saving the avs script as an avi however, VirtualDub crashes every time. Nevertheless, the resulting avi's are OK, it seems. Any clue to this strange behaviour?

This is a problem when running 32 bit AviSynth with certain filters under 64 bit Windows. One of the plugins has a memory corruption bug.

3. I do understand why for stabilization an adequate amount of cropping is required.

Image stabilization works by shifting the frame up, down, left, and/or right. Something has to be substituted at the edges of the frame. For example, if a frame needs to be shifted 4 pixels to the right something has to fill 4 pixels of the frame at the left edge. Some stabilizers simply fill with black borders. Some try to use other frames to fill the border more intelligently. This script fills the edge with a mirror image of the picture. But these filling methods are imperfect. An easy way to keep odd stuff from appearing at the edges of the frame is to crop the frame and (optionally) resize it back to the original size.

a. In most of my movies I do not need stabilization, so there I would prefer to see the full image.
Question: in order to achieve this, which values would I have to change and to what extent?

Set the cropping values to zero and disable the Depan() calls. Remove or disable all the lines in the STABILIZING section and add a line there that reads "stab = cropped_source" (without the quotation marks):

#STABILIZING
#....................................................................................................................................................................
#stab_reference= cropped_source.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()
#mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
##stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15).deflicker()
#stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15)
stab = cropped_source

b. In other parts, the movie is quite unstable (when someone else held the camera..) and I could use a much higher amount of stabilization.
Since I have no idea of the scale (e.g. is it 1-20 or 1-200?) I would highly appreciate your guidance here.

The comments say the units are pixels but I don't think that's correct. With a shaky video some frames are shifted much more than 15 pixels (the default values). Just play with different values until you see what you want.

4. John, in your demo Polavision clips I saw that the dirt has completely disappeared. They really look great. Although in my Dixons material, the script removes a very substantial part of the stains, I would like to make further improvements of course and I hope I may tap some more from your vast experience.
So, could you please indicate which lines (in your script) are concerned and what would be the maximum / minimum limits starting from the values in the present script. Again, I have no idea of the scale.

RemoveDirt in this circumstance cannot remove spots that appear in more than one frame (it can even add spots if the frame before and after a frame have a spot)*. You're not going get any more spot reduction. You'll just get more damage to the picture. The variable that controls the strength of spot removal is dirt_strength. The higher the value the more spot removal you get. You should set it the lowest value you can live with, ie, find a compromise that removes most of the spots but doesn't adversely effect other parts of the picture.

* This type of spot removal looks at three consecutive frames. If a spot appears in the middle frame but not in the prior or next frame it is considered a spot and is removed by replacing it with pixels from the prior and/or next frame. A spot that appears in two consecutive frames is assume to be part of the picture and isn't removed. This algorithm can produce spots in the middle frame when spots in the prior and next frame overlap.

[minimax]

Thank you johnmeyer and jagabo,
this will keep me busy for a few days!
Meanwhile another question: for a different project I shall be in need of a Scratch remover. The scratches are evidently caused by frequent projection, lengthwise on the film. Is there anything that could be added to the present script or would you know a better solution?

[jagabo]

Meanwhile another question: for a different project I shall be in need of a Scratch remover. The scratches are evidently caused by frequent projection, lengthwise on the film. Is there anything that could be added to the present script or would you know a better solution?

There's a DeScratch filter. I don't know much about it.
http://avisynth.org.ru/descratch/descratch.html