8mm Film Restoration

[TekVilla]

I have a major project going transferring several hundred 400' reels of 8mm and Super 8mm film to digital. I've got the wonderful Retro-8 scanner from Roger Evans and the RetroScan HD software, and this does a great job of scanning the films. I export the scans as Image Sequences (jpg or png) and open these in VirtualDub. I then use Neat Video to Denoise the frames and am getting excellent results.

My issue is that I would like to despot the sequences too. Being old and delicate I am reluctant to do much in the way of manual cleaning before scanning, and in any case much of the dirt cannot be removed this way having become well embedded over forty years. I have downloaded VideoFred's Film Restoring scripts and after much head scratching finally figured how to get VirtualDub to run them and again the results are good.

What I can't do is figure out how to edit the scripts so as to remove the 'before and after' comparison built into them. Obviously my requirement is simply to process the sequence so as to finish up with a single, cleaned, full frame image with no text overlay. If I comment out what I think is the relevant section the script fails, either with a message advising the line, or it just fails completely.

Can someone please advise me on how to do this? I'm really tearing my hair out over this, and I don't have much of it to spare. Many thanks.

[jagabo]

Look for a StackHorizontal() and remove it. Return only the "after" stream. Ie, look for something like:

StackHorizontal(video1, video2)

and change it to

return(video2)

You'll probably also want to remove the Subtitle() that labels the image.

[TekVilla]

Thanks for the quick response, at least it confirms I was working on the correct part of the script. There are in fact seven StackHorizontal entries which also contain the subtitle entry. I'm just not sure where to enter the new bit of code. Should I replace all the entries with just one instance of the new bit, or should I make seven entries?

I did experiment a bit and tried it both ways. The script failed both times and gave the error message 'I don't know what Video2 is'. I've attached a grab of this and also the relevant section of the script. Thanks again.

[jagabo]

My use of "video1" and "video2" was symbolic, not literal. Ie, StackHorizontal() was stacking two videos side by side. One was the original (symbolicly video1), the other was a processed video (symbolicly video2). You need to return that second video, whatever it's named. That part of the script sets up several comparisons. But the names of the processed videos are result1, result2, result3, and result4. So put

return(result1)

just before that part of the code. Or result2, result3, or result4, whichever you want.

[TekVilla]

Thank you. It seems to be working now. You have no doubt gathered I am completely new to scripting. I'm now going through the rest of the script and playing with the numbers to see the effect and can readily see it changes the results, so I think I'm on my way. Many thanks.

[manono]

If you're new to all this I'd say you're doing pretty well to get the script to run at all. Many that are much more experienced with AviSynth have found it very daunting. Good going.

[TekVilla]

What a nice thing to say, thank you. I have been involved in video production since the days of open reel in the early 1970s, so maybe that helps.

[2Bdecided]

I doubt you want to run VideoFred's script as-is anyway. It includes fixes for almost every problem your film could have. Your film probably doesn't have many of these problems and will be made worse by using them. Even on your small screen cap, it looks like it's unnecessarily cropping (as part of the stabilization - does your footage need stabilization?), changing the levels (to turn black into dark grey, and white into light grey), and (hard to see) probably sharpening. Did you need any of this stuff?

I think you need to understand the script so you can use just the parts that you require, and check the output after each part to see how well it's working in isolation.

VideoFred's script is a very complicated place to start trying to understand AVIsynth. If you want to get to the point where you can understand it, it's probably best to start with something simpler. e.g. here:
http://avisynth.nl/index.php/Main_Page#Usage

It will be time well spent, and save you time in the long run.

btw, if you edit AVIsynth script files using AVSedit or AvsP, the scripts+commands will be colour coded, which makes it easier to follow than in if you edit using Notepad.

Good luck!

Cheers,
David.
P.S. save your original captures - or at least lossless copies of them.

[TekVilla]

Thanks for your input on this. You are quite correct; I don't want to use all of VideoFred's script. My main requirement is for the dirt removal function, and it does seem to be doing this very well, although at present I have a very aggressive setting in place for this, but then the films are very dirty. As for the other things you mention I am going through the variables and tweaking them to see the effect. In most cases I am returning them to the default, i.e. Gamma back to 1.0, Black to 0, and White to 255.

I do take your point about understanding the scripting. The problem here is that, although I have looked at the Useage page many times and created 'simple' scripts from the examples shown, I have never (before yesterday) got any of them to run in VirtualDub. I feel sure that now I've got something running I will soon make progress with understanding the principles.

One thing I have noticed is that sometimes there is a space after an = sign, and sometimes not; here for example:

saturation=1.3 #for all outputs

gamma= 1.0 # for all outputs .

Is this significant and if so which should it be?

No need to worry about the preservation of the originals. RetroScan HD saves all scans into a protected folder and file format. These cannot be edited (although they can be deleted) and I therefore always have access to the original raw footage. I have to say the Retro-8 workflow has been very well planned and is very easy to run.

[jagabo]

My main requirement is for the dirt removal function, and it does seem to be doing this very well

Try RemoveSpots() or RemoveSpotsMC().

link: https://forum.videohelp.com/threads/363894-De-interlacing-Is-My-Workflow-Right?p=231634...=1#post2316342

example: https://forum.videohelp.com/threads/364731-How-can-I-remove-the-dirt-and-green-spots-fr...=1#post2323154

sample usage:

AviSource("filename.avi")
RemoveSpotsMC()

Be careful with them though. They sometimes remove spots that are supposed to be there. Especially in cartoons.

One thing I have noticed is that sometimes there is a space after an = sign, and sometimes not; here for example:

saturation=1.3 #for all outputs

gamma= 1.0 # for all outputs .

Is this significant and if so which should it be?

There's no significance. The result is the same whether there's a space or not.

As a beginning AviSynth user this may help you: when you don't explicitly name a stream the name "last" is used instead. So the above script is equivalent to:

last = AviSource("filename.avi")
last = RemoveSpotsMC(last)
return(last)

You can also explicitly name a stream. So the script is also equivalent to:

video = AviSource("filename.avi")
RemoveSpotsMC(video)

In this case the output of RemoveSpotsMC(video) is named "last" and in the absence of an explicit return() "last" is returned (output to the editor/encoder).

[TekVilla]

The Retro-8 is a marvelous bit of kit. The only problems I have had with it have been due to me making an incorrect setting, the machine itself has run completely reliably. Very early on I was having a problem with jitter and spoke to Roger Evans about this as I thought it might have been a fault. It turned out to be my fault for setting the pressure clip at too high a pressure. This seems to be the only setup item that is likely to be tricky, as getting the very light pressure required is not easy, although it is simple, if you see what I mean.

In short, I have had my Retro-8 for almost a year and have now run several thousand feet of film through it without any difficulty, once I mastered the pressure clip. It can be run completely unattended if you are confident you footage is good quality. I do find an occasional check on progress is useful as sometimes the racking goes out if the film wasn't threaded correctly in the camera or if a splice is slightly out of true. When it reaches the end of a roll the device stops automatically and turns off the lamp, and the software opens a dialog box for you to name and save the scan. It really couldn't be easier.

Roger Evans emailed me recently announcing a new high speed version of Retro-8 which will, apparently, scan a 400' reel in 30 minutes instead of 4 hours. More info here http://www.moviestuff.tv/moviestuff_home.html

I really can't recommend this unit highly enough. Let me know if you decide to get one.

[2Bdecided]

the films are very dirty

Can't they be physically cleaned (i.e. in the real world!) before you transfer them?

Professionals have ultrasonic film cleaners, and wet gate transfer techniques for hiding scratches. Amateurs have a bit of film cleaning solution and a lint-free cloth.

For most people the digital alternatives are more readily accessible than those professional approaches, but the digital clean-up is not always without its side effects. It usually works miracles though. The problems it introduces are not always easy to spot.

Cheers,
David.

[videofred]

Hi TekVilla and everybody else here,

The whole idea of my script is to change the different results with notepad or even faster, with the build-in Avisynth editor in VDMod.

For some reason, some people are having problems with selecting the results and removing the double screen.
It's simple: "resultS1" will output a stacked before/after file.
But "result1" will output the modified file only.
So, you only have to remove the "S".

As for dirt removing: you can use RemovedirtMC() as a stand alone, outside my script.

Simple:
Avisource(yourfile)
RemoveDirtMC(25) Where 25 is the strength, set it higher for more cleaning.

many greetings from Flanders, Belgium,
Fred.

[minimax]

Since approx. 10 years I've been digitizing my N-8 and S-8 movies from the 1960's - 80's,
using a converted Elmo-FPE double standard projector, running at 4 fps, and a Canon MVX-25i camcorder, firewire connected to my computer, using stop-motion for the conversion to AVI. (Off topic: at the time, lipsync sound was recorded on a camera-coupled cassette recorder which is now being re-united with the film via Adobe Premiere).
I'm pretty sure (cannot prove it) that the resulting avi's are progressive: there are no combing effects and each click in VirtualDub gives a new complete frame.
The hardware setup is fairly similar to VideoFred's and I have been quite satisfied with the results. That is: until I saw the absolutely fantastic results Fred has achieved with his Avisynth script !

Needless to say that, being a total newbie, I tried to train myself in using Avisynth and VirtualDubMod as a GUI and script editor. VDM works fine and appying filters is no problem.
Avisynth however, is an entirely different story: 6 weeks of absolutely fruitless efforts. This is what I did, among many other things:


I made a 15 sec. test movie called "Carnaval-0.avi" using VDub to convert the speed to 18fps and to make the necessary horizontal flip.
In Avisynth, simple script commands work allright e.g.
AviSource("D:\AVISYNTH\Exercises\Carnaval-0.avi", audio = False)
fadeout(20)

However, as soon as I try to load a plugin, I get nothing but error messages.

Example 1
Copy/pasting Fred's 2nd script, Blending, (properly adapted to my avi, I think) resulted in this message:

Avisynth open failure: LoadPlugin: unable to load "plugins/Deflicker.dll", error=0x7e (D:\Avisynth\Exercises\Carnaval-0, line 132)

Whatever I did in removing the next plugin or the next, I got a similar message.


Example 2
Loading just one of Fred's plugins

AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi").
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\Fred_restore\scripts\plugins\ RemoveDirt.dll")

causes this error message:

Avisynth open failure: LoadPlugin: unable to load " C:\Program Files (x86)\AviSynth 2.5\Fred_restore\scripts\plugins\ RemoveDirt.dll", error=0x7e
(D:\ D:\AVISYNTH\Exercises\Carnaval-0.avs, line 2)


Example 3
Loading a plugin from the Avisynth plugin file raises again another message

AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")

Error message:
Avisynth open failure:
The script's return value was not a video clip, (Is the undefined value).


Transferring everything to the same C:\ directory did not help:

AVISource("C:\Program Files (x86)\AviSynth 2.5\Carnaval-0.avi")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\MSharpen.dll")

as the same error message appeared again:
The script's return value was not a video clip, (Is the undefined value).

Obviously I am doing something terribly wrong and consequently I have 3 questions:

1. What should I do?
2. Where can I find step-by-step instructions on what to do in loading plugins?
3. Is Avisynth compatible at all with Windows 7 Pro 64-bit ?

Many thanks!

[johnmeyer]

Is Avisynth compatible at all with Windows 7 Pro 64-bit ?

I do all my work on an XP 32-bit computer, so I don't know for sure. However, many of the plugins for AVISynth have not been re-written for the 64-bit environment.

[jagabo]

Example 3
Loading a plugin from the Avisynth plugin file raises again another message

AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")

Error message:
Avisynth open failure:
The script's return value was not a video clip, (Is the undefined value).

Reverse the order of the two lines.

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")
AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")

[minimax]

Thank you johnmeyer, for your speedy comment!

Would there be a way to work around it or would it mean I have to fire up my old and slow XP-computer again?

Furthermore, Avisynth does come in a 64-bit version, but I seem to remember that during the installation of Avisynth I was explicitly advised to install the 32-bit version..

Would VideoFred's script run on a Avisynth 64-bit PC?

Sorry for all these questions, but it is pretty confusing.
Thanks again,
minimax

[jagabo]

32 bit AviSynth works just fine under 64 bit Windows. The 32 bit and 64 bit A/V systems are completely independent in Windows. You must use all 32 bit components when using 32 bit AviSynth: 32 bit AviSynth, 32 bit filters, 32 bit editor/encoder.

[minimax]

Thank you jagabo!

I reversed the order as suggested and now the avi runs without error messages.
However, the plugin does not seem to sort any effect. I hope you may have another suggestion to get it working.

Thanks as well for your comment regarding the 32/64 bit question. I have installed 32 bit versions of Avisynth, VDMod and AVSEdit. But how can one see whether a plugin is 32 or 64 bit?

Regards, minimax

[jagabo]

However, the plugin does not seem to sort any effect. I hope you may have another suggestion to get it working.

Loading the plugin simply makes it available. You have to call it to to use it:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")

AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")
RemoveDirt()

But AviSynth automatically loads plugins in its plugins folder. So you shouldn't need the LoadPlugin() line.

But how can one see whether a plugin is 32 or 64 bit?

You have to make sure you download the right version. If a site doesn't specify which it is -- it's probably 32 bit. I've never tried it, but I suspect you'll get an error if you try to load a 64 bit plugin into 32 bit AviSynth.

[minimax]

Thanks jagabo. Clearly, this is the type of advice I have been missing in the endless tutorials & threads I have been reading..
Nevertheless, it does not work yet. This is the code I saved:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")
AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")
RemoveDirt()

And this is the error message I got in return:



It seems I am missing something and I shall be looking forward to your further advice.

[manono]

Apparently your AVI isn't in either YV12 or YUY2. So, after the AVISource line add:

ConvertToYV12(Interlaced=True) #if it's interlaced. If it isn't, leave that part out.

[jagabo]

And this is the error message I got in return:

[Attachment 28216 - Click to enlarge]

It seems I am missing something and I shall be looking forward to your further advice.

So you are using a DV camcorder? And this is a DV AVI you are opening? PAL DV is natively YV12 so the decoder should be able to output YV12. There are some DV decoders that only output RGB, like Panasonic DV Codec, and those should be avoided. You want to use a DV decoder that can output YV12, like Cedocida.

Instead of manono's suggestion to use ConvertToYV12() try changing the AviSource() line to

AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi", pixel_type="YV12")

Then see if the script works.

You want to avoid conversion to RGB until you've verified that the YUV levels are legal. Otherwise super darks will be crushed and super brights will be blown out.

[minimax]

Thank you both, manono & jagabo.
Changing the code to this:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")
AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi", pixel_type="YV12")
RemoveDirt()

raised the same error message as before ('only YV12 and YUY2 are supported'):

Folowing manono's suggestion, as follows:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")
AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi")
ConvertToYV12() #if it's interlaced. If it isn't, leave that part out.
RemoveDirt()

caused this warning to appear in VDubMod:


I assume the clip is not interlaced, so I left the relevant part out. Leaving it in results in the same warning. How can I make sure whether it is interlaced or not?

Media Player could not handle it either and suggested that the required codec is not installed.

[jagabo]

Use VirtualDub instead of VirtualDubMod. It has internal support for YV12.

Also, if your video is interlaced (and DV usually is) you may need some special handling to keep the two fields separate.

[minimax]

Jagabo, sorry I forgot to reply to your question: yes, for digitizing my 8mm films I use a DV Camcorder (Canon MVX25i) and a stop motion application called Cinecap. At approx. 4 fps it builds an avi from a single shot of each movie frame. As far as I know the resulting avi is non-interlaced, but again, I'm not sure. I agree DV usually is interlaced, but this one does not seem to be.
Would you know a way to find out wheter the above avi is interlaced or not?

Meanwhile, I downloaded the Cedocida DV Codec 0.2.3 and installed "cedocida_dv_only.inf".
Regretfully, it does not make any difference, whether in VDubMod or in VDub, I still get the message: 'only YV12 and YUY2 color spaces are supported'.

[jagabo]

DV is usually interlaced but since you are using a very slow projection the two fields of each video frame are from the same film frame. So you can treat those as progressive. Presumably there will be some video frames that have fields from two different film frames (when the projector stepped from one film frame to the next) but you'll be getting rid of those.

Regarding Cedocida, go back to using the pixel_type="yv12" argument to AviSource. If that doesn't work go to Cecodida's configuration dialog and disable all outputs except YV12.

[minimax]

Indeed it looks like a progressive avi: if I click my way from keyframe to keyframe in VDub, the counter registers 1 frame for each click. No artefacts, no combing.
OK, I went back to this code:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirt.dll")
AVISource("D:\AVISYNTH\Exercises\Carnaval-0.avi", pixel_type="YV12")
RemoveDirt()

Then I tried to find the Cedocida dialog box. I have seen it on the Cedocida site so I know what you mean, but I am unable to locate it on my computer. Where should I look?

[jagabo]

Then I tried to find the Cedocida dialog box.

Start VirtualDub, select Video -> Compression, highlight Cedocida, press the Configure button, select options...

[minimax]

Continuing my internet search I finally found the Configuration dialog box. Did not realize it is part of VDub!
I changed the settings to those in te picture below. Are these correct?
Since I am still getting the same error message:
'only YV12 and YUY2 color spaces are supported'