I need help removing spots/scratches in this clip, please help?

[jrodefeld]

Hello,

I am attempting to improve the quality of the Jackie Chan film "Police Story". I am somewhat competent in using Avisynth but I am really bad at writing scripts. This video has a lot of spots and scratches and I want to remove them as completely as possible without effecting the other parts of the video.

I've tried using Despot in AVIsynth with only modest success.

Here is the video clip:

https://www.dropbox.com/s/9riztk9apb4qmn1/Police%20Story%20Segment%20one.m4v?dl=0


What I would REALLY appreciate is if any of you could take a quick look at the above clip and recommend a group of avisynth plugins and a basic script to remove the scratches and spots as much as possible.

Thanks.

[manono]

RemoveSpotsMC3X used in a script like this:

SeparateFields()
E=SelectEven().RemoveSpotsMC3X(0)
O=SelectOdd().RemoveSpotsMC3X(0)
Interleave(E,O)
Weave()

Does a pretty decent job without also removing too much detail. You can follow it up with some light sharpening if you like. It was modified from RemoveSpotsMC by jagabo and a search here will turn up the details.

[jagabo]

The source is progressive -- why the separate field handling? Did it just work better that way?

[manono]

Didn't I find that from something you wrote here? Anyway, the way I figure it, by separating the fields it makes the spots smaller allowing more of them to be removed. Am I wrong about that?

I didn't compare the results with not separating the fields, but that script seemed to work pretty darned well.

[jagabo]

I'm pretty sure that function was originally written for interlaced video, hence the deinterlace/reinterlace surrounding the call to it. Making the spots smaller also makes the real details smaller so I'm not sure it's a good idea. The OP should compare for himself and see which he prefers. Also try using just RemoveSpotsMC().

[BÌG•bµdd]

Hello manono , i am not sure about the artifact but it is a similar like spots and dots or vcr line .i was wondering if i leave a sample here could you please take a look i been reading on some places like here and doom10/doom9 but all those thread boggled my mind many threads i have being reading some of them are missing with filter dependencies so i was never able to get it work in past.but know as i see this thread i thought to post in here maybe i get some help..so pls try to get the solution i use avisynth and time is not a problem for me.
it's looking like the source is 480p with combo of vcr artifacts..
Sample : https://www.sendspace.com/file/xe4whn

[jagabo]

RemoveSpotsMC() or RemoveDirtMC() gets rid of most of the spots. It doesn't help with the splice marks at the top and bottom of some frames. Those are hard to get rid of with automated filters because they're at shot transitions and any filter strong enough to remove them will destroy the rest of the picture. With mostly still shots you can copy the previous or next frame over them. Or if you want to be really creative, copy on the top or bottom of the previous frame over them. Or, since it's only a few lines at the top and bottom of the frame, just crop them away. The clip could also use some stabilization to remove the film bounce.

Mpeg2Source("Dirty clip.d2v", CPU2="ooooxx", Info=3) 
Stab()
Crop(4,70,-12,-74)
TFM(d2v="D:\Downloads\Dirty clip.d2v") 
TDecimate() 
RemoveDirtMC(50, false)

You can get RemoveDirt() and RemoveDitMC() here:
https://forum.videohelp.com/threads/365089-8mm-Film-Restoration?p=2372066&viewfull=1#post2372066
Note that it requires several other filters. You can find them in VideoFred's package.

Try different values for RemoveDirtMC() until you find a setting you like. Too small a value will leave spots. Too high a value will remove some real detail.

The clip could also use a flicker reduction filter.

[BÌG•bµdd]

RemoveSpotsMC() or RemoveDirtMC() gets rid of most of the spots. It doesn't help with the splice marks at the top and bottom of some frames. Those are hard to get rid of with automated filters because they're at shot transitions and any filter strong enough to remove them will destroy the rest of the picture. With mostly still shots you can copy the previous or next frame over them. Or if you want to be really creative, copy on the top or bottom of the previous frame over them. Or, since it's only a few lines at the top and bottom of the frame, just crop them away. The clip could also use some stabilization to remove the film bounce.

Mpeg2Source("Dirty clip.d2v", CPU2="ooooxx", Info=3) 
Stab()
Crop(4,70,-12,-74)
TFM(d2v="D:\Downloads\Dirty clip.d2v") 
TDecimate() 
RemoveDirtMC(50, false)

You can get RemoveDirt() and RemoveDitMC() here:
https://forum.videohelp.com/threads/365089-8mm-Film-Restoration?p=2372066&viewfull=1#post2372066
Note that it requires several other filters. You can find them in VideoFred's package.

Try different values for RemoveDirtMC() until you find a setting you like. Too small a value will leave spots. Too high a value will remove some real detail.

The clip could also use a flicker reduction filter.

Thanks a lot mate.
I am checking dat thread for Dependencies now then i'll do some test latter, your test work nice on spots and scratch but i doubt on TFM+Decimate like that in script won't perform nice i doubt it may cuz to this artifact.
https://forum.videohelp.com/images/imgfiles/IXpgqHL.png?1

[jagabo]

That chroma artifact is caused by treating interlaced YV12 as progressive YV12. This is very often caused by VirtualDub -- it treats all incoming YV12 as progressive. TFM().TDecimate() won't produce that unless you screw up the chroma channels before calling them. Or if the video is screwed up before you got it.

[poisondeathray]

One way to "automate" replacing the "dirty" splices for the scene changes is to use scselect from the original removedirt. If a scenechange is detected, it will replace with the previous frame if before, next frame if after .

.
.
.
source=last

prev = source.selectevery(1,-1)
next = source.selectevery(1,1)
source.SCSelect(next,prev,source,dfactor=2.0)

The dfactor is the threshold for detection, and might need to be adjusted slightly

[jagabo]

Oops. I just realized a made a serious mistake in my earlier script. I had Stab() running before TFM() and TDecimate(). It should run after them.

Mpeg2Source("Dirty clip.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="D:\Downloads\Dirty clip.d2v") 
TDecimate() 
Stab()
Crop(4,70,-12,-74)
RemoveDirtMC(50, false)

There was very little motion in the clip so the problem wasn't too obvious.

[jagabo]

One way to "automate" replacing the "dirty" splices for the scene changes is to use scselect from the original removedirt. If a scenechange is detected, it will replace with the previous frame if before, next frame if after .

.
.
.
source=last

prev = source.selectevery(1,-1)
next = source.selectevery(1,1)
source.SCSelect(next,prev,source,dfactor=2.0)

The dfactor is the threshold for detection, and might need to be adjusted slightly

Nice. I think it could be simplified to:

prev = selectevery(1,-1)
next = selectevery(1,1)
SCSelect(next,prev,last,dfactor=2.0)

or even:

SCSelect(selectevery(1,1),selectevery(1,-1),last,dfactor=2.0)

[BÌG•bµdd]

thanks for your time i got the script working fine now thanks for pointing me right direction jagabo and highly oblige with other members reply as well gonna test them soon.i been trying your above script and i changed a bit in the script but not whether that was right or wrong but as soon as it do clean the image i think it's no problem.here is my tested output result
https://www.sendspace.com/file/sygrgu

DGDecode_mpeg2source("C:\Users\Administrator\Desktop\Dirty clip.d2v", cpu=4, info=3)

ColorMatrix(hints=true, threads=0)
source=RemoveDirtMC(50, false)

prev = source.selectevery(1,-1)
next = source.selectevery(1,1)
source.SCSelect(next,prev,source,dfactor=2.0)
Stab()
crop(4, 70, -6, -66)
Smoothcontrast
mvdegrain
Dehalo_alpha
spline36resize(720,352)
Undot() # Minimal Noise

[jagabo]

Your video has the wrong aspect ratio. It should be 1.85:1. And it has lots of very un-natural looking filtering artifacts.

[BÌG•bµdd]

oops..
jagabo i don't how to get exact 1:85.1 a/ratio (look like i missed it to learn ),mostly i follow the megui returned dar after cropping the black border from the video ,but after encoding the video clip i get a/ratio above 1.90.7 and some other similar values of dar in megui .
sometime i use these dar (below), please can u clear is it safe to use or not?
33:14 1280x544
40:17 1920x816

[jagabo]

You resized to 720x352 near the end of your script. That's about a 2:1 aspect ratio (720/352 ~= 2.045) when using square pixel encoding.

One way to always get the aspect ratio right is to first resize the VOB frame to the correct square pixel aspect ratio then crop away the borders. DVD is always either 4:3 or 16:9. So for 4:3 NTSC DVD resize to 640x480 or 720x540 (or any other 4:3 ratio of width:height). For 16:9 NTSC DVD resize to 854x480 or 720x404 (both of those are approximately 16:9), then crop away the black borders. It's best to keep your final frame size an integer multiple of 4 (aka, mod 4) because some codecs don't like mod 2 or mod 1.

Another easy way to get the aspect ratio correct is to keep the source video's pixel aspect ratio when you encode. Crop away any black borders then encode with a pixel aspect ratio of 8:9 for 4:3 NTSC DVD, 32:27 for 16:9 NTSC DVD. The only problem with this is that some players don't repect the pixel aspect ratio flag so it may not play with the correct display aspect ratio on some players.