Need help with Virtualdub's SpotRemover

[unclescoob]

Hi Everyone,

My current project involves cleaning up some old cartoons from a DVD source. The production company simply did a master tape transfer to DVD without any restoration.

After alot of research and trial/error, I finally discovered what I think is the perfect filter for this project: Spotremover (Virtualdub). The filter works wonders however, there are three issues that I'm currently having trouble with:

1. Ghosting in some scenes: Based on what I've gathered, temporal smoothers cause this (unless there's something else I'm not aware about?). How can I adjust the settings in this filter to completely eliminate this artifact AND keep the spot-removing aspect intact?

2. In scene changes, you can see a few seconds of dirt in the beginning of the scene and suddenly...voila! the filter kicks in: I've read that some dirt-removal filters generally do not clean the very beginning and end of a frame IF it's after a scene change. But I want to adjust it so that the very beginning of every scene (hence, the entire episode) is cleaned.

3. Random rectangles around particular areas in the post-processed video: Is there any way to eliminate this?

Thank you guys for your time, I really appreciate the advice .

[poisondeathray]

1. Ghosting in some scenes: Based on what I've gathered, temporal smoothers cause this (unless there's something else I'm not aware about?). How can I adjust the settings in this filter to completely eliminate this artifact AND keep the spot-removing aspect intact?

You can try adjusting the temporal smoothing settings. No you can't have both. This is usually an artifact of temporal smoothing

Instead of temporal smoothing, you can try stack a different filter for "cleaning"

2. In scene changes, you can see a few seconds of dirt in the beginning of the scene and suddenly...voila! the filter kicks in: I've read that some dirt-removal filters generally do not clean the very beginning and end of a frame IF it's after a scene change. But I want to adjust it so that the very beginning of every scene (hence, the entire episode) is cleaned.

Not possible with any temporal based spot/dirt filters. The method in how they work is they look for differences between frames. On a hard scene change, the preceding frame is completely different, so no "spots" are detected

Similarly, if you have duplicate frames (very common in animation) this will cause "dirty" frames to get passed, because the algorithm doesn't "see" the artifacts as something to remove - they are too similar between neighboring frames. You usually have to decimate the dupes first

Another approach is to insert a duplicate of the 3nd frame behind 1st frame of sequence to engage the algorithm, then delete it later. Similarly for the last, you would insert a dupe of the 2nd last frame at the very end, then delete it later.

Or you could use spatial filters on dupe frames or end/beginning frames of a sequence

3. Random rectangles around particular areas in the post-processed video: Is there any way to eliminate this?

Are you in debug mode?





There are other options in avisynth as well, like removedirtmc, removedirt, despot, several others.

You may want to try spatial filters, again dozens of options to choose from

There are also many filters geared towards anime restoration; rarely do you have issues only limited to "spots" in animation. There is no reason you should limit yourself to 1 type of filter. If you post a video sample, then people can give you suggestions as to filters , settings , or approaches you might want to try

[jagabo]

1. Ghosting in some scenes: Based on what I've gathered, temporal smoothers cause this (unless there's something else I'm not aware about?).

That might be from a blend deinterlace or PAL/NTSC conversion. If so, you might be able to remove it with AviSynth and SRestore().

2. In scene changes, you can see a few seconds of dirt in the beginning of the scene and suddenly...voila! the filter kicks in: I've read that some dirt-removal filters generally do not clean the very beginning and end of a frame IF it's after a scene change. But I want to adjust it so that the very beginning of every scene (hence, the entire episode) is cleaned.

It won't be possible on the very first frame after a scene change. There's no way to know if it's a spot of dirt or if it's supposed to be there.

Posting a sample is your best bet.

[unclescoob]

Hey Poison,

Ok, regarding the temporal smoothing - are you familiar with which setting on SpotRemover I can tinker with in order to lower the temporal setting a bit? I've tweaked around with it a bit but the settings do not specify "temporal", as with other filters.

Regarding the rectangles, I unchecked the "debug" mode and still am getting this. But I'll try it again tonight and see. What are those rectangles all about? They seem to zero in on areas being cleaned.

Regarding dirt on the beginning of new frames-- so spatial filters can help eliminate this?

I apologize, as I'm getting familiar with Virtualdub.

[poisondeathray]

unclescoob - I don't use SpotRemover too often, but I'm fairly certain those "rectangles" are from the "debug mode". Do you see them in the output pane before exporting ? Do you have the latest version downloaded?

For the smoothing, the settings are in the "smoothing" tab. You can try reducing the strength. I try avoid temporal smoothing unless the source is that badly damaged. It's really a last ditch effort IMO

Regarding dirt on the beginning of new frames-- so spatial filters can help eliminate this?

You're not going to like it - but there is no "easy" way to effectively treat the beginning/end frames of a sequence using any temporal cleaner. Even the ones that have scenecut detection enabled treat them "differently" than other frames, so they aren't cleaned as well. Even a spatial filter applied only on those frames means you need to isolate or mark them manually as being "treated" (as to avoid overtreating the other frames). So there is no "automatic" easy 1 click solution for this, especially not in vdub. I posted some techniques you can try to workaround this above

jagabo brought up a good point about ghosting - it could also be from blends in the source

[unclescoob]

The rectangles, yes..I see them in the output pane. And I do have the latest version, as I thought this was an issue and double-checked.

Jagaboo, the source video does not have "ghosting". Just the edited version in the output pane. I had this type of trouble when I tried working with an Avidemux noise filter...until I lowered the temporal setting and then the "ghosting" disappeared.

However, is it possible to accomplish the PAL/NTSC conversion issue in Virtualdub?

[poisondeathray]

Note there are 2 debug checkboxes, one in the smoothing tab, one in the spot removal tab. The spot removal tab is the important one to uncheck. (I just checked quickly and rectangles appear when it is checked)

There is also a help .html file that comes with spotremover . It suggests to lower the intensity smooth strength to reduce the blurring (ghosting)

[unclescoob]

Beautiful! I knew there was something to this. I'm going to give this a shot when I get home. As far as the scene changes is concerned...well I guess it's only a few seconds and I can live with that.

Thank you very much for your help guys. As I said I'm new to video restoration but believe me, I'm not one to think that those "one-click" fixes do the trick. I have no problem going through the steps to restore my movies.

Poison- I might try your approach on inserting duplicate frames. I think I have an idea what you mean. But this is possible with Virtualdub, correct?

Nonetheless, you cleared things up on the issue for me. I appreciate the help!!

[poisondeathray]

RE: duplicate frame insertion: It's important that the inserted "duplicate" is a bit farther on not an immediate neighbor. It has to be "similar" enough that it's not completely different (like a scenechange), but not 100% identical or otherwise no detection will occur

so instead of frame A,B,C, <scenechange> 0,1,2,3 it would be

A,B,C,A <scenechange> 2,0,1,2,3

If you inserted a B instead of the A, or a 1 instead of a 2, you would have two neighboring identical frames and that won't work.

I highly suspect your animation would have duplicate frames in the regular sequence as well. It's very rare to have older animation with perfect cadence. If you don't remove the duplicates, dirty frames will be passed through that escape detection.

It's difficult to do in any application . You can script it in avisynth, but an easier way might be to decompress to image format (like png) and manually assemble it (vdub will import sequentially numbered png sequences) , apply the filter(s), and delete the inserts later

[unclescoob]

Definitely going to give it a try.

Thanks alot for the tip.

[the Prodigy]

an easier way might be to decompress to image format (like png) and manually assemble it (vdub will import sequentially numbered png sequences) , apply the filter(s), and delete the inserts later[/QUOTE]

Can you explain from step 1 how it doing please?

[poisondeathray]

an easier way might be to decompress to image format (like png) and manually assemble it (vdub will import sequentially numbered png sequences) , apply the filter(s), and delete the inserts later

in vdub, with the video loaded, file=> export => image sequence

[the Prodigy]

But how I can extract the image with a sopts from the video? and how I make image sequences?

[jagabo]

But how I can extract the image with a sopts from the video? and how I make image sequences?

Poisondeathray just told you how to do that. Then, after many months of editing the individual images to remove spots, you can put them all back together into a video by opening the first one in VirtualDub. It will automatically append subsequent images if you leave them sequentially numbered.

[the Prodigy]

OK thank you very much

[unclescoob]

Interesting that this topic was revived, as I have been experimenting with this method lately, but am still a bit confused on the re-opening the images in V-dub:

So here are the steps that I have taken:

1. Open the video in Virtualdub

2. Mark the segment that I want to edit frame by frame

3. Extract that segment as png image files into a folder.

4. Edit the frames

(now here's where it gets sloppy)

5. Open the first frame in V-dub - Here's what happens: The images do open as a video, however it plays at 10fps. Not the 24fps as the original video. What gives?

I have one more question about this, but would like to resolve this first.

Thanks

[jagabo]

5. Open the first frame in V-dub - Here's what happens: The images do open as a video, however it plays at 10fps. Not the 24fps as the original video. What gives?

How is VirtualDub supposed to know the original frame rate? It's just a series of still images.

Video -> Frame Rate... -> Change Frame Rate to (fps)...

[unclescoob]

Video -> Frame Rate... -> Change Frame Rate to (fps)...

Whoah, that's all??? And the video will play as the original source?

[unclescoob]

Sorry, should be more specific:

Other than colorspace conversion from YV12 to RGB(32)?, is there anything else that would change the quality of the new video when you convert from uncompressed to png, back to uncompressed?

[poisondeathray]

Other than colorspace conversion from YV12 to RGB(32)?, is there anything else that would change the quality of the new video when you convert from uncompressed to png, back to uncompressed?

No compression losses: PNG is lossless RGB compression . The losses are purely from rounding errors, chroma up&down sampling and colorspace conversion

Part of the colorspace conversion problem is Rec601 matrix will be used unless otherwise specfied (so you will clip superbrights & darks if you don't "legalize" them beforehand) . But from a retail DVD source, this is unlikely to be a big problem . It's a bigger problem with video footage

[unclescoob]

I was considering adding a simple script before opening in Virtualdub:

ConverttoRGB32()

However, I also understand that if I change the input colorspace settings in V-dub to YUY, there is no need to do this script.

Is this true?

[poisondeathray]

I was considering adding a simple script before opening in Virtualdub:

ConverttoRGB32()

However, I also understand that if I change the input colorspace settings in V-dub to YUY, there is no need to do this script.

Is this true?

Yes, you can change the input and output colorspace settings in vdub to whatever you want, but eitherway exporting to PNG will be RGB (it will be converted to RGB somewhere)

Note, there were some issues with vdub's colorspace settings - they were low quailty, low bit depth conversions leading to posterization - I don't know if they have fixed the issues in the latest releases

If you do your conversions in the script, you more control over the matrix used, the chroma sampling algorithm used.

Round tripping YUV=>RGB=>YUV is usually better with point resized chroma (bicubic is used by default in avisynth 2.6.x) but there are pros and cons to whatever method you choose (eitherway you ' re going to lose some quality, it's a tradeoff between aliasing and blurred edges but for most people , they might not even notice the difference. Picky people will know for sure)

[poisondeathray]

You can do some tests yourself, but I've done a bunch, and the method Gavino suggested in this thread seems better overall , least amount of loss , aliasing or blurring. (there are some images and test videos for comparison, you will see how less sharp the "regular" YV12=>RGB=>YV12 conversion is) . For things like cartoon edges , text edges , you will notice the difference . The video sample from a dvd menu had thin red text , so it's really obvious

This makes use of YV24 (YUV 4:4:4) avisynth 2.6.x and point resized chroma . It won't work in 2.5.x

http://forum.doom9.org/showthread.php?t=164737

For Progressive

ConvertToYV24(chromaresample="point") 
MergeChroma(PointResize(width, height, 0, 1)) 
ConvertToRGB32() ... 
# filtering in RGB32 
ConvertToYV12(chromaresample="point")

[unclescoob]

What if I save it as a .png image with Avisynth instead?

[poisondeathray]

What if I save it as a .png image with Avisynth instead?

You can do that too, with imagewriter()
http://avisynth.org/mediawiki/ImageWriter

But you still have the same issues with YUV<=>RGB colorspace conversions.

YUV<=>RGB conversions are lossy. Period.

It' s usually not a huge issue for live action content (most people won't even notice, even if you look frame by frame), but for cartoons, text, graphics, with clean color edges you can notice the deterioration more easily . You definitely want to minimize the number of conversions . If you have to use RGB filters or workflow, plan it way ahead of time so you do the excursion once only. The more conversions you do , the more it deteroriates (even when using lossless formats, it has nothing to do with compression losses)

[unclescoob]

Yeah, considering I'm working with cartoon content, I think I'll just stick with one colorspace.

Thanks.

[happierhannah]

3. Random rectangles around particular areas in the post-processed video: Is there any way to eliminate this?

I, too, was having trouble with rectangles in my output video after using the filter. But if you change the settings in basic and select "disable spot removal" the problem should be gone. I'm not sure if it actually takes care of the spots still or not. But even if it doesn't, the rest of the plugin still works wonders and you can have a clean output (no rectangles).

[unclescoob]

WOW, thanks! I mean by this point I've graduated to motion compensation denoising, but I'll give it a shot.