I've been capturing old Hi8 tapes lately, many of them using the method of capturing the same clip several times and taking a median out of it. This eliminates all sorts of glitches and noise really well.
There was no median function in AviSynth, and although you can script one using other plugins, I thought I'd try making a bespoke median filter. I don't really have any experience coding for AviSynth, but I did manage to come up with something. The plugin supports between 3 and 25 input clips. Bugs may still remain.
You can use the plugin like this:
See also the readme or posts down below for other possible uses.Code:Median(clip1,clip2,clip3..., chroma=true/false)
I have also made the files available for download here.
Any feedback is welcome!
+ Reply to Thread
Results 1 to 15 of 15
Thread: Median() plugin for AviSynth
Last edited by ajk; 14th Feb 2014 at 13:37. Reason: Updated attachment
Just as a reference, median of 3 and 5 using existing filters:
I don't have multiple caps available right now but as a demonstration of effectiveness I used the filter on a very noisy video that was mostly still, taking the median of every 5 frames:
AviSource("Noisey Video.avi") v1=SelectEvery(5) v2=Trim(1,0).SelectEvery(5) v3=Trim(2,0).SelectEvery(5) v4=Trim(3,0).SelectEvery(5) v5=Trim(4,0).SelectEvery(5) StackHorizontal(SelectEvery(5), Median(v1,v2,v3,v4,v5))
Last edited by jagabo; 13th Feb 2014 at 10:30.
Here is an example of one of the actual tapes I've been working on. Severe "comets" or whatever you want to call them, plus just ordinary noise. Five captures eliminates almost everything, a few glitches here and there do get through though. Dunno if I can be bothered to go up to seven or nine captures with this one, but I have many other tapes to go...
Note: video clip shows a fish being gutted so if you are sensitive to that sort of thing, skip the video. This part just happened to be some of the worst quality on the tape.
Mentioned this already here http://forum.videohelp.com/threads/362340-New-plugin-that-s-helpful
There's a lot more that could be done here. This technique is also used in other areas like astrophotography. There's a variation called sigma denoise, where you throw out extreme values and average the rest. Average reduces noise quicker than median, so a combo would work best. You could probably also do it in far less passes with some masking (at least for the comets). Some problems are, the horizontal line jitter, which can make some weird artefacts with median, and essentially a horizontal blur with averaging, and dropped frames.
You can extend the technique to multiple tapes/laserdiscs of the same content. This would remove all noise in the playback chain.
I would have thought it were a given (at least among all here) that one must use captures that don't have dropped frames, varying edits, framerates or runtimes, and that do have a stable timebase (via TBC, etc), and same base resolution/AR (not combining P&S with Letterboxed, etc. Also these are why mixing PAL with NTSC copies would likely go very wrong).
Nice, clearcut examples, BTW.
Cool thanks, hadn't noticed your thread when I made mine I thought I'd post this both here and on doom9 just for redundancy, so if my webspace or one of the forums is not available in the future, there's yet another copy.
I think the best approach depends on the nature of the noise/defect. White dropouts like in my tapes wipe out the entire underlying image, so averaging will not do any good. The best cure is to take a pure median out of as many samples as possible.
But for noise on top of a still recognizable image a combined approach should indeed work very well. Perhaps a MedianAverage() function call that will throw out a specified number of extremes and average the remaining samples?
Ok, I will give you some of my ideas and you can (hopefully) update your plugin, since I'm unlikely to make my own plugin anytime soon.
More info here http://www.digitalfaq.com/forum/video-restore/2734-averaging-multiple-captures-2.html
Here's the formulas for how much the different techniques reduce noise:
average - sqr(n)
median - sqr(2n/pi)
clip average - sqr(n-2)
The last one, clip average, refers to a proposed method where the 2 most extreme values are thrown out. This method would be best at 6 captures.
I'll have to explain my ideas better for you. For each pixel, sort the values, then average the array except the first and last index. This eliminates the impulse noise (the comets), and black edges problem, but has better noise reduction of the remaining gaussian noise than pure median.
I think the ideal plugin would let you choose how many extreme values to throw away, and choose between pure median or pure average. Anyhow, there's a script for this already in that thread. I'll have to think this through better when I have time.
I've already made a plugin for reducing comets in just two captures. It throws out the more extreme value with some logic that prefers a luma value closer to the middle range (luma=126) to avoid black/white pixels.
With masking, like the existing depulse (1-pass only) plugin, there's a spatial search for white lines. There's at least n-1 captures that are perfectly fine, and you should find those pixels and average them, also in 95% of the pixels they are all fine, so use the better averaging of noise. Am I making any sense?
You also have to be careful with these kinds of manipulations, as you can cause an overall brightness/contrast change.
Also, pure averaging does work on comets, it just makes the white lines "faded", but ultimately it would eliminate them.
I'd like to make profiles of comets sometime, to see exactly how many pixels long they are and if the leading edge has any transparency. In fact you could profile them in just a section of two captures then use the profile to create an ideal mask for the best 1 or 2 pass de-comet.
Last edited by jmac698; 14th Feb 2014 at 04:53.
# Discard the highest two values and keep the lowest MedianBlend(clip1,clip2,clip3, low=0, high=2)
# Discard the lowest two values and keep the highest MedianBlend(clip1,clip2,clip3, low=2, high=0)
# Discard the lowest and highest values, average the two remaining MedianBlend(clip1,clip2,clip3,clip4, low=1, high=1)
I've also extended the processing to allow for up to 25 input clips.
I quite like your sample, could I have short (~1s) clips of those 5 captures, from the exact same time? In original quality, for example lossless UTVideo. I'm writing a program now to extract the comet profile, as I said it's something I've always been curious about.
If anyone else has samples, the ideal sample has no dropped frames, obvious comets, uncompressed codec, samples from both PAL and NTSC tapes, a variety of luma values across the frame, stable image (use of TBC), and properly captured with no clipping of the levels (no value more than 234, yes, 234), and at least 3 passes.
You posted at the same time as me, I am delighted to see your updated results. I think the "medianblend" final example looks fantastic, the noise is really reduced. I believe that with masking, those final "half-comets" could be eliminated. I'm happy to work with you on this. I'd also like to see a comparison between medianblend and pure median, to confirm the formula's predictions of better denoising with medianblend.
Here is the result of a pure median of five captures (same as in the video clip earlier but full resolution and in PNG) for comparison:
In this particular case I'm more interested in de-cometing than denoising - I'm anyway going to run a light pass of a motion compensated denoiser further down the line. But do keep up with the experiments, there are many kinds of uses for these functions, I hope
I have an idea for auto-profiling. Take the first frame of each clip if >2, and find cases for all pixels where there is one extreme value, call that a comet. A simple way to find an extreme value is to sort, find the smallest delta between any two values, and the 3rd is extreme if it's more than x* the difference. I don't know exactly right now, what if it's nearly noise-free and the delta is around 5? Then a change of double, or 10 would be a comet?
This is a clustering algorithm, and the standard algorithm is the k-means http://en.wikipedia.org/wiki/K-means_clustering but you could probably come up with your own logic, again by having profiles of what comets are you would have more ideas. Also have to remember people can have wrong recording levels etc. so nothing should refer to absolute luma values.
Ok, I'd have to say the medianblend is slightly less noisy than pure median. The only disadvantage is the half-comets. As for denoising, certainly it would be better for, ahem "detail" if you got the "real" picture without noise rather than an artificially denoised picture.
Sure. But there will always be noise from the CCD and elsewhere that isn't coming from the playback/capture signal path and can be eliminated like this. So I'm usually going to need a denoising pass in any case, even if the capture itself were "perfect".
I like the idea of a pure median because it's essentially failure-free, whereas masking is generally really hard to get just right. But a median blend of six captures could work well, at least any remaining comets would be very faint.