VideoHelp Forum
+ Reply to Thread
Results 1 to 21 of 21
Thread
  1. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    I've been capturing old Hi8 tapes lately, many of them using the method of capturing the same clip several times and taking a median out of it. This eliminates all sorts of glitches and noise really well.

    There was no median function in AviSynth, and although you can script one using other plugins, I thought I'd try making a bespoke median filter. I don't really have any experience coding for AviSynth, but I did manage to come up with something. The plugin supports between 3 and 25 input clips. Bugs may still remain.

    You can use the plugin like this:

    Code:
    Median(clip1,clip2,clip3..., chroma=true/false)
    See also the readme or posts down below for other possible uses.

    I have also made the files available for download here.

    Any feedback is welcome!
    Attached Files
    Last edited by ajk; 15th Mar 2014 at 06:56. Reason: Updated attachment
    Quote Quote  
  2. Just as a reference, median of 3 and 5 using existing filters:
    http://forum.videohelp.com/threads/340963-Best-quality-and-speed-video-denoisers-2011?...=1#post2122313

    I don't have multiple caps available right now but as a demonstration of effectiveness I used the filter on a very noisy video that was mostly still, taking the median of every 5 frames:

    Code:
    AviSource("Noisey Video.avi") 
    
    v1=SelectEvery(5)
    v2=Trim(1,0).SelectEvery(5)
    v3=Trim(2,0).SelectEvery(5)
    v4=Trim(3,0).SelectEvery(5)
    v5=Trim(4,0).SelectEvery(5)
    
    StackHorizontal(SelectEvery(5), Median(v1,v2,v3,v4,v5))
    Obviously, this only works on the still parts of the picture (though the woman was pretty still during the 5 frames pictured). Unfiltered on the left, median (of 5) on the right:

    Name:  medianof5.png
Views: 184
Size:  531.1 KB
    Last edited by jagabo; 13th Feb 2014 at 09:30.
    Quote Quote  
  3. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Here is an example of one of the actual tapes I've been working on. Severe "comets" or whatever you want to call them, plus just ordinary noise. Five captures eliminates almost everything, a few glitches here and there do get through though. Dunno if I can be bothered to go up to seven or nine captures with this one, but I have many other tapes to go...

    Note: video clip shows a fish being gutted so if you are sensitive to that sort of thing, skip the video. This part just happened to be some of the worst quality on the tape.
    Attached Thumbnails Click image for larger version

Name:	median.jpg
Views:	42
Size:	337.4 KB
ID:	23557  

    Attached Files
    Quote Quote  
  4. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    Mentioned this already here http://forum.videohelp.com/threads/362340-New-plugin-that-s-helpful

    There's a lot more that could be done here. This technique is also used in other areas like astrophotography. There's a variation called sigma denoise, where you throw out extreme values and average the rest. Average reduces noise quicker than median, so a combo would work best. You could probably also do it in far less passes with some masking (at least for the comets). Some problems are, the horizontal line jitter, which can make some weird artefacts with median, and essentially a horizontal blur with averaging, and dropped frames.

    You can extend the technique to multiple tapes/laserdiscs of the same content. This would remove all noise in the playback chain.
    Quote Quote  
  5. Member Cornucopia's Avatar
    Join Date: Oct 2001
    Location: Deep in the Heart of Texas
    Search Comp PM
    I would have thought it were a given (at least among all here) that one must use captures that don't have dropped frames, varying edits, framerates or runtimes, and that do have a stable timebase (via TBC, etc), and same base resolution/AR (not combining P&S with Letterboxed, etc. Also these are why mixing PAL with NTSC copies would likely go very wrong).

    Nice, clearcut examples, BTW.

    Scott
    Quote Quote  
  6. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    @jmac698

    Cool thanks, hadn't noticed your thread when I made mine I thought I'd post this both here and on doom9 just for redundancy, so if my webspace or one of the forums is not available in the future, there's yet another copy.

    I think the best approach depends on the nature of the noise/defect. White dropouts like in my tapes wipe out the entire underlying image, so averaging will not do any good. The best cure is to take a pure median out of as many samples as possible.

    But for noise on top of a still recognizable image a combined approach should indeed work very well. Perhaps a MedianAverage() function call that will throw out a specified number of extremes and average the remaining samples?
    Quote Quote  
  7. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    Ok, I will give you some of my ideas and you can (hopefully) update your plugin, since I'm unlikely to make my own plugin anytime soon.

    More info here http://www.digitalfaq.com/forum/video-restore/2734-averaging-multiple-captures-2.html

    Here's the formulas for how much the different techniques reduce noise:
    average - sqr(n)

    median - sqr(2n/pi)

    clip average - sqr(n-2)

    The last one, clip average, refers to a proposed method where the 2 most extreme values are thrown out. This method would be best at 6 captures.

    I'll have to explain my ideas better for you. For each pixel, sort the values, then average the array except the first and last index. This eliminates the impulse noise (the comets), and black edges problem, but has better noise reduction of the remaining gaussian noise than pure median.

    I think the ideal plugin would let you choose how many extreme values to throw away, and choose between pure median or pure average. Anyhow, there's a script for this already in that thread. I'll have to think this through better when I have time.

    I've already made a plugin for reducing comets in just two captures. It throws out the more extreme value with some logic that prefers a luma value closer to the middle range (luma=126) to avoid black/white pixels.

    With masking, like the existing depulse (1-pass only) plugin, there's a spatial search for white lines. There's at least n-1 captures that are perfectly fine, and you should find those pixels and average them, also in 95% of the pixels they are all fine, so use the better averaging of noise. Am I making any sense?

    You also have to be careful with these kinds of manipulations, as you can cause an overall brightness/contrast change.

    Also, pure averaging does work on comets, it just makes the white lines "faded", but ultimately it would eliminate them.

    I'd like to make profiles of comets sometime, to see exactly how many pixels long they are and if the leading edge has any transparency. In fact you could profile them in just a section of two captures then use the profile to create an ideal mask for the best 1 or 2 pass de-comet.

    http://www5e.biglobe.ne.jp/~hoe/dv-and-movie/tipsxxx/index.html
    depulse

    English readme
    http://forum.doom9.org/archive/index.php/t-151426.html
    Last edited by jmac698; 14th Feb 2014 at 03:53.
    Quote Quote  
  8. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Originally Posted by jmac698 View Post
    Also, pure averaging does work on comets, it just makes the white lines "faded", but ultimately it would eliminate them.
    Well yes, technically it will, but just imagine how many captures you would need to average to get rid of a white streak on a (near) black background. I don't think it's practical to use a pure average without any median component in the process. And at least on some of my tapes there are surprisingly often comets in the same place in different captures, which would make it even worse.

    Originally Posted by jmac698 View Post
    I've already made a plugin for reducing comets in just two captures. It throws out the more extreme value with some logic that prefers a luma value closer to the middle range (luma=126) to avoid black/white pixels.
    That is a cool idea. It probably works quite well most of the time. Though again, if you have a white streak on a black background, won't it be a bit random what gets included?

    Originally Posted by jmac698 View Post
    The last one, clip average, refers to a proposed method where the 2 most extreme values are thrown out. This method would be best at 6 captures.

    I'll have to explain my ideas better for you. For each pixel, sort the values, then average the array except the first and last index. This eliminates the impulse noise (the comets), and black edges problem, but has better noise reduction of the remaining gaussian noise than pure median.

    I think the ideal plugin would let you choose how many extreme values to throw away, and choose between pure median or pure average. Anyhow, there's a script for this already in that thread. I'll have to think this through better when I have time.
    I've updated my plugin now to include MedianBlend(). This is a more configurable median function that should cater for most of those ideas. Some examples:

    Code:
    # Discard the highest two values and keep the lowest
    
    MedianBlend(clip1,clip2,clip3, low=0, high=2)
    This is essentially a minimum function. This is the result with my earlier test capture:

    Name:  min.png
Views: 154
Size:  801.4 KB

    Code:
    # Discard the lowest two values and keep the highest
    
    MedianBlend(clip1,clip2,clip3, low=2, high=0)
    This is the equivalent maximum function.

    Name:  max.png
Views: 151
Size:  784.4 KB

    Code:
    # Discard the lowest and highest values, average the two remaining
    
    MedianBlend(clip1,clip2,clip3,clip4, low=1, high=1)
    And this should be what you described. In the case of my clips some half-intensity comets remain (more are visible in the video than in this particular frame).

    Name:  blend.png
Views: 159
Size:  689.8 KB

    I've also extended the processing to allow for up to 25 input clips.
    Quote Quote  
  9. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    @ajk
    I quite like your sample, could I have short (~1s) clips of those 5 captures, from the exact same time? In original quality, for example lossless UTVideo. I'm writing a program now to extract the comet profile, as I said it's something I've always been curious about.

    If anyone else has samples, the ideal sample has no dropped frames, obvious comets, uncompressed codec, samples from both PAL and NTSC tapes, a variety of luma values across the frame, stable image (use of TBC), and properly captured with no clipping of the levels (no value more than 234, yes, 234), and at least 3 passes.
    Quote Quote  
  10. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Sure. Here are the five captures that I have. I don't have the UT codec installed so this is in Lagarith YUY2.
    Attached Files
    Quote Quote  
  11. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    @ajk
    p.s.
    You posted at the same time as me, I am delighted to see your updated results. I think the "medianblend" final example looks fantastic, the noise is really reduced. I believe that with masking, those final "half-comets" could be eliminated. I'm happy to work with you on this. I'd also like to see a comparison between medianblend and pure median, to confirm the formula's predictions of better denoising with medianblend.
    Quote Quote  
  12. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Here is the result of a pure median of five captures (same as in the video clip earlier but full resolution and in PNG) for comparison:

    Name:  med.png
Views: 153
Size:  714.7 KB

    In this particular case I'm more interested in de-cometing than denoising - I'm anyway going to run a light pass of a motion compensated denoiser further down the line. But do keep up with the experiments, there are many kinds of uses for these functions, I hope
    Quote Quote  
  13. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    I have an idea for auto-profiling. Take the first frame of each clip if >2, and find cases for all pixels where there is one extreme value, call that a comet. A simple way to find an extreme value is to sort, find the smallest delta between any two values, and the 3rd is extreme if it's more than x* the difference. I don't know exactly right now, what if it's nearly noise-free and the delta is around 5? Then a change of double, or 10 would be a comet?

    This is a clustering algorithm, and the standard algorithm is the k-means http://en.wikipedia.org/wiki/K-means_clustering but you could probably come up with your own logic, again by having profiles of what comets are you would have more ideas. Also have to remember people can have wrong recording levels etc. so nothing should refer to absolute luma values.
    Quote Quote  
  14. Member
    Join Date: Sep 2011
    Location: NTSC
    Search Comp PM
    Ok, I'd have to say the medianblend is slightly less noisy than pure median. The only disadvantage is the half-comets. As for denoising, certainly it would be better for, ahem "detail" if you got the "real" picture without noise rather than an artificially denoised picture.
    Quote Quote  
  15. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Sure. But there will always be noise from the CCD and elsewhere that isn't coming from the playback/capture signal path and can be eliminated like this. So I'm usually going to need a denoising pass in any case, even if the capture itself were "perfect".

    I like the idea of a pure median because it's essentially failure-free, whereas masking is generally really hard to get just right. But a median blend of six captures could work well, at least any remaining comets would be very faint.
    Quote Quote  
  16. Member PuzZLeR's Avatar
    Join Date: Oct 2006
    Location: Toronto Canada
    Search Comp PM
    This thread could use a reviving.

    First off, thumbs up AJK. Nice plugin.

    On three caps, using MedianBlend, I have virtually perfect results removing white specks with parameters low=0,high=2 and the same success with low=2,high=0 on black spots.

    As for handling both polar extremes for the same script - white and black dropouts together in the captures - I did try low=1,high=1, and the plugin still does well, but there's a slight "averaging" in some cases.

    I could keep diminishing the artifacts, or "averaging them out", to near zero, but that would require more captures, which I'd rather avoid.

    However, I am still trying to understand how this happens on a "median" plugin, even if it will "average" (as the readme says) the remaining ones when there's only one remaining. An average of one = median = the same pixel/clip unchanged. No? Could this be a bug, or did I miss something?

    (Again, talking about removing dropouts on captures that have both white and black dropouts - boy, the excitement never ends with my Panasonic AG-1980.)
    Last edited by PuzZLeR; 15th Mar 2014 at 02:51.
    Been away for a while and busy with work the last few months so I had no time for forums. My apologies for any emails I couldn't get to in time - missed you all! :-)
    Quote Quote  
  17. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Originally Posted by PuzZLeR View Post
    This thread could use a reviving.

    First off, thumbs up AJK. Nice plugin.
    Thanks, nice to hear someone is putting it to good use

    On three caps, using MedianBlend, I have virtually perfect results removing white specks with parameters low=0,high=2 and the same success with low=2,high=0 on black spots.
    This sounds correct. With three caps, essentially the function boils down to a minimum or maximum function and there is no averaging element involved. You get the lightest (or darkest) pixel out of each set of three.

    As for handling both polar extremes for the same script - white and black dropouts together in the captures - I did try low=1,high=1, and the plugin still does well, but there's a slight "averaging" in some cases.
    Hmm. Well basically with three caps MedianBlend() with those parameters should be exactly the same as just a Median() out of the three - there is nothing left to average. Could you do a Subtract(a,b) between the Median() and MedianBlend() and see if there is any difference? I can't think of any reason why there would be, it should be running the exact same code in each case. In fact, MedianBlend() probably doesn't make much sense with less than four captures except in particular cases where you only have dark OR light noise.

    Of course, running a median on three captures will always have some appearance of "averaging" or smoothing out the video. After all, it's not just attacking the dark or light spots - pixel values such as 100, 101 and 102 will be processed too and you end up with the middle value. Maybe it's just that? You could limit the processing to only dark and light areas with some masktools magic if necessary.

    Still, if you feel something is not right, give me a couple of examples and I will look into it
    Quote Quote  
  18. Member PuzZLeR's Avatar
    Join Date: Oct 2006
    Location: Toronto Canada
    Search Comp PM
    Hi AJK, thanks for your response. I am not home this weekend, but would be more than happy to provide some examples by tomorrow.
    Been away for a while and busy with work the last few months so I had no time for forums. My apologies for any emails I couldn't get to in time - missed you all! :-)
    Quote Quote  
  19. Member PuzZLeR's Avatar
    Join Date: Oct 2006
    Location: Toronto Canada
    Search Comp PM
    Hi AJK. I took another (much longer) look at it, and the moral of the story is that if one is going to run multiple captures, they should make that special effort to capture all of them under very similar settings for median methods to work effectively.

    That was my problem. The plugin, however is amazing.

    Originally Posted by ajk
    Of course, running a median on three captures will always have some appearance of "averaging" or smoothing out the video. After all, it's not just attacking the dark or light spots - pixel values such as 100, 101 and 102 will be processed too and you end up with the middle value. Maybe it's just that?
    Pretty much.

    I processed three captures that had certain segments within that had different brightness levels. This was due to three caps of a 3hr+ tape (almost 10 combined capture hours) that had different recorded programs in it. I was adjusting the proc amp throughout during capture for the needs of the different programs, but didn't realize that the caps would not be equal in the end.

    I was wondering why some segments worked, and others were "averaging". Well, they weren't "averaging", it was the plugin replacing a dropout with the brigher pixels of the brighter capture, hence the appearance of "averaging" with a dropout.

    You can have a look if you're interested:
    clip1 - almost clear but has a black dropout in frame 6
    clip2 - moderate dropouts - but has a distinct white dropout in frame 4
    clip3 - more "dirtier", yet the clip has more brightness, which caused the problem

    When you run either MedianBlend(clip1,clip2,clip3,low=1,high=1) or Median(clip1,clip2,clip3) the black dropout from clip1 is reduced since it was in clip2 somewhat, but you will see what looks like an "averaging" of the white dropout in frame 4. This is actually the brighter pixels of clip3 placed in there, NOT a dropout averaged like I thought. This is what confused me. My bad.

    Instead of one big gulp, i will capture in smaller doses now to ensure I have very similar settings for all captures for the plugin to be effective.

    Originally Posted by ajk
    Hmm. Well basically with three caps MedianBlend() with those parameters should be exactly the same as just a Median() out of the three - there is nothing left to average. Could you do a Subtract(a,b) between the Median() and MedianBlend() and see if there is any difference? I can't think of any reason why there would be, it should be running the exact same code in each case.
    You are correct. In all my tests, including this example, Subtract(MedianBlend(clip1,clip2,clip3,low=1,high= 1),Median(clip1,clip2,clip3)) does produce the gray screen of equality.

    The plugin works as expected on my end.

    Originally Posted by ajk
    Thanks, nice to hear someone is putting it to good use
    Oh, for sure. For those of us who need it, this is a blessing. I personally will put it to work in the coming weeks on tapes that I've delayed. They always looked great on my Panasonic AG-1980, but since the machine is aging and has side-effects (such as dropouts), and replacing such a unit in 2014 is not a viable option, your plugin is the ultimate capture/restoration solution for it.

    Thank you so much for this little gem!
    Attached Files
    Been away for a while and busy with work the last few months so I had no time for forums. My apologies for any emails I couldn't get to in time - missed you all! :-)
    Quote Quote  
  20. Member
    Join Date: Dec 2005
    Location: Finland
    Search Comp PM
    Originally Posted by PuzZLeR View Post
    Hi AJK. I took another (much longer) look at it, and the moral of the story is that if one is going to run multiple captures, they should make that special effort to capture all of them under very similar settings for median methods to work effectively.
    Indeed - I always capture the same tape (or segment) three or five or however many times with absolutely the same settings, just rewinding the tape. I don't want to shut down VirtualDub, reboot the system, or anything in between if at all possible. Otherwise it's nearly certain that some setting will have been reset or changed Good that you have it sorted now, I'll still have a look at your samples just out of interest.

    Oh, for sure. For those of us who need it, this is a blessing. I personally will put it to work in the coming weeks on tapes that I've delayed. They always looked great on my Panasonic AG-1980, but since the machine is aging and has side-effects (such as dropouts), and replacing such a unit in 2014 is not a viable option, your plugin is the ultimate capture/restoration solution for it.

    Thank you so much for this little gem!
    Excellent to hear! I can't take the credit for the method, of course, as it has been possible to do these things in AviSynth with scripting and other plugins, but I wanted something that is simple to use and works on any kind of input. And it's also fun to work on little projects like this
    Quote Quote  
  21. Member PuzZLeR's Avatar
    Join Date: Oct 2006
    Location: Toronto Canada
    Search Comp PM
    Originally Posted by ajk
    Indeed - I always capture the same tape (or segment) three or five or however many times with absolutely the same settings, just rewinding the tape. I don't want to shut down VirtualDub, reboot the system, or anything in between if at all possible. Otherwise it's nearly certain that some setting will have been reset or changed Good that you have it sorted now, I'll still have a look at your samples just out of interest.
    The thing is that alot of my tapes have multiple recordings, even as small as dozens of music videos, which each need different treatment with a proc amp. That was the problem I experienced earlier - when trying to do the WHOLE thing before the next capture - some captures will come out different. For those tapes, I will just capture, and recapture, smaller segments at a time to better ensure a true median in processing. To get rid of those dropouts it's worth it to shorten the workflow's cycle.

    Originally Posted by alk
    Excellent to hear! I can't take the credit for the method, of course, as it has been possible to do these things in url=http://www.videohelp.com/tools/Avisynth]AviSynth[/url] with scripting and other plugins, but I wanted something that is simple to use and works on any kind of input. And it's also fun to work on little projects like this
    You sure can take credit for this when alot of the other ones fall short in many ways. Personally, the medianblur package wasn't for me - I just don't get the options. And this averaging and clensing thing? Don't get it in a solution at the capture level, or mixing in when a true median is required, or even in damaging other parts of the video in compromise - that is IF it's effective at all in removing anything. And many need further components, that can take forever to find, for system32, etc. And I have no appetite to sludge through long boring scripts on doom9 (in which most never work anyway). Your plugin was simple and to the point, and gets the job done without useless showing off, and is fun to use.
    Been away for a while and busy with work the last few months so I had no time for forums. My apologies for any emails I couldn't get to in time - missed you all! :-)
    Quote Quote  



Similar Threads