VideoHelp Forum
+ Reply to Thread
Page 1 of 3
1 2 3 LastLast
Results 1 to 30 of 70
Thread
  1. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    I've been capturing old Hi8 tapes lately, many of them using the method of capturing the same clip several times and taking a median out of it. This eliminates all sorts of glitches and noise very effectively. And, unlike any denoising method that only acts on one clip, this process can actually dig out the true image from underneath the noise without guessing or interpolating what should be there.

    There is no median function in AviSynth, and although you can script one using other plugins, I thought I'd try making a bespoke filter. I don't really have any experience coding for AviSynth, but I have managed to come up with something that works. The plugin supports between 3 and 25 input clips. Bugs may still remain.

    The simplest way to use the plugin is like this:

    Code:
    Median(clip1, clip2, clip3, ...)
    When using this method to suppress noise from multiple analog captures, it is imperative that all the clips match exactly. If they don't, you will get a median of different images, and the resulting image is quite likely garbage. It is easy to roughly line up captures using Trim() to make sure the all begin at the same point in time, but due to frames dropped during capture, it is fairly likely that they will not remain 100% in sync over longer periods of time.

    To help with this, Median() has some basic image matching functionality that will attempt to select the best matching frames for each output frame. You can activate this processing by adding:

    Code:
    Median(clip1, clip2, clip3, sync=10, debug=true)
    And now the filter will look at ten frames forward and back for all of the clips. Activating the debug output is not mandatory but it will help to estimate what is a reasonable sync radius to use. You can of course set it high to be on the safe side, but the more images the filter needs to look at, the slower it will run.

    You can also use the filter on one clip in a temporal manner with TemporalMedian(), or combine median and average functionality with MedianBlend(). See also the readme or posts down below for other ways to use the filters included in the plugin.



    I have also made the files available for download here.

    Any feedback is welcome!
    Image Attached Files
    Last edited by ajk; 16th Jan 2016 at 08:17. Reason: Updated attachment
    Quote Quote  
  2. Just as a reference, median of 3 and 5 using existing filters:
    http://forum.videohelp.com/threads/340963-Best-quality-and-speed-video-denoisers-2011?...=1#post2122313

    I don't have multiple caps available right now but as a demonstration of effectiveness I used the filter on a very noisy video that was mostly still, taking the median of every 5 frames:

    Code:
    AviSource("Noisey Video.avi") 
    
    v1=SelectEvery(5)
    v2=Trim(1,0).SelectEvery(5)
    v3=Trim(2,0).SelectEvery(5)
    v4=Trim(3,0).SelectEvery(5)
    v5=Trim(4,0).SelectEvery(5)
    
    StackHorizontal(SelectEvery(5), Median(v1,v2,v3,v4,v5))
    Obviously, this only works on the still parts of the picture (though the woman was pretty still during the 5 frames pictured). Unfiltered on the left, median (of 5) on the right:

    Click image for larger version

Name:	medianof5.png
Views:	1473
Size:	531.1 KB
ID:	23552
    Last edited by jagabo; 13th Feb 2014 at 10:30.
    Quote Quote  
  3. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Here is an example of one of the actual tapes I've been working on. Severe "comets" or whatever you want to call them, plus just ordinary noise. Five captures eliminates almost everything, a few glitches here and there do get through though. Dunno if I can be bothered to go up to seven or nine captures with this one, but I have many other tapes to go...

    Note: video clip shows a fish being gutted so if you are sensitive to that sort of thing, skip the video. This part just happened to be some of the worst quality on the tape.
    Image Attached Thumbnails Click image for larger version

Name:	median.jpg
Views:	300
Size:	337.4 KB
ID:	23557  

    Image Attached Files
    Quote Quote  
  4. Mentioned this already here http://forum.videohelp.com/threads/362340-New-plugin-that-s-helpful

    There's a lot more that could be done here. This technique is also used in other areas like astrophotography. There's a variation called sigma denoise, where you throw out extreme values and average the rest. Average reduces noise quicker than median, so a combo would work best. You could probably also do it in far less passes with some masking (at least for the comets). Some problems are, the horizontal line jitter, which can make some weird artefacts with median, and essentially a horizontal blur with averaging, and dropped frames.

    You can extend the technique to multiple tapes/laserdiscs of the same content. This would remove all noise in the playback chain.
    Quote Quote  
  5. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    I would have thought it were a given (at least among all here) that one must use captures that don't have dropped frames, varying edits, framerates or runtimes, and that do have a stable timebase (via TBC, etc), and same base resolution/AR (not combining P&S with Letterboxed, etc. Also these are why mixing PAL with NTSC copies would likely go very wrong).

    Nice, clearcut examples, BTW.

    Scott
    Quote Quote  
  6. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    @jmac698

    Cool thanks, hadn't noticed your thread when I made mine I thought I'd post this both here and on doom9 just for redundancy, so if my webspace or one of the forums is not available in the future, there's yet another copy.

    I think the best approach depends on the nature of the noise/defect. White dropouts like in my tapes wipe out the entire underlying image, so averaging will not do any good. The best cure is to take a pure median out of as many samples as possible.

    But for noise on top of a still recognizable image a combined approach should indeed work very well. Perhaps a MedianAverage() function call that will throw out a specified number of extremes and average the remaining samples?
    Quote Quote  
  7. Ok, I will give you some of my ideas and you can (hopefully) update your plugin, since I'm unlikely to make my own plugin anytime soon.

    More info here http://www.digitalfaq.com/forum/video-restore/2734-averaging-multiple-captures-2.html

    Here's the formulas for how much the different techniques reduce noise:
    average - sqr(n)

    median - sqr(2n/pi)

    clip average - sqr(n-2)

    The last one, clip average, refers to a proposed method where the 2 most extreme values are thrown out. This method would be best at 6 captures.

    I'll have to explain my ideas better for you. For each pixel, sort the values, then average the array except the first and last index. This eliminates the impulse noise (the comets), and black edges problem, but has better noise reduction of the remaining gaussian noise than pure median.

    I think the ideal plugin would let you choose how many extreme values to throw away, and choose between pure median or pure average. Anyhow, there's a script for this already in that thread. I'll have to think this through better when I have time.

    I've already made a plugin for reducing comets in just two captures. It throws out the more extreme value with some logic that prefers a luma value closer to the middle range (luma=126) to avoid black/white pixels.

    With masking, like the existing depulse (1-pass only) plugin, there's a spatial search for white lines. There's at least n-1 captures that are perfectly fine, and you should find those pixels and average them, also in 95% of the pixels they are all fine, so use the better averaging of noise. Am I making any sense?

    You also have to be careful with these kinds of manipulations, as you can cause an overall brightness/contrast change.

    Also, pure averaging does work on comets, it just makes the white lines "faded", but ultimately it would eliminate them.

    I'd like to make profiles of comets sometime, to see exactly how many pixels long they are and if the leading edge has any transparency. In fact you could profile them in just a section of two captures then use the profile to create an ideal mask for the best 1 or 2 pass de-comet.

    http://www5e.biglobe.ne.jp/~hoe/dv-and-movie/tipsxxx/index.html
    depulse

    English readme
    http://forum.doom9.org/archive/index.php/t-151426.html
    Last edited by jmac698; 14th Feb 2014 at 04:53.
    Quote Quote  
  8. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Originally Posted by jmac698 View Post
    Also, pure averaging does work on comets, it just makes the white lines "faded", but ultimately it would eliminate them.
    Well yes, technically it will, but just imagine how many captures you would need to average to get rid of a white streak on a (near) black background. I don't think it's practical to use a pure average without any median component in the process. And at least on some of my tapes there are surprisingly often comets in the same place in different captures, which would make it even worse.

    Originally Posted by jmac698 View Post
    I've already made a plugin for reducing comets in just two captures. It throws out the more extreme value with some logic that prefers a luma value closer to the middle range (luma=126) to avoid black/white pixels.
    That is a cool idea. It probably works quite well most of the time. Though again, if you have a white streak on a black background, won't it be a bit random what gets included?

    Originally Posted by jmac698 View Post
    The last one, clip average, refers to a proposed method where the 2 most extreme values are thrown out. This method would be best at 6 captures.

    I'll have to explain my ideas better for you. For each pixel, sort the values, then average the array except the first and last index. This eliminates the impulse noise (the comets), and black edges problem, but has better noise reduction of the remaining gaussian noise than pure median.

    I think the ideal plugin would let you choose how many extreme values to throw away, and choose between pure median or pure average. Anyhow, there's a script for this already in that thread. I'll have to think this through better when I have time.
    I've updated my plugin now to include MedianBlend(). This is a more configurable median function that should cater for most of those ideas. Some examples:

    Code:
    # Discard the highest two values and keep the lowest
    
    MedianBlend(clip1,clip2,clip3, low=0, high=2)
    This is essentially a minimum function. This is the result with my earlier test capture:

    Click image for larger version

Name:	min.png
Views:	1381
Size:	801.4 KB
ID:	23592

    Code:
    # Discard the lowest two values and keep the highest
    
    MedianBlend(clip1,clip2,clip3, low=2, high=0)
    This is the equivalent maximum function.

    Click image for larger version

Name:	max.png
Views:	1362
Size:	784.4 KB
ID:	23593

    Code:
    # Discard the lowest and highest values, average the two remaining
    
    MedianBlend(clip1,clip2,clip3,clip4, low=1, high=1)
    And this should be what you described. In the case of my clips some half-intensity comets remain (more are visible in the video than in this particular frame).

    Click image for larger version

Name:	blend.png
Views:	1380
Size:	689.8 KB
ID:	23594

    I've also extended the processing to allow for up to 25 input clips.
    Quote Quote  
  9. @ajk
    I quite like your sample, could I have short (~1s) clips of those 5 captures, from the exact same time? In original quality, for example lossless UTVideo. I'm writing a program now to extract the comet profile, as I said it's something I've always been curious about.

    If anyone else has samples, the ideal sample has no dropped frames, obvious comets, uncompressed codec, samples from both PAL and NTSC tapes, a variety of luma values across the frame, stable image (use of TBC), and properly captured with no clipping of the levels (no value more than 234, yes, 234), and at least 3 passes.
    Quote Quote  
  10. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Sure. Here are the five captures that I have. I don't have the UT codec installed so this is in Lagarith YUY2.
    Image Attached Files
    Quote Quote  
  11. @ajk
    p.s.
    You posted at the same time as me, I am delighted to see your updated results. I think the "medianblend" final example looks fantastic, the noise is really reduced. I believe that with masking, those final "half-comets" could be eliminated. I'm happy to work with you on this. I'd also like to see a comparison between medianblend and pure median, to confirm the formula's predictions of better denoising with medianblend.
    Quote Quote  
  12. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Here is the result of a pure median of five captures (same as in the video clip earlier but full resolution and in PNG) for comparison:

    Click image for larger version

Name:	med.png
Views:	1359
Size:	714.7 KB
ID:	23601

    In this particular case I'm more interested in de-cometing than denoising - I'm anyway going to run a light pass of a motion compensated denoiser further down the line. But do keep up with the experiments, there are many kinds of uses for these functions, I hope
    Quote Quote  
  13. I have an idea for auto-profiling. Take the first frame of each clip if >2, and find cases for all pixels where there is one extreme value, call that a comet. A simple way to find an extreme value is to sort, find the smallest delta between any two values, and the 3rd is extreme if it's more than x* the difference. I don't know exactly right now, what if it's nearly noise-free and the delta is around 5? Then a change of double, or 10 would be a comet?

    This is a clustering algorithm, and the standard algorithm is the k-means http://en.wikipedia.org/wiki/K-means_clustering but you could probably come up with your own logic, again by having profiles of what comets are you would have more ideas. Also have to remember people can have wrong recording levels etc. so nothing should refer to absolute luma values.
    Quote Quote  
  14. Ok, I'd have to say the medianblend is slightly less noisy than pure median. The only disadvantage is the half-comets. As for denoising, certainly it would be better for, ahem "detail" if you got the "real" picture without noise rather than an artificially denoised picture.
    Quote Quote  
  15. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Sure. But there will always be noise from the CCD and elsewhere that isn't coming from the playback/capture signal path and can be eliminated like this. So I'm usually going to need a denoising pass in any case, even if the capture itself were "perfect".

    I like the idea of a pure median because it's essentially failure-free, whereas masking is generally really hard to get just right. But a median blend of six captures could work well, at least any remaining comets would be very faint.
    Quote Quote  
  16. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    This thread could use a reviving.

    First off, thumbs up AJK. Nice plugin.

    On three caps, using MedianBlend, I have virtually perfect results removing white specks with parameters low=0,high=2 and the same success with low=2,high=0 on black spots.

    As for handling both polar extremes for the same script - white and black dropouts together in the captures - I did try low=1,high=1, and the plugin still does well, but there's a slight "averaging" in some cases.

    I could keep diminishing the artifacts, or "averaging them out", to near zero, but that would require more captures, which I'd rather avoid.

    However, I am still trying to understand how this happens on a "median" plugin, even if it will "average" (as the readme says) the remaining ones when there's only one remaining. An average of one = median = the same pixel/clip unchanged. No? Could this be a bug, or did I miss something?

    (Again, talking about removing dropouts on captures that have both white and black dropouts - boy, the excitement never ends with my Panasonic AG-1980.)
    Last edited by PuzZLeR; 15th Mar 2014 at 03:51.
    I hate VHS. I always did.
    Quote Quote  
  17. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Originally Posted by PuzZLeR View Post
    This thread could use a reviving.

    First off, thumbs up AJK. Nice plugin.
    Thanks, nice to hear someone is putting it to good use

    On three caps, using MedianBlend, I have virtually perfect results removing white specks with parameters low=0,high=2 and the same success with low=2,high=0 on black spots.
    This sounds correct. With three caps, essentially the function boils down to a minimum or maximum function and there is no averaging element involved. You get the lightest (or darkest) pixel out of each set of three.

    As for handling both polar extremes for the same script - white and black dropouts together in the captures - I did try low=1,high=1, and the plugin still does well, but there's a slight "averaging" in some cases.
    Hmm. Well basically with three caps MedianBlend() with those parameters should be exactly the same as just a Median() out of the three - there is nothing left to average. Could you do a Subtract(a,b) between the Median() and MedianBlend() and see if there is any difference? I can't think of any reason why there would be, it should be running the exact same code in each case. In fact, MedianBlend() probably doesn't make much sense with less than four captures except in particular cases where you only have dark OR light noise.

    Of course, running a median on three captures will always have some appearance of "averaging" or smoothing out the video. After all, it's not just attacking the dark or light spots - pixel values such as 100, 101 and 102 will be processed too and you end up with the middle value. Maybe it's just that? You could limit the processing to only dark and light areas with some masktools magic if necessary.

    Still, if you feel something is not right, give me a couple of examples and I will look into it
    Quote Quote  
  18. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    Hi AJK, thanks for your response. I am not home this weekend, but would be more than happy to provide some examples by tomorrow.
    I hate VHS. I always did.
    Quote Quote  
  19. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    Hi AJK. I took another (much longer) look at it, and the moral of the story is that if one is going to run multiple captures, they should make that special effort to capture all of them under very similar settings for median methods to work effectively.

    That was my problem. The plugin, however is amazing.

    Originally Posted by ajk
    Of course, running a median on three captures will always have some appearance of "averaging" or smoothing out the video. After all, it's not just attacking the dark or light spots - pixel values such as 100, 101 and 102 will be processed too and you end up with the middle value. Maybe it's just that?
    Pretty much.

    I processed three captures that had certain segments within that had different brightness levels. This was due to three caps of a 3hr+ tape (almost 10 combined capture hours) that had different recorded programs in it. I was adjusting the proc amp throughout during capture for the needs of the different programs, but didn't realize that the caps would not be equal in the end.

    I was wondering why some segments worked, and others were "averaging". Well, they weren't "averaging", it was the plugin replacing a dropout with the brigher pixels of the brighter capture, hence the appearance of "averaging" with a dropout.

    You can have a look if you're interested:
    clip1 - almost clear but has a black dropout in frame 6
    clip2 - moderate dropouts - but has a distinct white dropout in frame 4
    clip3 - more "dirtier", yet the clip has more brightness, which caused the problem

    When you run either MedianBlend(clip1,clip2,clip3,low=1,high=1) or Median(clip1,clip2,clip3) the black dropout from clip1 is reduced since it was in clip2 somewhat, but you will see what looks like an "averaging" of the white dropout in frame 4. This is actually the brighter pixels of clip3 placed in there, NOT a dropout averaged like I thought. This is what confused me. My bad.

    Instead of one big gulp, i will capture in smaller doses now to ensure I have very similar settings for all captures for the plugin to be effective.

    Originally Posted by ajk
    Hmm. Well basically with three caps MedianBlend() with those parameters should be exactly the same as just a Median() out of the three - there is nothing left to average. Could you do a Subtract(a,b) between the Median() and MedianBlend() and see if there is any difference? I can't think of any reason why there would be, it should be running the exact same code in each case.
    You are correct. In all my tests, including this example, Subtract(MedianBlend(clip1,clip2,clip3,low=1,high= 1),Median(clip1,clip2,clip3)) does produce the gray screen of equality.

    The plugin works as expected on my end.

    Originally Posted by ajk
    Thanks, nice to hear someone is putting it to good use
    Oh, for sure. For those of us who need it, this is a blessing. I personally will put it to work in the coming weeks on tapes that I've delayed. They always looked great on my Panasonic AG-1980, but since the machine is aging and has side-effects (such as dropouts), and replacing such a unit in 2014 is not a viable option, your plugin is the ultimate capture/restoration solution for it.

    Thank you so much for this little gem!
    Image Attached Files
    I hate VHS. I always did.
    Quote Quote  
  20. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Originally Posted by PuzZLeR View Post
    Hi AJK. I took another (much longer) look at it, and the moral of the story is that if one is going to run multiple captures, they should make that special effort to capture all of them under very similar settings for median methods to work effectively.
    Indeed - I always capture the same tape (or segment) three or five or however many times with absolutely the same settings, just rewinding the tape. I don't want to shut down VirtualDub, reboot the system, or anything in between if at all possible. Otherwise it's nearly certain that some setting will have been reset or changed Good that you have it sorted now, I'll still have a look at your samples just out of interest.

    Oh, for sure. For those of us who need it, this is a blessing. I personally will put it to work in the coming weeks on tapes that I've delayed. They always looked great on my Panasonic AG-1980, but since the machine is aging and has side-effects (such as dropouts), and replacing such a unit in 2014 is not a viable option, your plugin is the ultimate capture/restoration solution for it.

    Thank you so much for this little gem!
    Excellent to hear! I can't take the credit for the method, of course, as it has been possible to do these things in AviSynth with scripting and other plugins, but I wanted something that is simple to use and works on any kind of input. And it's also fun to work on little projects like this
    Quote Quote  
  21. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    Originally Posted by ajk
    Indeed - I always capture the same tape (or segment) three or five or however many times with absolutely the same settings, just rewinding the tape. I don't want to shut down VirtualDub, reboot the system, or anything in between if at all possible. Otherwise it's nearly certain that some setting will have been reset or changed Good that you have it sorted now, I'll still have a look at your samples just out of interest.
    The thing is that alot of my tapes have multiple recordings, even as small as dozens of music videos, which each need different treatment with a proc amp. That was the problem I experienced earlier - when trying to do the WHOLE thing before the next capture - some captures will come out different. For those tapes, I will just capture, and recapture, smaller segments at a time to better ensure a true median in processing. To get rid of those dropouts it's worth it to shorten the workflow's cycle.

    Originally Posted by alk
    Excellent to hear! I can't take the credit for the method, of course, as it has been possible to do these things in url=http://www.videohelp.com/tools/Avisynth]AviSynth[/url] with scripting and other plugins, but I wanted something that is simple to use and works on any kind of input. And it's also fun to work on little projects like this
    You sure can take credit for this when alot of the other ones fall short in many ways. Personally, the medianblur package wasn't for me - I just don't get the options. And this averaging and clensing thing? Don't get it in a solution at the capture level, or mixing in when a true median is required, or even in damaging other parts of the video in compromise - that is IF it's effective at all in removing anything. And many need further components, that can take forever to find, for system32, etc. And I have no appetite to sludge through long boring scripts on doom9 (in which most never work anyway). Your plugin was simple and to the point, and gets the job done without useless showing off, and is fun to use.
    I hate VHS. I always did.
    Quote Quote  
  22. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    Time for a little update.

    Using a median of several captures is effective at removing all kinds of issues, but it relies on the captures matching exactly. In principle this is easy, all you need to do is Trim() each clip to start from the same point. In practice, however, analog captures are rarely all identical due to possible frame drops and other timing issues.

    Making sure several captures match can be a very tedious task if done by manual means. So I thought I'd try to automate the process by implementing a frame comparison feature into Median(). This is how you might use it:

    Code:
    Median(clip1, clip2, clip3, clip4, clip5, sync = 3)
    By adding the sync parameter Median() will, for each frame in clip1, search for the best matching frame in clips 2-5 up to a distance of 3 frames back or ahead.

    To keep the impact on processing time down frames will not be compared pixel-by-pixel by default. But if you have a challenging set of clips, you can add the parameter exact = true and then the frames will be compared in their entirety.

    The test version is available here: Median-experimental.zip

    This experimental version will also output debug information that can be seen using DebugView: https://technet.microsoft.com/en-us/sysinternals/debugview.aspx

    Just run DebugView and stepping through your .avs script you should see log entries such as:

    Code:
    median: syncing frame 2 with clip: 3/5	
    median: offset: -3, difference: 3.470222	
    median: offset: -2, difference: 3.470222	
    median: offset: -1, difference: 1.463862	
    median: offset: +0, difference: 3.373855	
    median: offset: +1, difference: 4.656302	
    median: offset: +2, difference: 5.664156	
    median: offset: +3, difference: 7.346792	
    median: best match with: -1
    Any testing and feedback of this feature would be greatly appreciated!
    Quote Quote  
  23. Wow, that is a really ambitious undertaking. I'll be interested to hear whether it works. I gave up on the multiple capture process because it is so extraordinarily tedious, not only waiting for the multiple captures to finish, but then when I found myself spending hours trying to track down where one or more capture lost sync, and then fixing it, I was never tempted to try it again.

    For most projects, where someone gives me 20-30 cassettes to transfer (typical project) multiple captures are out of the question. However, for that wedding video, or something else that is extraordinarily precious, the extra quality and lack of artifacts that this approach provides still makes it an important tool to have available.

    Thanks for doing this!
    Last edited by johnmeyer; 12th Nov 2015 at 11:38. Reason: Hit save, rather than preview, and hadn't finished.
    Quote Quote  
  24. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    This is totally awesome. Thank you!

    And, just to comment:

    Time for a little update.
    Much debate has been about "which filters are best for VHS caps". Your median filter would be my vote, and this update should secure it.

    Originally Posted by ajk View Post
    Using a median of several captures is effective at removing all kinds of issues, but it relies on the captures matching exactly. In principle this is easy, all you need to do is Trim() each clip to start from the same point. In practice, however, analog captures are rarely all identical due to possible frame drops and other timing issues.
    Yes, simply lining them up with Trim() to a designated common frame is not enough. At least not for most captures.

    The problem (as I mentioned in other threads) is indeed dropped frames, even when using a good VCR, good tape, TBC, fast and dedicated computer, good hard drive, buffers, etc. It just happens.

    Just because an application like VirtualDub doesn't report them does not mean they're not there. Yes. You may have dropped frames.

    And this is due to tape drift, or stretch, or a number of other physical factors beyond what other preventative measures may be to avoid this, and furthermore compounded on bad tapes, or recording "seams" where content (on the tape) has been paused/stopped midway, recorded over, etc. Your multiple captures are very likely to go out of sync in these cases.

    Many have been capturing for many years and have no clue they have dropped frames, and may be horrified to reveal how many dropped frames their captures may have, and the only real way to catch this is with comparing multiple captures.

    Analog to digital is indeed a chaotic transition.

    Originally Posted by ajk
    Making sure several captures match can be a very tedious task if done by manual means. So I thought I'd try to automate the process by implementing a frame comparison feature into Median().
    For the record, it could be corrected manually. But yes, can be tedious.

    Just line them up with Trim(), stack them with StackHorizontal() to view them side by side, and scrub through and look for misaligned video. A good clue is a scene change, where one capture changed a frame ahead of the others. That means that that particular capture has lost a frame earlier on, and you have to back up to find it, then some trial and error to zero in on where there's a missing frame.

    When you do find that dropped frame, just use DuplicateFrame() to artificially implement the extra frames so they sync. Or, if it's an extra frame, just eliminate it with DeleteFrame().

    Having only a drop or two is not so bad. Either it's an easy fix, or it can even be ignored since the consequences are rather frivolous.

    However...

    Having many drops will produce an output with horrible artifacts, or would need plenty of time to fix. For example, say you have 5 captures of a two hour production, and each has one dropped frame about every 10 minutes. This means that you will have to do this hunting for dropped frames 60 times - this can take a good part of an afternoon to do, so this is where your plugin's update is most welcome.
    Originally Posted by ajk
    Any testing and feedback of this feature would be greatly appreciated!
    I look forward to testing it, and, of course for personal use later on. Much appreciated. I just need a few days/weeks to run some longer captures as I have none outstanding at the moment, but will get on to it right away!
    I hate VHS. I always did.
    Quote Quote  
  25. Two quick comments, both of which I've made before, but bear repeating in this context.

    1. The multiple capture technique only removes noise introduced during playback. Anything that is already on the tape will not be removed. And, with VHS, there is a lot of junk that is on the tape from the moment the tape is captured. Thus, you will almost certainly still need to apply other filters even after you've spent all that time doing multiple captures. And, since these filters will also deal with the noise introduced during playback, the improvement from this method may not be that much with tapes that have lots of other problems.

    Put another way, multiple capture is a great way to deal with tapes that are in great shape, where the predominant problem is the noise issues introduced during playback.

    2. When I read this:

    Many have been capturing for many years and have no clue they have dropped frames, and may be horrified to reveal how many dropped frames their captures may have,
    I once again am reminded about other threads where people try to help others with all sorts of problems introduced by capture cards. I myself, ten years ago, gave up using my ATI Radeon 8500 capture card because no matter what capture software I used, I had problems with levels, dropped frames, etc. When everything did work, with the stars and moon aligning, the captures were really good. But I could never rely on this happening. By contrast, even with the 4:1:1 issues of DV, when I used the passthrough on my camcorder, and later purchased a VHS deck with DV capture built in, I never dropped another frame and never had any levels issues (other than 4:1:1). So, even though I know that many in this forum vehemently disagree with me, this is why I keep recommending using DV capture for VHS transfers for people who report having problems with dropped frames and levels when capturing using a capture card.

    I'm sure more modern, and more expensive capture cards are much better, so please don't take this comment as a blanket statement about all captures cards.
    Quote Quote  
  26. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    @johnmeyer

    I ran out of patience with one of my tapes, which is why I am trying to get the computer to do it for me. There weren't that many dropped frames in the hour or so of footage - you'd most likely not notice anything wrong just viewing any of them regularly - but with five captures it still meant a heck of a lot of trimming and splicing.

    My own use for this are mainly Hi8 tapes, which seem particularly prone to the random white comets some of my samples posted earlier in this thread exhibit. Maybe it's just my camcorder, although I have two and both do this. Anyway it's great if it works for VHS and other sources too.


    @PuzZLeR

    I have actually given some thought to completely remedying the dropped frame situation. It is in principle possible to look at a number of captures and figure out if one of them is missing a frame, and then insert a copy from one of the other clips. But the complexity involved in referencing each video stream with all the others is on a whole other level from what I have now. So for the time being, clip1 will be the master and the others will be matched to it.


    I mainly developed this thing for my own needs. It has been about a year and a half with no messages on this thread, or on Doom9, so I really have no idea if anyone (aside from PuzZLeR) is using this filter or not. If anyone is, please do post some samples, it'd be great to see some successful restorations
    Quote Quote  
  27. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    Originally Posted by johnmeyer
    The multiple capture technique only removes noise introduced during playback. Anything that is already on the tape will not be removed. And, with VHS, there is a lot of junk that is on the tape from the moment the tape is captured. Thus, you will almost certainly still need to apply other filters even after you've spent all that time doing multiple captures. And, since these filters will also deal with the noise introduced during playback, the improvement from this method may not be that much with tapes that have lots of other problems.

    Put another way, multiple capture is a great way to deal with tapes that are in great shape, where the predominant problem is the noise issues introduced during playback.
    Oh, I agree. This is not the end-all-fix-all. You'd likely still need some work to do afterwards as well to remove the more "permanent" issues.

    However, there's enough issues to deal with without playback quirks. There's no reason to also add the random issues too. After using this plug-in the rest of the way is so much easier. You may not be there, but this plugin clears alot of the way for sure.

    And yes, when the tape is clean, and relatively problem-free, this plugin is gold. I don't really do commercial tapes, but I've used this plugin on some professional shoots, like weddings, etc, and the quality has been impressive, and those that I've done it for have raved about how good it looks - even better than the tape.
    I hate VHS. I always did.
    Quote Quote  
  28. Member PuzZLeR's Avatar
    Join Date
    Oct 2006
    Location
    Toronto Canada
    Search Comp PM
    Originally Posted by ajk;
    I have actually given some thought to completely remedying the dropped frame situation. It is in principle possible to look at a number of captures and figure out if one of them is missing a frame, and then insert a copy from one of the other clips. But the complexity involved in referencing each video stream with all the others is on a whole other level from what I have now. So for the time being, clip1 will be the master and the others will be matched to it.
    I see the mechanics you're using.

    I've used the plugin for 3 or 5 captures, depending. Any one of them, or even any subsequent if I chose to do so, is under similar circumstances no better or worse than the other, and could have easily been designated as a "Master", so using clip1 is good enough.

    Ok, if one is really picky, thinking that maybe the first capture is problematic as the tape may still be sticky or whatnot, just use another capture instead as the Master, or just rename the files. However, capturing under exact conditions produces very similar quirks in similar distribution.

    As per the referencing you're doing, let me comment. Just duplicating a frame adjacent to the dropped one was always good enough, and never showed in the end result. I've done some real closeups, but a very miniscule halo on one measly un-noticeable frame was the worst it got, and not even. And all this referencing does anyway is just add a duplicate of another capture, which doesn't really make much improvement anyway, so I didn't bother with the extra work. Either way was good enough for me when it came to fixing a solo dropped frame in the midst of thousands of good ones.

    Ok, an insertion of a good frame from another capture is still better than a duplicate of an adjacent frame to fix a drop. But if you had to do it for 60 hunts manually, as my earlier example, you'd spend a weekend instead of an afternoon for maybe a 0.0005% benefit for the final production. If your update does this - great - as long as I don't have to do it.

    Originally Posted by ajk
    I mainly developed this thing for my own needs. It has been about a year and a half with no messages on this thread, or on Doom9, so I really have no idea if anyone (aside from PuzZLeR) is using this filter or not. If anyone is, please do post some samples, it'd be great to see some successful restorations
    Does this make me your biggest fan?

    It reminds of my H.264 SD blu-ray guide. It served me well, but I do understand it's quite niche, so maybe this is too.

    But such techniques are used in the sciences, such as astronomy, and more people should be encouraged to use multiple captures and median methods, even at the professional level. It would be a shame otherwise.

    Yes, I have used the plugin plenty since, and in full habit, and I thank you. But yes, I too would like to see what other people have done with it, and any new ideas in the process.
    Last edited by PuzZLeR; 12th Nov 2015 at 14:40.
    I hate VHS. I always did.
    Quote Quote  
  29. [QUOTE=PuzZLeR;2418663]
    Originally Posted by ajk;
    Ok, an insertion of a good frame from another capture is still better than a duplicate of an adjacent frame to fix a drop. But if you had to do it for 60 hunts manually, as my earlier example, you'd spend a weekend instead of an afternoon for maybe a 0.0005% benefit for the final.
    FWIW, if you have access to Sony Vegas Pro, one of the great things about that NLE is its scripting language. I have developed 100+ scripts that are really great for things like this. For instance, I have a one-keypress script that lets me replace the frame showing in the preview with a duplicate of the previous frame.

    Bad frame? Press a key: gone.

    It would be just as easy to replace the current frame with a duplicate of the frame on another capture located on any other track in the project. The script could be modified in a second to permit using a frame that is several frames ahead or behind. And, since I can also create dialogs, I could have a dialog pop up that would let you choose which track, and how much offset you wanted for your frame replacement.

    If anyone is using Vegas, just tell me what you need and I'll post the appropriate script.
    Quote Quote  
  30. Member
    Join Date
    Dec 2005
    Location
    Finland
    Search Comp PM
    @PuzZLeR

    Yeah, clip1 is of course just used for convenience here, and the master clip can be any of the captures. If one appears to have less dropped frames than the others, that's the best one to use.

    You probably are the biggest fan of this plugin, so how about some examples from past work


    @johnmeyer

    Using Vegas or any GUI based tool is certainly a great deal better for aligning clips than AviSynth (I have Movie Studio but not Vegas Pro, unfortunately). But what I had in mind is a completely automated solution. Let me see if I can explain.

    Currently, the system I have implemented in the plugin looks at a frame from the first input clip (clip1). It then looks at all the other clips, comparing not only with the frame with the same number, but also n earlier frames and n later frames. For each, it calculates a difference indicator (0% to 100%, 0% being an identical frame). Then, for calculating the output, it will use whichever frames have been deemed to be the best matching.

    Now, as it stands, if clip1 is missing a frame which exists in all the other clips, all those others will simply be discarded. But if we were to compare each clip to every other clip, it would be easy to see that a frame in fact exists in four out of five (or however many) clips, and should be included in the output.

    However this is not trivial to code into a plugin, at least not in a way that is not excessively slow. I dug out the script for my most troublesome tape, and even that had only about 10 or so dropped or duplicated frames over the whole hour. So this isn't a feature that deserves a lot of effort, at least for now.
    Quote Quote  



Similar Threads