VideoHelp Forum
+ Reply to Thread
Results 1 to 21 of 21
Thread
  1. Hi everyone,

    I'm helping a friend with converting couple of old VHS-Tapes he recorded using his Camcorder. While most captures were decent, one specific tape is pretty bad. It suffers from a lot of noise, but also comets. Attaching an example (I picked small section where the comets are easy to detect due to the background).

    I was checking couple of old threads, and was trying DePulse() (playing around with the values to L=180, and D=30). While it does remove some of them, it doesn't remove those on the far left side. Another recommendation here was to try RemoveSpots(), but I couldn't find a plugin name like so. I did find RemoteDirt, maybe it's the same one?
    I tried to use RemoveDirt() port DePulse(), but that made things worse (and alone it doesn't do much). Ideas?

    Thanks!
    Image Attached Files
    Quote Quote  
  2. I've had pretty good luck with RemoveDirt or the motion-compensated version, RemoveDirtMC. The former is a DLL and the later is a function which uses that DLL along with MVTools2 to achieve better resuls.
    Quote Quote  
  3. I think I tried RemoveDirt(). If I recall correctly, RemoveDirt() is a Function you have to manually add and calls RestoreMotionBlocks. I will check it again once home, but I think It didn't do much.
    Quote Quote  
  4. Member DB83's Avatar
    Join Date
    Jul 2007
    Location
    United Kingdom
    Search Comp PM
    I do not know if a avisynth filter exists (would be surprised if there was not one) but I have had good results using a Median filter for comets.
    Quote Quote  
  5. That's would work if the noise isn't identical on multiple videos right? I need to check if that's indeed the case.
    Quote Quote  
  6. Originally Posted by Okiba View Post
    I think I tried RemoveDirt(). If I recall correctly, RemoveDirt() is a Function you have to manually add and calls RestoreMotionBlocks. I will check it again once home, but I think It didn't do much.
    No, it is not that complicated. This is the RemoveDirtMC function which uses the RemoveDirt DLL. You can simply use RemoveDirt, and call it as shown within this function.

    Code:
    function RemoveDirtMC_old(clip,int limit, bool "_grey")
    {
      _grey=default(_grey, false)
      limit = default(limit,6)
      i=MSuper(clip,pel=2)
      fvec = MAnalyse(i,isb=false, blksize=16, delta=1, truemotion=true)
      bvec = MAnalyse(i,isb=true,  blksize=16, delta=1, truemotion=true)
      backw = MFlow(clip,i,bvec)
      forw  = MFlow(clip,i,fvec)
      clp=interleave(forw,clip,backw)
      clp=clp.RemoveDirt(limit,_grey)
      clp=clp.SelectEvery(3,1)
      return clp
    }
    Quote Quote  
  7. Then I probably picked the wrong plugin/wrong version. I grabbed it from here. Where I try to run "RemoveDirt()", AvsPMod auto-complete it and show me the parameters, but AviSynth will complain there is no function named "RemoveDirt()". What I did around it (which was probably wrong) was to take the RemoveDirt that mentioned on the Plugin page - paste into my script, and call it:

    Code:
    function RemoveDirt(clip input, bool "_grey", int "repmode") 
    {
        _grey=default(_grey, false)
    	repmode=default(repmode, 16)
    	clmode=17
    	clensed=Clense(input, grey=_grey, cache=4)
    	sbegin = ForwardClense(input, grey=_grey, cache=-1)
    	send = BackwardClense(input, grey=_grey, cache=-1)
    	alt=Repair(SCSelect(input, sbegin, send, clensed, debug=true), input, mode=repmode, modeU = _grey ? -1 : repmode ) 
    	restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
    	corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, debug=false, noise=10, noisy=12, grey=_grey)
    	return RemoveGrain(corrected, mode=clmode, modeU = _grey ? -1 : clmode )
    }
    It works. But the results were just OKish. It seems the script you shared is different. Is it possible the DLL only work with AviSynth and not with AviSynth+? or am I'm missing something here?

    EDIT: Seems like the latest version (0.9.2) doesn't come with RemoveDirt() buildin. You have to use the older version if you want it or use the script above.
    Last edited by Okiba; 16th Nov 2020 at 16:43.
    Quote Quote  
  8. Originally Posted by Okiba View Post
    Then I probably picked the wrong plugin/wrong version. I grabbed it from here. Where I try to run "RemoveDirt()", AvsPMod auto-complete it and show me the parameters, but AviSynth will complain there is no function named "RemoveDirt()". What I did around it (which was probably wrong) was to take the RemoveDirt that mentioned on the Plugin page - paste into my script, and call it:

    Code:
    function RemoveDirt(clip input, bool "_grey", int "repmode") 
    {
        _grey=default(_grey, false)
    	repmode=default(repmode, 16)
    	clmode=17
    	clensed=Clense(input, grey=_grey, cache=4)
    	sbegin = ForwardClense(input, grey=_grey, cache=-1)
    	send = BackwardClense(input, grey=_grey, cache=-1)
    	alt=Repair(SCSelect(input, sbegin, send, clensed, debug=true), input, mode=repmode, modeU = _grey ? -1 : repmode ) 
    	restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
    	corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, debug=false, noise=10, noisy=12, grey=_grey)
    	return RemoveGrain(corrected, mode=clmode, modeU = _grey ? -1 : clmode )
    }
    It works. But the results were just OKish. It seems the script you shared is different. Is it possible the DLL only work with AviSynth and not with AviSynth+? or am I'm missing something here?

    EDIT: Seems like the latest version (0.9.2) doesn't come with RemoveDirt() buildin. You have to use the older version if you want it or use the script above.
    That function is RemoveDirt() when using the current build. Include it in your script or put in in your plugins folder as an AVSI file.
    Quote Quote  
  9. Member
    Join Date
    May 2005
    Location
    Australia-PAL Land
    Search Comp PM
    I have comets like yours on a tape. When I played it in a different VCR, no comets.
    Quote Quote  
  10. That function is RemoveDirt() when using the current build. Include it in your script or put in in your plugins folder as an AVSI file.
    Yep. That's what I did. It didn't solve all the comets, and I didn't like it being aggressive with the modification (I assume that's because it calls RemoveGrain()). I fiddled with the inner parameters a bit (repmode, checked what Restore() do, removed RemoveGrain() etc), but nothing was able to eliminate all the comets. Three's a lot of parameters to play with, so perhaps I need to keep trying.


    I have comets like yours on a tape. When I played it in a different VCR, no comets.
    That's indeed mostly the cast for me too, but on this specific tape, comets are showing on 3 different VCRs.
    Quote Quote  
  11. Member DB83's Avatar
    Join Date
    Jul 2007
    Location
    United Kingdom
    Search Comp PM
    You do appear somewhat dismissive of the help that people are offering - they do know what they write about.


    And a function is just that - it is a container for the tools that actually do the work. Remove parts of that and the function no longer functions. Simple as that.
    Quote Quote  
  12. Dismissive? that wasn't my initiation and I'm sorry if anything I said made you feel that. I have nothing but respect to everyone here helping a total stranger on the internet. I have no doubt most people here know much more then I am (see my previous threads on this forum), and jagabo and johnmeyer specifically helped me so many times. So again - nothing but respect

    And to the subject - the reason I tried to tweak the "RemoveDirt" function is because the author mentioned the script was used during tests. I assumed the fact that "RemoveDirt()" is no longer part of the Dll, was because the author wish only to supply the basic functions (RestoreMotionBlocks and CSelect), and wish you to wrap them based on your own needs (and he shared him own tests example on how to use them). Running RemoveDirt() suggested by the author didn't solve all the comets, so I was checking the method being called internally (RestoreMotionBlocks) and it has many parameters you can tweak, like noise level, tolerance, dist etc. I'm trying to play around with them see if I can achieve better results. I'm avoiding changing any of the internal logic like the Cleanse, Forwardclense etc.
    Quote Quote  
  13. I haven't used RemoveSpotsMC in years. And what I had didn't work with my modern AviSynth+ installation. But I pieced together some old scripts and came up with:

    Code:
    function RemoveSpots(clip input, bool "_grey", int "repmode") 
    {
        _grey=default(_grey, false)
    	repmode=default(repmode, 16)
    
    	clensed=Clense(input, grey=_grey, cache=4)
    	sbegin = ForwardClense(input, grey=_grey, cache=-1)
    	send = BackwardClense(input, grey=_grey, cache=-1)
    	alt=Repair(SCSelect(input, sbegin, send, clensed, debug=true), input, mode=repmode, modeU = _grey ? -1 : repmode ) 
    	restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
    	corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, debug=false, noise=10, noisy=12, grey=_grey)
    	return corrected
    }
    
    function RemoveSpotsMC(clip, int "repmode", bool "_grey")
    {
      _grey=default(_grey, false)
      repmode = default(repmode, 6)
    
      i=MSuper(clip,pel=2)
      bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
      fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
      backw = MFlow(clip,i,bvec)
      forw  = MFlow(clip,i,fvec)
      clp=interleave(backw,clip,forw)
      clp=clp.RemoveSpots(repmode=repmode, _grey=_grey)
      clp=clp.SelectEvery(3,1)
      return clp
    }
    
    LWlibavVideoSource("Drops.avi") 
    AssumeTFF()
    src = last
    
    Separatefields()
    even = SelectEven().RemoveSpotsMC(repmode=2, _grey=false)
    odd = SelectOdd().RemoveSpotsMC(repmode=2, _grey=false)
    Interleave(even,odd)
    #RemoveSpotsMC(repmode=2, _grey=false)
    
    Weave()
    StackHorizontal(src.SeparateFields(), last.SeparateFields())
    This removes most of the comets and a lot of other noise (not necessarily a good thing). Some of the remaining comets are because of duplicate fields/frames in the cap. Some of those can be eliminated by re-enabling the commented out RemoveSpotsMC() after the Interleave(). You can play around with the repmode parameter to see what works better/worse. I don't really understand what it does.
    Quote Quote  
  14. Thank you for spending time piecing up those old scripts jagabo! Will be home later and will give it a try

    Some of the remaining comets are because of duplicate fields/frames in the cap.
    Interesting. It's the same setup I used up until now. What makes this capture different? the quality of the tape? DePulse with debug=true marks pretty neatly what it detects to be comets, and some sections are unidentifiable. It's probably those duplicated frames.

    to know what's being fixed, an no matter how much I played with the parameters, some comets were not being addressed. It's probably because of that.
    That explains some of the issues I have with some comets. Some comets were never getting address (or even marked in debug mode) in all the attempt I tried.
    Quote Quote  
  15. With:

    Code:
    LWlibavVideoSource("Drops.avi") 
    AssumeTFF()
    SeparateFields()
    Subtract(last, last.Trim(1,0))
    you can see that there are duplicate fields at 50 and 51, 83 and 84, 181 and 182, etc. This often happens when a device misses a field -- it substitutes a second copy of the previous field instead. Note that Subtract() subtracts one input from the other (pixel by pixel) and adds 126. So when the two frames are identical you are left with a flat medium grey image. The duplicates here aren't 100 percent identical -- there are very small differences. That indicates this is happening before the capture device. Something (TBC?) digitized the video, duplicated a field, then output analog again. A small amount of noise in that analog output is showing up in the cap and hence in the Subtract() result.
    Quote Quote  
  16. The script you shared jagabo indeed have more impact compared to the "RemoveDirt()" function. But perhaps, I haven't been using RemoveDirt() correctly. Perhaps it too require separating the fields and weaving them back. Using repmode=2 indeed get rid of ALL the comets, but it's feels too aggressive on noise as you mentioned. I was checking the Repair documentation to figure how the difference modes work, but it was too technical for me to understand. What I did is just try them. 14 was almost flawless (minimal amount of comets) - and I didn't feel it too aggressive on noise, so that's sounds like a good solution. I doubt my friend will be able to tell the difference between modes. Anyhow, I stored a Lossless copy for him, so perhaps in the future we can have another go if he will mind.

    That indicates this is happening before the capture device. Something (TBC?) digitized the video, duplicated a field, then output analog again.
    Oh, I see. It's probably because it's a DVD Combo. It probably fiddles with the frames. I have another VCR deck here with just VCR and component out. I tried it out as I'm pretty sure the signal is not being digitized there. The results were more Comets, and I wasn't able to remove them with just DePulse again, had again be more aggresive with Repair mode, so I figure I'll stick with the Combo as the overall image quality is better.

    So I think I'm pretty settled with the results. Thank you very much everyone! I appreciate your help!
    Quote Quote  
  17. You can reduce the general noise reduction of RemoveSpots() while still removing most of the comets by using a mask to overlaying the cleaned image only the areas where comets were removed.

    Code:
    function RemoveSpots(clip input, bool "_grey", int "repmode") 
    {
        _grey=default(_grey, false)
    	repmode=default(repmode, 16)
    
    	clensed=Clense(input, grey=_grey, cache=4)
    	sbegin = ForwardClense(input, grey=_grey, cache=-1)
    	send = BackwardClense(input, grey=_grey, cache=-1)
    	alt=Repair(SCSelect(input, sbegin, send, clensed, debug=true), input, mode=repmode, modeU = _grey ? -1 : repmode ) 
    	restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
    	corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, debug=false, noise=10, noisy=12, grey=_grey)
    	return corrected
    }
    
    function RemoveSpotsMC(clip, int "repmode", bool "_grey")
    {
      _grey=default(_grey, false)
      repmode = default(repmode, 6)
    
      i=MSuper(clip,pel=2)
      bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
      fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
      backw = MFlow(clip,i,bvec)
      forw  = MFlow(clip,i,fvec)
      clp=interleave(backw,clip,forw)
      clp=clp.RemoveSpots(repmode=repmode, _grey=_grey)
      clp=clp.SelectEvery(3,1)
      return clp
    }
    
    
    LWlibavVideoSource("Drops.avi") 
    AssumeTFF()
    src = last # remember the original source for comparison at the end
    Separatefields()
    
    before = last # remember original fields
    
    # heavy noise reduction and comet removal
    even = SelectEven().RemoveSpotsMC(repmode=14, _grey=false)
    odd = SelectOdd().RemoveSpotsMC(repmode=14, _grey=false)
    Interleave(even,odd)
    rs = RemoveSpotsMC(repmode=14, _grey=false)
    
    # build a mask of the removed comets by subtracting the cleaned field from the original
    cmask=Subtract(before, rs).BilinearResize(width/3,height).mt_binarize(110).PointResize(width,height).Invert().mt_expand(chroma="-128").Blur(1.0)
    # only overlay the comets
    Overlay(before, rs, mask=cmask)
    
    # use the next line to view the intermediate results
    #return(Interleave(before, last, cmask, rs)) # before any cleanup, only comets removed, the comet mask, the full RemoveSpotsMC result
    
    Weave()
    
    #Show original and cleaned with QTGMC()
    StackHorizontal(src.QTGMC(), last.QTGMC())
    You can play around with the mt_binarize() threshold to catch more or less of the faint comet tails (and other noise). Or you can use more traditional noise reduction. Also, the Overlay doesn't have to be all or nothing. You can manipulate the mask so that a percentage of the general noise reduction is applied too.
    Quote Quote  
  18. It sounds like you are making progress with RemoveDirt. If you give up on that, I used Despot for fixing comets (I used to call them "tears") many years ago. This was the script I developed. The only downside is that tuning Despot is a bitch.

    This script is designed for interlaced video.
    Code:
    function Remove_TearsI(clip source) 
    {
    
    ml           = 100 # mask scale
    scene_change = 200 # scene change
    
    fields=source.SeparateFields()
    even_fields=selecteven(fields)
    odd_fields =selectodd(fields)
    
    even_result_fields = RemoveTears(even_fields,ml,scene_change)
    odd_result_fields  = RemoveTears(odd_fields, ml,scene_change)
    
    #=================
    #Put even and odd fields back together into interlaced video
    
    interleave(even_result_fields,odd_result_fields)
    weave()
    
    } # End Function RemoveTearsI
    
    function RemoveTears(clip source, int ml, int scene_change) 
    {
    show=0
    tsmooth=0
    p1percent=2
    dilate=20
    maxpts=0
    p2=12
    mthres=16
    p1=24
    pwidth=100
    pheight=12
    mwidth=7
    mheight=5
    merode=33
    interlaced=false
    seg=1
    sign=0
    ranked=true
    
    super = MSuper(source,hpad=16, vpad=16, pel=2, sharp=1)
    
    vb = MAnalyse(super,isb = true,  blksize=8,overlap=4,search=3,dct=0) # backward vectors 
    vf = MAnalyse(super,isb = false, blksize=8,overlap=4,search=3,dct=0) # forward vectors 
    
    cb = MFlow(source,super,vb,thSCD1=scene_change) # next compensated backward 
    cf = MFlow(source,super,vf,thSCD1=scene_change) # previous compensated forward
    
    sadf = MMask(super,vf, ml=ml,kind=1,gamma=1, thSCD1 = scene_change)
    msadf= sadf.Binarize()           # binary inverted forward SAD mask
    
    sadb = MMask(super,vb, ml=ml, gamma=1, kind=1, thSCD1 = scene_change) # backward SAD mask
    msadb= sadb.Binarize() # binary inverted backward SAD mask
    
    msad = Logic(msadf,msadb,"OR") # combined inverted SAD mask
    msad = msad.Expand() # expanded inverted SAD mask
    msadi = Interleave(msad, msad, msad) # interleaved 3-frame inverted SAD mask
    
    Interleave(cf,source,cb) # interleave forward compensated, source, and backward compensated
    
    DeSpot(show=show,extmask=msadi,tsmooth=tsmooth,p1percent=p1percent,dilate=dilate,maxpts=maxpts,p2=p2,\
           mthres=mthres,p1=p1,pwidth=pwidth,pheight=pheight,mwidth=mwidth,mheight=mheight,merode=merode,\
           interlaced=interlaced,seg=seg,sign=sign,ranked=ranked).SelectEvery(3,1)
    
    }
    Quote Quote  
  19. You can reduce the general noise reduction of RemoveSpots() while still removing most of the comets by using a mask to overlaying the cleaned image only the areas where comets were removed.

    Brilliant. That sustainably improved the harshness of the noise reduction. 130 mt_binarize was the sweet spot for this tape. Thank you jagabo!
    As a side note, I compared the "RemoteDirt()" script suggested in the Official page, to the one you suggested ("RemoveSpots"), and To me, RemoveSpots looks better. Also, RemoveSpots() wasn't able to get rid of all the comets - but running DePulse along with RemoveSpots() removed the rest.

    This was the script I developed. The only downside is that tuning Despot is a bitch.
    Will give it a go once I'm home. Thank you for sharing it But yea, There's seems to be a lot of parameter you need to setup, I'm not sure what most do and what the proper values should be. I will do some reading on Despot when I'm home. Thanks johmeyer!
    Quote Quote  
  20. Also, that BilinearResize(width/3,height) was used to blur away small dots. You can change that to a smaller or higher with to allow less or more small black dots to be cleaned too.
    Quote Quote  
  21. Ermm, will give it a try. Thank you!
    I also had some success by running DePulse() twice if anyone will find it useful for his case.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!