VideoHelp Forum
+ Reply to Thread
Results 1 to 7 of 7
Thread
  1. [edit]Here is a link to a better version of the script, later in this thread: Better Version of Script

    Long ago I created a script that I used to find individual blank frames. I needed this because my NLE (Vegas) has a nasty habit of creating random blank frames during certain types of renders, and you don't know if you have the problem without a tool which can look at every single frame.

    I then found I needed a script to find "flash" frames because I do movie film transfers, and because of the way home movie cameras work, the first frame of every single scene is horribly overexposed. These therefore need to be found and removed.

    As time went on, I found all sorts of other situations where a video can contain a single frame which does not match either adjacent frame.

    I recently had a request for this script, so I went back and cleaned it up a bit and added some comments. The main tool used for detection is the YDifference function built into AVISynth. It compares each pixel in the current frame to each corresponding pixel in the adjacent frame. The metric "blows up" whenever there is a big difference in lots of the pixels, something that happens at scene changes, but also when there is a single-frame corruption. What's more, the metrics blow up looking both backwards and forwards when there is an individual bad frame, whereas they only blow up in one direction at a scene change. Therefore, this function will not do anything at scene changes (which is what you want).

    Other types of detection could easily be substituted, using either the other stat functions built into AVISynth, or by using the myriad of compare functions built into StainlessS' excellent RT_Stats package.

    Here's the script. I hope it helps a few people.

    Code:
    #Find And (Optionally) Fix Bad Frames
    #John Meyer - December 12, 2016
    
    #This script detects single bad frames. You have two options of what to do.
    #You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad".
    #As an alternative, you can configure it to automatically replace bad frames with a new
    #frame that is interpolated from its neighbors. This replacement is often near-perfect (no guarantees, however...)
    
    #This script will fail if the bad frame happens immediately before or after a scene change.
    #This script will also fail to find a bad frame if there is more than one bad frame in a row.
    
    #It works very well for finding both blank frames and also "flash" frames (like those caused
    #by a photographer's flash). It will also find single frames which have lots of 
    #static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like
    #a film frame that wasn't properly registered in the film gate, or an analog
    #video frame that lost vertical sync.
    
    #When using VirtualDub, to create the text file containing the bad frame numbers, first uncomment that code block.
    #Then, select "Run Video Analysis Pass" in the VirtualDub File menu.
    
    #The script uses ratios of the metrics for the current frame to the same metrics 
    #on the two adjacent frames. Under normal circumstances, the metrics should be quite
    #similar, and therefore the ratio should be very near to unity (i.e., 1.00). 
    
    #Run through the video with the "script" variable enabled, and look at the metrics
    #in order to determine an optimum threshold value. A larger threshhold will
    #catch fewer bad frames, and a lower threshold will eventually create false positives. 
    #The replacement code works well enough that if you end up replacing a few frames that are
    #actually good, you probably won't notice it.
    
    #You need to un-comment the WriteFileIf lines to actually write the frame numbers to a file.
    #You need to un-comment the script lines to cause the metrics to appear on screen in order to 
    #determine the optimum badthreshold value (2 is a good starting point, however).
    #You need to un-comment the ReplaceBadI lines to automatically replace bad frames with interpolated frames.
    
    #I recommend only having one of these three code blocks enabled at any one time.
    
    #-----------------------------
    loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
    global badthreshold = 2
    showdot = false           # set to true to add "***" to each replacment frame (for troubleshooting)
    filename = "e:\Bad.txt"
    
    source=AVISource("e:\fs.avi").convertTOYV12().killaudio()
    
    
    #TEMPORARILY remove comments from the following block in order to show the metrics. 
    /*
    script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) \
             / YDifferenceFromPrevious( selectevery(source, 1, -1) )) + \
             "\nNext Ratio        = " + String( YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )), lsp=0)"""
    final=Scriptclip(source, script)
    return final
    */
    
    
    #Uncomment the code in the next two lines to create a file which contains the frame numbers of all bad frames
    
    /*
    WriteFileIf(source, filename, "YDifferenceFromPrevious(source) / YDifferenceFromPrevious( selectevery(source, 1, -1) ) > badthreshold && \
               YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )>badthreshold", "current_frame", append = false)
    */
    
    
    #Add comments to the two lines of code below to stop replacement of
    #each bad frame with a one that is interpolated from adjacent frames.
    #The "I" in the function name stands or Interlaced, because this will
    #work with interlaced video (as well as progressive)
    
    output=ReplaceBadI(source,showdot)
    return output
    
    
    #------------------------------
    
    function ReplaceBadI (clip c, bool showdot)
    {
      even = c.SeparateFields().SelectEven()
      super_even = showdot ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) 
      vfe=manalyse(super_even,truemotion=true,isb=false,delta=2)
      vbe=manalyse(super_even,truemotion=true,isb=true,delta=2)
      filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)
    
      odd  = c.SeparateFields().SelectOdd()
      super_odd = showdot ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) 
      vfo=manalyse(super_odd,truemotion=true,isb=false,delta=2)
      vbo=manalyse(super_odd,truemotion=true,isb=true,delta=2)
      filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)
    
      Interleave(filldrops_e,filldrops_o)
      Replacement = Weave()
    
      global original = c
    
      fixed = ConditionalSelect(c, "\
                         Prev = YDifferenceFromPrevious(original)" + \
                         chr(13) + "
                         Prev1 = YDifferenceFromPrevious(SelectEvery(original,1,-1))" + \
                         chr(13) + "
                         Next = YDifferenceToNext(original)" + \
                         chr(13) + "
                         Next1 = YDifferenceToNext(SelectEvery(original,1,1))" + \
                         chr(13) + "
                         Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                         original, selectevery(Replacement,1,-1))
    
      return fixed
    }
    Last edited by johnmeyer; 13th Dec 2016 at 15:11. Reason: added link to better version of script, later in this thread
    Quote Quote  
  2. Member
    Join Date
    Jan 2013
    Location
    Florida
    Search Comp PM
    Thanks for this John. Just last night I went through a reel from a new batch of transfers, and picked out a lot of flash frames. While I don't see them after every scene change, I do see them on a very big majority of them.

    Looking forward to taking it for it a spin.
    Quote Quote  
  3. Originally Posted by Bruce/Fl View Post
    Thanks for this John. Just last night I went through a reel from a new batch of transfers, and picked out a lot of flash frames. While I don't see them after every scene change, I do see them on a very big majority of them.

    Looking forward to taking it for it a spin.
    For film flash frames this particular version of the script may, or may not, do the trick. I did mention film flash frames in my OP, but that was more as a way of describing how I got to this point. This particular script is oriented entirely towards finding an individual rogue frame. The film flash frame can be like this, but sometimes it is not. In particular, as you know if you've done film transfers, on some scenes the flash is quite subtle and sometimes quite strong. In other instances, there are several over-exposed frames in a row, not just one, with the exposure gradually trending back to normal. If the flash is not strong or if there are multiple over-exposed frames, then this script fails for the reasons I stated in post #1.

    Film flash frames are actually rather tricky because of the way amateur film works. For instance, when designing a flash frame script you have to go through all four permutations of over- and under-exposure for the current scene and the next scene. As one example, if the current scene is already over-exposed, and the next scene underexposed, then the flash over-exposure on the first frame of the dark scene might actually look pretty close to the last frame of the current scene, and you are left with what amounts to a scene change, something the script is designed to ignore.

    For film flash frames, I worked with StainlessS over at doom9.org to develop other scripts. None of them work very reliably. So, what I usually do is simply use a scene detection script, create a file that contains all the frame numbers where something is found, and then use my NLE to quickly go to each frame number and inspect what is going on at this point. Unfortunately this is probably the only way to do this because each scene is different, and often requires editing on both sides of the scene change. This is due to the fact that sometimes the flash is so large that it infects the last frame of the previous scene (the light bled through to the adjacent frames during exposure). Another problem is that a lot of amateurs would jerk the camera as they took their finger or thumb off the spring-loaded camera on/off button, so even without any substantial flash, you need to remove the last several frames at the end of a scene. Finally, I've had some cameras that take as long as four frames for the shutter to get up to speed, especially on cold days. So, between all this, I often have to remove as many as five frames at a scene boundary, if the content of the film lets me do this (some times I just have to keep the flash in order to preserve a precious, unique frame).

    Finally, back to flash frames: this script works perfectly to detect and remove flashes that result when a still photographer takes an indoor picture. Therefore, it works great for wedding receptions, etc.
    Quote Quote  
  4. Here is a better version of the original script, revised to cleaned up to make it look prettier, but with many fixes and improvements. You can now control the script simply by setting the variables in the "variable block" at the beginning of the script.
    Code:
    #Find And (Optionally) Fix Bad Frames
    #John Meyer - December 13, 2016
    #Rev. 2.0
    #Thanks to Gavino and StainlessS for making the script more professional.
    
    #This script detects single bad frames. 
    
    #You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad".
    #You can also configure it to automatically replace each bad frame with a new
    #frame interpolated from its neighbors. 
    
    #This script will fail if the bad frame happens immediately before or after a scene change.
    #This script will also fail to find a bad frame if there is more than one bad frame in a row.
    
    #It works very well for finding both blank frames and also "flash" frames (like those caused
    #by a photographer's flash). It will also find single frames which have lots of 
    #static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like
    #a film frame that wasn't properly registered in the film gate, or an analog
    #video frame that lost vertical sync.
    
    #When using VirtualDub, to create the text file containing the bad frame numbers,
    #select "Run Video Analysis Pass" in the VirtualDub File menu. If you are simultaneously
    #creating a fixed video file, you don't need to do this because the file will
    #be created simultaneously as the fixed video file is created.
    
    #The script uses ratios of the metrics for the current frame to the same metrics 
    #on the two adjacent frames. Under normal circumstances, the metrics should be quite
    #similar, and therefore the ratio should be very near to unity (i.e., 1.00). 
    
    #Run through the video with the "METRICS" variable set to "True" and look at the metrics
    #in order to determine an optimum threshold value. A larger threshhold will
    #catch fewer bad frames, and a lower threshold will eventually create false positives. 
    #The replacement code works well enough that if you end up replacing a few frames that are
    #actually good, you probably won't notice it.
    
    
    #-----------------------------
    loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
    
    #Control script operation by changing the following values :
    #=====================================================================
    VideoFile           = "E:\fs.avi"
    global badthreshold = 2              # Set METRICS=TRUE to determine best value
    METRICS             = FALSE          # TRUE will show Metrics ONLY (i.e., TRUE overrides all other selctions)
    SHOWDOT             = FALSE          # TRUE will add "***" to each replacment frame (for troubleshooting)
    REPLACE             = TRUE           # TRUE will replace each bad frame with a one that is interpolated from adjacent frames
    FILEWRITE           = TRUE           # TRUE will create a file which contains the frame numbers of all bad frames
    filename            = "E:\Bad.txt"   # Set to name and location where you want the frame numbers stored
    #=====================================================================
    
    source = AVISource(VideoFile).convertTOYV12().killaudio()
    
    script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) /
             \ Max(YDifferenceFromPrevious( selectevery(source, 1, -1)),0.00001) ) + 
             \ "\nNext Ratio        = " + String( YDifferenceToNext(source) /
             \ Max(YDifferenceToNext(selectevery(source, 1, 1)),0.00001)), lsp=0)"""
    MetClip = Scriptclip(source, script)
    
    FileFixed = (FILEWRITE)
        \ ? WriteFileIf(source, filename, "
        \ YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) 
        \ > badthreshold && YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)
        \ > badthreshold", "current_frame", append = false) : Source
    
    output = (METRICS) ? MetClip : (REPLACE) ? ReplaceBadI(FileFixed,showdot) : FileFixed
    return output
    
    #------------------------------
    
    function ReplaceBadI (clip c, bool SHOWDOT)
    {
      even        = c.SeparateFields().SelectEven()
      super_even  = SHOWDOT ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) 
      vfe         = manalyse(super_even,truemotion=true,isb=false,delta=2)
      vbe         = manalyse(super_even,truemotion=true,isb=true,delta=2)
      filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)
    
      odd         = c.SeparateFields().SelectOdd()
      super_odd   = SHOWDOT ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) 
      vfo         = manalyse(super_odd,truemotion=true,isb=false,delta=2)
      vbo         = manalyse(super_odd,truemotion=true,isb=true,delta=2)
      filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)
    
      Interleave(filldrops_e,filldrops_o)
      Replacement = Weave()
    
      fixed       = ConditionalSelect(c, "
                    Prev  = YDifferenceFromPrevious()
                    Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001)
                    Next  = YDifferenceToNext()
                    Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001)
                    Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                    c, selectevery(Replacement,1,-1))
    
      return fixed
    }
    Last edited by johnmeyer; 13th Dec 2016 at 16:25. Reason: Added more divide by zero checks
    Quote Quote  
  5. Here's a link to a very short, small test video showing how the script automatically removes photographer flashes at a wedding:

    Photo Flash Removal

    There are two flashes, a fraction of a second apart, about midway through this five-second clip.

    Here is one frame from that clip, with the left side showing the before, and the right side showing the video which results from the script's automatic replacement:



    The image on the right is a completely synthesized frame.

    The one thing I did find out -- and I completely expected this -- is that the script does not work for certain types of photo flashes. The problem is that modern strobes often fire multiple times, especially during their "pre-flash" routine. As I said in my notes in the script, the script is not designed to handle two bad frames in a row. However, when the bad frame is all by itself, the script does really good things.
    Quote Quote  
  6. I have a video that has some still shots with MPEG 2 encoding artifacts. It generates false positives because you are testing ratios. Of course, it doesn't matter when you're replacing frames with neighbors but it does when generating a list of flash frames. It makes sense to add an absolute difference test in addition to the relative differences.

    Code:
    WriteFileIf(source, "flashframes.txt", "(YDifferenceFromPrevious(source) > 5) && (YDifferenceFromPrevious(source) / YDifferenceFromPrevious( selectevery(source, 1, -1) ) > badthreshold && \
               YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )>badthreshold)", "current_frame", append = false)
    In the particular case an absolute diff of 5 worked, eliminating the false positives.
    Quote Quote  
  7. Interesting idea to add an additional metric. When I worked with StainlessS on some scene detection code, he ended up creating several different thresholds.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!