VideoHelp Forum
+ Reply to Thread
Results 1 to 17 of 17
Thread
  1. Hi guys,

    Let say we can see visually that video has frame loss at 1min 27 sec, is there any software that can detect or alert this frame drop,
    E.g at 1mins 27sec of the video, we had visually spot a frame loss on the video, similar as the moving pendulum.

    is there a way for any software or script to detect same frame drop interval (1min 27 sec) as what i have seen visually?


    E.g of moving pendulum having frame drop.
    Image
    [Attachment 47021 - Click to enlarge]


    Is there also software to detect that the moving object is out of sync due to video lag/freeze
    Eg of moving two moving pendulum out of sync due to video lag.
    Image
    [Attachment 47022 - Click to enlarge]
    Last edited by helpme101; 26th Oct 2018 at 06:27.
    Quote Quote  
  2. Generally, no. But... When capture devices miss a frame they often repeat the previous frame. Duplicate frames are easily detected in AviSynth. Under some circumstances you might be able to subtracts frames from some earlier period. If a frame is missing the difference will be large and that can be detected. Can you post a sample video?
    Quote Quote  
  3. Hi,

    Thank for your respond.

    This video show a good example of drop frame, drop frame basically mean frame loss or skip frame, the video wont be smooth
    and can visually see a skip movement,

    In this video 0:23 to 0:26 playback in slow motion , the car is moving step by step, but sudden at the pink line indicator it skip to 2 steps instead of 1.
    https://www.youtube.com/watch?v=hDaBgVPXIyw

    Concept is similar as the first moving pendulum gif, it wont be smooth movement.


    Is there a software that can quick scan this drop frame issue, mean at 0:25 in this video had a a drop frame, any software to able scan that frame drop at 0:25 mins
    without the need to manually visually the video for this drop frame issue.
    Quote Quote  
  4. Everyone here knows what video with missing frames looks like. In order to make a judgement on whether certain techniques will work with your video we need a sample of it.
    Quote Quote  
  5. Member Budman1's Avatar
    Join Date
    Jul 2012
    Location
    NORTHWEST ILLINOIS, USA
    Search Comp PM
    Usually dropped packets from streaming video usually causes a jump in DTS/PTS time that can be checked with ffprobe but FFprobe is not a speed demon so 'Quick Scan' would be not so quick unless you know approximately the area of skip.

    Image
    [Attachment 47026 - Click to enlarge]


    In the case of you sample there is no gap because I assume this is a manufactured video since 25.666667 / durartion (.041667) is approximately frame 616.

    the following is from one of my videos with just what you mention.
    Image
    [Attachment 47027 - Click to enlarge]
    Last edited by Budman1; 27th Oct 2018 at 00:31. Reason: added examle
    Quote Quote  
  6. I have GIF animations turned off in my browser so I didn't realize the samples images in the first post were animated! Are these the actual videos you're working with?
    Quote Quote  
  7. It is quite similar. The lst pendulum showing obvious frame loss
    . The video show quick frame loss. Mean to say that sometime need playback 2 to 3 times focusing on moving pendulum and able observe a quick frame skip ...
    Quote Quote  
  8. Originally Posted by helpme101 View Post
    It is quite similar. The lst pendulum showing obvious frame loss
    . The video show quick frame loss. Mean to say that sometime need playback 2 to 3 times focusing on moving pendulum and able observe a quick frame skip ...
    How similar ?

    If it's as simple as the 1st pendulum, yes . An avisynth script would be YDifferenceFromPrevious based on a threshold with WriteFileIf . For example you can loop the 1st gif and make a video, and test the script - and it works with 100% accuracy - it will print out the frame numbers where there is a "jump" to a log file



    But not with any accuracy in general situations , like random videos

    You can conduct a similarity comparison to detect jumps, but usually there will be too many false positives to be useful . e.g. a large % frame difference might indicate a "jump" or dropped frame. BUT - a scenechange, or whip pan, or camera flash, or person jumping on a trampoline in/out of the frame, etc... can just as easily be detected as a large frame difference, but they aren't real "jumps" or dropped frames. Those are "normal" .

    For continuous scenes , you can track objects with motion tracking, and a large jump could indicate a large movement such as the pendulum jump . But again , many potential issues, scenechanges, or "actual" jumps (sometimes motion is supposed to be non linear and not smooth, eg. a rocket accelerating, where the delta is increasing)
    Quote Quote  
  9. Originally Posted by poisondeathray View Post
    If it's as simple as the 1st pendulum, yes . An avisynth script would be YDifferenceFromPrevious based on a threshold
    But even that simple case the method doesn't work very well. When there are no missing frames there is very little motion at times, a lot of motion at other times. A missing frame when there is little motion may return a smaller diff than times where there is a lot of motion. You could only reliably detect missing frames where there is a lot of motion. You would need to compare the amount of motion relative to the amount of motion in nearby frames (and this assumes motions aren't so large as to just look like noise). And even if this does work it would only alert you to the first dropped frame -- all frames after that will be different.

    For comparing the motion of two pendulums one's first thought is to subtract one image from the other. Visually... early in the video the two pendulums are in sync:

    Image
    [Attachment 47058 - Click to enlarge]


    On the left is a crop of the left pendulum, the middle a crop of the right pedulum, and the right the two image subtracted (in RGB: Image1 - Image2 + 126 so as not to lose values less than 0) with AviSynth's RGBDifference printed at the top. With the given video the two pendulums don't always line up perfectly when they are in sync so there is some "noise" which you can see here.

    Later in the video they are obviously out of sync and the errors are larger:

    Image
    [Attachment 47059 - Click to enlarge]


    A further complication when comparing the two pendulums is that the motions overlap -- at least in the sample video. And at times they can extend beyond the edges of the frame. The circular outlines mark the approximate extents of the two pendulums motions:

    Image
    [Attachment 47053 - Click to enlarge]


    So you can't just subtract the left "half" of the image from the right.

    Another way I was thinking of detecting missing frames is to use motion prediction. Say, at frame N use frames N-1 and N+1 to predict where the pendulum should be. Then compare that to where it actually appears. If a frame is missing the pendulum will be in a different location than predicted. But motion prediction doesn't work well with the samples. And even if it did, the motions aren't linear and constant so it will often be slightly off.

    There are many many details like this that determine whether the differences will be detectable by regular computation. A neural network might have better luck.
    Quote Quote  
  10. Originally Posted by jagabo View Post
    Originally Posted by poisondeathray View Post
    If it's as simple as the 1st pendulum, yes . An avisynth script would be YDifferenceFromPrevious based on a threshold
    But even that simple case the method doesn't work very well. When there are no missing frames there is very little motion at times, a lot of motion at other times. A missing frame when there is little motion may return a smaller diff than times where there is a lot of motion. You could only reliably detect missing frames where there is a lot of motion. You would need to compare the amount of motion relative to the amount of motion in nearby frames (and this assumes motions aren't so large as to just look like noise).
    Yes - and there are dozens of other variables that could screw it up too . Even more, for more complex cases. I mentioned some situations that are problematic above for false positives above - sometimes the large motions are actually part of the animation and wanted but will be detected as a drop

    But if was that simple case exactly (or very similar) and you make a loop for a test video - it does work. That's why I asked "how similar?" in response to his "quite similar". Because the condition is set up for > some threshold . So those small motions will not cross that threshold. We are looking for "big" jumps

    You can refine it with things like : min/max etc.. IF previous frame > something AND next frame < something AND ismoving > diff than something ,etc.. to better define the conditions . Maybe you have to satisfy multiple conditions on multiple frames for it to count as a "drop". But even complex , multiconditional script will not be accurate enough for typical videos in my experience. This method simply does not work for general cases and there are way too many variables to mess things up

    But a static segment, pendulum at rest, and supposed to be at rest, no camera movement - every frame really is the same. A drop there cannot really be detected by any method. But actual drops there can lead to AV desync and might actually be important (or not)

    And even if this does work it would only alert you to the first dropped frame -- all frames after that will be different.
    Yes, and there is no reliable way to know *how many* frames are missing, unless you have linear, reliable motion or some other collaborating information. The 1st gif isn't completely linear, there is easing in the motion. Can you tell me with your eyes exactly how many frames were missing in that 1st gif ?

    If you have a running back weaving in and out on a football field, or someone doing a random dance - there is no reliable way to determine how many frames would be dropped between 2 frames because of random direction changes. Not even a neural net or human eye can not necessarily do it accurately. There are cues like background elements, maybe the stadium, maybe the football field markings that might help - but in an animation with no background clues, it's basically impossible unless the motion is linear, predictable. His 1st gif eased into motion , but we don't know afterwards what is going on. There is no information on the "after" endpoint . Does it speed up, slow down ? The drop is right at the end, so we can't figure out how many frames are missing.

    But I guess it depends on what he is planning to so with the information, is indicating the 1st frame of "n" drops enough ? ie. so there is/are 1 or more drops, now what are you going to do about it?

    Eitherway - some background context or real examples would help
    Last edited by poisondeathray; 30th Oct 2018 at 20:38.
    Quote Quote  
  11. Here is my looped test video generation , it's 1.mp4 below

    Code:
    FFVideoSource("video frame loss.gif")
    ConvertToYV12()
    Loop(10)

    detection script
    Code:
    LSmashVideoSource("1.mp4")
    
    #ConditionalFilter(last,last,last , "YDifferenceFromPrevious", ">", "2.5", show=true)
    WriteFileIf(last, "F:\log.txt", "YDifferenceFromPrevious>2.5", "current_frame" )
    The way I like to do it is preview the ConditionalFilter, and change the arguments with show=true (to see the overlay), change the values if need be or modify the arguments, then copy it into the WriteFileIF line


    results of the log
    100
    200
    300
    400
    500
    600
    700
    800
    900
    Image Attached Files
    Quote Quote  
  12. Hi, from the result of log is it printing frame 100, 200 etc detect the frame drop?
    Is there anyway to print timing, eg from the 1.mp4 gif video . between 3 sec to 4sec interval show the video frame loss, will it be able to print in time interval min/sec etc?

    results of the log
    100
    200
    300
    400
    500
    600
    700
    800
    900
    Quote Quote  
  13. Divide by the currentframe by framerate to get seconds.
    Quote Quote  
  14. hi


    how about able this sample video link
    At 2-3 sec showing very quick frame loss. sometime need view a few times . Will it able to detect?
    Quote Quote  
  15. Originally Posted by helpme101 View Post
    how about able this sample video link
    At 2-3 sec showing very quick frame loss. sometime need view a few times . Will it able to detect?
    This is not as simple as the gif

    How would you define "video frame loss" in that video?

    Because there are :

    1) hard duplicate frames (encoded duplicates),

    2) as well as dropped frames,

    3) as well as frame blending, (+/- motion blur, but actual blending too)

    4) as well as VFR (variable frame rate) timing (some single frames are duplicated because of timecodes)

    #4 is possibly because of re-encoding , you would need to try that on the original video

    Another reason to use the original, original video is compression artifacts - they can interfere with metrics and detection.


    You have to be able to define what is a "loss" in order to set the detection parameters


    Usually with actual recordings , the problem is duplicate frames, as described in post #2. The lag or drop in recording produces a duplicate "placeholder" frame to keep the timing
    Last edited by poisondeathray; 31st Oct 2018 at 22:25.
    Quote Quote  
  16. You can use OpenCV or similar library to compute motion monotonicity.

    Simple example:https://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/

    Perhaps in your case you need to separate your problem on sub-problems e.g.: object extraction and matching, matched object motion analysis.
    Quote Quote  
  17. Originally Posted by helpme101 View Post
    Hi guys,

    Let say we can see visually that video has frame loss at 1min 27 sec, is there any software that can detect or alert this frame drop,
    E.g at 1mins 27sec of the video, we had visually spot a frame loss on the video, similar as the moving pendulum.
    8 Ball Pool Google Hangouts Omegle
    is there a way for any software or script to detect same frame drop interval (1min 27 sec) as what i have seen visually?


    E.g of moving pendulum having frame drop.
    Image
    [Attachment 47021 - Click to enlarge]


    Is there also software to detect that the moving object is out of sync due to video lag/freeze
    Eg of moving two moving pendulum out of sync due to video lag.
    Image
    [Attachment 47022 - Click to enlarge]
    there a software that can quick scan this drop frame issue, mean at 0:25 in this video had a a drop frame, any software to able scan that frame drop at 0:25 mins
    without the need to manually visually the video for this drop frame issue.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!