Let say we can see visually that video has frame loss at 1min 27 sec, is there any software that can detect or alert this frame drop,
E.g at 1mins 27sec of the video, we had visually spot a frame loss on the video, similar as the moving pendulum.
is there a way for any software or script to detect same frame drop interval (1min 27 sec) as what i have seen visually?
E.g of moving pendulum having frame drop.
[Attachment 47021 - Click to enlarge]
Is there also software to detect that the moving object is out of sync due to video lag/freeze
Eg of moving two moving pendulum out of sync due to video lag.
[Attachment 47022 - Click to enlarge]
+ Reply to Thread
Results 1 to 17 of 17
Last edited by helpme101; 26th Oct 2018 at 06:27.
Generally, no. But... When capture devices miss a frame they often repeat the previous frame. Duplicate frames are easily detected in AviSynth. Under some circumstances you might be able to subtracts frames from some earlier period. If a frame is missing the difference will be large and that can be detected. Can you post a sample video?
Thank for your respond.
This video show a good example of drop frame, drop frame basically mean frame loss or skip frame, the video wont be smooth
and can visually see a skip movement,
In this video 0:23 to 0:26 playback in slow motion , the car is moving step by step, but sudden at the pink line indicator it skip to 2 steps instead of 1.
Concept is similar as the first moving pendulum gif, it wont be smooth movement.
Is there a software that can quick scan this drop frame issue, mean at 0:25 in this video had a a drop frame, any software to able scan that frame drop at 0:25 mins
without the need to manually visually the video for this drop frame issue.
Everyone here knows what video with missing frames looks like. In order to make a judgement on whether certain techniques will work with your video we need a sample of it.
Usually dropped packets from streaming video usually causes a jump in DTS/PTS time that can be checked with ffprobe but FFprobe is not a speed demon so 'Quick Scan' would be not so quick unless you know approximately the area of skip.
[Attachment 47026 - Click to enlarge]
In the case of you sample there is no gap because I assume this is a manufactured video since 25.666667 / durartion (.041667) is approximately frame 616.
the following is from one of my videos with just what you mention.
[Attachment 47027 - Click to enlarge]
Last edited by Budman1; 27th Oct 2018 at 00:31. Reason: added examle
I have GIF animations turned off in my browser so I didn't realize the samples images in the first post were animated! Are these the actual videos you're working with?
It is quite similar. The lst pendulum showing obvious frame loss
. The video show quick frame loss. Mean to say that sometime need playback 2 to 3 times focusing on moving pendulum and able observe a quick frame skip ...
If it's as simple as the 1st pendulum, yes . An avisynth script would be YDifferenceFromPrevious based on a threshold with WriteFileIf . For example you can loop the 1st gif and make a video, and test the script - and it works with 100% accuracy - it will print out the frame numbers where there is a "jump" to a log file
But not with any accuracy in general situations , like random videos
You can conduct a similarity comparison to detect jumps, but usually there will be too many false positives to be useful . e.g. a large % frame difference might indicate a "jump" or dropped frame. BUT - a scenechange, or whip pan, or camera flash, or person jumping on a trampoline in/out of the frame, etc... can just as easily be detected as a large frame difference, but they aren't real "jumps" or dropped frames. Those are "normal" .
For continuous scenes , you can track objects with motion tracking, and a large jump could indicate a large movement such as the pendulum jump . But again , many potential issues, scenechanges, or "actual" jumps (sometimes motion is supposed to be non linear and not smooth, eg. a rocket accelerating, where the delta is increasing)
For comparing the motion of two pendulums one's first thought is to subtract one image from the other. Visually... early in the video the two pendulums are in sync:
[Attachment 47058 - Click to enlarge]
On the left is a crop of the left pendulum, the middle a crop of the right pedulum, and the right the two image subtracted (in RGB: Image1 - Image2 + 126 so as not to lose values less than 0) with AviSynth's RGBDifference printed at the top. With the given video the two pendulums don't always line up perfectly when they are in sync so there is some "noise" which you can see here.
Later in the video they are obviously out of sync and the errors are larger:
[Attachment 47059 - Click to enlarge]
A further complication when comparing the two pendulums is that the motions overlap -- at least in the sample video. And at times they can extend beyond the edges of the frame. The circular outlines mark the approximate extents of the two pendulums motions:
[Attachment 47053 - Click to enlarge]
So you can't just subtract the left "half" of the image from the right.
Another way I was thinking of detecting missing frames is to use motion prediction. Say, at frame N use frames N-1 and N+1 to predict where the pendulum should be. Then compare that to where it actually appears. If a frame is missing the pendulum will be in a different location than predicted. But motion prediction doesn't work well with the samples. And even if it did, the motions aren't linear and constant so it will often be slightly off.
There are many many details like this that determine whether the differences will be detectable by regular computation. A neural network might have better luck.
But if was that simple case exactly (or very similar) and you make a loop for a test video - it does work. That's why I asked "how similar?" in response to his "quite similar". Because the condition is set up for > some threshold . So those small motions will not cross that threshold. We are looking for "big" jumps
You can refine it with things like : min/max etc.. IF previous frame > something AND next frame < something AND ismoving > diff than something ,etc.. to better define the conditions . Maybe you have to satisfy multiple conditions on multiple frames for it to count as a "drop". But even complex , multiconditional script will not be accurate enough for typical videos in my experience. This method simply does not work for general cases and there are way too many variables to mess things up
But a static segment, pendulum at rest, and supposed to be at rest, no camera movement - every frame really is the same. A drop there cannot really be detected by any method. But actual drops there can lead to AV desync and might actually be important (or not)
And even if this does work it would only alert you to the first dropped frame -- all frames after that will be different.
If you have a running back weaving in and out on a football field, or someone doing a random dance - there is no reliable way to determine how many frames would be dropped between 2 frames because of random direction changes. Not even a neural net or human eye can not necessarily do it accurately. There are cues like background elements, maybe the stadium, maybe the football field markings that might help - but in an animation with no background clues, it's basically impossible unless the motion is linear, predictable. His 1st gif eased into motion , but we don't know afterwards what is going on. There is no information on the "after" endpoint . Does it speed up, slow down ? The drop is right at the end, so we can't figure out how many frames are missing.
But I guess it depends on what he is planning to so with the information, is indicating the 1st frame of "n" drops enough ? ie. so there is/are 1 or more drops, now what are you going to do about it?
Eitherway - some background context or real examples would help
Last edited by poisondeathray; 30th Oct 2018 at 20:38.
Here is my looped test video generation , it's 1.mp4 below
FFVideoSource("video frame loss.gif") ConvertToYV12() Loop(10)
LSmashVideoSource("1.mp4") #ConditionalFilter(last,last,last , "YDifferenceFromPrevious", ">", "2.5", show=true) WriteFileIf(last, "F:\log.txt", "YDifferenceFromPrevious>2.5", "current_frame" )
results of the log
Hi, from the result of log is it printing frame 100, 200 etc detect the frame drop?
Is there anyway to print timing, eg from the 1.mp4 gif video . between 3 sec to 4sec interval show the video frame loss, will it be able to print in time interval min/sec etc?
results of the log
Divide by the currentframe by framerate to get seconds.
How would you define "video frame loss" in that video?
Because there are :
1) hard duplicate frames (encoded duplicates),
2) as well as dropped frames,
3) as well as frame blending, (+/- motion blur, but actual blending too)
4) as well as VFR (variable frame rate) timing (some single frames are duplicated because of timecodes)
#4 is possibly because of re-encoding , you would need to try that on the original video
Another reason to use the original, original video is compression artifacts - they can interfere with metrics and detection.
You have to be able to define what is a "loss" in order to set the detection parameters
Usually with actual recordings , the problem is duplicate frames, as described in post #2. The lag or drop in recording produces a duplicate "placeholder" frame to keep the timing
Last edited by poisondeathray; 31st Oct 2018 at 22:25.
You can use OpenCV or similar library to compute motion monotonicity.
Perhaps in your case you need to separate your problem on sub-problems e.g.: object extraction and matching, matched object motion analysis.