Hi guys,
Let say we can see visually that video has frame loss at 1min 27 sec, is there any software that can detect or alert this frame drop,
E.g at 1mins 27sec of the video, we had visually spot a frame loss on the video, similar as the moving pendulum.
is there a way for any software or script to detect same frame drop interval (1min 27 sec) as what i have seen visually?
E.g of moving pendulum having frame drop.
[Attachment 47021 - Click to enlarge]
Is there also software to detect that the moving object is out of sync due to video lag/freeze
Eg of moving two moving pendulum out of sync due to video lag.
[Attachment 47022 - Click to enlarge]
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays!
+ Reply to Thread
Results 1 to 17 of 17
Thread
-
Last edited by helpme101; 26th Oct 2018 at 06:27.
-
Generally, no. But... When capture devices miss a frame they often repeat the previous frame. Duplicate frames are easily detected in AviSynth. Under some circumstances you might be able to subtracts frames from some earlier period. If a frame is missing the difference will be large and that can be detected. Can you post a sample video?
-
Hi,
Thank for your respond.
This video show a good example of drop frame, drop frame basically mean frame loss or skip frame, the video wont be smooth
and can visually see a skip movement,
In this video 0:23 to 0:26 playback in slow motion , the car is moving step by step, but sudden at the pink line indicator it skip to 2 steps instead of 1.
https://www.youtube.com/watch?v=hDaBgVPXIyw
Concept is similar as the first moving pendulum gif, it wont be smooth movement.
Is there a software that can quick scan this drop frame issue, mean at 0:25 in this video had a a drop frame, any software to able scan that frame drop at 0:25 mins
without the need to manually visually the video for this drop frame issue. -
Everyone here knows what video with missing frames looks like. In order to make a judgement on whether certain techniques will work with your video we need a sample of it.
-
Usually dropped packets from streaming video usually causes a jump in DTS/PTS time that can be checked with ffprobe but FFprobe is not a speed demon so 'Quick Scan' would be not so quick unless you know approximately the area of skip.
[Attachment 47026 - Click to enlarge]
In the case of you sample there is no gap because I assume this is a manufactured video since 25.666667 / durartion (.041667) is approximately frame 616.
the following is from one of my videos with just what you mention.
[Attachment 47027 - Click to enlarge]Last edited by Budman1; 27th Oct 2018 at 00:31. Reason: added examle
-
I have GIF animations turned off in my browser so I didn't realize the samples images in the first post were animated! Are these the actual videos you're working with?
-
It is quite similar. The lst pendulum showing obvious frame loss
. The video show quick frame loss. Mean to say that sometime need playback 2 to 3 times focusing on moving pendulum and able observe a quick frame skip ... -
How similar ?
If it's as simple as the 1st pendulum, yes . An avisynth script would be YDifferenceFromPrevious based on a threshold with WriteFileIf . For example you can loop the 1st gif and make a video, and test the script - and it works with 100% accuracy - it will print out the frame numbers where there is a "jump" to a log file
But not with any accuracy in general situations , like random videos
You can conduct a similarity comparison to detect jumps, but usually there will be too many false positives to be useful . e.g. a large % frame difference might indicate a "jump" or dropped frame. BUT - a scenechange, or whip pan, or camera flash, or person jumping on a trampoline in/out of the frame, etc... can just as easily be detected as a large frame difference, but they aren't real "jumps" or dropped frames. Those are "normal" .
For continuous scenes , you can track objects with motion tracking, and a large jump could indicate a large movement such as the pendulum jump . But again , many potential issues, scenechanges, or "actual" jumps (sometimes motion is supposed to be non linear and not smooth, eg. a rocket accelerating, where the delta is increasing) -
But even that simple case the method doesn't work very well. When there are no missing frames there is very little motion at times, a lot of motion at other times. A missing frame when there is little motion may return a smaller diff than times where there is a lot of motion. You could only reliably detect missing frames where there is a lot of motion. You would need to compare the amount of motion relative to the amount of motion in nearby frames (and this assumes motions aren't so large as to just look like noise). And even if this does work it would only alert you to the first dropped frame -- all frames after that will be different.
For comparing the motion of two pendulums one's first thought is to subtract one image from the other. Visually... early in the video the two pendulums are in sync:
[Attachment 47058 - Click to enlarge]
On the left is a crop of the left pendulum, the middle a crop of the right pedulum, and the right the two image subtracted (in RGB: Image1 - Image2 + 126 so as not to lose values less than 0) with AviSynth's RGBDifference printed at the top. With the given video the two pendulums don't always line up perfectly when they are in sync so there is some "noise" which you can see here.
Later in the video they are obviously out of sync and the errors are larger:
[Attachment 47059 - Click to enlarge]
A further complication when comparing the two pendulums is that the motions overlap -- at least in the sample video. And at times they can extend beyond the edges of the frame. The circular outlines mark the approximate extents of the two pendulums motions:
[Attachment 47053 - Click to enlarge]
So you can't just subtract the left "half" of the image from the right.
Another way I was thinking of detecting missing frames is to use motion prediction. Say, at frame N use frames N-1 and N+1 to predict where the pendulum should be. Then compare that to where it actually appears. If a frame is missing the pendulum will be in a different location than predicted. But motion prediction doesn't work well with the samples. And even if it did, the motions aren't linear and constant so it will often be slightly off.
There are many many details like this that determine whether the differences will be detectable by regular computation. A neural network might have better luck. -
Yes - and there are dozens of other variables that could screw it up too . Even more, for more complex cases. I mentioned some situations that are problematic above for false positives above - sometimes the large motions are actually part of the animation and wanted but will be detected as a drop
But if was that simple case exactly (or very similar) and you make a loop for a test video - it does work. That's why I asked "how similar?" in response to his "quite similar". Because the condition is set up for > some threshold . So those small motions will not cross that threshold. We are looking for "big" jumps
You can refine it with things like : min/max etc.. IF previous frame > something AND next frame < something AND ismoving > diff than something ,etc.. to better define the conditions . Maybe you have to satisfy multiple conditions on multiple frames for it to count as a "drop". But even complex , multiconditional script will not be accurate enough for typical videos in my experience. This method simply does not work for general cases and there are way too many variables to mess things up
But a static segment, pendulum at rest, and supposed to be at rest, no camera movement - every frame really is the same. A drop there cannot really be detected by any method. But actual drops there can lead to AV desync and might actually be important (or not)
And even if this does work it would only alert you to the first dropped frame -- all frames after that will be different.
If you have a running back weaving in and out on a football field, or someone doing a random dance - there is no reliable way to determine how many frames would be dropped between 2 frames because of random direction changes. Not even a neural net or human eye can not necessarily do it accurately. There are cues like background elements, maybe the stadium, maybe the football field markings that might help - but in an animation with no background clues, it's basically impossible unless the motion is linear, predictable. His 1st gif eased into motion , but we don't know afterwards what is going on. There is no information on the "after" endpoint . Does it speed up, slow down ? The drop is right at the end, so we can't figure out how many frames are missing.
But I guess it depends on what he is planning to so with the information, is indicating the 1st frame of "n" drops enough ? ie. so there is/are 1 or more drops, now what are you going to do about it?
Eitherway - some background context or real examples would helpLast edited by poisondeathray; 30th Oct 2018 at 20:38.
-
Here is my looped test video generation , it's 1.mp4 below
Code:FFVideoSource("video frame loss.gif") ConvertToYV12() Loop(10)
detection script
Code:LSmashVideoSource("1.mp4") #ConditionalFilter(last,last,last , "YDifferenceFromPrevious", ">", "2.5", show=true) WriteFileIf(last, "F:\log.txt", "YDifferenceFromPrevious>2.5", "current_frame" )
results of the log
100
200
300
400
500
600
700
800
900 -
Hi, from the result of log is it printing frame 100, 200 etc detect the frame drop?
Is there anyway to print timing, eg from the 1.mp4 gif video . between 3 sec to 4sec interval show the video frame loss, will it be able to print in time interval min/sec etc?
results of the log
100
200
300
400
500
600
700
800
900 -
This is not as simple as the gif
How would you define "video frame loss" in that video?
Because there are :
1) hard duplicate frames (encoded duplicates),
2) as well as dropped frames,
3) as well as frame blending, (+/- motion blur, but actual blending too)
4) as well as VFR (variable frame rate) timing (some single frames are duplicated because of timecodes)
#4 is possibly because of re-encoding , you would need to try that on the original video
Another reason to use the original, original video is compression artifacts - they can interfere with metrics and detection.
You have to be able to define what is a "loss" in order to set the detection parameters
Usually with actual recordings , the problem is duplicate frames, as described in post #2. The lag or drop in recording produces a duplicate "placeholder" frame to keep the timingLast edited by poisondeathray; 31st Oct 2018 at 22:25.
-
You can use OpenCV or similar library to compute motion monotonicity.
Simple example:https://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/
Perhaps in your case you need to separate your problem on sub-problems e.g.: object extraction and matching, matched object motion analysis. -
Similar Threads
-
is there software to determine moving object having frame drop or lag
By helpme101 in forum ProgrammingReplies: 2Last Post: 27th Oct 2018, 00:50 -
How to Determine Original DVD Frame Rate & Frame Type
By lomaidala in forum Video ConversionReplies: 30Last Post: 3rd Jul 2017, 08:42 -
How to determine Blu-ray frame rates?
By tdaly in forum Newbie / General discussionsReplies: 3Last Post: 3rd Sep 2016, 13:03 -
VirtualDub - Drop Frame
By harky in forum Capturing and VCRReplies: 22Last Post: 12th Jun 2016, 12:25 -
Replacing moving object in VirtualDub
By vadim1508 in forum EditingReplies: 17Last Post: 29th Nov 2014, 20:51