VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Results 1 to 24 of 24
Thread
  1. Hi,
    I'm searching for a way to detect lost frames/stuttering in videos I've recorded with a capture card (mkv, avc). Maybe some tool that would scan the video and create a list showing when these errors occur in the video.

    I've found out that MPC-BE can display a graph (press Ctrl+j), which shows spikes when the video file stutters. I can also activate a statistics windows (press Ctrl+4). Here I can see that the values for "Sync Offset" and "Jitter" change drastically when there's stuttering.
    Sadly, MPC-BE does not have an option to log these values.

    So, is there any other way to detect and log such errors in a video?

    Click image for larger version

Name:	error2.png
Views:	54
Size:	615.7 KB
ID:	50688Click image for larger version

Name:	error1.png
Views:	42
Size:	284.3 KB
ID:	50689
    These are the graph/statistics in MPC-BE when there's an error in the video.

    Click image for larger version

Name:	ok.png
Views:	43
Size:	685.4 KB
ID:	50690
    These are the graph/statistics in MPC-BE when there's no error in the video.
    Quote Quote  
  2. MPCBE is showing you playback statistics, not properties of the video.

    AviSynth can log duplicate frames with something like:

    Code:
    WriteFileIf("Dups.txt", "YDifferenceFromPrevious<0.5", "current_frame", flush=true)
    I've seen scripts that attempt to log skipped frames by looking for differences between frames that are unusually large -- but less than most scene change thresholds.
    Quote Quote  
  3. VirtualDub(2) menu: Go->next drop frame.
    I don't know what the discovery of dropped frame is based on though. Duplicates as a substitute of skipped frames?
    Quote Quote  
  4. I believe VirtualDub's frame drop detection only detects D frames in AVI files. Some capture programs, when they detect that a frame hasn't show up in time, insert a D frame in the cap. That is a special frame type that indicates a dropped frame (it requires very few bits) and the decoder usually just substitutes a copy of the last frame. It's not an actual copy of the last frame (which requires lots of bits).
    Quote Quote  
  5. Originally Posted by jagabo View Post
    I believe VirtualDub's frame drop detection only detects D frames in AVI files. Some capture programs, when they detect that a frame hasn't show up in time, insert a D frame in the cap. That is a special frame type that indicates a dropped frame (it requires very few bits) and the decoder usually just substitutes a copy of the last frame. It's not an actual copy of the last frame (which requires lots of bits).
    I see, thanks. I have always been wondering how this works.
    Quote Quote  
  6. I know that MPC-BE only shows playback statistics, but using these statistics stuttering can be detected very easily while playing/watching the video.

    Logging the affected frames through AviSynth sounds good to me, but unfortunately, I have very little experience with their script language. The mentioned code does not work.
    Using

    DirectShowSource("00.mkv")
    WriteFileIf("Dups.txt", "YDifferenceFromPrevious<0.5", "current_frame", flush=true)


    it just creates a file which looks like this

    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    .
    .
    .


    VirtualDub2 does not detect any dropped frames in my videos. When I use the "next drop frame" function, I receive the message "No next dropped frame found".

    However, I know that the video file is faulty, because no matter which media player I use, the stuttering always occurs at exactly the same frames.
    Quote Quote  
  7. It's probably VFR drops (low fps sections) if its in sync . It should be 50 fps CFR if captured properly, but it's 12.86 avg in the "bad" file

    Avisynth won't detect that with that script, only the actual encoded frames. (Things like actually encoded duplicates it will detect - but you have sections that are missing, and "held together" in sync by timecodes, telling the player to display the frame longer - effectively lower fps sections)

    You can extract the timecodes (e.g. mkvextract for mkv , or mp4fpsmod for mp4) , then use a v2 to v1 timecode converter (e.g. tcconv) to convert them into human readable (v1) format
    Quote Quote  
  8. I took a shot at doing exactly this, in this thread over in doom9,org:

    Automatically fix dups followed (eventually) by drops

    Finding duplicates, especially if they are exact, is pretty easy using YDifference, as already mentioned. Finding drops is much more difficult because the difference metrics between frames after only one single frame is removed gets to be pretty tough, depending on the motion. To illustrate the point, if you put the camera on a tripod and take a video indoors of an empty room, if you remove a frame, you cannot possibly detect it.

    However, the approach I developed does sort of work, despite that, because when you erroneously replace a frame where there is no motion, and that frame didn't really need to be replaced, the replacement algorithm is so perfect that the new frame looks just fine.

    So, try it out and see what you think. Use the last script that I posted, not the first one, before I improved it.
    Quote Quote  
  9. Originally Posted by Shadowlord View Post
    I know that MPC-BE only shows playback statistics, but using these statistics stuttering can be detected very easily while playing/watching the video.
    I don't think you understand. The statistics you are getting from MPCBE have to do with timing -- when a frame is supposed to be displayed vs. when it actually displayed. For example, if a frame takes too long to decompress it will be displayed at a later time than it was supposed to -- that timing error that is reported. It cannot detect actual duplicate or dropped frames in a video file.

    Originally Posted by Shadowlord View Post
    Logging the affected frames through AviSynth sounds good to me, but unfortunately, I have very little experience with their script language. The mentioned code does not work.
    Using

    DirectShowSource("00.mkv")
    WriteFileIf("Dups.txt", "YDifferenceFromPrevious<0.5", "current_frame", flush=true)


    it just creates a file which looks like this

    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    .
    .
    .
    Lower the threshold from 0.5 to 0.1 or whatever value eliminates false positives but still shows true duplicates. Also be aware that a video may contain many essentially duplicate frames. For example in a fade to black all the black frames may look like duplicates. Or in a still shot the frames may look like duplicates -- depending on how much film grain was retained during compression (often the first thing to go). And conversely, compression artifacts may make true duplicate frames look different.

    DirectShowSource() may work for this but it should generally be used as a last resort (it's not always frame accurate). The best thing for you to do would be to download the LSMASH source filter for AviSynth and use LSmashVideoSource() for MP4 files and LWlibavVideoSource() for others.
    Last edited by jagabo; 30th Oct 2019 at 10:32.
    Quote Quote  
  10. Thanks, but after taking a closer look at the solutions, I am not sure that they are leading in the right direction.
    As described at the beginning, I would refer to the errors as "lost" or "dropped" frames (I don't know what the correct term is). They are not "duplicate" frames.

    To make things even more complicated: The video is 50 fps. The original source is 25 fps, so every frame is a duplicate frame. Of course, this doesn't cause stuttering when playing.

    Therefore, I believe jagabo's code and Johnmeyer's script won't work for this problem, because I'm not trying to find duplicate frames.


    Originally Posted by jagabo View Post
    I don't think you understand. The statistics you are getting from MPCBE have to do with timing -- when a frame is supposed to be displayed vs. when it actually displayed. For example, if a frame takes too long to decompress it will be displayed at a later time than it was supposed to -- that timing error that is reported. It cannot detect actual duplicate or dropped frames in a video file.
    Yes, I know that MPC-BEs statistics are based on timing. "The graph shows how much the audio/video timing differs from the ideal situation" - but nevertheless the graph is always showing big spikes when there's an error in the video that I want to detect. I don't know what causes these timing errors.


    I've uploaded a small sample, maybe this makes things clearer.
    Quote Quote  
  11. If all you need to do is find and replace duplicate frames (you probably don't want to remove them because that will ruin audio sync) then Filldrops is the function you want. I posted a version for interlaced video here:

    FillDropsI()

    Here's the same function, but for progressive video:

    Code:
    function filldrops (clip c)
    {
      super=MSuper(c,pel=2)
      vfe=manalyse(super,truemotion=true,isb=false,delta=1)
      vbe=manalyse(super,truemotion=true,isb=true,delta=1)
      filldrops = mflowinter(c,super,vbe,vfe,time=50)
      fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
      return fixed
    }
    You can change how similar the two adjacent frame must be in order for them to be considered duplicates by changing the number after the "lessthan" word in the next to the last line. It is currently set to 0.1. If you change the to 0.0, only absolute duplicates will be detected and replaced with a motion-estimated frame.

    This function is nice because it fixes the problem and doesn't require you to output a list of frames and then do something else. If you truly want a list of frame numbers, you'll have to change the conditionalfilter line.
    Quote Quote  
  12. Originally Posted by Shadowlord View Post
    I've uploaded a small sample, maybe this makes things clearer.
    That video is variable frame rate and has bad time codes. In the places where you see the frame lock up and jump there are discontinuities in the time code. Here's an example starting at frame 101

    Code:
     type,  time,  duration, type, frame_number
    -------------------------------------------
    video,  2080,  0.020000,    P,   101
    video,  2100,  0.020000,    P,   102
    video,  2120,  0.020000,    P,   103
    video,  2141,  0.020000,    P,   104
    video,  2160,  0.020000,    P,   105
    video,  2180,  0.020000,    P,   106
    video,  2200,  0.020000,    P,   107
    video,  2220,  0.020000,    P,   108
    video,  2440,  0.020000,    P,   109
    video,  2461,  0.020000,    I,   110
    video,  2480,  0.020000,    P,   111
    video,  2500,  0.020000,    P,   112
    video,  2520,  0.020000,    P,   113
    video,  2540,  0.020000,    P,   114
    video,  2560,  0.020000,    P,   115
    video,  2580,  0.200000,    P,   116
    video,  2720,  0.020000,    P,   117
    video,  2742,  0.020000,    P,   118
    video,  2760,  0.020000,    P,   119
    video,  2780,  0.020000,    P,   120
    Frame_number is the displayed frame number. You can see the table includes frames 101 to 120. Time in that table is the time at which the frame should be displayed (in milliseconds from the start of the video). You can see it stepping at intervals of 20 ms from frame 101 to frame 108, 2080 ms to 2220 ms (except for a minor glitch at frame 104 which is 1 ms off). But thenthere's a big jump to 2440 at frame 109, a 220 millisecond jump. So when the player gets to frame 108 it displays that frame for 220 ms instead of 20 ms. If you step through the actual video frames you can see there is a big jump in motion between frames 108 and 109. Several frames of the original video were dropped between what's now frame 108 and 109. Later you see another jump at frame 117, 140 ms.

    <edit>
    So it may not be right to say there are errors in the time code. If there are 10 frames (5 unique, 5 duplicates) missing between frame 108 and 109 the time codes are right.

    In fact, filling the "gaps" with motion interpolated frames leads to nice smooth motion. Sample with original video on the left and the fixed version on the right attached.

    But what you really need to do is get the caps right (no dropped frames) to start with.
    </edit>
    Image Attached Files
    Last edited by jagabo; 30th Oct 2019 at 22:55.
    Quote Quote  
  13. Originally Posted by Shadowlord View Post
    So, is there any other way to detect and log such errors in a video?
    You could for example analyze the file with ffprobe (included in the ffmpeg package):
    Code:
    ffprobe.exe -show_frames -print_format xml "sample.mkv" >output.xml
    Quote Quote  
  14. Alternate ffprobe command line with just relevent data:

    Code:
    ffprobe -v quiet -pretty -print_format csv -show_entries "frame=coded_picture_number,pkt_pts,pkt_duration,pict_type," -select_streams v:0 input.mkv >output.csv
    Quote Quote  
  15. @johnmeyer:
    I've tried your Filldrops-script, but I don't see any improvement when applied to my sample.mkv. Maybe I'm doing something wrong?

    Code:
    LWLibavVideoSource("sample.mkv")
    filldrops()
    
    
    function filldrops (clip c)
    {
      super=MSuper(c,pel=2)
      vfe=manalyse(super,truemotion=true,isb=false,delta=1)
      vbe=manalyse(super,truemotion=true,isb=true,delta=1)
      filldrops = mflowinter(c,super,vbe,vfe,time=50)
      fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
      return fixed
    }
    Saved as fix.avs
    then run ffmpeg -I fix.avs new.mkv



    @jagabo:
    Looking at the time code of my sample video does indeed help to see if there are missing/dropped frames. But how can I detect uneven intervals in longer videos? Is there a way to list only uneven intervals?

    What did you use to fill these gaps with motion interpolated frames in the sample.mkv.avs.mkv? Was this achieved with an Avisynth filter/plugin?
    Quote Quote  
  16. Originally Posted by Shadowlord View Post
    Filldrops-script, but I don't see any improvement
    It won't work well when multiple consecutive frames are missing. It will only insert one frame per jump.

    Originally Posted by Shadowlord View Post
    Looking at the time code of my sample video does indeed help to see if there are missing/dropped frames. But how can I detect uneven intervals in longer videos? Is there a way to list only uneven intervals?
    The easiest way would be to write a quick program to subtract successive time codes (from ffprobe) and output the frame number and number of missing frames when the time exceeds a threshold (25 ms?). If you use a script like the one below it would make sense for the program to output the actual AviSynth script.

    Originally Posted by Shadowlord View Post
    What did you use to fill these gaps with motion interpolated frames in the sample.mkv.avs.mkv? Was this achieved with an Avisynth filter/plugin?
    I used AviSynth and one of the motion interpolation filters, InsertFramesMC(). That's a modification of RX(). Both can be found in these forums. Be warned, this type of motion interpolation works well with panning shots but can introduce ugly artifacts with large and/or complex motions.

    Code:
    LWLibavVideoSource("sample.mkv") 
    AssumeFPS(50)
    
    InsertFramesMC(117, 6)
    InsertFramesMC(109, 10)
    InsertFramesMC(101, 3)
    There was at least on small jump before frame 101, and lots of jumps after 117.
    Quote Quote  
  17. I finally looked at the video. It has the wrong speed (50 fps) and is therefore duplicating each frame. That is easy to fix.

    Once that is fixed, there are then both jumps and dups, exactly the thing the script I linked to above was designed to fix. The Filldrops() scripts will be useless with this video. In the script I linked to (over at doom9.org), you'll need to use a pretty big window (lots of frames) because the dups and drops are separated by such a large number of frames.

    The cause of this is probably a lousy capture card. If you can re-capture, try to tune the software and hardware to avoid the drops. What usually creates this is a capture where a frame is dropped so, to keep the sound in sync, a later frame is duplicated, in order to keep the frame count correct. This can also happen if you use a screen capture utility (e.g., Camtasia) to capture an online video. Some streaming sites use variable frame rate which can also produce this effect which is why I think Jagabo went off in that direction.
    Quote Quote  
  18. Frame drops and subsequent duplicates to maintain sync during capturing with PC are often caused by temporary system overload (CPU, HDD access and write speed etc.).
    Try to disable tasks and processes running in the background like virusscanner, downloads and preparation activities for Windows updates, telemetry, updates of apps etc.
    Quote Quote  
  19. There are time code gaps of as many as 33 frames in the sample video:

    Code:
    frame_number  missing_frames
    
       101              3
       109             10
       117              6
       286              2
       299             10
       331             14
       332              1
       338              1
       374             12
       412             27
       531              2
       557             16
       566              2
       894             33
       901              6
       907              6
       917             16
       937             30
    Here's an example of InsertFramesMC to fill the gaps. Again, original (converted to CRF) on left, filled on right. You'll see some of the artifacts motion interpolation creates later in the clip. Other motion interpolators might work a little better.
    Image Attached Files
    Quote Quote  
  20. Thank you all!
    I can't write a program to check the time codes, but I was able to do something like this in Excel, which I will use to check my videos.

    I will also take a closer look at the scripts and filters for filling the gaps, but of course it is more important to capture without any errors in the first place. I believe the errors were caused by a poor HDD access/write speed. I've switched to a faster disk now.
    Quote Quote  
  21. I wrote a quick dirty program that spit out commands ready to be pasted into an AviSynth script:

    Code:
    InsertFramesMC(   101,  3)
    InsertFramesMC(   112, 10)
    InsertFramesMC(   130,  6)
    InsertFramesMC(   305,  2)
    InsertFramesMC(   320, 10)
    InsertFramesMC(   362, 14)
    InsertFramesMC(   377,  1)
    InsertFramesMC(   384,  1)
    InsertFramesMC(   421, 12)
    InsertFramesMC(   471, 27)
    InsertFramesMC(   617,  2)
    InsertFramesMC(   645, 16)
    InsertFramesMC(   670,  2)
    InsertFramesMC(  1000, 33)
    InsertFramesMC(  1040,  6)
    InsertFramesMC(  1052,  6)
    InsertFramesMC(  1068, 16)
    InsertFramesMC(  1104, 30)
    The sample video in post #19 used that script.
    Quote Quote  
  22. Originally Posted by jagabo View Post
    I wrote a quick dirty program that spit out commands ready to be pasted into an AviSynth script
    Interesting! May I download this program somewhere?
    Quote Quote  
  23. It's a quick dirty program, not very robust, with little error handling, and it only handles 50 fps videos. Just a proof of concept really. Don't blame me if it crashes on unusual input. The input must be in the form from the ffprobe command I gave earlier:

    Code:
    ffprobe -v quiet -pretty -print_format csv -show_entries "frame=coded_picture_number,pkt_pts,pkt_duration,pict_type," -select_streams v:0 input.mkv >output.csv
    The CSV file should look like:

    Code:
    frame,0,20,I,0
    frame,20,20,P,1
    frame,40,20,P,2
    frame,60,20,P,3
    frame,80,20,P,4
    frame,100,20,P,5
    frame,120,20,P,6
    frame,140,20,P,7
    frame,160,20,P,8
    frame,180,20,P,9
    frame,200,20,I,10
    frame,220,20,P,11
    frame,240,20,P,12
    By default the program sends output to the console. To run the program with output to a file:
    Code:
    tcc output.csv >inserts.txt
    The output will look like:

    Code:
    InsertFramesMC(   101,  3) #    101   3
    InsertFramesMC(   112, 10) #    109  10
    InsertFramesMC(   130,  6) #    117   6
    InsertFramesMC(   305,  2) #    286   2
    InsertFramesMC(   320, 10) #    299  10
    InsertFramesMC(   362, 14) #    331  14
    InsertFramesMC(   377,  1) #    332   1
    InsertFramesMC(   384,  1) #    338   1
    InsertFramesMC(   421, 12) #    374  12
    InsertFramesMC(   471, 27) #    412  27
    InsertFramesMC(   617,  2) #    531   2
    InsertFramesMC(   645, 16) #    557  16
    InsertFramesMC(   670,  2) #    566   2
    InsertFramesMC(  1000, 33) #    894  33
    InsertFramesMC(  1040,  6) #    901   6
    InsertFramesMC(  1052,  6) #    907   6
    InsertFramesMC(  1068, 16) #    917  16
    InsertFramesMC(  1104, 30) #    937  30
    You must use an AviSynth source filter that outputs VFR frames (ie, does not automatically insert dups for longer duration frames). I use LWlibavVideoSource(). And you must AssumeFPS(50). A full AviSynth script will look like:

    Code:
    LWLibavVideoSource("sample.mkv") 
    AssumeFPS(50)
    
    InsertFramesMC(   101,  3) #    101   3
    InsertFramesMC(   112, 10) #    109  10
    InsertFramesMC(   130,  6) #    117   6
    InsertFramesMC(   305,  2) #    286   2
    InsertFramesMC(   320, 10) #    299  10
    InsertFramesMC(   362, 14) #    331  14
    InsertFramesMC(   377,  1) #    332   1
    InsertFramesMC(   384,  1) #    338   1
    InsertFramesMC(   421, 12) #    374  12
    InsertFramesMC(   471, 27) #    412  27
    InsertFramesMC(   617,  2) #    531   2
    InsertFramesMC(   645, 16) #    557  16
    InsertFramesMC(   670,  2) #    566   2
    InsertFramesMC(  1000, 33) #    894  33
    InsertFramesMC(  1040,  6) #    901   6
    InsertFramesMC(  1052,  6) #    907   6
    InsertFramesMC(  1068, 16) #    917  16
    InsertFramesMC(  1104, 30) #    937  30
    The numbers after the hash marks (comments, for reference) are the original frame numbers and the number of frames to be inserted. Note that after each call to InsertFramesMC() all later frame numbers are increased by the number of inserted frames. The first call to InsertFramesMC() adds three frames -- so the second call to InsertFramesMC() is specified at frame 112, not the original frame number of 109. So you can't remove a line (or lines) without modifying all the subesequent lines.

    A 64 bit exe and ANSI C source code is attached.
    Image Attached Files
    Quote Quote  
  24. Originally Posted by jagabo View Post
    It's a quick dirty program, not very robust, with little error handling, and it only handles 50 fps videos. Just a proof of concept really. Don't blame me if it crashes on unusual input.
    Thanks for the program and the instructions on using it!
    Quote Quote  



Similar Threads