Is there a way to check for missing frames in a corrupt video?

[NoobInDenial]

Hello,
I was recording a video with a laptop webcam where it suddenly crashed and ended up with a corrupt video file. I was also simultaneously recording the screen through another monitor using a remote desktop viewer with OBS. So I know it crashed at the 39 min 24 sec timestamp. I then repaired the footage using ffmpeg and recover_mp4 tools but the recovered file only has a length of 37 min 27 sec. However, comparing it to the remote OBS recording, the repaired file seems to have the full footage from start to finish. So why has the video been sped up? Does this mean I lost roughly 2 minutes of frames at random times in the video? I don't notice any jerky movements and the frames seem smooth and consistent. The audio is also missing, but for me having the audio isn't really important.

So my question is - is there any way for me to know if the corrupted file lost any frames? If so, how can I check how many frames are missing and, most importantly, at exactly what point in the video? I was thinking maybe it's possible to view the contents of the file with a hex editor or something and check how many frames it's supposed to have. This was a personal project that involved lots of movement in the video so each frame is actually valuable to me.

I tried using other repair tools like Wondershare, but it's the trial version so I'm only allowed to have a preview. The result also seems to be the same from start to finish except it shows a length of 39 min 30 sec this time, and it fixed the audio.

I hope the details of this post isn't too confusing. I'm not the best at explaining things. Any help will be appreciated. Thank you!

[Subtitles]

If your file extension is *.ts then I would suggest using TS Doctor (Trial version has no restrictions and works for 30 days).

[NoobInDenial]

Thank you for the reply. Will it be possible to convert a corrupt mp4 to .ts? I couldn't find any free way to convert my mp4 file to .ts yet because my file is over 600mb. And I have no idea how to use TS doctor. Will it be able to let me know exactly when the video has missing frames?

[Subtitles]

Thank you for the reply. Will it be possible to convert a corrupt mp4 to .ts? I couldn't find any free way to convert my mp4 file to .ts yet because my file is over 600mb. And I have no idea how to use TS doctor. Will it be able to let me know exactly when the video has missing frames?

If the original file extension is not *.ts then there is no benefit to convert it to *.ts

[NoobInDenial]

Oh I see. Well, are there any other options? I imported the repaired mp4 file into TS doctor and found that it has around 70200 frames. It also says the video is 39 mins this time. I definitely know there are missing frames since a video of 39 min 24 sec should roughly have more than 70900 frames. Now I just need to know at what time they are missing. Any suggestions?

[poisondeathray]

Oh I see. Well, are there any other options? I imported the repaired mp4 file into TS doctor and found that it has around 70200 frames. It also says the video is 39 mins this time. I definitely know there are missing frames since a video of 39 min 24 sec should roughly have more than 70900 frames. Now I just need to know at what time they are missing. Any suggestions?

Not necessarily, you're making some assumptions about the frame rate of the recording which may not be correct

OBS recordings can be VFR (variable frame rate) , and the recording framerate can fluctuate. Even if you didn't crash, they can be VFR and appear to be missing frames (or not) if you examined the encoded frames, frame by frame, but ignored the presentation timestamps. Sometimes recordings can have real dropped frames as well

If the "repaired" video file assumed a constant frame rate higher than the average of the actual timestamps of the "uncrashed" video (assume that you had one), then it could appear to be sped up, and the duration shorter .

There is no way to know for certain. Even a simultaneous recording of the same event can have different framecounts, and different timestamps, yet all be in sync with the same duration

[NoobInDenial]

Ok, that makes sense. This is kind of something I wondered myself, that is whether or not a healthy recording can just naturally have missing frames. You're right about the fact that my repaired video has been sped up because FFmpeg and recover_mp4 had the fps option set to 31.25 during the repair process. Normally my webcam would record an average of 29.83fps.

I changed the fps settings for the repair several times and imported the results into Davinci Resolve to see the total number of frames the file has. It appears that every time I lower the fps, the result has a larger frame count, at least according to Davinci Resolve. For example, if the fps is 29.00, I get more than 73000 frames. If the fps is 31, I get around 69000 frames. The repaired file from Wondershare Recoverit also shows a different frame count of around 71000. However, FFmpeg always tells me that the frame count is 70224 for each result. When I view each of these files with TS Doctor it also tells me that the frame count is 70224. So it seems that Davinci Resolve is showing me different frame counts for each result.

So I'm a bit confused. I'm not sure which of these have the accurate frame count. Is this just a problem with Davinci Resolve showing the wrong frame count? I feel like 73000 is way more than it should be. Does it add extra frames somehow? Shouldn't it always be the same since all of these repair results are technically the same file but with different fps? Obviously I'm not an expert on this so I apologize if I'm wrong.

[poisondeathray]

Ok, that makes sense. This is kind of something I wondered myself, that is whether or not a healthy recording can just naturally have missing frames. You're right about the fact that my repaired video has been sped up because FFmpeg and recover_mp4 had the fps option set to 31.25 during the repair process. Normally my webcam would record an average of 29.83fps.

I changed the fps settings for the repair several times and imported the results into Davinci Resolve to see the total number of frames the file has. It appears that every time I lower the fps, the result has a larger frame count, at least according to Davinci Resolve. For example, if the fps is 29.00, I get more than 73000 frames. If the fps is 31, I get around 69000 frames. The repaired file from Wondershare Recoverit also shows a different frame count of around 71000. However, FFmpeg always tells me that the frame count is 70224 for each result. When I view each of these files with TS Doctor it also tells me that the frame count is 70224. So it seems that Davinci Resolve is showing me different frame counts for each result.

So I'm a bit confused. I'm not sure which of these have the accurate frame count. Is this just a problem with Davinci Resolve showing the wrong frame count? I feel like 73000 is way more than it should be. Does it add extra frames somehow? Shouldn't it always be the same since all of these repair results are technically the same file but with different fps? Obviously I'm not an expert on this so I apologize if I'm wrong.

Probably none of them do, or ffmpeg is the closest. What you want is the number of unique frames.

The number of total frames is irrelevant, if some method is used to "scale" the number of frames. Most NLE's like resolve will do this

If you insert duplicates in order to achieve a frame number, it's kind of pointless. Even worse case is if you drop real unique frames to achieve "x" framerate number . NLE's insert duplicates (or blends) or drop frames to conform the asset to the project time line settings - this is "normal" behaviour for video editing for "normal" files . You don't want that, and you definitely don't want to drop good frames

[johnmeyer]

I have no idea what you are trying to do, even after skimming through this thread. However, if you need a script to detect missing frames ("drops"), I developed one which looks for duplicates and drops. You could probably adapt the code to just look for missing frames.

Automatically fix dups followed (eventually) by drops

[NoobInDenial]

Ok, that makes sense. This is kind of something I wondered myself, that is whether or not a healthy recording can just naturally have missing frames. You're right about the fact that my repaired video has been sped up because FFmpeg and recover_mp4 had the fps option set to 31.25 during the repair process. Normally my webcam would record an average of 29.83fps.

I changed the fps settings for the repair several times and imported the results into Davinci Resolve to see the total number of frames the file has. It appears that every time I lower the fps, the result has a larger frame count, at least according to Davinci Resolve. For example, if the fps is 29.00, I get more than 73000 frames. If the fps is 31, I get around 69000 frames. The repaired file from Wondershare Recoverit also shows a different frame count of around 71000. However, FFmpeg always tells me that the frame count is 70224 for each result. When I view each of these files with TS Doctor it also tells me that the frame count is 70224. So it seems that Davinci Resolve is showing me different frame counts for each result.

So I'm a bit confused. I'm not sure which of these have the accurate frame count. Is this just a problem with Davinci Resolve showing the wrong frame count? I feel like 73000 is way more than it should be. Does it add extra frames somehow? Shouldn't it always be the same since all of these repair results are technically the same file but with different fps? Obviously I'm not an expert on this so I apologize if I'm wrong.

Probably none of them do, or ffmpeg is the closest. What you want is the number of unique frames.

The number of total frames is irrelevant, if some method is used to "scale" the number of frames. Most NLE's like resolve will do this

If you insert duplicates in order to achieve a frame number, it's kind of pointless. Even worse case is if you drop real unique frames to achieve "x" framerate number . NLE's insert duplicates (or blends) or drop frames to conform the asset to the project time line settings - this is "normal" behaviour for video editing for "normal" files . You don't want that, and you definitely don't want to drop good frames

So Davinci Resolve is adding duplicates or blends? I check by moving to the next frame by pressing the arrow keys and the frames don't look like duplicates to me. And this is right after I drag and drop the media into the program's window without adding it into the timeline or anything. Is there any way to know which frames are the ones fabricated by Resolve?

I also compared each different repair results in Resolve to count how many frames there are within short intervals during the same part of a video. For example, if the interval is 5 seconds, I get around 150 frames counted for each video. I hope I'm explaining this properly, but basically, despite having different number of total frame counts and fps, Resolve is showing me the same frame count within these intervals for each video. So it doesn't seem to be adding more frames for this specific part of the video. Should I keep testing this method out with more random parts of the video to find where it adds the extra frames?

[poisondeathray]

So Davinci Resolve is adding duplicates or blends? I check by moving to the next frame by pressing the arrow keys and the frames don't look like duplicates to me. And this is right after I drag and drop the media into the program's window without adding it into the timeline or anything.

That section might be close to the FPS of the timeline settings - if that's the case, no additional manipulations are applied to that section

All NLE's use similar methods . They all use CFR timelines. All assets are converted to those CFR timeline settings - because you need to keep things like sync and to be able to edit - it's almost impossible to edit VFR on a VFR timeline

Simple example. You have a 30 FPS CFR video. You put it on a 60 FPS timeline. The frames are doubled, and you have the same duration . If the timeline was 15 FPS, you would drop every 2nd frame.

e.g. Lets say you have a 29.83 fps (average) video. You put it on a 30 FPS timeline. In a section that has say, 33 fps average, it will drop frames to achieve that target 30 FPS CFR. This is very bad if you're trying to "recover" a video. But if you want to avoid dropping frames and use a higher FPS timeline, you will get duplicates inserted. That's usually not desireable either

Is there any way to know which frames are the ones fabricated by Resolve?

I also compared each different repair results in Resolve to count how many frames there are within short intervals during the same part of a video. For example, if the interval is 5 seconds, I get around 150 frames counted for each video. I hope I'm explaining this properly, but basically, despite having different number of total frame counts and fps, Resolve is showing me the same frame count within these intervals for each video. So it doesn't seem to be adding more frames for this specific part of the video. Should I keep testing this method out with more random parts of the video to find where it adds the extra frames?

No, and I wouldn't use Resolve for this . You already saw that the results changed when the "repair" fps was different - that should be enough to tell you that path is unreliable

And you will not get the "real correct" results, because it's impossible to determine what the "real result" really was with a VFR recording, unless you had a time machine and didn't crash. Again - 2 exactly same hardware setups, recording exactly the same event, will not have exactly the same frames - They might start and end at the same time, but the sample points in between will be different (ie. different frames)

If you had a CFR recording, then you have some valid assumptions , and you could figure out what was missing

[NoobInDenial]

I have no idea what you are trying to do, even after skimming through this thread. However, if you need a script to detect missing frames ("drops"), I developed one which looks for duplicates and drops. You could probably adapt the code to just look for missing frames.

Automatically fix dups followed (eventually) by drops

I have a corrupt mp4 file and originally I asked if there was a way to know how many missing frames there are out of how much it's supposed to have if it wasn't "corrupt". But it's not really important to get the exact number of frames lost. What I really want is to know WHEN the frames went missing within the duration of the video. I'm learning now that maybe it's not possible to do this. I'm not sure if your script can help me achieve this but it may come in handy some other time. I'll definitely check it out, thank you!

[Subtitles]

You mentioned using TS Doctor in above posts.
Open your file, click on Prepare Cutting and then click on OK
Save New file
When it finishes, it creates log file and text file. Inspect these files and it will give you a list of when the errors started.
If it finds no errors then you will not get any useful information from these files.

[NoobInDenial]

No, and I wouldn't use Resolve for this . You already saw that the results changed when the "repair" fps was different - that should be enough to tell you that path is unreliable

And you will not get the "real correct" results, because it's impossible to determine what the "real result" really was with a VFR recording, unless you had a time machine and didn't crash. Again - 2 exactly same hardware setups, recording exactly the same event, will not have exactly the same frames - They might start and end at the same time, but the sample points in between will be different (ie. different frames)

If you had a CFR recording, then you have some valid assumptions , and you could figure out what was missing

What tool/software do you recommend that will actually give accurate frame counts? It should be able to let
me know how many frames there are between specific times as well as allow me to move frame by frame. What would also be ideal is if there is a program that can compare two videos and see which ones are identical frames and which ones are different.

Does VLC show an accurate total frame count? I can check with the statistics window but I would have to play the video from start to finish without touching anything since that would mess the statistics up. By the end, VLC always says the video has 70,188 frames, including the lost frames, rather than the 70,224 FFmpeg tells me.

What I'm trying now is using the OBS recording as the real-time timestamp to compare it with the repaired video. I'm doing so with different sections of the video as well as checking how many frames there are from the beginning of the video to a certain point.

For example, if there is a point in the video where I wave my hand, and it was at the 10 min "real-time" mark, it should have maybe around 18,000 total frames if it was recorded with 30fps. By looking at this exact moment in any of the repaired results, if it has a similar amount of frames to 18,000, then can I assume it has kept most of the frames?

I know you mentioned that because of the VFR this isn't going to be reliable but this is the closest way that I can think of to manually find when the frames are missing. Maybe if I stick with the realistic fps that my webcam records with, such as anything between 29.8fps to 30fps, I'll be able to calculate a rough total frame count number for any given time in the video. But I have no idea if this is even an effective method. I guess I'm looking for large differences in the frame count comparisons but what number would suggest it's significant? Would 100 or more frames be enough?

In one case, the frame count up to the 20 min "real-time" mark was close to 36,000, which seemed reasonable. But after a few mins in the video, the differences between the frame comparisons were getting larger, such as 300 frames. Does this mean this is where the video started to lose frames? Again, I don't even know if this is working out or not.

[poisondeathray]

What tool/software do you recommend that will actually give accurate frame counts?

A program that scans and indexes the frames with byte offsets, such as DGIndexNV (if you have a nvidia card) will give an accurate frame count. That will be the low level, more accurate analysis of the actual video frames. Conversely, programs that automatically tell you the frame count immediately will not necessarily be accurate - because they do not scan through the file - they will be making assumptions based on headers , duration, declared fps, timestamp information in the video bitstream and container - but those assumptions might not be correct, especially in a damaged/ perhaps recovered file

It should be able to let me know how many frames there are between specific times as well as allow me to move frame by frame.

A problem that you potentially face is that timing will be different than the original, because the repair program probably assumed a constant framerate to the value you set (e.g. 31.25 you mentioned for one of them), unless it was able to recover the original timestamps - but then it shouldn't "need" you to enter anything in the first place ... So probably you do have not the exact timestamps of the original . You might be able to roughly sync up some points, but you have 2 VFR recordings (maybe 1 damaged / recovered) - and keep in mind when you try to sync something up (probably in a NLE), that you're not getting the "real" results, but rather a CFR converted representation - with either dropped or duplicated inserted frames.

With avisynth, you can display with an overlay the CFR and VFR times with the ffms2 plugin , using FFVideoSource, and navigate in a GUI such as avspmod . I do not think it would be useful for you in terms of "times" , because your recovered timestamps are unlikely to be the original. Even though the framecount will be the same, the duration will change with whatever FPS you assumed in the repair program. I know you don't care about audio, but this will make it more difficult to compare both recordings (and the fact that they probably both record VFR in the first place even without the crash). But at least you will have frame accuracy of the "physically" recorded frames (not the additional layer of confusion caused by the inserted duplicated or dropped new frames from a NLE)

What would also be ideal is if there is a program that can compare two videos and see which ones are identical frames and which ones are different.

Between frames within 1 video, or between 2 videos ?

How "identical" ? Because "Identical" frames will be not be truly "identical" in your case because of lossy compression differences.

Does VLC show an accurate total frame count?

Not sure, but I doubt it

What I'm trying now is using the OBS recording as the real-time timestamp to compare it with the repaired video. I'm doing so with different sections of the video as well as checking how many frames there are from the beginning of the video to a certain point.

For example, if there is a point in the video where I wave my hand, and it was at the 10 min "real-time" mark, it should have maybe around 18,000 total frames if it was recorded with 30fps. By looking at this exact moment in any of the repaired results, if it has a similar amount of frames to 18,000, then can I assume it has kept most of the frames?

That 's probably the best you could do for a fair estimate. It would also help if you knew details about "typical" recordings. eg. What where the lowest average framerate recordings, are they always 29.something ? Do you ever get ones with a lower average FPS? Something like 26.75?

I know you mentioned that because of the VFR this isn't going to be reliable but this is the closest way that I can think of to manually find when the frames are missing. Maybe if I stick with the realistic fps that my webcam records with, such as anything between 29.8fps to 30fps, I'll be able to calculate a rough total frame count number for any given time in the video. But I have no idea if this is even an effective method. I guess I'm looking for large differences in the frame count comparisons but what number would suggest it's significant? Would 100 or more frames be enough?

In one case, the frame count up to the 20 min "real-time" mark was close to 36,000, which seemed reasonable. But after a few mins in the video, the differences between the frame comparisons were getting larger, such as 300 frames. Does this mean this is where the video started to lose frames? Again, I don't even know if this is working out or not.

I'm not sure you can do anything else - because you cannot distinguish - even manually - between which frames were missing due to VFR, or because of the crash . Unless it was a different situation e.g. you had a intact VFR recording , and a 2nd version of that crashed when say you were copying it to a 2nd drive

Timestamp recording VFR generally means missing frames, or lower average FPS than if you had recorded CFR. Notice most (or probably all) of all your "good" recordings have a lower average framerate than the "target" (probably 30 fps) . But if you *knew* that your recordings were *always* greater than 29.9 something, then that method is probably more accurate predictor than if you had recordings that varied more in their average FPS

[NoobInDenial]

You mentioned using TS Doctor in above posts.
Open your file, click on Prepare Cutting and then click on OK
Save New file
When it finishes, it creates log file and text file. Inspect these files and it will give you a list of when the errors started.
If it finds no errors then you will not get any useful information from these files.

It gave no errors, which is what I expected. I've attached the log file just in case you'd like to view it. I don't really know what "Prepare Cutting" does since I'm new to TS Doctor. And I'm not sure what to be looking for in the log files.

[NoobInDenial]

A program that scans and indexes the frames with byte offsets, such as DGIndexNV (if you have a nvidia card) will give an accurate frame count. That will be the low level, more accurate analysis of the actual video frames. Conversely, programs that automatically tell you the frame count immediately will not necessarily be accurate - because they do not scan through the file - they will be making assumptions based on headers , duration, declared fps, timestamp information in the video bitstream and container - but those assumptions might not be correct, especially in a damaged/ perhaps recovered file

Well I guess this won't work if I don't have a nvidia card? I found a program called DJV player and it does what I'm looking for. But judging from what you said it might not be accurate. The only indicator that it might be is that it's pretty laggy so it's using lots of resources to do something. It also says the total frame count of the video is 70,224, which is the same as what FFmpeg and TS doctor tells me. This is probably the "true" frame count

A problem that you potentially face is that timing will be different than the original, because the repair program probably assumed a constant framerate to the value you set (e.g. 31.25 you mentioned for one of them), unless it was able to recover the original timestamps - but then it shouldn't "need" you to enter anything in the first place ... So probably you do have not the exact timestamps of the original . You might be able to roughly sync up some points, but you have 2 VFR recordings (maybe 1 damaged / recovered) - and keep in mind when you try to sync something up (probably in a NLE), that you're not getting the "real" results, but rather a CFR converted representation - with either dropped or duplicated inserted frames.

With avisynth, you can display with an overlay the CFR and VFR times with the ffms2 plugin , using FFVideoSource, and navigate in a GUI such as avspmod . I do not think it would be useful for you in terms of "times" , because your recovered timestamps are unlikely to be the original. Even though the framecount will be the same, the duration will change with whatever FPS you assumed in the repair program. I know you don't care about audio, but this will make it more difficult to compare both recordings (and the fact that they probably both record VFR in the first place even without the crash). But at least you will have frame accuracy of the "physically" recorded frames (not the additional layer of confusion caused by the inserted duplicated or dropped new frames from a NLE)

I don't pay attention to the timestamps of the repaired video. I just look at the events/visuals in the video to match them up while using the OBS recording as the "real-time" timestamp to calculate the frame counts.

31.25 fps was the default that the recovery tool sets every time I try to repair it. Then I found out that I can change the fps to whatever I wish. This obviously changes the duration of the video but the duration doesn't matter since all I want is to count frames. If I set the fps to 29.7 then the repaired video becomes 39min 24sec, which is how long the recording was up until the crash. But we know doing this is pointless because it may not even have been 29.7fps in the first place. Plus, the repaired video probably had any duplicate frames deleted by FFmpeg. So even though this repaired video and the OBS recording starts and finishes at the same time, they are definitely not in sync. However, the repaired audio has the full 39min 24sec duration and syncs well with the OBS recording, which is what I expected.

What would also be ideal is if there is a program that can compare two videos and see which ones are identical frames and which ones are different.

Between frames within 1 video, or between 2 videos ?

How "identical" ? Because "Identical" frames will be not be truly "identical" in your case because of lossy compression differences.

On second thought, nevermind this, it wouldn't help me. I was thinking along the lines of comparing two of the same video except one is an NLE edit that probably has duplicates and such. I can probably accomplish this by comparing DJV player and Davinci Resolve together, assuming DJV is accurate.

That 's probably the best you could do for a fair estimate. It would also help if you knew details about "typical" recordings. eg. What where the lowest average framerate recordings, are they always 29.something ? Do you ever get ones with a lower average FPS? Something like 26.75?

Typical recordings are usually 29.83 on average but it can sometimes go up to 30fps. There are times when it goes down to 29.7
but that is when I noticed lag that skipped a few frames while recording. If my corrupt recording was 29.7fps on average, then
70,224 total frames means I lost almost nothing. But I didn't notice any lag while I was recording. And it was not under low-light conditions, which also significantly drops the framerate. Should I just assume the best and say that I lost no frames because it went down to 29.7? I don't know... it just feels too optimistic to me. My belief is that I'm probably around 300 frames short of how much it should have, which is about 10 seconds missing scattered randomly throughout the video.

So, if they are lost throughout random parts of the video, did it happen after the crash because it wasn't able to process/save the file properly? Or did it already happen while it was recording? I think it's important to know the answer to this. If the frames were already lost while it was recording, then they were just "naturally" gone to begin with. If not, then all of the frames were originally captured but some got lost as a result of the crash.

[poisondeathray]

It also says the total frame count of the video is 70,224, which is the same as what FFmpeg and TS doctor tells me. This is probably the "true" frame count

That is probably what you have or very close. Sometimes programs (e.g. ffmpeg) will drop the 1st 2 frames, but they might not be 100% complete frames in the first place. An example would be a file that had an open-gop

So even though this repaired video and the OBS recording starts and finishes at the same time, they are definitely not in sync. However, the repaired audio has the full 39min 24sec duration and syncs well with the OBS recording, which is what I expected.

Right, and as I mentioned earlier, even if you didn't crash, an intact recording might start and end the same, but the frames in the middle might be slightly different sample points. That's the nature of VFR . Some frames display longer or shorter than others - even if you had the original uncrashed correct frames (or assume you were able to recovery 100% all the frames) - unless you have the original timestamp information , it wouldn't sync up. Even if you had the timestamp information, you might find certain parts "slightly off" and snap back into sync a bit later.

Should I just assume the best and say that I lost no frames because it went down to 29.7? I don't know... it just feels too optimistic to me. My belief is that I'm probably around 300 frames short of how much it should have, which is about 10 seconds missing scattered randomly throughout the video.

How did you arrive at "300" frames ?

It sounds like you didn't lose too many either to VFR "normal" losses, or to crash. But look on the bright side - It could have been MUCH worse

So, if they are lost throughout random parts of the video, did it happen after the crash because it wasn't able to process/save the file properly? Or did it already happen while it was recording? I think it's important to know the answer to this. If the frames were already lost while it was recording, then they were just "naturally" gone to begin with. If not, then all of the frames were originally captured but some got lost as a result of the crash.

I would expect a crash to affect the frames near the time of the crash, such as an interrupted recording - not "randomly" 1 or 2 here or there

If you suspect there are missing frames and you looked and compared and they are "randomly " missing here and there - they are probably the "normal" missing frames from VFR recording, not from the crash

[NoobInDenial]

How did you arrive at "300" frames ?

It sounds like you didn't lose too many either to VFR "normal" losses, or to crash. But look on the bright side - It could have been MUCH worse

Apologies for responding a bit late but I'm still very much keen on this subject! I mentioned before that usually the "healthy" recordings on average are 29.8 to 30fps. So I just estimated that a 39min 24sec video should have around 70,500 to 70,900 frames. Generously, I would be missing at least 300 since mine only has 70,224. Worst case scenario would be around 700. But yes, I agree that it could have been much worse.

So far I've been working on the "testings" that I mentioned in an earlier post. I'll just reiterate: my goal was to find where the missing frames were dropped by taking precise time measurements of my OBS "real-time" recording and comparing it with my repaired video to calculate the number of frames that go by during those time measurements. I used DJV player to look at the frame numbers and Davinci Resolve to measure the time of the OBS recording. Because the OBS recording is 60fps, I set Resolve to measure the time in 60fps. After I was almost finished with the testing, I realized that the measurements would be inaccurate since Resolve adds extra frames to the video. So I decided to start over using Avidemux instead to measure the time in milliseconds. I don't know which would be a more accurate measurement of a video in terms of timing, milliseconds or frames per second? Either way, both ended up having similar results.

I'm not exactly sure if I can make a concrete conclusion with my testing, but overall I don't think I found anything out of the ordinary or anything groundbreaking. I was probably losing maybe 1 or 2 frames every 7 to 8 seconds of the video, which would amount to roughly 300 frames.

I know it's normal for VFR recordings to naturally drop frames but what does it exactly mean to "lose" a frame? Were they captured on display while it was recording but did not make it into the output, or were they never there to begin with? For instance, say I blinked at one point while recording with my webcam as well as recording the screen with OBS. The webcam video ends up not having that specific moment but the OBS recording does. This has happened a couple of times before, where the video output had missing frames that OBS was able to pick up. Shouldn't I expect all frames that were shown on screen while recording to be saved? Does it do this every time it lags or is it a rare glitch? Do all cameras have this issue or just my laptop webcam?

[poisondeathray]

I don't know which would be a more accurate measurement of a video in terms of timing, milliseconds or frames per second? Either way, both ended up having similar results.

For video, you have discrete sampling. For CFR it's clear - each frame is displayed in a 30/1 fps CFR video for ~33.33333ms

But the FPS is variable for VFR recordings, so the FPS fluctuates . Each frame has a timestamp indicating when it is displayed, and for how long . The average FPS for VFR depends on what time horizon you calculate it. If you take an average over 1 minute, the FPS value might be different than 10 minutes. For CFR it's always the same

But your VFR recordings are always compromised (even non crashed ones) - ie. they have fewer frames than their CFR equivalent. Therefore you have less accuracy, gaps in sampling

I know it's normal for VFR recordings to naturally drop frames but what does it exactly mean to "lose" a frame? Were they captured on display while it was recording but did not make it into the output, or were they never there to begin with?

It means they were never recorded, for whatever reason. There can be different underlying reasons for this, including sensor scan rate (e.g. lighting changes), hardware insufficiency for different components - eitherway the end result is the same

For instance, say I blinked at one point while recording with my webcam as well as recording the screen with OBS. The webcam video ends up not having that specific moment but the OBS recording does. This has happened a couple of times before, where the video output had missing frames that OBS was able to pick up. Shouldn't I expect all frames that were shown on screen while recording to be saved? Does it do this every time it lags or is it a rare glitch? Do all cameras have this issue or just my laptop webcam?

This is a normal occurrence for VFR - different sample points, dropped frames

Recall what I wrote earlier

Again - 2 exactly same hardware setups, recording exactly the same event, will not have exactly the same frames - They might start and end at the same time, but the sample points in between will be different (ie. different frames)

Also ~30fps or about 33.3 ms, is not a very fast sampling rate. In North America, a typical computer display, or TV will be 60Hz , or 59.94Hz . You're only capturing about 1/2 the data on the screen in the first place


But this can be "typical" for CFR as well. It won't be as bad at the VFR case, but 2 CFR recordings of the same event won't necessarily sync up exactly in terms of frame sampling points, unless they were genlocked. Consumer equipment usually does not have this capability

https://en.wikipedia.org/wiki/Genlock

[NoobInDenial]

Oof sorry once again for the late response

But the FPS is variable for VFR recordings, so the FPS fluctuates . Each frame has a timestamp indicating when it is displayed, and for how long . The average FPS for VFR depends on what time horizon you calculate it. If you take an average over 1 minute, the FPS value might be different than 10 minutes. For CFR it's always the same

That would explain why a measurement over a time of 1 minute had more varying results of frame counts for each calculation, as a longer interval means multiplying a larger number with all the possible fps (29.8, 29.9, 30 etc). I mainly stuck with measuring 1 to 5 second intervals and they usually gave me results where there were only 1 or 2 frame differences.

Again - 2 exactly same hardware setups, recording exactly the same event, will not have exactly the same frames - They might start and end at the same time, but the sample points in between will be different (ie. different frames)

Does this even include when OBS is capturing the screen of the webcam app during the recording? I still expect they should be the same since recording the screen isn't like recording the same event with a different camera or hardware.

Also ~30fps or about 33.3 ms, is not a very fast sampling rate. In North America, a typical computer display, or TV will be 60Hz , or 59.94Hz . You're only capturing about 1/2 the data on the screen in the first place

So this is probably not the case, but does this mean that the webcam, despite having the settings to record at 30fps, will record in 60fps but then will drop half the frames while saving/processing the output file? I thought about this possibility before but didn't think it was very logical at all, so forgive me if I'm misunderstanding.

When I was comparing the 2 different recordings frame by frame I didn't notice any "new" different frames from each other. It's only when a webcam recording had certain lag spikes and it is at this moment when it has a missing frame that OBS was able to catch. I assumed this lag is due to a hardware issue and isn't considered a typical "normal" occurrence since this rarely happens. Maybe it has happened before where I didn't notice it, but from what I remember this only ever happened once or twice before and it was a long time ago.

I guess a concern for me now is whether other cameras will have the same behavior. If this is normal, how will I be able to trust cameras ever again? I'm the type of person who values and obsesses over how perfect the recording should be and that every moment should be included. I know the human brain can't register and remember EVERY single frame that was displayed at the time of recording, but when it has lag spikes the individual frames actually become slow enough for you to recall them mentally.

https://en.wikipedia.org/wiki/Genlock

This seems very interesting. I don't think I've ever heard of this before honestly. I doubt that I fully understand all the details after reading about it but it's always great to learn more about these things.

[poisondeathray]

Again - 2 exactly same hardware setups, recording exactly the same event, will not have exactly the same frames - They might start and end at the same time, but the sample points in between will be different (ie. different frames)

Does this even include when OBS is capturing the screen of the webcam app during the recording? I still expect they should be the same since recording the screen isn't like recording the same event with a different camera or hardware.

When I was comparing the 2 different recordings frame by frame I didn't notice any "new" different frames from each other. It's only when a webcam recording had certain lag spikes and it is at this moment when it has a missing frame that OBS was able to catch. I assumed this lag is due to a hardware issue and isn't considered a typical "normal" occurrence since this rarely happens. Maybe it has happened before where I didn't notice it, but from what I remember this only ever happened once or twice before and it was a long time ago.

Yes, because of the nature of VFR - the recorded frames will be offset. They might "appear" the same - because your screen only refreshes probably at 60Hz, but the frames will be offset by some ms value, this is expected. But by the same token, some frames might "appear" different.

For real time , say 10.000 seconds, OBS might record a frame at 10.001 seconds, but the webcam at 10.004 seconds. But they "look" like the same frame because the display has "quantized" the data (whatever it is) in the first place to 16.67ms intervals - that is the smallest increment limited by the display refresh rate unless you had a faster refresh rate. That frame is displayed for 16.67ms, so any recording device will show the same frame when recording between that interval.

That screen data has a relatively LOW sampling rate in the first place, compared to human vision as if you witnessed a live event. Consider high FPS gameplay (people play at >120 FPS these days), or a sports event like 100m photo finish where there is <0.001 s difference. The events captured are many times more accurate with high speed cameras

I guess a concern for me now is whether other cameras will have the same behavior. If this is normal, how will I be able to trust cameras ever again? I'm the type of person who values and obsesses over how perfect the recording should be and that every moment should be included. I know the human brain can't register and remember EVERY single frame that was displayed at the time of recording, but when it has lag spikes the individual frames actually become slow enough for you to recall them mentally.

99.9% of consumer cameras today will behave the same, but many web cams can record 60fps average

It would be better to record higher FPS with duplicates so you don't miss any real unique data or frames. If the MIN fps is > the actual data FPS rate, you won't miss anything

So it depends what your actual data is, and where the bottlenecks are . eg. If your actual data is sent at <30FPS MAX (eg. maybe it's a internet feed from another person's webcam) and you record a higher fps with a min fps always >30fps, you will never miss any real data.

But it was some gameplay at 140-150fps average, and you had a 300Hz display, and you recorded at 30FPS min, you will miss a lot of different frames and real data