Virtualdub2 removing every D Frames

[Hunk91]

Hello everybody !
I'm facing a problem. I recorded a video with an action camera, and the result is an avi file with a resolution of 1280x720 at 30fps progressive frames.

The problem is that there are a lot of duplicate frames and they are labeled as D frames where are the unique frames are labeled as K frames.

I searched for an easy solution to remove those duplicate frames but they are not working (remove frames virtualdub plugins, mpdecimate option with ffmpeg).

The only way I found, is deleting them manualy with the keyboard delete key. It's a very long process as the video is 23 minutes long.

Do you have an easier solution ? What would you recommend ?

Thanks a lot


Best regards

[johnmeyer]

If you merely delete duplicate frames, your audio will lose sync.

I have an AVISynth script I wrote which will replace duplicates with a frame that is motion estimated, using the two unique frames on either side of the duplicate.

Also, do you have any idea why an action cam video would have duplicates?? That should not happen. Have you, perhaps, used the wrong frame rate in your NLE? For instance, depending on what NLE you are using, if you drop 24 fps video into a 30 fps timeline, the NLE may duplicate the frames.

If there is a pattern to the duplicates you should be able to use IVTC (Inverse Telecine) to delete those duplicates. You will then up with video at a different frame rate, but if you use that frame rate, the audio should stay in sync.

[davexnet]

How did the footage get all those duplicates in the first place?

[Hunk91]

Thanks you johnMeyer and for your reply. I have to check with IVTC.
The audio out of synch is not a problem as I will remove the audio and add music instead.
The pattern I see is there's a duplicate every 3 frames ( i mean frame 1 and 2 are unique and frame 3 is a duplicate of frame 2 and then againt 2 unique frames and then a duplicate and so on)
The camera I use is a Very low budget action camera but is OK for what I want to archive.

Best regards

[poisondeathray]

The pattern I see is there's a duplicate every 3 frames ( i mean frame 1 and 2 are unique and frame 3 is a duplicate of frame 2 and then againt 2 unique frames and then a duplicate and so on)

In vdub2, you can try video => frame rate => process every third frame (decimate by 3)

or in ffmpeg

-vf decimate=cycle=3

That should leave you a 20 fps video . If there are no big jumps in motion, that's probably what it should be. You can try motion interpolation after decimation for a smoother video (20fps is "choppy")


But if the duplicate was a dropped frame, then interpolating a new frame at the duplicate location would likely be the correct approach for 30fps . This is probably unlikely if the pattern is every 3 frames and consistent

If you still have problems , post a video sample

[johnmeyer]

You have described a telecine pattern. This is used to allow 24 fps film or progressive video to play in a 30 fps project.

I think VD has an inverse telecine filter. I use AVISynth with TFM/TDecimate.

You might get more help if you identify which action camera you were using and, if you know it, what setting you used. Also, drop the video onto MediaInfo and post the results.

[Hunk91]

Thank you all for your advices!
I finaly made it by using an avisynth script.Thanks to a post from StainlessS at Doom9.org .

The patern of sequence is 2 uniques frames followed by 1 duplicate frame. Since I needed to erase the 3rd frame in a group of 3 frames.

Here is the script I used :

AviSource("video.avi")
SelectEvery(30,0,1,3,4,6,7,9,10,12,13,15,16,18,19,21,22,24,25,27,28)

So in the end my video was 20 fps per second. With using the interpolate plugin from virtualdub I could rebuild a 30 fps video file.

I hope this could help others in the futur if they are facing a similar issue.

Best regards

[johnmeyer]

Well, if it works for you, then that's great. I would never use SelectEvery() on something like this because the cadence often gets modified at scene changes. If you instead use TFM() with TDecimate, you'll avoid any of those problems. However, if you are happy with the result, that is all that counts.

[jagabo]

AviSource("video.avi")
SelectEvery(30,0,1,3,4,6,7,9,10,12,13,15,16,18,19,21,22,24,25,27,28)

Since your pattern repeats exactly all you needed was:

AviSource("video.avi")
SelectEvery(3,0,1)

[rgr]

Also, do you have any idea why an action cam video would have duplicates?? That should not happen.

This is quite normal in low light. The camera increases the exposure time but duplicates the frames.

[rgr]

The audio out of synch is not a problem as I will remove the audio and add music instead.

If there is no regularity, you can use this (better):
http://avisynth.nl/index.php/Dup

or this (worse):
http://avisynth.nl/index.php/DeDup

[rgr]

So in the end my video was 20 fps per second. With using the interpolate plugin from virtualdub I could rebuild a 30 fps video file.

Use RIFE() at the end of the script. You'll have 40fps and much better smoothness.

[Hunk91]

Thanks you Jagabo and rgr for your advices.

Jagabo your script is much more simpler ! I never got the logic with those functions where frames are involved.

Rgr thank you for the rife function idea, I will give it a try and test it soon.

As for the action camera it is from the brand Homeday (a subdivision of French low cost and cheap items Shop named Gifi, a kind of Dollar store) it is the Xpert model released many years ago.

It can record at a resolution up to : 1280x720 at 30fps (but the true frame rate is 20fps and I think the resolution is upscaled). It is a realy cheap camera (technology speaking but also financialy) . I paid it 10 euros, I wasn't expecting much of it, just for training how to use such a camera. For underwater footages (my main use) it is a good start (not perfect, but if there's a leak in the case and the camera gets damaged I won't lost a lot of money.
I plan to buy a much better camera for next summer so those frame rate issues won't be more present.

I think the frame rate issue on this camera is also caused by the fact that this camera doesn't have any Electronic Image Stabilization and when I'm panning too fast or if it's too shacky this could cause duplicated frames.
The pattern is not always the same, but for my Last shooting I was using a diving rig stabilizer which helped a lot.

Best regards

[jagabo]

Jagabo your script is much more simpler ! I never got the logic with those functions where frames are involved.

Since you are working with a pattern that repeats every 3 frames you want to use SelectEvery(3,....). This breaks the clip into consecutive groups of 3 frames, (0,1,2) (3,4,5) (6,7,8)... As far as SelectEvery() is concerned each group contains frames 0,1,2. Since the third frame (#2) is a duplicate you only want to keep frames 0 and 1, hence SelectEvery(3, 0, 1). I.e, out every group of 3 frames keep only frames 0 and 1.

[Hunk91]

Thank you Jagabo, for the explanation.

So I tried Dedup as advised by rgr and it did a great job at removing every dupciated frames!!. I compound those erased duplicated frames with Interframe and got an easy interpolation.

For those who would be interested here is the script I used :

FFMpegSource2("video.avi")
ConvertToYV12
DupMC(log="log.txt")
InterFrame(NewNum=30000, NewDen=1000, Cores=2)

For those wondering why I used FFMpegSource instead of AviSource, it is because the video codec is MJPG. I can use AviSource only if I install a MJPEG codec.

Edit : I Just checked and FFMpegSource removes automaticaly my duplicates ! So no need of DeDup.

 
FFMpegSource2("video.avi")
ConvertToYV12
InterFrame(NewNum=30000, NewDen=1000, Cores=2)

[jagabo]

Edit : I Just checked and FFMpegSource removes automaticaly my duplicates !

Yes, ffmpegsource doesn't include the dups by default. This is also true of soft pulldown in MPEG files. Other filters like AviSource and LSMASH include the dups.

So no need of DeDup.

 
FFMpegSource2("video.avi")
ConvertToYV12
InterFrame(NewNum=30000, NewDen=1000, Cores=2)

That will work if the duplicates aren't placeholders for missing frames. Ie, if the non duplicates are evenly spaced at 1/20 second intervals. If the dups are placeholders for missing frames the remaining frames are spaced at 1/30 and 2/30 second intervals giving uneven motion at 20 fps. You can synthesize the missing frames like this:

 
FFMpegSource2("video.avi")
ConvertToYV12()
InterFrame(FrameDouble=true, Cores=2) # double the frame rate
SelectEvery(4, 0,2,3)

Given frames:

0       1       3...    (frame 2 is missing)

after InterFrame you have:

0  0.5  1   2   3...  (the synthesized frames are created with motion halfway between their surrounding frames)
                      (0.5 i halfway between 0 and 1, 2 is halfway between 1 and 3)

Hence

SelectEvery(4, 0,2,3).

will leave you with frames 0 and 1 of the original video and the synthesized frame 2.

[Hunk91]

Edit : I Just checked and FFMpegSource removes automaticaly my duplicates !

Yes, ffmpegsource doesn't include the dups by default. This is also true of soft pulldown in MPEG files. Other filters like AviSource and LSMASH include the dups.

So no need of DeDup.

 
FFMpegSource2("video.avi")
ConvertToYV12
InterFrame(NewNum=30000, NewDen=1000, Cores=2)

That will work if the duplicates aren't placeholders for missing frames. Ie, if the non duplicates are evenly spaced at 1/20 second intervals. If the dups are placeholders for missing frames the remaining frames are spaced at 1/30 and 2/30 second intervals giving uneven motion at 20 fps. You can synthesize the missing frames like this:

 
FFMpegSource2("video.avi")
ConvertToYV12()
InterFrame(FrameDouble=true, Cores=2) # double the frame rate
SelectEvery(4, 0,2,3)

Given frames:

0       1       3...    (frame 2 is missing)

after InterFrame you have:

0  0.5  1   2   3...  (the synthesized frames are created with motion halfway between their surrounding frames)
                      (0.5 i halfway between 0 and 1, 2 is halfway between 1 and 3)

Hence

SelectEvery(4, 0,2,3).

will leave you with frames 0 and 1 of the original video and the synthesized frame 2.

Thank you once again Jagabo for your detailed answer.
I will try with your recommandations. The results I had with the basic command for Interframe were good and even a spectacular smoothness when I tried for 60 fps (but there were some little strange side effects).

[jagabo]

spectacular smoothness when I tried for 60 fps (but there were some little strange side effects).

When motions are too large or too complex motion interpolation can't figure it out. RIFE usually works better then InterFrame.

http://avisynth.nl/index.php/RIFE