Decimate odd patterned duplicate frames in video

[raphaelt]

Hi everyone,

First off, a warning: I'm not an expert in video in general or with the tools I have on hand, so please excuse me if I am not clear in the way I present my problem. Here goes:

I have a collection of mp4 video files which I located on the Internet that all seem to have duplicate frames in a pattern that produces a lurching movement I'm hoping to eliminate. But because I'm far from an expert, I don't recognize the pattern or how to eliminate it, and it doesn't seem to correspond to the frame rate that I can see.

The videos are mp4s with H264 encoding, 29.97 fps progressive.

From playing back the video frame by frame, I have seen this unusual alternating pattern which lasts 72 frames before resetting:

1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d 1 2d

I hope that's clear. The "d" stands for a duplicate of the frame before, and the numbers for unique frames. As you can hopefully see, there is first a sequence of single frames with duplicates (15 unique doubled frames), followed by a series of two unique frames and then one duplicate (a total of 42 frames with the dupes). This pattern then resets back to the beginning and repeats, through the whole video. With dupes, the whole sequence lasts for 72 frames.

I don't know how the sequence begins at the start of the video, because the first few seconds are black, and I'm relying on visual clues alone to determine the pattern of duplicates. So I just arbitrarily started the sequence at the beginning of the single frames with duplicates section. This repeating pattern could in reality start and finish at any point, but this 72 frame pattern does seem to continuously repeat throughout. If there's a more sophisticated way of analyzing the sequence of duplicates in these videos, please let me know. From what I can tell, all the videos in this collection, all of which are H264 mp4s, have this exact pattern, so I assume that the original encoder did some kind of botched framerate conversion from the original source when he or she produced this collection.

By the way, this is no secret collection of videos. They're freely available (and public domain) episodes of Ozzie and Harriet produced by one "kjs" at archive.org. Here's a link to one of the videos so that you can download it and test it if it would help you see what I'm describing: http://www.archive.org/download/TheAdventuresOfOzzieHarriet_642/180_FREE_FLOWERS.mp4

My goal is to decimate the duplicate frames and then produce a DVD compliant framerate of 23.98 or 29.97 fps, if at all possible (using an interpolation filter if necessary). Barring that, I could produce a non-standard XVID or something, for playing on XVID-capable DVD video players. My problem is that I don't know where in this 72-frame pattern the video actually starts, or what tools and settings to use to pull out the dupes. Any guidance or insights would be warmly appreciated!

Thanks in advance!

Raphael

[manono]

You've got problems beyond just the repeat frames. A lot of the frames are blended and there's no way to undo the blending. In addition, they're blending two unique frames together and it plays jerky because of that also, and not just because of the repeated frames. In addition, you're wrong about it having a constant 72 frame cycle. If that were true I could begin anywhere and count 72 frames and the number of duplicate frames should be the same . And I get dupe frame counts all over the place, I've counted 4 different 72 frame cycles and get 72/29, 72/29, 72/33, and 72/35. There is maybe a cycle after which it begins to repeat the same sequence, but it's much larger than 72 frames. And because I don't feel like trying to figure it out, I can't find any way to remove just the repeat frames.

My suggestion is to leave the 29.97fps framerate alone and just go ahead and do whatever you plan on doing. Also, it was just downsized to 480x320 and plays with the wrong aspect ratio, the same 3:2 as the 720x480 source, I suppose. So, everyone looks slightly fat. If making a DVD you do a straight resize to 720x480 and encode at 4:3 and 29.97fps. Otherwise resize to some 1.33:1 ratio, ones such as 480x360, 512x384, or 640x480, if the plan is to make an AVI, perhaps including cropping away of the black bars. Or at least set a par value so it gets resized by the player to the correct aspect ratio.

[raphaelt]

Drat. I was worried it would be this hopeless. When I took a sampling of random sections of a number of the videos (there are about 45 in all) I ran into the same 72 frame cycle each time, but I suppose I should have looked in more depth in just one video. I didn't realize the cycle varied as you discovered. That pretty much shoots it, I guess.

I did notice the blended frames, the severity of which vary from clip to clip -- I also noticed that in some cases the blended frames had interlacing problems. I hoped that if I eliminated the duped frame problem it would at least improve if not solve the quality issue.

A shame this person went through all the trouble of creating these videos for people and then botched the transcoding.

But thanks for taking the trouble to give me your advice. I appreciate the help.

Raphael

You've got problems beyond just the repeat frames. A lot of the frames are blended and there's no way to undo the blending. In addition, they're blending two unique frames together and it plays jerky because of that also, and not just because of the repeated frames. In addition, you're wrong about it having a constant 72 frame cycle. If that were true I could begin anywhere and count 72 frames and the number of duplicate frames should be the same . And I get dupe frame counts all over the place, I've counted 4 different 72 frame cycles and get 72/29, 72/29, 72/33, and 72/35. There is maybe a cycle after which it begins to repeat the same sequence, but it's much larger than 72 frames. And because I don't feel like trying to figure it out, I can't find any way to remove just the repeat frames.

My suggestion is to leave the 29.97fps framerate alone and just go ahead and do whatever you plan on doing. Also, it was just downsized to 480x320 and plays with the wrong aspect ratio, the same 3:2 as the 720x480 source, I suppose. So, everyone looks slightly fat. If making a DVD you do a straight resize to 720x480 and encode at 4:3 and 29.97fps. Otherwise resize to some 1.33:1 ratio, ones such as 480x360, 512x384, or 640x480, if the plan is to make an AVI, perhaps including cropping away of the black bars. Or at least set a par value so it gets resized by the player to the correct aspect ratio.