Smart interleaving?

[-Habanero-]

I have two of the same videos in different frame rates where one has unique frames that the other doesn't. I'm trying to combine them to create the best quality footage there is. I thought of just simply adjusting the framerate as needed to match them up, interleave and then Dedup to get rid of the duplicates would cut it but it didn't because the random frame drop from both has some frames appearing before or after others.

[poisondeathray]

Is there something else missing from your description ? Why not just keep the unique frame video ? What does the missing frame video have that the unique frame video doesn't ?

If there is no defining distinguishing characteristic to use in a detection script - (ie. how would you determine which frame from which video is "better" ? ) , you'd have to manually mix & match - easier to do in a NLE or something with a timeline. If they are otherwise the same video, you can use changefps() to create duplicates in the lower FPS video and line them up

[-Habanero-]

Both videos are sort of jerky because of the dropped frames but the fact that each contain legit frames that the other doesn't looked encouraging. The truly original source is on a Laserdisc out there somewhere. These two re-encoded sources are a low bitrate with two different codecs, one looks h263 the other h264. There's no brightness or contrast difference (ok maybe a slight one) between them. The only thing different is the resolution and actual frame rate whatever it might be. They both have duplicate frames but in different spots.

Is there something else missing from your description ? Why not just keep the unique frame video ? What does the missing frame video have that the unique frame video doesn't ?

An example:

video 1 frames
A B B D E F G

video 2 frames
A B C D E E G

I wondered if there was an automatic way to match the duplicates from first video with a non-duplicate from the other.

[poisondeathray]

Duh I totally misread that...

One way you might do that is conditionalfilter() , or conditionalselect(). If a duplicate is detected according to a threshold value, then select from the other video. So you would still need to align the frames with changefps() or something similar and resize to equivalent dimension . But I think that will only work with duplicate frames.

[poisondeathray]

An example:

video 1 frames
A B B D E F G

video 2 frames
A B C D E E G

I wondered if there was an automatic way to match the duplicates from first video with a non-duplicate from the other.

Here is an example using conditionalfilter() solving your example above. I'm sure you can figure it out when you look at it. Obviously this is an impossibly "clean" example, so on the real videos you have to adjust the threshold for noise which will affect "duplicate" detection etc..., then change it to show=false for the real thing

a=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("A", x=160, y=160, size=40)
b=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("B", x=160, y=160, size=40)
c=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("C", x=160, y=160, size=40)
d=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("D", x=160, y=160, size=40)
e=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("E", x=160, y=160, size=40)
f=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("F", x=160, y=160, size=40)
g=blankclip(length=1, width=360, height=360, pixel_type="yv12", fps=24).subtitle("G", x=160, y=160, size=40)

vid1=a+b+b+d+e+f+g

vid2=a+b+c+d+e+e+g

conditionalfilter(vid1, vid2, vid1, "YDifferenceFromPrevious", "<", "0.1", show=true)

This uses YDifferenceFromPrevious for the runtime logic. You specific videos might have a better runtime parameter for detection

[-Habanero-]

Duh I totally misread that...

Not your fault, I was a little vague.

I have never heard of this conditional filter. It seems to work in principle but fails when the alignment eventually goes out of bounds and makes the video look like a misaligned interleave.

May I ask why you did the a= b= c= stuff in your script? I was about to ask for a way to make conditional filter look beyond just one single frame to fix the misalignment but it seems I didn't get something from your script.

[poisondeathray]

The a,b,c stuff was to generate the 2 test clips as per your example. In that example, A, D, G are aligned.

If you cannot align it with other manipulations mentioned above, then this method won't work as is, you'd have to modify it or try something else.

You can add other logical conditions to satisfy, but you need figure out a better description of what is going on or describe the characteristics more clearly. Put them on a timeline in a NLE and look at the pattern

[-Habanero-]

Wait, the letters I posted were frames, not videos. I only have two videos. Conditionalfilter makes the video really jerky unless I trim the second video by 4 frames and then the first 150 frames are good until the misaligned jerkiness begins again.
I guess I can do this a little at a time. The entire video is only like 2500 frames anyway.

[poisondeathray]

Yes, it generates 2 test videos like in your example . It was to illustrate how conditionalfilter() works

vid1=a+b+b+d+e+f+g => ABBDEFG

vid2=a+b+c+d+e+e+g => ABCDEEG

return vid1 and you will see A,B,B,D,E,F,G frames . return vid2 and you will see A,B,C,D,E,E,G frames . The frames are labelled with subtitle()




If your actual videos are CFR videos and in sync, and have the same duration but with different fps (but different random frame drops between them), that means the lower fps video has fewer frames, when you align the first frame (use trim() )

If you use changefps() on the lower fps video, the that will add duplicate frames and they both will have the same framecount now, and be aligned on the start frame and end frame (but not necessarily in the middle)

If the start and end are not aligned, something else is missing in your description, like they are not in sync, or have different duration, or VFR

[-Habanero-]

Ah, I understand. I thought you were describing a method more advanced with the multiple blank clips because I wanted to look at a range higher than just one frame. This conditionalfilter wasn't perfect but it more or less did the job. Figuring out why this frame or the other every now and then didn't match up was too confusing and annoying and I'm not being paid to fix this guy's video so in the end I did what I could and the result still looks better than the original.
Thank you.