Decimate/De-duplicate H.264 video losslessly?

[tubagod]

I'm wondering if it's possible to decimate/de-duplicate H.264 video losslessly?

My input file is a screen recording from Quicktime at a constant framerate of 60FPS. But there are tons of duplicated frames. Most of the recorded video is going more like 20FPS, though it varies, and sometimes it stays on one screen for several seconds. Obviously, my output file will need to be VFR.

I ran it through FFMPEG using this command:

ffmpeg -i input.mov -vf mpdecimate -vsync vfr -strict -2 output.mov

This created a much smaller file, and it successfully deduped the frames. But it re-encoded the whole video, so the end result is definitely lossless, compared to the input file.

I don't understand enough about the inner workings of H.264 (or codecs in general) to know if what I'm asking is possible. If it isn't, I can live with it. But I'd like to know for sure before I give up.

[Bernix]

Can you provide some information about your input file (resolution, bitrate) and vision of your output file size or content of file for example if it is capture webpage or so.
Because if enough reference frames can helps a lot and duplicated frames do not take much space in h264 or h265.

Bernix

[CursedLemon]

I have to wonder if there would be issues decimating a lossy video file, as they're loaded with B and P-frames and in some editing functions that creates issues. For example, you can't trim videos without re-encoding on a B or P-frame, you have to do it on an I-frame.

If you have a fast/big hard drive, maybe try doing your initial captures in YUV?

Also, if your CRF setting for your capture is something like 25-ish, that really ought to create a small, usable file, regardless of any duplicate frames.

[poisondeathray]

ffmpeg -i input.mov -vf mpdecimate -vsync vfr -strict -2 output.mov

This created a much smaller file, and it successfully deduped the frames. But it re-encoded the whole video, so the end result is definitely lossless, compared to the input file.

That should be lossy compared to the input file. Filesize would usually be be much larger if lossless

The short answer is usually "no" with usual h.264 varieties. And definitely "no" unless you used uncompressed or a lossless codec to re-encode

The reason is -vf anything in ffmpeg decodes and uncompresses the video. So the filesize would normally be much larger, even if you lose 90% of the frames

If original used I-frame compression (intra only, each frame is individually encoded) , then theoretically you could do this losslessly and keep the original compression. Then roughly you would reduce the filesize by the %of frames dropped (e.g. if you reduce 50% frames, the filesize would rop 50% approximately, because the stream isn't decompressed, then recompressed . You kept the original stream compression) . But this isn't possible with ffmpeg using -vf something, you'd have to use a smart editor that can pass through frames (stream copy)

But the majority of usually h.264 used uses long GOP or temporal compression. Frames rely on information from adjacent frames. So you cannot drop single frames where every you want. Only groups of frames which can be between ~1-250 typically. When you decode to uncompressed frames, each frame is an I-frame, so you can manipulate at will. But then you use the original compression. If you recompress the video using a lossy codec, you lose quality = generation loss. If you use an lossless compression codec, it will be usually be much larger than the original (it's lossless compared to the uncompressed version, not the compressed version)

[tubagod]

Thanks for the detailed response, especially @poisondeathray.

It's a screen recording of video that was originally VFR. (I know, not ideal.) I used the "Screen Recording" tool in QuickTime to capture it.

I DO understand the basic difference between I-frames, B-frames, and P-frames, but I'm not sure how to determine whether my video file uses only I-frame compression. (Since it was screen-recorded in real time, I would assume, at least, that it doesn't use any B-frame compression, unless QuickTime's screen recorder quickly goes back and processes the forward-frame information, shortly after real-time.)

How would I determine whether it uses only I-frame compression? Can ffmpeg tell me that?

[CursedLemon]

Unless you are using x264's 0-CRF setting (I think, anyway), you will have B and P frames in your vidjas.

[poisondeathray]

How would I determine whether it uses only I-frame compression? Can ffmpeg tell me that?

ffprobe can

But quicktime doesn't use intra compression if you used default settings , you already know the answer

If you changed the "key frame rate" in the quicktime dialog to "all" or every "1" frames - then it would be intra

You still can't use ffmpeg to decimate losslessly (-vf anything...decompresses stream, you lose original compression). And if you decimate manually, you will be out of sync unless you have some mechanism to compensate for the timecodes

If you're concerted about quality loss, people generally capture at a higher quality level, sometimes even lossless, knowing that they will edit/process the file and incur some quality loss for a final distribution format later

Unless you are using x264's 0-CRF setting (I think, anyway), you will have B and P frames in your vidjas.

crf 0 is lossless but can use temporal compression too (lossless temporal, that's actually the default setting. You have to force --keyint 1 for intra encoding) .

[CursedLemon]

crf 0 is lossless but can use temporal compression too (lossless temporal, that's actually the default setting. You have to force --keyint 1 for intra encoding) .

Ah, gotcha!

[Bernix]

I tested i frame compression x265. And crf0 is much bigger than original (obviously). But when sets to lossless, it creates Frame that is about crf5 size. But it is only to mention. I think it will be similar to x264. Don't know. IMHO anything bellow 10 is useless for final video. 10 is also huge file. I dont think you recognize crf16 with good other settings, like Motion estimation method set to Uneven multi hexagon, Subpixel refinement to 9, 8+ reference frames and similar.

[tubagod]

Thanks for the replies. I have one more dumb question...

Once I've converted it to VFR video, accepting that I'll have a bit of quality loss, would still be possible for me to remove some frames after the fact without incurring additional quality loss? Or would I have to do that before converting?

I'll give you the specific scenario:

I've got a bunch of videos that are screen-recordings of live-stream video. There are a few spots where the screen cuts to black and shows a buffering message for half a second or so. I'd like to convert them in bulk, but it would be nice if I could retain the option to get rid of the "Buffering" screens after the fact, without losing additional quality AGAIN.

Is there an editor that lets me manually remove frames from a VFR video without re-encoding? (I realize that might be dependent on the position of the I-frames. But if there's a cutaway in a VFR video, I'm thinking maybe that would trigger an I-frame automatically in most cases. It might depend on the encoder, but that seems to make sense to me.)

If that is possible, can I just extend the previous frame out longer? (If not, the audio would get out-of-sync. But if there's an editor that can losslessly cut the video that way without extending the previous frame, I'm not as worried if it needs to reencode the audio to do that.)

Anyway, if that sounds possible, what editor should I play around with to test my options?

[CursedLemon]

Use Avidemux and scroll through the video by keyframe, cropping on those points wherever works best (you might not get exactly clean cuts on the transitions to black). Then you can select "copy" for both the video and audio outputs, and save to an MKV container.

[poisondeathray]

Thanks for the replies. I have one more dumb question...

Once I've converted it to VFR video, accepting that I'll have a bit of quality loss, would still be possible for me to remove some frames after the fact without incurring additional quality loss? Or would I have to do that before converting?

Not really. But it depends on how you "converted it to VFR video" . If you use a lossless intermediate (again large filesizes) , then you can avoid additional generation lossless

I've got a bunch of videos that are screen-recordings of live-stream video. There are a few spots where the screen cuts to black and shows a buffering message for half a second or so. I'd like to convert them in bulk, but it would be nice if I could retain the option to get rid of the "Buffering" screens after the fact, without losing additional quality AGAIN.

Is there an editor that lets me manually remove frames from a VFR video without re-encoding? (I realize that might be dependent on the position of the I-frames. But if there's a cutaway in a VFR video, I'm thinking maybe that would trigger an I-frame automatically in most cases. It might depend on the encoder, but that seems to make sense to me.)

If that is possible, can I just extend the previous frame out longer? (If not, the audio would get out-of-sync. But if there's an editor that can losslessly cut the video that way without extending the previous frame, I'm not as worried if it needs to reencode the audio to do that.)

Anyway, if that sounds possible, what editor should I play around with to test my options?

Another problem is 99.9999% of "video editors" use CFR timelines. You cannot edit VFR video without extreme difficulty or lots of manual manipuation of timecodes

That's why the usual way things are done is CFR recording, edit and whatever manipulations while it's still CFR (e.g. color correction, editing, annotations, graphics whatever... ) , then you can use VFR for the final distribution. This way you minimize generation losses.