Encoding just the cut portion up to the key frame during editing.

[Budman1]

Does anyone know of a way to encode just the portion from where you cut a video up to the next key frame? It's easy to get the key frame list but I am wondering if the frames before this (at the cut) can be encoded with the same format as the rest of the video, up to the next key frame, possibly with something like trim.

Playing around with ffmpeg and avisynth hasn't worked yet for me but it seem like there should be a way. Any ideas would be most appreciated.

Thanks.

[Baldrick]

I would check how mkv cutter works. See http://forum.selur.de/topic220-mkv-cutter-tool-to-cut-h264-mkv-files-frame-accurate.html

[smrpix]

If you're doing a lot of this, commercial products like VideoReDo and TMPGenc Smart Renderer do exactly this. If it's a one-off, they both have free trials.

[Budman1]

Actually, im looking for a method that I can incorporate into a new editor I am writing. Ffmpeg, avisynth, CLI, or basic/C+ would be great. Sorry I was not more explanatory in my original post. Thank you.

[smrpix]

Ah! There's a lot of demand for that. Best of luck.

[hello_hello]

I'd love to know how programs make it work for x264 encoded video.

If you use -stitchable in the x264 command line (I leave it enabled by default, because you never know) and if you use the same encoder settings (different bitrates or CRF values are okay) then you can encode video in sections and split it and append it to your hearts content, but when it comes to re-encoding a section of an existing video encoded with a different x264 version and trying to match the encoder settings in order to re-encode a section of it.... I've rarely had any luck getting that to work, in respect to being able to appended the re-encoded section to the rest of it.

[Budman1]

Thx yo everyone for the valuable information. I will continue to experiment and display the results if I find a way to accomplish this for various containers..

[Selur]

Looking at how MKVCutter does it:

here's a general overview how MKVCutter works:

• analyze content with MKVInfo (to get the key frame positions)
• analyze content with MediaInfo (to get some general data needed for audio reencoding and preview)
• create AvisynthSkript to preview the content (simple script that calls FFVideoSource on the input)
• let user set which parts of the content he wants
• create audio and video cut lists for mkvmerge, create trim calls
• let user set the output name and the temp folder
• call mkvmerge with the video cut lists to create a bunch of temporary files (here the cuts are always on key frames)
• analyze one of the not reencoded parts with h264_parse to improve compatibility
• create avisynth scripts using the trim calls and LWLibAVSource to decode the GOPs that need reencoding (smart rendering)
• create x264 call to reencode the content provided by the avisynth scripts
• call x264 to reencode
• when all GOPs that need to be reencoded are reencoded, call mkvmerge to cut the audio (here attachments, subtitles&co do not get disabled any therefore should still be available)
• merge the new audio video and all the reencoded and kept GOPs

the main points are:
a. figure out all the gops that you need/want to reencode
b. split the source into 'before cut-out', 'cut-out' and 'after cut-out'
c. reencode the 'cut-out' (or a portion of it)
d. merge 'before cut-out', 'reencoded cut-out' and 'after cut-out'

Assuming you know either the time codes or the frame numbers of the frames (and your source is cfr and not broken somehow) you need to reencode:
'a.' can be done with ffprobe and some filtering:

ffprobe -show_frames -pretty "path to input"

delivers a list of all the frames their types and time codes
-> filtering to get the gops start&end time codes should be easy
'b.' can be done with ffmpeg using something along the lines of:

ffmpeg -i inputfile -ss [start] -t [duration] -c copy outputfile

three times to get the desired parts 'before cut-out', 'cut-out' and 'after cut-out'
'c.' can also be done with ffmpeg; not posting an example line here, since that should be easy. Only thing you need to be sure is which encoding settings to use during reencoding to be still compatible to the source. (MKVCutter uses mediainfo and h264_parse for this, but nowadays one might be able to use ffprobe or the libav-library for this. In case the source is always x264, mediainfo should be enough since it basically shows you the original encoding command line)
'd.' can be done with ffmpeg, see: https://trac.ffmpeg.org/wiki/Concatenate

-> whole thing isn't that complicated and can be done for years (only wrote MKV Cutter as a proof-of-concept since nobody reacted to the general idea of it back in 2010).

Cu Selur

[shekh]

What happens with audio at the same time? The cut-out part must satisfy both given duration and the codec paging, but you cannot do both, correct?
For example if your video cut-out part is just 1 frame of 30fps, the corresponding audio @48kHz should be 1600 samples, but the codec maybe only accepts pages of 1024 samples. Is this resolved somehow? (I have vague idea about formats)

[Selur]

from experience: both mkvmerge and ffmpeg can handle the audio at the same time without a problem

[shekh]

I suspect this works because nobody cares of ~20ms desync. Neither I see it as major problem, just curious.

[Selur]

20ms is really hard to spot during normal viewing,... (and that delay would only be for one gop)

[shekh]

As I understand the delay must propagate to entire "after-cut-out" segment. To concatenate a-b-c where b has added length it must push c further in timeline.
Yes 20ms is small but it can lead to bigger issue if the process is repeated multiple times (how about connecting 100 segments).
(all this is pure speculation without confirmed experiment)

[Selur]

Assuming you are correct doing a few cuts should fast result in a clear visible delay, since I normally do around 4 to 8 cuts when I use MKVCutter and I never encountered such a problem my guess it that the mkvmerge and ffmpeg both are clever enough not let this happen.
If you have the time you could try MKVCutter and remove 100 segments and see what happens,.... (iirc. both ffmpeg and mkvmerge keep a/v synch during cut and merges)

[shekh]

Interesting. I`ll give it a try

[hello_hello]

I'm absolutely certain MKVToolNix retains the existing audio/video sync when appending MKVs. I think some programs (VirtualDub?) just append the video and audio without allowing for gaps, but MKVToolNix seems more clever. I'm not sure about ffmpeg.

Try this:
Open an MKV with MKVTooNix and split it. I'll call those parts A and B.
Open B and set a negative audio delay or 1000ms and remux. I'll call that C.
Open C and set a positive audio delay of 1000ms and remux. That'll be D and it should now have a second of silence at the beginning due to the "missing" audio, but from there it'll be in sync.
Append A and D. You should have the original MKV back with synced audio throughout, except for the 1000ms of silence where it was spit.

[shekh]

@hello_hello
The example reveals nothing because it combines back original parts without changing anything, only throwing away approximately 1000ms of audio.
What is still interesting is whether the silence is exactly 1000ms. If it is, then there is something smart going on, otherwise this is trivial.

[hello_hello]

As I understand the delay must propagate to entire "after-cut-out" segment. To concatenate a-b-c where b has added length it must push c further in timeline.

@hello_hello
The example reveals nothing because it combines back original parts without changing anything, only throwing away approximately 1000ms of audio.
What is still interesting is whether the silence is exactly 1000ms. If it is, then there is something smart going on, otherwise this is trivial.

Admittedly the odds of it having removed exactly 1000ms of audio are pretty slim, so you'd hope the audio delay wouldn't be exactly 1000ms after removing it as I suggested. There's no such thing as negative delays for MKV, so MKVToolNix will always remove more audio than required when applying a negative delay, rather than less, then apply a positive audio delay to keep the sync exact. A similar thing happens when splitting. The second segment will usually have a small positive audio delay, and I assume the audio in the first segment will be longer than the video by the same amount. If it wasn't, you couldn't join the segments together again without some sort of gap/glitch.

I tried an experiment which seemed to confirm MKVToolNix will keep the audio sync when appending. After splitting an MKV (segments A and B), the second segment (B) had an audio delay of 8ms. I applied a -8ms audio delay to it and remuxed which gave me a new second segment with no delay (C). Finally, I appended (A) and (C).

Assuming there'd need to be a small adjustment to the video to compensate for the audio overlap I created (or hoped I'd created), I extracted the video timecodes and checked them where I'd originally split (the first keyframe after the 1 minute point). I used a 25fps video to make it easier to see what happened as the frames should always be 40ms apart.

60880
60920
60960
61000
61040
61080
61120
61160
61200
61240
61280
61320
61368
61408
61448
61488
61528
61568
61608
61648
61688
61728
61768
61808
61848


I could have created an audio gap instead, in which case I'd probably need to check the audio timecodes, and if I knew the exact size of each audio frame maybe I could have been a bit more scientific, but the evidence so far suggests you can trust MKVToolNix to do the right thing.

[Budman1]

Update:
@Selur... Making some headway using the method for MKV but tailoring it to other formats. Working on deciding which frames go into which portion before cutout, cut out or after cutout and which parts get recoded. Thanks for the explanation.

Thanks to everyone for the other suggestions which I am incorporating as need also.

[Selur]

Happy that helped a bit.
Should basically work that way for every format that has a gop structure.
The tricky part when doing 'smart-rendering' normally is to figure out which encoding settings to use to stay compatible with the existing file.

[Selur]

btw. if you figure out a way to reliably determine compatible encoding settings for H.265 files, would be nice if you could share them

[Budman1]

H.265 is something I definitely want to build into my program. Right now I have used ffmpeg to get the Key frame locations:

ffmpeg.exe -i "C:\Users\xxx\Desktop\video.mp4" -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 2 -f null NUL -loglevel debug 2>&1| for /f "tokens=4,8,9 delims=. " %d in ('findstr "pict_type:I"') do echo %d %e.%f>>"keyframe_list.txt"

Yields something like this:

n:0 t:0.080000
n:116 t:4.720000
n:292 t:11.760000
n:329 t:13.240000
n:368 t:14.800000
n:618 t:24.800000
n:868 t:34.800000
n:1031 t:41.320000
n:1100 t:44.080000

MediaInfo CLI can be interrogated for the setting of H.264. So far MP4 has successfully been encoded just the portion after cut and preceding the key frame. Then all parts were concatenated and played fine in PotPlayer and tested ok with AtomicParsley. I'm sure problems will arise with other profiles like MP4 High@L3 vs High@L4.1.

Anyone know any good reference links for H.265 settings in FFMpeg?

Thanks
Budman1

[Selur]

MediaInfo CLI can be interrogated for the setting of H.264

That is only usable for x264 encoded content. For random content mediainfo is lacking sometimes crucial data (since reading the headers isn't enough).

Anyone know any good reference links for H.265 settings in FFMpeg?

Only useful ffmpeg documenation in this regard is the source code to my knowledge.
In general it's using '-x265-params' with the normal parameters from x265 (without the leading '--') separated by colons.