encoding to use concat demuxer

[s1nbad]

I am hoping someone can give some advice on a solution with requirements that come in two parts.

PART ONE
I have a large number of downloaded files predominantly H264 encoded in mp4 and mkv containers where I need to extract small master video clips.
The clips require exact timing (ms) but are only around 10secs long. I would like to maintain the original quality and do not require any existing Audio or Metadata.
I will not be able to split the file on a key frame so plan to use ffmpeg to re-encode without frozen frames.
I am not concerned about size of the files or time it takes to create the master clips although I wish to maintain the original quality.

PART TWO
I will use ffmpeg concat demuxer to quickly join the small 'Master' video clips from PART ONE into larger single videos.
This operation will be automated and will be required to be done constantly - time is of the essence
The ffmpeg website states - "All files must have the same streams (same codecs, same time base, etc.)."

any advice on the best "standardised" encoding to use for all the extractions ?

[smrpix]

Scrubbing the source clips to find the sections you want is what takes the time. Why not use a regular NLE like Premiere to mark your cutpoints, drop them in a timeline and then output? Skip the whole making new master clips with ffmpeg phase. Or is there something about your needs I'm not understanding correctly?

[Cornucopia]

Yep, sounds like you want to EDIT without using an EDITOR. Besides being convoluted, it isn't recommended because it is less efficient. And you were the one who said time was of the essence.


Scott

[s1nbad]

thanks for the help - I have all the sections I need in a text file I can therefore run a script and have ffmpeg run for days if required but.... it is the recombining the clips in any number and any order needs to be fast for my app and using comcat remux seems the best solution. I agree with the idea of an editor that is how I have all the sections required already but mangaging the thousands of source files has now become too onerous - the clips represent less than 1% of the data size.

Think this is the best solution but want to get that first run correct

[jagabo]

any advice on the best "standardised" encoding to use for all the extractions ?

Which codec you use depends on your needs. Lossless all-keyframe codecs can be truly lossless and easily concatenated but the files will be very large. Slightly smaller are near lossless codecs like ProRes. If you need much more compression you can use x264 with the --stitchable switch.

[s1nbad]

any advice on the best "standardised" encoding to use for all the extractions ?

Which codec you use depends on your needs. Lossless all-keyframe codecs can be truly lossless and easily concatenated but the files will be very large. Slightly smaller are near lossless codecs like ProRes. If you need much more compression you can use x264 with the --stitchable switch.

Thanks for your help - not sure what further info is needed beyond what i have described. The number one need is the ability to concatenate very rapidly. To do this the documentation states - All files must have the same streams (same codecs, same time base, etc.). Unfortunately it does not go on to explain the "etc.".

From a quick search I see a number of issues with concat demux of prores files because of indexing issues - i cannot validate this.

I am not worried about file size since the clips are only 10s each file. So, as you suggest, lossless all-keyframe codecs seems to be a good choice for me but could you recommend one ? Would I need to standardise the output size also ? Would the codec chosen also have the same 'time base' ?

any help would be useful

[jagabo]

Some lossless video codecs: UT Video Codec, Lagarith, Huffyuv, x264 (with qp=0) But beware, not all codecs work in all containers. Not all programs deal with all lossless codecs. And you didn't specify whether you needed support for RGB, YUV, chroma subsampling (4:4:4, 4:2:2, 4:1:1, 4:2:0), 8 bit, 10 bit, 12 bit, 16 bit, interlaced video, etc.