Concatenating 2 files with diffrent encoder setting (x264)

[ben45]

Hello
As you know matching x264's encoder settings isn't easy. Is there any way to concatenate mkv files with different encoder setting?
I have splitted 1.mkv to four parts and want to re-encode one part and append to other parts.

[poisondeathray]

matching settings doesn't ensure that it will join seamlessly. Even if you used the same encoder version, exact same settings , often there is a glitch at the seams

You can use --stitchable on the segments, but it usually has to be used on all the individual segments , which defeats the purpose in you scenario . The other way is to set a different --sps-id in the re-encoded section, then binary join the elementary streams. There are several threads detailing this procedure, but it's slightly complicated for most people

[ben45]

matching settings doesn't ensure that it will join seamlessly. Even if you used the same encoder version, exact same settings , often there is a glitch at the seams

You can use --stitchable on the segments, but it usually has to be used on all the individual segments , which defeats the purpose in you scenario . The other way is to set a different --sps-id in the re-encoded section, then binary join the elementary streams. There are several threads detailing this procedure, but it's slightly complicated for most people

Thanks for reply
Could you please help me more with --sps-id?

[poisondeathray]

I posted more detailed instructions before, either on this board or doom9 . Use search.

Basically you set a different value than the other segments. The default is "0" if you never specified it with the other segments , so if you set a value like 2 or 3, it will work

[ben45]

I posted more detailed instructions before, either on this board or doom9 . Use search.

Basically you set a different value than the other segments. The default is "0" if you never specified it with the other segments , so if you set a value like 2 or 3, it will work

That's it?

Please post doom9's links here

[ben45]

I did it! clip appended without any error or warning but there's problems on some TV players. Here's the steps
1. muxed .mkv file to mp4
2.demuxed and splited h264 stream from mp4 file with ffmpeg. Here's the script that i used

for %%a in ("*.mp4") do "C:\ffmpeg\bin\ffmpeg.exe"  -i "%%a" -segment_times 240,480,720,10800 -vcodec copy -bsf h264_mp4toannexb -f segment -an "test-%%05d.h264"
pause

3. encoded part 2 with new setting
4. Joined h264 streams with DOS copy command

copy /b 1.h264 + 2.h264 + 3.h264 + 4.h264 output.h264

5.Muxed all h264 streams with mkvmerge to .mkv

But there's a problem on playback! screen freezes for a while ( about 0.5 s ) on encoded part ( tested on Sumsung HDTV )
I have checked necessary settings like SAR and FPS and matched them but problem not solved

sorry for my bad english

[poisondeathray]

Possibly ffmpeg didn't segment correctly, or problems with some of your steps. Most common cause is not cutting properly on IDR frames. How did you mux mkv to mp4 ? That step isn't necessary

Post original video, or cut a segment in original container at least a few seconds, or few GOP's longer than 1,2,3 from the original . (forget about 4 for now)

[ben45]

Possibly ffmpeg didn't segment correctly, or problems with some of your steps. Most common cause is not cutting properly on IDR frames. How did you mux mkv to mp4 ? That step isn't necessary

Post original video, or cut a segment in original container at least a few seconds, or few GOP's longer than 1,2,3 from the original . (forget about 4 for now)

Thanks for reply dear poisondeathray

You're right. The 1st step isn't necessary. I have splited original .mkv to h264 streams but problem still not solved

I use this script to cut h264 stream

for %%a in ("*.mkv") do "C:\ffmpeg\bin\ffmpeg.exe"  -i "%%a" -segment_times 240,480,720,1080 -vcodec copy -bsf h264_mp4toannexb -f segment -an "test-%%05d.h264"
pause

Is there any problem with this script?

Original video's encoder setting

cabac=1 / ref=12 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=925 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=62500 / vbv_bufsize=62500 / nal_hrd=vbr / filler=0 / ip_ratio=1.40 / aq=1:1.00

Encoded part's setting

cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x3:0x3 / me=dia / subme=1 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc=2pass / mbtree=0 / bitrate=1200 / ratetol=2.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

[poisondeathray]

possibly something wrong, sometimes ffmpeg can be buggy with splitting especially if there are open GOP's. Possibly wrong encoding settings, many things can be wrong

[ben45]

possibly something wrong, sometimes ffmpeg can be buggy with splitting especially if there are open GOP's. Possibly wrong encoding settings, many things can be wrong

I use latest version of ffmpeg
https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20161010-1bda0ee-win64-static.zip

I have used -sps-id 2 to encode appended part

[poisondeathray]

mediainfo doesn't give you all the information, you need a stream analyser to see some other things

also, --sps-id allows you to join segments with different encoding settings, but you should still try to roughly match them

[ben45]

mediainfo doesn't give you all the information, you need a stream analyser to see some other things

also, --sps-id allows you to join segments with different encoding settings, but you should still try to roughly match them

Please check this analysis report

Part1.mkv (original)

==================== NAL ====================

 forbidden_zero_bit : 0 

 nal_ref_idc : 3 

 nal_unit_type : 7 ( Sequence parameter set ) 

======= SPS =======

 profile_idc : 100 

 constraint_set0_flag : 0 

 constraint_set1_flag : 0 

 constraint_set2_flag : 0 

 constraint_set3_flag : 0 

 constraint_set4_flag : 0 

 constraint_set5_flag : 0 

 reserved_zero_2bits : 0 

 level_idc : 41 

 seq_parameter_set_id : 0 

 chroma_format_idc : 1 

 residual_colour_transform_flag : 0 

 bit_depth_luma_minus8 : 0 

 bit_depth_chroma_minus8 : 0 

 qpprime_y_zero_transform_bypass_flag : 0 

 seq_scaling_matrix_present_flag : 0 

 log2_max_frame_num_minus4 : 1 

 pic_order_cnt_type : 0 

   log2_max_pic_order_cnt_lsb_minus4 : 3 

   delta_pic_order_always_zero_flag : 0 

   offset_for_non_ref_pic : 0 

   offset_for_top_to_bottom_field : 0 

   num_ref_frames_in_pic_order_cnt_cycle : 0 

 num_ref_frames : 12 

 gaps_in_frame_num_value_allowed_flag : 0 

 pic_width_in_mbs_minus1 : 79 

 pic_height_in_map_units_minus1 : 33 

 frame_mbs_only_flag : 1 

 mb_adaptive_frame_field_flag : 0 

 direct_8x8_inference_flag : 1 

 frame_cropping_flag : 1 

   frame_crop_left_offset : 0 

   frame_crop_right_offset : 0 

   frame_crop_top_offset : 0 

   frame_crop_bottom_offset : 5 

 vui_parameters_present_flag : 1 

=== VUI ===

 aspect_ratio_info_present_flag : 1 

   aspect_ratio_idc : 1 

     sar_width : 0 

     sar_height : 0 

 overscan_info_present_flag : 0 

   overscan_appropriate_flag : 0 

 video_signal_type_present_flag : 0 

   video_format : 0 

   video_full_range_flag : 0 

   colour_description_present_flag : 0 

     colour_primaries : 0 

   transfer_characteristics : 0 

   matrix_coefficients : 0 

 chroma_loc_info_present_flag : 0 

   chroma_sample_loc_type_top_field : 0 

   chroma_sample_loc_type_bottom_field : 0 

 timing_info_present_flag : 1 

   num_units_in_tick : 1001 

   time_scale : 48000 

   fixed_frame_rate_flag : 1 

 nal_hrd_parameters_present_flag : 1 

 vcl_hrd_parameters_present_flag : 0 

   low_delay_hrd_flag : 0 

 pic_struct_present_flag : 0 

 bitstream_restriction_flag : 1 

   motion_vectors_over_pic_boundaries_flag : 1 

   max_bytes_per_pic_denom : 0 

   max_bits_per_mb_denom : 0 

   log2_max_mv_length_horizontal : 11 

   log2_max_mv_length_vertical : 11 

   num_reorder_frames : 2 

   max_dec_frame_buffering : 12 

=== HRD ===

 cpb_cnt_minus1 : 0 

 bit_rate_scale : 0 

 cpb_size_scale : 1 

   bit_rate_value_minus1[0] : 976561 

   cpb_size_value_minus1[0] : 1953124 

   cbr_flag[0] : 0 

 initial_cpb_removal_delay_length_minus1 : 18 

 cpb_removal_delay_length_minus1 : 12 

 dpb_output_delay_length_minus1 : 8 

 time_offset_length : 0

Part2.mkv (encoded and appended to part1.mkv )

==================== NAL ====================

 forbidden_zero_bit : 0 

 nal_ref_idc : 3 

 nal_unit_type : 7 ( Sequence parameter set ) 

======= SPS =======

 profile_idc : 100 

 constraint_set0_flag : 0 

 constraint_set1_flag : 0 

 constraint_set2_flag : 0 

 constraint_set3_flag : 0 

 constraint_set4_flag : 0 

 constraint_set5_flag : 0 

 reserved_zero_2bits : 0 

 level_idc : 31 

 seq_parameter_set_id : 2 

 chroma_format_idc : 1 

 residual_colour_transform_flag : 0 

 bit_depth_luma_minus8 : 0 

 bit_depth_chroma_minus8 : 0 

 qpprime_y_zero_transform_bypass_flag : 0 

 seq_scaling_matrix_present_flag : 0 

 log2_max_frame_num_minus4 : 0 

 pic_order_cnt_type : 0 

   log2_max_pic_order_cnt_lsb_minus4 : 2 

   delta_pic_order_always_zero_flag : 0 

   offset_for_non_ref_pic : 0 

   offset_for_top_to_bottom_field : 0 

   num_ref_frames_in_pic_order_cnt_cycle : 0 

 num_ref_frames : 4 

 gaps_in_frame_num_value_allowed_flag : 0 

 pic_width_in_mbs_minus1 : 79 

 pic_height_in_map_units_minus1 : 33 

 frame_mbs_only_flag : 1 

 mb_adaptive_frame_field_flag : 0 

 direct_8x8_inference_flag : 1 

 frame_cropping_flag : 1 

   frame_crop_left_offset : 0 

   frame_crop_right_offset : 0 

   frame_crop_top_offset : 0 

   frame_crop_bottom_offset : 5 

 vui_parameters_present_flag : 1 

=== VUI ===

 aspect_ratio_info_present_flag : 1 

   aspect_ratio_idc : 1 

     sar_width : 0 

     sar_height : 0 

 overscan_info_present_flag : 0 

   overscan_appropriate_flag : 0 

 video_signal_type_present_flag : 1 

   video_format : 5 

   video_full_range_flag : 0 

   colour_description_present_flag : 1 

     colour_primaries : 2 

   transfer_characteristics : 2 

   matrix_coefficients : 1 

 chroma_loc_info_present_flag : 0 

   chroma_sample_loc_type_top_field : 0 

   chroma_sample_loc_type_bottom_field : 0 

 timing_info_present_flag : 1 

   num_units_in_tick : 1001 

   time_scale : 48000 

   fixed_frame_rate_flag : 1 

 nal_hrd_parameters_present_flag : 0 

 vcl_hrd_parameters_present_flag : 0 

   low_delay_hrd_flag : 0 

 pic_struct_present_flag : 0 

 bitstream_restriction_flag : 1 

   motion_vectors_over_pic_boundaries_flag : 1 

   max_bytes_per_pic_denom : 0 

   max_bits_per_mb_denom : 0 

   log2_max_mv_length_horizontal : 11 

   log2_max_mv_length_vertical : 11 

   num_reorder_frames : 2 

   max_dec_frame_buffering : 4 

=== HRD ===

 cpb_cnt_minus1 : 0 

 bit_rate_scale : 0 

 cpb_size_scale : 0 

   bit_rate_value_minus1[0] : 0 

   cpb_size_value_minus1[0] : 0 

   cbr_flag[0] : 0 

 initial_cpb_removal_delay_length_minus1 : 0 

 cpb_removal_delay_length_minus1 : 0 

 dpb_output_delay_length_minus1 : 0 

 time_offset_length : 0

[ben45]

Maybe there's a problem with split script. I can't open .h264 files on Avisynth ( directshowsource: unable to determine the duration of the video )

[poisondeathray]

There are multiple points of possible failure - maybe inproper cutting (IDR boundaries) , non matched settings on important values - especially level is important for HW players are sensitive - your 4.1 vs. 3.1 , and improper HRD VBV values can cause HW player to stutter. The latter is almost impossible to fix with common software - you need a smart renderer segment encoder that obeys HRD and VBV constraints - there are no cheap ones - only enterprise level BD software is capable, and they only do BD compliant streams, not "generic" encodes like yours

1) I would first make sure the cuts are proper on IDR boundaries, no open GOP's, then try matching settings (esp. level) more closely first

2) if it still doesn't work on the samsung without pause, maybe try stripping the HRD info on the other segment streams before joining with roozhou's ffmpeg patch tool (your bitrate is low, it' s unlikely you will encounter a true HW buffering issue)

If you still can't get it working post the video sample of original requested earlier

Maybe there's a problem with split script. I can't open .h264 files on Avisynth ( directshowsource: unable to determine the duration of the video )

It's an elementary stream - you usually can't open it with directshowsource without a container. You would need ES AVC source filter like DGSource (not free). DGAVCIndex (free) might work, but it's depreciated and has problems with some streams

[ben45]

There are multiple points of possible failure - maybe inproper cutting (IDR boundaries) , non matched settings on important values - especially level is important for HW players are sensitive - your 4.1 vs. 3.1 , and improper HRD VBV values can cause HW player to stutter. The latter is almost impossible to fix with common software - you need a smart renderer segment encoder that obeys HRD and VBV constraints - there are no cheap ones - only enterprise level BD software is capable, and they only do BD compliant streams, not "generic" encodes like yours

1) I would first make sure the cuts are proper on IDR boundaries, no open GOP's, then try matching settings (esp. level) more closely first

2) if it still doesn't work on the samsung without pause, maybe try stripping the HRD info on the other segment streams before joining with roozhou's ffmpeg patch tool (your bitrate is low, it' s unlikely you will encounter a true HW buffering issue)

If you still can't get it working post the video sample of original requested earlier

Maybe there's a problem with split script. I can't open .h264 files on Avisynth ( directshowsource: unable to determine the duration of the video )

It's an elementary stream - you usually can't open it with directshowsource without a container. You would need ES AVC source filter like DGSource (not free). DGAVCIndex (free) might work, but it's depreciated and has problems with some streams

I have changed 4.1 to 3.1 and set HRD to VBR but problem still not solved

[poisondeathray]

Does it play ok in software player (computer) without the pause ?

Try to match other settings too, like reference frames etc,,

Setting --nal-hrd vbr won't doing anything by itself. You also have to set --vbv-bufsize and --vbv-maxrate (the oriignal had both set to 62500), but current information of the patched in piece is not going to match the other segments, which would cause some HW decoders to pause. (you encoded the piece #2 as is, not from a whole video, so the removal information is going to different) . That's why stripping the info from the other segments with roozhou's tool might overcome this problem (modern TV players don't even need VBV constraints for low bitrate encodes like yours, since your average is <2Mbps)

[ben45]

Does it play ok in software player (computer) without the pause ?

Try to match other settings too, like reference frames etc,,

Setting --nal-hrd vbr won't doing anything by itself. You also have to set --vbv-bufsize and --vbv-maxrate (the oriignal had both set to 62500), but current information of the patched in piece is not going to match the other segments, which would cause some HW decoders to pause. (you encoded the piece #2 as is, not from a whole video, so the removal information is going to different) . That's why stripping the info from the other segments with roozhou's tool might overcome this problem (modern TV players don't even need VBV constraints for low bitrate encodes like yours, since your average is <2Mbps)

Yes. Plays whithout any problem on my laptop (potplayer , kmplayer , mpc-pc)

The whole story behind using sps-id is to avoid matching setting ( It's not impossible but takes time! a lot! )
Seems FFmpeg is not the problem. I splited clip with mkvmerge (without demuxing h264 stream) and re-encoded part2.mkv. Plays well on pc but still there's problem on tv playback

[poisondeathray]

Does it play ok in software player (computer) without the pause ?

Try to match other settings too, like reference frames etc,,

Setting --nal-hrd vbr won't doing anything by itself. You also have to set --vbv-bufsize and --vbv-maxrate (the oriignal had both set to 62500), but current information of the patched in piece is not going to match the other segments, which would cause some HW decoders to pause. (you encoded the piece #2 as is, not from a whole video, so the removal information is going to different) . That's why stripping the info from the other segments with roozhou's tool might overcome this problem (modern TV players don't even need VBV constraints for low bitrate encodes like yours, since your average is <2Mbps)

Yes. Plays whithout any problem on my laptop (potplayer , kmplayer , mpc-pc)

The whole story behind using sps-id is to avoid matching setting ( It's not impossible but takes time! a lot! )
Seems FFmpeg is not the problem. I splited clip with mkvmerge (without demuxing h264 stream) and re-encoded part2.mkv. Plays well on pc but still there's problem on tv playback

But as I said earlier, even matching exact settings, exact x264 binary version, sometimes still causes glitch at the seams. That's the only reason to use --sps-id - when all other methods fail. If all you wanted to do is match settings and you have mediainfo, you can use selur's mis2x264 which will generate the commandline from the source mediainfo automatically

[ben45]

Does it play ok in software player (computer) without the pause ?

Try to match other settings too, like reference frames etc,,

Setting --nal-hrd vbr won't doing anything by itself. You also have to set --vbv-bufsize and --vbv-maxrate (the oriignal had both set to 62500), but current information of the patched in piece is not going to match the other segments, which would cause some HW decoders to pause. (you encoded the piece #2 as is, not from a whole video, so the removal information is going to different) . That's why stripping the info from the other segments with roozhou's tool might overcome this problem (modern TV players don't even need VBV constraints for low bitrate encodes like yours, since your average is <2Mbps)

Yes. Plays whithout any problem on my laptop (potplayer , kmplayer , mpc-pc)

The whole story behind using sps-id is to avoid matching setting ( It's not impossible but takes time! a lot! )
Seems FFmpeg is not the problem. I splited clip with mkvmerge (without demuxing h264 stream) and re-encoded part2.mkv. Plays well on pc but still there's problem on tv playback

But as I said earlier, even matching exact settings, exact x264 binary version, sometimes still causes glitch at the seams. That's the only reason to use --sps-id - when all other methods fail. If all you wanted to do is match settings and you have mediainfo, you can use selur's mis2x264 which will generate the commandline from the source mediainfo automatically

I have about 1200 clips. Extracting and matching encoder setting will take forever! That's why i'm trying to use --sps-id method
Any other solution dear poisondeathray?

[poisondeathray]

You should be able to batch process everything, but you can't even get 1 to work properly (on the Samsung) yet, so you have to figure that out first

But one potential problem you're going to have batching this is ffmpeg splits are not perfect and the IDR placement in the source videos

If you've done the other steps correctly, I would try stripping the HRD info next . You would strip the original before splitting

[ben45]

You should be able to batch process everything, but you can't even get 1 to work properly (on the Samsung) yet, so you have to figure that out first

But one potential problem you're going to have batching this is ffmpeg splits are not perfect and the IDR placement in the source videos

If you've done the other steps correctly, I would try stripping the HRD info next . You would strip the original before splitting

Thanks! I really appreciate your help

[ben45]

You should be able to batch process everything, but you can't even get 1 to work properly (on the Samsung) yet, so you have to figure that out first

But one potential problem you're going to have batching this is ffmpeg splits are not perfect and the IDR placement in the source videos

If you've done the other steps correctly, I would try stripping the HRD info next . You would strip the original before splitting

Good news! i have tested another clip with no HRD=VBR header and the result was OK! matching encoder settings works! Now removing HRD info is the only problem
I have used this tool to remove SEI and HRD info from .mkv file
http://forum.doom9.org/showthread.php?t=152419

Plays well on pc but Samsung TV can't recognize coded "Video codec not supported"
Is there any "easy" way to replace SEI info? seem removing this data isn't safe

[poisondeathray]

Did you use roozhou's tool on the elementary stream ? It works better on ES. (ie. demux the original video, then patch it)

[poisondeathray]

I have changed 4.1 to 3.1 and set HRD to VBR but problem still not solved

1) BTW, it's the opposite; the original was L4.1 , the re-encoded was L3.1 . I would change your re-encoded to L4.1

You can mostly use different settings, but there are a few that are important to match for HW decoders , such as --sar , --level

2) Check mediainfo view=>text of the a) original, and b) the small re-encoded piece, and c) the joined piece . Make sure the fourcc's match (sometimes HW player doens't like a certain fourcc like "h264 , H264, X264, x264 , AVC1, avc1, etc..." , sometimes they are even case sensitive . Otherwise you can force the fourcc to match

3) You can also try similar HRD VBV settings, but if that was indeed the reason for the Samsung failure, it probably won't be close enough to re-encode a small section. ie. you would need to re-encode whole video using the similar settings , split the same section, then concat it back in . It's preferrable to do it the "easier" way if you can get it to work

[ben45]

Did you use roozhou's tool on the elementary stream ? It works better on ES. (ie. demux the original video, then patch it)

No. I think splitting with mmg is safer so....I found another way! TSmuxeR removes all SEI data without any problem. I matched all encoder settings and now file plays without any problem.
Now i'm looking for a script to automate mkvmerge's append
https://forum.videohelp.com/threads/380814-Mkvmerge-batch-append

[pandy]

Not sure if this help but try different container - TS allow to do such things - how it will be visible is more related to decoder.