What's wrong with this video frame?

[Compositor]

I'm reencoding some videos and one of them is giving me problems. It's consistent and I've narrowed down the location, so I've been able to cut out that segment of the video from the source and the result to show you.

"source" is from the original video.
"result" is from the reencode.
"x265 Hybrid Custom 3" is the settings I used in Hybrid to encode the result.

Duration and frame numbers are referenced from Aegisub.

At frame 53 (2.213 seconds) , something happens to the encoding. The timecodes in the result become off, there's 2 milisecond difference compared to the source. Also, the audio goes dead. I've tried encoding the video before and after this frame seperately and then patching them back together, but that 2ms deviation is still there. Now, 2ms may not seem like much and I can mux in the audio from the source again, but I would prefer a single glitch-free video. And, for all I know, this may be a sign of a bigger problem in the source video.

So, can anyone help me?

[poisondeathray]

Probably in the way you cut it or processed it. The audio is 2s 148ms (short)

If you remux new video, old audio, it's fine

The xml is probably not enough info, you'd need a log file to see what really happened . Or explain in more detail your steps

[Compositor]

When I first encoded the video, I did not cut it. And I noticed that, at that frame, the timecode was short two miliseconds. That defecit remains even if I cut the video at that frame without audio.

[poisondeathray]

The reason is you're using a different timebase when you're re-encoding, so there is a slight jitter in the timecodes. If you want it exactly the same, you can copy the timecodes from the source. What is important is the number of frames (if they match or not)

[Compositor]

If the timecodes deviated from the start of the video, I can see that. But would this jitter only happen at a certain frame?

I've reencoded several videos using this setup already, and this is the first time this has happened.

[poisondeathray]

If the timecodes deviated from the start of the video, I can see that. But would this jitter only happen at a certain frame?

I've reencoded several videos using this setup already, and this is the first time this has happened.

It's not just one frame. Many frames have ms difference. Look at the timecodes. The first one is frame two. Maybe the one you mentioned is the first frame that has gets rounded to two . There is at least one that has a delta of three

The timebase and timescale used are different (24000 vs 48000), that's the reason for the jitter between source and encode. A few ms fwd / back and it evens out. It doesn't really matter for playback, you can't hear +/- 2ms difference. Hell, compressed audio has way higher delay. If you want if perfect, remux the timecodes.

[hech54]

I've had the same scenario while trying to replace audio on an mpeg2 video (music video). No matter what I did, the visible glitch at one point in the video caused the rest to be out of sync. The original audio played perfectly. I wasn't re-encoding the video, just replacing audio.
Sometimes weird stuff just happens.

[Compositor]

I've done some more digging, and I now believe the reencoded video was actually correcting the glitch on the source video. That frame is 44ms after the previous frame, while all other frames before and after it are 42ms, just like other videos I've worked on before. I have another video that is identical at this point of the show, and it indicates the timecodes should be 2ms less than on the source.

[poisondeathray]

The difference here is there are no glitches when you encode his sample. Source is ok, audio is ok - so he must have messed up the encode somehow and truncated the audio. There is up to 3ms jitter on frames because of the timebase and timescale differences. But if you remux timecodes, it's identical

eg

ffmpeg -i source.mkv -pix_fmt yuv420p10le -c:v libx265 -crf 20 -c:a copy ffmpeg.mkv

ffmpeg.mkv has "jitter" , so maybe up to 3ms difference on some frames

ffmpeg_timecodes.mkv uses source timecodes, so it's identical 0ms differences in terms of timing

src_timecodes_v2.txt is his source.mkv timecodes
res_timecodes_v2.txt is his result.mkv timecodes

[poisondeathray]

I've done some more digging, and I now believe the reencoded video was actually correcting the glitch on the source video. That frame is 44ms after the previous frame, while all other frames before and after it are 42ms, just like other videos I've worked on before. I have another video that is identical at this point of the show, and it indicates the timecodes should be 2ms less than on the source.

what glitch in the source ?

There is jitter in the timecodes of that source. (VFR time does not match CFR time) . But wasn't that what you wanted (to match the source) ?

[Selur]

Using the source and doing the following:

• extract audio&video:
  call:
  

  mkvextract tracks "C:\Users\Selur\Desktop\source.mkv" 1:"E:\Output\iId_2_aid_0_lang_ja_DELAY_65ms_08_38_37_2910_01.flac" 0:"E:\Output\source_08_38_37_2910_03.264"

  output:
  

  Extracting track 0 with the CodecID 'V_MPEG4/ISO/AVC' to the file 'E:\Output\source_08_38_37_2910_03.264'. Container format: AVC/h.264 elementary stream
  Extracting track 1 with the CodecID 'A_FLAC' to the file 'E:\Output\iId_2_aid_0_lang_ja_DELAY_65ms_08_38_37_2910_01.flac'. Container format: raw data
  Progress: 100%

• extract timecodes:
  

  mkvextract timecodes_v2 "C:\Users\Selur\Desktop\source.mkv" 0:"E:\Temp\timeCode_08_39_50_511.tc"

  output:
  

  Progress: 100%

• multiplexing audio, video and timecodes into a new file:
  call:
  

  mkvmerge --ui-language en -o "E:\Output\source.mkv" -d 0 --default-track 0:yes --default-duration 0:24000/1001fps --aspect-ratio-factor 0:1 --fourcc 0:MP4V --no-chapters --compression -1:none --forced-track 0:yes --no-audio --no-subtitles --timecodes 0:"E:\Temp\timeCode_08_39_50_511.tc" "E:\Output\source_08_38_37_2910_03.264" --track-name 0:"FLAC 2.0" --language 0:ja --default-track 0:yes --sync 0:65 --forced-track 0:no -a 0 --compression -1:none --no-video --no-subtitles --no-chapters "E:\Output\iId_2_aid_0_lang_ja_DELAY_65ms_08_38_37_2910_01.flac"

  output:
  

  mkvmerge v30.1.0 ('Forever And More') 64-bit
  'E:\Output\source_08_38_37_2910_03.264': Using the demultiplexer for the format 'AVC/h.264'.
  +-> Parsing the FLAC file. This can take a LONG time.
  +-> Pre-parsing FLAC file: 100%
  'E:\Output\iId_2_aid_0_lang_ja_DELAY_65ms_08_38_37_2910_01.flac': Using the demultiplexer for the format 'FLAC'.
  'E:\Output\source_08_38_37_2910_03.264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
  'E:\Output\iId_2_aid_0_lang_ja_DELAY_65ms_08_38_37_2910_01.flac' track 0: Using the output module for the format 'FLAC'.
  The file 'E:\Output\source.mkv' has been opened for writing.
  Progress: 100%
  The cue entries (the index) are being written...
  Multiplexing took 0 seconds.

everything seems fine, but the output is async.
-> No clue what's really going wrong here. :/

[Compositor]

what glitch in the source ?

I believe the video was appended at that frame. From my experience, such appending may lead to a delay of 1ms if it was just video and 20ms if it had audio as well at the joined frame.

[poisondeathray]

what glitch in the source ?

I believe the video was appended at that frame. From my experience, such appending may lead to a delay of 1ms if it was just video and 20ms if it had audio as well at the joined frame.

Not for video . Not if it was done correctly. Video frames are independent and separate from the timecode information (there is a "declared" framerate in the video stream header, but that is overidden by the timecodes. The latter takes precedence). All that really matters is the framecount

For some types of lossy compressed audio, possibly, if it wasn't done correctly. Lossy compressed audio can have a delay and if it's not accounted for, the audio can get shifted. It can be worse if you have multiple segments appended. But for uncompressed audio and flac - no