Are timecode files needed when extracting audio to use in a different video

[beany]

Hi everyone, I'm new to this forum.

I have a question about timecode files that I was hoping someone could help me with. I tried searching google, but I cannot find an answer to my question.

I have many videos for the sole purpose of extracting the English, french, and spanish audio and subtitles. These are mostly if not all MKV files. I will be erasing the video after extracting the audio and subtitles, and I will be using the audio and subtitles on other videos that are currently only in English and/or Japanese. Some of these videos may have the same length and framerate as the videos that the audio and subs are being extracted from, and others may not.

For now all I want to do is to extract the audio and subtitles, and I want to make sure I get everything from the files that I will need that will allow me in the future to be able to successfully use the audio and subs with another video. So right now I only need to extract the audio and video so I can erase those hundreds of gigs of videos from my hard drive. I am using MKVToolNix to do this with the gMKVExtractGUI. I am also writing down the length and framerate of the original videos that I will be erasing.

My issue is that I have discovered that there are timecode files in mkv files that can be extracted along with an audio file (and I think even a subtitle file) and I am unsure if these timecode files will have any use if I am going to be erasing the video.

Something that is probably confusing me is that I took a course in college called Computer assisted sound design (which wasn't very good) and one day the teacher showed us a device that syncronized timestamps between a device and the computer during recording to ensure everything was in sync, but there were no timecode files that I was aware of. I'm guessing that may be a completely different thing from what these timecode files in mkv files are used for.

So my questions are:

Are these timecode files that are connected to the audio files (and I think even subtitle files) actually of any use if I am going to just erase the video and use the audio and subtitles with another video that may or may not be the same length and framerate?

Are there any types of video containers other than mkv that also have timcode files?

Also, it would be nice if someone could also tell me what these timecode files are generally used for so that I can understand why it would or wouldn't be useful in my situation.

Edit: changed reference to MKVExtractGUI to the correct program gMKVExtractGUI

[sneaker]

Audio codecs work with frames. E.g. an 48kHz AC3 frame is 32ms long. Within the Matroska container the timing for every one of these frames is stored. Timecode files are basically a way of storing those timings in a separate text file. In 99.9% of cases you do not need to create and keep timecode files for audio tracks as every audio frame will simply directly follow its predecessor (i.e. the frame rate is constant). You would only need them if the frame rate were not constant, i.e. if 2 (or more) audio frames overlap or have a gap between them. (Sometimes the timing of the first frame is not 0 or not identical to the timing of the first video frame - we simply call it "delay".)

You never need separate timecode files for subtitles because subtitle formats always include timings (they never have a constant framerate, e.g. like 1 line every 1 second but they are tied to the timings of the dialog).



Now the question is: do your files have gaps/overlaps? If you rip them directly from DVD/Blu-Ray: no. If they are from some kind of webcam/phone cam/screen recording: maybe.

[El Heggunte]

sneaker's answer is very-goode

Now the question is: do your files have gaps/overlaps? If you rip them directly from DVD/Blu-Ray: no. If they are from some kind of webcam/phone cam/screen recording: maybe.

AFAIK the only kind of audio which frequently contains gaps and overlaps are the WMA streams generated by some screen-capture applications. And in such cases, instead of dealing with timecode files, the best thing to do is recompress the WMA stream to a "normal" and sane format, because thus far only the ASF container is effectively suitable for Windows Media Audio.

[beany]

Sorry for taking so long to reply, I work in the evening up to 3am and I'm taking a break now. I'll be able to respond again in a few more hours if anyone replies again.

sneaker, I want to thank you very much for you response. You have really explained very clearly why I would not need to keep the timecode file 99.99% of the time. And since all the videos I have are either directly from DVD's or Blu-rays or just encodes of them, I assume that 100% of the audio I am working with will not require me to save any timecode files. And no, none of my videos are from phones or webcams or cam recordings of movies, but it's good to know to watch out for that in the future.

But if I did have a video where the audio files frame rate was variable rather than constant, how would I check if that were the case? and would the timecode file even be of any use in that case since I am erasing the video? I was thinking before starting this thread that most probably the timecode file for an audio file would be useless without the original video it was syncronized with and without the timecode file for the video, but I see now that if information about an audio file with variable framerate is kept in the timecode file, then maybe it's useful. But I wouldn't know yet how to use that timecode file to help synchronize the audio with a different video.

I forgot to mention in my initial post that I am also taking note of the Delay for each audio file which gMKVExtractGUI gives 2 values for the delay, which it calls "Delay" and "Effective Delay". There is a way to set those values to be used in naming the file when it is saved which makes things easier. I'm not sure what the difference is between the 2 values when they happen to be different, but I'll figure that out later.

Also thanks for the info on the subtitles. That is exactly what I was thinking since I've opened subtitle files with text editors to edit them and have seen how each line is marked with a time for when to display it in the video. But gMKVExtractGui gives me timecode files for subtitles which made me wonder if there is something I'm not aware of with the timing of subtitles that requires a timecode file. It's really nice to have confirmation on that from someone who knows what they are talking about.

El Heggunte, thank you for the information on the wma streams generated by some screen capture programs. It's good to know what to do in that situation. But again I am wondering, how would I discover whether or not the audio in the video is a variable frame rate with gaps and overlaps.

[El Heggunte]

..........
But again I am wondering, how would I discover whether or not the audio in the video is a variable frame rate with gaps and overlaps.

By decoding the compressed audio to an uncompressed .WAV file and checking whether their respective durations differ too much. "Normally" a difference of ± 0.5 seconds can be considered already too much.

NOTICE, this was entirely true some years ago, when the only ways to properly decode Windows Media Audio were BassAudioSource() (an Avisynth plugin) and Winamp's Disk File Writer. But today I really don't know whether tools like ffmpeg and MEncoder have been updated to deal correctly with WMA... Thanks to my suggestions, Lord_Mulder (a Doom9 moderator) fixed his tool wma2wav, but I haven't bothered to check whether the maintainers of ffmpeg and MEncoder copied his recepy

[beany]

Thank you El Heggunte, it's good to know that converting to a wave file will allow me to see the gaps and overlaps, but I don't quite understand how I can see the frames of the audio to check if there are any frames that are shorter, or what program will show me that. Could you recommend a program and explain what sort of "view" in the program will show me audio frames separation.

Also it seems a bit time consuming checking this way when I am just saving the audio for later use, even though that is a good way to do it when I am synchronizing the audio with another video. Is there not a way to see if it is a varialble frame rate (even if I cannot see if there are gaps and overlaps) by looking at the timecode file, or by using ffprobe or another program to analyse the audio either while it is still combined with the video, or after it is extracted.

There is one last thing I hope someone could answer for me that wasn't directly answered. Even though a timecode file may have important information in it for an audio file that is a variable framerate, can that timecode file still be used or useful if I erase the video and use the audio with another video? Or do I need to use the original video along with the timecode file for the original video to make use of the timecode file for the audio? In other words, are the 2 timecode files for the audio and video connnected in a way that they are useless without each other even though there may be useful information in the audio timecode file?

[sneaker]

There is one last thing I hope someone could answer for me that wasn't directly answered. Even though a timecode file may have important information in it for an audio file that is a variable framerate, can that timecode file still be used or useful if I erase the video and use the audio with another video? Or do I need to use the original video along with the timecode file for the original video to make use of the timecode file for the audio? In other words, are the 2 timecode files for the audio and video connnected in a way that they are useless without each other even though there may be useful information in the audio timecode file?

Hard to answer. I think in general you start with a container file like Matroska. Within that container video, audio and subtitles are in sync. There is NEVER a guarantee you can just replace/take out certain tracks and have them be in sync with other files. I would even say the chance you can take an audio track with gaps/overlaps and simply mux it with another video and have them be in sync is very low. Because gaps/overlaps indicate some kind of problem, usually.
In the end we can usually only do 1 thing: mux the tracks. Then open them in a player and use our eyes and ears to see/hear if they fit.

[sneaker]

I forgot to mention in my initial post that I am also taking note of the Delay for each audio file which gMKVExtractGUI gives 2 values for the delay, which it calls "Delay" and "Effective Delay". There is a way to set those values to be used in naming the file when it is saved which makes things easier. I'm not sure what the difference is between the 2 values when they happen to be different, but I'll figure that out later.

My guess is "effective delay" is the difference of the start timecodes of video and audio. I.e. if video starts at 100ms on the virtual timeline, audio starts at 350ms on the virtual timeline, then "effective delay" is 250ms, "delay" is 350ms.

[beany]

Sneaker, thanks for the effective delay explanation. It makes sense even if it is just a guess, and it helps a lot to understand what it could be. I'd be willing to bet you are correct.

About your post before that, I never expected any audio to be in sync with a new video so easily, but if there are gaps and overlaps in the audio from a variable framerate, even though I would expect to have to either stretch or cut and move over the audio a bit around those gaps and overlaps as I listen through the entire audio in an editor that shows me the new video playing at the same time; I was wondering if the timecode file would be able to be used in any way to help with the gaps and overlaps in the audio before I go through the audio to sync it with the new video.

In other words, I imagine if the timecode file can be used, it would have to be by muxing the audio with the new video using a matroska container, and at the same time I think there is a way to use MKVToolnix to process the timecode file while muxing the audio to the new video. Then if that were to do anything, I'm wondering if maybe the audio would have less problems when later editing it to sync it with the video. But I was thinking that the original video as welll as the timecode file for the original video may both be needed to make use of the timecode file for the audio, and that a timecode file is probably only used in conjunction with another timecode file that it is already in sync with, such as the 2 timecode files for the audio and video that are already in sync with each other.

So since I am erasing the original video and using the audio with a different video that is better quality and from a completely different version DVD or Blu-ray, is there any reason for me to keep the timecode file for just the audio, even when the audio is a variable frame rate? and if yes, how would I use the timecode file, and what can I expect it to help with? or am I correct, and a single timecode file for just the audio is useless?