m314ss – Find, verify and auto-sync subtitles using the video's audio

[mohaanymo]

Hi guys,

I've been working on a small open-source tool called m314ss.

The main idea is simple:

Give it a video + an SRT, and it tries to answer: does this subtitle actually match and sync with this video?

It doesn't rely on filenames, release names, hashes, or metadata to decide that. It analyzes the video's audio and compares speech onsets with the subtitle cue timings.

Behind that is an alignment algorithm that tests thousands of possible offsets and looks for a statistically strong match. If the subtitle really belongs to the video, it can also figure out the timing offset and automatically sync it.

So you can simply do:

m314ss -srt subtitle.srt -video movie.mkv

and it will either sync it or reject it if it doesn't fit the video.

I also added SubDL and OpenSubtitles support, so it can search candidates automatically, test them using the same algorithm, and only keep one that actually matches.

It's written in Go and only needs FFmpeg.

GitHub:
https://github.com/mohaanymo/m314ss

It's still new, so I'd really appreciate people testing it on different movies, TV shows, languages, strange releases, bad encodes, etc.

I'm especially interested in finding cases where the matcher gets it wrong.

[lomero]

great news, might be useful sometimes. but load also vtt? i have some site that shared video with subs as vtt file

[Quint]

Interesting, thanks! Can it also align single subtitles, or only move ALL to a global offset?

[weiLahw9]

Disclose your use of AI already. That's the bare minimum.

[lomero]

honestly, thinking about it, i've never found no sync from audio and subs. Rather have found a wrong sync from video and audio with audio delay

[mohaanymo]

great news, might be useful sometimes. but load also vtt? i have some site that shared video with subs as vtt file

yeah I am willing to make it supports multiple subtitle formats

[mohaanymo]

Interesting, thanks! Can it also align single subtitles, or only move ALL to a global offset?

that's an interesting question. The short answer is: for now it uses a single global offset.

but the reason is that the tool was originally designed around a slightly different question: "is this actually the correct subtitle file for this video?"
so right now it looks at the subtitle cues as a whole and tries to find a consistent relationship between them and the speech in the audio. If it can find a strong match with one offset, it can be confident that this is the right subtitle and apply that offset to the whole file. if different parts need completely different corrections, that also makes it harder to confidently say that this is actually the correct subtitle file.

I think per-cue syncing could be added later, but I'd probably only want to do that after we're already confident that this is the correct subtitle for the video. Then it becomes a sync problem rather than a "does this subtitle even belong to this video?" problem.

[mohaanymo]

Disclose your use of AI already. That's the bare minimum.

you already asked me about AI in the m314dl thread, and I answered you. there is nothing I'm trying to "disclose" or hide.

and when you said "That's the bare minimum", actually I don't understand, the bare minimum for WHO?!
you're free to dislike AI-assisted development, just as anyone is free not to use the tool. but I don't accept the premise that using AI requires me to put a warning label on every project I release. (should I also add a label that I used nvim to create the project!!!)

I understand the code I publish, I maintain it, and I take responsibility for it.

and about m314ss, I have created my self from scratch to solve a problem I've been facing (same as m314dl), the idea and an algorithm I designed myself. I worked out the core approach for determining whether subtitles actually align with a video's audio and how the synchronization should be calculated. AI helped me implement, iterate, clean up code, and get the tool finished faster. that's a development tool, not a substitute for understanding what I'm building.

If you think the algorithm is flawed, the implementation is wrong, or you've found a case where m314ss fails, I'm genuinely interested. That's useful technical criticism and exactly what I posted the project here for.

so, repeatedly making the discussion about the fact that I use AI, after I've already answered you about it, doesn't tell us anything about whether the software is actually good or bad.

[mohaanymo]

honestly, thinking about it, i've never found no sync from audio and subs. Rather have found a wrong sync from video and audio with audio delay

yeah, that's a different case. m314ss currently uses the audio as the reference, so it assumes the audio itself is correctly synced with the video.

if the audio track has a delay relative to the video, m314ss can detect that the speech and subtitles don't line up, but from those two alone it can't really know which one is wrong the subtitles or the audio.

so you're right, audio/video desync would be a different problem. Interesting case though.

[davexnet]

What about the use case where the subtitle is otherwise OK but drifts further out as it goes along?
For example, the SRT was timed to match a 25 fps video and the video we have is 24 fps?

[Quint]

Interesting, thanks! Can it also align single subtitles, or only move ALL to a global offset?

I think per-cue syncing could be added later, but I'd probably only want to do that after we're already confident that this is the correct subtitle for the video. Then it becomes a sync problem rather than a "does this subtitle even belong to this video?" problem.

Could be very useful. We very often deal with subtitles (older movies) that have a hundred of small delays because meant for slightly different sources. The same problem still exists for the soundtracks, you have to fix by hand in these cases. There is only one thing I know that tries to deal with this (Vapoursynth, temporal align), but this doesn't work in most cases.
To have this temporal align for subtitles would be a nice thing.

What about the use case where the subtitle is otherwise OK but drifts further out as it goes along?
For example, the SRT was timed to match a 25 fps video and the video we have is 24 fps?

This you can fix easily with SubtitleEdit or other Editors.