Hi everyone,

Last month, I came across stinkybread's AVSync repository and tried using it to sync a few video files. The concept was great, but the results weren't ideal when working with low-quality source videos containing the dub I wanted to retime onto a high-quality reference video.

The original AVSync approach extracts scene-change keyframes, matches them visually between videos, sets timeline anchors, and time-stretches the audio between those points. However, when working with heavily compressed or low-resolution sources, visual keyframe matching produced very few reliable matches, leading to sync issues.

Realizing that both my low-quality source file and the high-quality reference file contained matching original-language audio tracks (e.g., Japanese or English), I realized we could compare those original audio streams directly instead.

Working in VS Code with GitHub Copilot, I developed and refined this audio-to-audio strategy:

Cross-correlate matching original-language audio streams to build an editorial "recipe" (detecting FPS differences, cuts, missing eyecatches, and reference-only sections).
Apply that exact edit recipe to the foreign dub track and retimed subtitles.
Mux the corrected dub and subtitles back into the high-quality reference video container.
After extensive real-world testing (including a successful 53-episode batch run), the audio-to-audio workflow has proven very reliable for production use.

You can check out the fork here:
https://github.com/Lowitle/avsync

What's next:
I plan to apply what I've learned from the audio-to-audio pipeline back to the original video-to-video concept, making visual matching more robust for cases where matching audio tracks aren't available.

I would love to get your thoughts, test results, or feedback!

Thanks for reading,
Lowitle