Progressive out of sync audio

[SushiSky]

I have a video that has audio that gets progressively out of sync. Upon reading a lot of other threads, I tried fixing frame rates and changing tempos in Audacity, but it doesn't work. I had paired an english audio stream taken from a low quality anime release to pair it with a high quality raw release. Any help would be appreciated

[SushiSky]

Bump

[manono]

What are the framerates of your two sources? What are the lengths of your two sources? Are you sure that they both have the same number of frames - that one doesn't have any extra material such as studio logos, different credits, additional scenes, etc.?

[SushiSky]

I believe one of them has a frame rate of 29 fps and the other with the hd video stream has a 23.976 fps. But even after changing the frame rates, the audio didn't work. The duration of the video and the audio differs by about 7-8 seconds. I checked them both, but it seems to be identical. I tried cutting the end off, hoping to cause a linear sync problem, which would be easier to fix

[Selur]

audio might be streched in one of the sources,.. video might be vfr,...
-> post
a.

mediainfo --full "path to file"

output of the two sources
b.

mkvinfo -v "Path to file"

output of the two sources
this way it's possible to see whether the audio is stretched and the video if vfr or not,...

[SushiSky]

First Source
Video
Count :
248
Count of stream of this kind :
1
Kind of stream :
Video
Kind of stream :
Video
Stream identifier :
0
StreamOrder :
0
ID :
1
ID :
1
Unique ID :
3986910091
Format :
AVC
Format/Info :
Advanced Video Codec
Format/Url :
http://developers.videolan.org/x264.html
Commercial name :
AVC
Format profile :
High@L3.1
Format settings :
CABAC / 5 Ref Frames
Format settings, CABAC :
Yes
Format settings, CABAC :
Yes
Format settings, ReFrames :
5
Format settings, ReFrames :
5 frames
Internet media type :
video/H264
Codec ID :
V_MPEG4/ISO/AVC
Codec ID/Url :
http://ffdshow-tryout.sourceforge.net/
Codec :
V_MPEG4/ISO/AVC
Codec :
AVC
Codec/Family :
AVC
Codec/Info :
Advanced Video Codec
Codec/Url :
http://ffdshow-tryout.sourceforge.net/
Codec profile :
High@L3.1
Codec settings :
CABAC / 5 Ref Frames
Codec settings, CABAC :
Yes
Codec_Settings_RefFrames :
5
Duration :
1438689
Duration :
23mn 58s
Duration :
23mn 58s 689ms
Duration :
23mn 58s
Duration :
00:23:58.689
Width :
852
Width :
852 pixels
Height :
480
Height :
480 pixels
Pixel aspect ratio :
1.000
Display aspect ratio :
1.775
Display aspect ratio :
16:9
Frame rate mode :
VFR
Frame rate mode :
Variable
Frame rate :
23.976
Frame rate :
23.976 fps
Original frame rate :
29.970
Original frame rate :
29.970 fps
Frame count :
34494
Resolution :
8
Resolution :
8 bits
Colorimetry :
4:2:0
Color space :
YUV
Chroma subsampling :
4:2:0
Bit depth :
8
Bit depth :
8 bits
Scan type :
Progressive
Scan type :
Progressive
Interlacement :
PPF
Interlacement :
Progressive
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Writing library :
x264 - core 104 r1683 62997d6
Writing library :
x264 core 104 r1683 62997d6
Writing library/Name :
x264
Writing library/Version :
core 104 r1683 62997d6
Encoding settings :
cabac=1 / ref=5 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=0.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=0 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc=crf / mbtree=0 / crf=24.0 / qcomp=1.00 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
Default :
Yes
Default :
Yes
Forced :
No
Forced :
No
Audio #1
Count :
217
Count of stream of this kind :
2
Kind of stream :
Audio
Kind of stream :
Audio
Stream identifier :
0
Stream identifier :
1
StreamOrder :
1
ID :
2
ID :
2
Unique ID :
1699543183
Format :
AAC
Format/Info :
Advanced Audio Codec
Commercial name :
AAC
Format profile :
HE-AACv2 / HE-AAC / LC
Format settings, SBR :
Yes (Implicit)
Format settings, SBR :
Yes (Implicit)
Format settings, PS :
Yes (Implicit)
Format settings, PS :
Yes (Implicit)
Codec ID :
A_AAC
Codec :
AAC LC-SBR-PS
Codec :
AAC LC-SBR-PS
Codec/Family :
AAC
Duration :
1438678
Duration :
23mn 58s
Duration :
23mn 58s 678ms
Duration :
23mn 58s
Duration :
00:23:58.678
Channel(s) :
2 / 1 / 1
Channel(s) :
2 channels / 1 channel / 1 channel
Channel positions :
Front: L R / Front: C / Front: C
Channel positions :
1/0/0
Sampling rate :
48000 / 48000 / 24000
Sampling rate :
48.0 KHz / 48.0 KHz / 24.0 KHz
Samples count :
69056544
Compression mode :
Lossy
Compression mode :
Lossy
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Delay relative to video :
0
Video0 delay :
0
Default :
Yes
Default :
Yes
Forced :
No
Forced :
No
Audio #2
Count :
217
Count of stream of this kind :
2
Kind of stream :
Audio
Kind of stream :
Audio
Stream identifier :
1
Stream identifier :
2
StreamOrder :
2
ID :
3
ID :
3
Unique ID :
2289655193
Format :
AAC
Format/Info :
Advanced Audio Codec
Commercial name :
AAC
Format profile :
HE-AACv2 / HE-AAC / LC
Format settings, SBR :
Yes (Implicit)
Format settings, SBR :
Yes (Implicit)
Format settings, PS :
Yes (Implicit)
Format settings, PS :
Yes (Implicit)
Codec ID :
A_AAC
Codec :
AAC LC-SBR-PS
Codec :
AAC LC-SBR-PS
Codec/Family :
AAC
Duration :
1438678
Duration :
23mn 58s
Duration :
23mn 58s 678ms
Duration :
23mn 58s
Duration :
00:23:58.678
Channel(s) :
2 / 1 / 1
Channel(s) :
2 channels / 1 channel / 1 channel
Channel positions :
Front: L R / Front: C / Front: C
Channel positions :
1/0/0
Sampling rate :
48000 / 48000 / 24000
Sampling rate :
48.0 KHz / 48.0 KHz / 24.0 KHz
Samples count :
69056544
Compression mode :
Lossy
Compression mode :
Lossy
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Delay relative to video :
0
Video0 delay :
0
Language :
ja
Language :
Japanese
Language :
Japanese
Language :
ja
Language :
jpn
Language :
ja
Default :
No
Default :
No
Forced :
No
Forced :
No

[SushiSky]

2nd source with the hd video
Video
Count :
248
Count of stream of this kind :
1
Kind of stream :
Video
Kind of stream :
Video
Stream identifier :
0
ID :
1
ID :
1
Format :
AVC
Format/Info :
Advanced Video Codec
Format/Url :
http://developers.videolan.org/x264.html
Commercial name :
AVC
Format profile :
High@L4.1
Format settings :
CABAC / 4 Ref Frames
Format settings, CABAC :
Yes
Format settings, CABAC :
Yes
Format settings, ReFrames :
4
Format settings, ReFrames :
4 frames
Internet media type :
video/H264
Codec ID :
avc1
Codec ID/Info :
Advanced Video Coding
Codec ID/Url :
http://www.apple.com/quicktime/download/standalone.html
Codec :
AVC
Codec :
AVC
Codec/Family :
AVC
Codec/Info :
Advanced Video Codec
Codec/Url :
http://developers.videolan.org/x264.html
Codec/CC :
avc1
Codec profile :
High@L4.1
Codec settings :
CABAC / 4 Ref Frames
Codec settings, CABAC :
Yes
Codec_Settings_RefFrames :
4
Duration :
1429928
Duration :
23mn 49s
Duration :
23mn 49s 928ms
Duration :
23mn 49s
Duration :
00:23:49.928
Bit rate :
2008000
Bit rate :
2 008 Kbps
Maximum bit rate :
7011208
Maximum bit rate :
7 011 Kbps
Width :
1280
Width :
1 280 pixels
Height :
720
Height :
720 pixels
Pixel aspect ratio :
1.000
Display aspect ratio :
1.778
Display aspect ratio :
16:9
Rotation :
0.000
Frame rate mode :
CFR
Frame rate mode :
Constant
Frame rate :
23.976
Frame rate :
23.976 fps
Frame count :
34284
Resolution :
8
Resolution :
8 bits
Colorimetry :
4:2:0
Color space :
YUV
Chroma subsampling :
4:2:0
Bit depth :
8
Bit depth :
8 bits
Scan type :
Progressive
Scan type :
Progressive
Interlacement :
PPF
Interlacement :
Progressive
Bits/(Pixel*Frame) :
0.091
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Stream size :
358960192
Stream size :
342 MiB (91%)
Stream size :
342 MiB
Stream size :
342 MiB
Stream size :
342 MiB
Stream size :
342.3 MiB
Stream size :
342 MiB (91%)
Proportion of this stream :
0.90880
Writing library :
x264 - core 65 r1069M ed32ad2
Writing library :
x264 core 65 r1069M ed32ad2
Writing library/Name :
x264
Writing library/Version :
core 65 r1069M ed32ad2
Encoding settings :
cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=umh / subme=8 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=1 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc=2pass / bitrate=2008 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=50000 / vbv_bufsize=50000 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
Encoded date :
UTC 2009-01-06 06:16:56
Tagged date :
UTC 2009-01-06 06:17:13
Audio
Count :
217
Count of stream of this kind :
1
Kind of stream :
Audio
Kind of stream :
Audio
Stream identifier :
0
ID :
2
ID :
2
Format :
AAC
Format/Info :
Advanced Audio Codec
Commercial name :
AAC
Format profile :
LC
Format settings, SBR :
No (Explicit)
Format settings, SBR :
No (Explicit)
Codec ID :
40
Codec :
AAC LC
Codec :
AAC LC
Codec/Family :
AAC
Codec/CC :
40
Duration :
1429993
Duration :
23mn 49s
Duration :
23mn 49s 993ms
Duration :
23mn 49s
Duration :
00:23:49.993
Bit rate mode :
VBR
Bit rate mode :
Variable
Bit rate :
197872
Bit rate :
198 Kbps
Maximum bit rate :
250616
Maximum bit rate :
251 Kbps
Channel(s) :
2
Channel(s) :
2 channels
Channel positions :
Front: L R
Channel positions :
2/0/0
Sampling rate :
48000
Sampling rate :
48.0 KHz
Samples count :
68639664
Frame count :
67031
Compression mode :
Lossy
Compression mode :
Lossy
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Delay relative to video :
0
Video0 delay :
0
Stream size :
35370344
Stream size :
33.7 MiB (9%)
Stream size :
34 MiB
Stream size :
34 MiB
Stream size :
33.7 MiB
Stream size :
33.73 MiB
Stream size :
33.7 MiB (9%)
Proportion of this stream :
0.08955
Encoded date :
UTC 2009-01-06 06:17:12
Tagged date :
UTC 2009-01-06 06:17:13

[Selur]

for the future: learn to post properly, either use 'code'- or 'quote'-tags or better attach a file with the content.
looking at the stuff you posted it looks like:
1. fps of the video stream in the 1st container is 23.976fps, but the fps of the container indicates 29.97
(-> playback speed is boosted for 23.976 to 29.97 fps)
2. seems like your 1st video stream might also be vfr
3. both containers do not have a/v delays
=> did you try to stretch the audio playback through the container flags? (without reencoding, but through remuxing?)

Problem audio might still be streched and the video of the 1st container might be vfr.
-> if either of this is true, you will probably not get it fixed without looking at the time codes

4. Frame count : 34494 source 1, Frame count : 34284 source two
34494 - 34284 = 210 ~ 9 seconds!!
-> your sources do not match, so there are bound to be some a/v synch issues, only way to fix this is probably with a lot of work inside an NLE.


Cu Selur

[SushiSky]

So if I managed to remove these frames and make the two source's video streams the same, all I need to do next would be to change the frame rates?

[SushiSky]

I would also appreciate if anyone could tell me what exactly to do based on the media info output

[Selur]

So if I managed to remove these frames and make the two source's video streams the same, all I need to do next would be to change the frame rates?

if the frame are the same you can either, change the frame rate of the video or adjust the playback speed of the audio,.. (assuming audio in not stretched and video is not vfr)

[SushiSky]

If the video is in fact vfr, would that make it impossible to sync?

[Selur]

You could probably try to convert the video to cfr while keeping the audio synch and then extract the adjusted audio stream and modify it that it matches the new video stream,... like I wrote before this is probably a job for a NLE tool and a lot of time, since you would have constantly need to check the synching,.. so in theory not impossible just really a lot of work, in praxis not really feasible.... since you probably would have to study the time codes and adjust the audio in a way that it will match the new video stream that,..
-> personally, I would drop the project since I wouldn't have the time to figure out how to do this the best way,..

But you never know may be I'm missing something and there's someone who knows an easy way to fix this.

Cu Selur

[SushiSky]

what software do you recommend for changing the video to cfr? I would like to hear any other ideas you have as regards to fixing this. I appreciate all your help

[Selur]

Personally I would probably to for Avisynth and read up on the vfr-to-cfr tutorials that are out there,...
(alternatively I would probably adjust my own software to handle the case)

[SushiSky]

But I don't get how changing it to cfr would affect the sync issues

[Selur]

If the file is cfr and still synch (and the frame count matches) you can simply adjust the playback speed,... the point is to convert to cfr and keep a/v synch,..

[SushiSky]

I have found a source that provides the same episode, with a constant frame rate.

Video
Count :
251
Count of stream of this kind :
1
Kind of stream :
Video
Kind of stream :
Video
Stream identifier :
0
Inform :
English, 812 Kbps, 480*272 (16:9), at 24.000 fps, AVC (Baseline@L2.1) (2 Ref Frames)
ID :
1
ID :
1
Format :
AVC
Format/Info :
Advanced Video Codec
Format/Url :
http://developers.videolan.org/x264.html
Commercial name :
AVC
Format profile :
Baseline@L2.1
Format settings :
2 Ref Frames
Format settings, CABAC :
No
Format settings, CABAC :
No
Format settings, ReFrames :
2
Format settings, ReFrames :
2 frames
Internet media type :
video/H264
Codec ID :
avc1
Codec ID/Info :
Advanced Video Coding
Codec ID/Url :
http://www.apple.com/quicktime/download/standalone.html
Codec :
AVC
Codec :
AVC
Codec/Family :
AVC
Codec/Info :
Advanced Video Codec
Codec/Url :
http://developers.videolan.org/x264.html
Codec/CC :
avc1
Codec profile :
Baseline@L2.1
Codec settings :
2 Ref Frames
Codec settings, CABAC :
No
Codec_Settings_RefFrames :
2
Duration :
1437417
Duration :
23mn 57s
Duration :
23mn 57s 417ms
Duration :
23mn 57s
Duration :
00:23:57.417
Bit rate mode :
VBR
Bit rate mode :
Variable
Bit rate :
812243
Bit rate :
812 Kbps
Maximum bit rate :
4005888
Maximum bit rate :
4 006 Kbps
Width :
480
Width :
480 pixels
Height :
272
Height :
272 pixels
Pixel aspect ratio :
1.000
Display aspect ratio :
1.765
Display aspect ratio :
16:9
Rotation :
0.000
Frame rate mode :
CFR
Frame rate mode :
Constant
Frame rate :
24.000
Frame rate :
24.000 fps
Frame count :
34498
Resolution :
8
Resolution :
8 bits
Colorimetry :
4:2:0
Color space :
YUV
Chroma subsampling :
4:2:0
Bit depth :
8
Bit depth :
8 bits
Scan type :
Progressive
Scan type :
Progressive
Interlacement :
PPF
Interlacement :
Progressive
Bits/(Pixel*Frame) :
0.259
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Stream size :
145941464
Stream size :
139 MiB (87%)
Stream size :
139 MiB
Stream size :
139 MiB
Stream size :
139 MiB
Stream size :
139.2 MiB
Stream size :
139 MiB (87%)
Proportion of this stream :
0.86686
Language :
en
Language :
English
Language :
English
Language :
en
Language :
eng
Language :
en
Encoded date :
UTC 2012-06-20 09:16:53
Tagged date :
UTC 2012-06-20 09:28:06
Buffer size :
4014080
Color primaries :
BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
Transfer characteristics :
BT.709-5, BT.1361
Matrix coefficients :
BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
Audio
Count :
217
Count of stream of this kind :
1
Kind of stream :
Audio
Kind of stream :
Audio
Stream identifier :
0
Inform :
English, 121 Kbps (128 Kbps), 44.1 KHz, 2 channels, AAC (LC)
ID :
2
ID :
2
Format :
AAC
Format/Info :
Advanced Audio Codec
Commercial name :
AAC
Format profile :
LC
Codec ID :
40
Codec :
AAC LC
Codec :
AAC LC
Codec/Family :
AAC
Codec/CC :
40
Duration :
1437360
Duration :
23mn 57s
Duration :
23mn 57s 360ms
Duration :
23mn 57s
Duration :
00:23:57.360
Bit rate mode :
CBR
Bit rate mode :
Constant
Bit rate :
121488
Bit rate :
121 Kbps
Nominal bit rate :
128000
Nominal bit rate :
128 Kbps
Channel(s) :
2
Channel(s) :
2 channels
Channel positions :
Front: L R
Channel positions :
2/0/0
Sampling rate :
44100
Sampling rate :
44.1 KHz
Samples count :
63387576
Frame count :
61902
Compression mode :
Lossy
Compression mode :
Lossy
Delay :
0
Delay :
00:00:00.000
Delay, origin :
Container
Delay, origin :
Container
Delay relative to video :
0
Video0 delay :
0
Stream size :
21827728
Stream size :
20.8 MiB (13%)
Stream size :
21 MiB
Stream size :
21 MiB
Stream size :
20.8 MiB
Stream size :
20.82 MiB
Stream size :
20.8 MiB (13%)
Proportion of this stream :
0.12965
Language :
en
Language :
English
Language :
English
Language :
en
Language :
eng
Language :
en
Encoded date :
UTC 2012-06-20 09:16:54
Tagged date :
UTC 2012-06-20 09:28:06

[Selur]

okay, this source has:

Frame rate : 24.000 fps Frame count : 34498

your hd source has:

Frame rate : 23.976 fps
Frame count : 34284

so you need to compare these two, and remove the audio&video parts where they differ and then audio should be interchangeable,..
-> you probably need a NLE-tool for this (no, I can't recommend one and I never tried such a thing, but I suspect others have tried this before and might be able to help)

Cu Selur

[SushiSky]

One last question :3
Isn't 24fps and 23.976 fps gonna cause some serious progressive audio sync problems over time?

[poisondeathray]

One last question :3
Isn't 24fps and 23.976 fps gonna cause some serious progressive audio sync problems over time?

Yes it will go out of sync over time

If you have a 24FPS CFR source, and 23.976FPS CFR source (and they are the same video, with the same frames and same framecount), you can either change the framerate of the video to match or stretch/shrink the audio to match

Since your recent video is CFR, you can do the audio in audacity (it will be a linear transformation)

You cannot do this properly with VFR, because in VFR streams, the audio is the same (it's CFR), but it's the timecodes that tell the media player to play the video faster or slower in sections (non linear)

okay, this source has:
Code:
Frame rate : 24.000 fps Frame count : 34498
your hd source has:
Code:
Frame rate : 23.976 fps Frame count : 34284

But linear tranformation won't work for this, because the videos are clearly different. You need to match up exact same video frames (i.e same video) as Selur says. But even cutting & editing in NLE, you will get audio "blips" where you make cuts. It's not ideal.

[SushiSky]

So with cutting and editing in NLE with the cfr sources, the video and audio will be synced? In that case do you know of any programs for mac that can display frames of two videos in a time line?

[poisondeathray]

FCP or Premiere Pro

If the frames are identical , matched up, and the fps CFR and is corrected (same fps 23.976 vs 23.976), then essentially you have the same cut of the video - then it should be in sync . You should be able to swap video or audio between the 2 versions of the video

You have to make sure it's the same cut of the video. They can be different versions (containing other scenes), or edited differently. Then you have to manually sync each section

But if the HD video is VFR (then it's likely missing duplicate frames that the CFR version has, also notice it has fewer frames - this suggests it could be VFR ) , it won't work this way. NLE's only work with CFR (they don't read VFR timecodes, at least not the type in MKV or MP4) . Check the timecodes to be sure .