HD TS to HD DivX/AC3 audio sync issue

[Xesdeeni]

I have the first of a bunch of HD movies recorded from ABC that I want to convert to HD DivX so they will fit on single DVDRs and play on my Sigma Designs based player (Z500). ABC apparently plays games with their telecine, so with some help I was able to determine that the original frame rate for this particular movie was 23.2603 fps (ech!). Anyway, after inversing the telecine with AVISynth and plugins (DGIndex/DGDecode, tivtc, etc.), I converted the video to DivX and muxed in the original AC3 using VirtualDubMod. The resulting video looked great and the audio started out in sync, but by the end it was a second or so out.

The original TS is 1:19:37.90 in length.
The AC3 extracted with DGIndex is 1:19:37.95.
The AVISynth script (inversing the telecine) is 1:19:37.88.
The resulting AVI is 1:19:37.84 in length.

So I'm at a loss to understand why the audio is so far out of sync. And while I can manually adjust the audio, this means a lot of extra work for each movie, without me really knowing the root cause.

Any suggestions?

Xesdeeni

[FulciLives]

When I use to use a PCI TV tuner type capture card I would never have the audio in sync unless I used the AUDIO as the MASTER STREAM. The result is a video file that was in-sync with the audio but the video file would always be "off" a bit. What I mean is that the video was never exactly 29.970fps but it would often be instead something like 29.969 or 29.968 or 29.967 etc.

So I would add AssumeFPS(29.970, true) to the end of my script. Thus it would give me a "standard" 29.970fps but this would require that the audio be changed as well. Since my audio was always PCM WAV this was not a problem. I would simply run the script through TMPGEnc Plus and the SSRC plug-in would change the audio. I would end up with a new PCM WAV audio file that matched the corrected 29.970fps video.

Why did I bring all that up?

Well I am thinking that the IVTC is what has caused the problem here in your case. I mean 23.2603fps is a rather "odd" number. Maybe doing a "fix" somewhat similar to my old capture card "fix" will change the video enough to match the audio. Of course you probably don't want to adjust the audio as in my case but just the video. Tis a thought ...

I suppose this leads to another question ... is it out-of-sync playing the AviSynth file back in VirtualDub(Mod) before trying to convert or only after converting or ... ???

Also can you post the AviSynth script you used? Is the script outputting at 23.2603fps or did you try to change it to 23.976fps etc.

- John "FulciLives" Coleman

[Xesdeeni]

When I use to use a PCI TV tuner type capture card I would never have the audio in sync unless I used the AUDIO as the MASTER STREAM....I would end up with a new PCM WAV audio file that matched the corrected 29.970fps video.

I don't know which PCI card you used, but in some cases, the audio was being recorded with the computer's audio card, while the video was being recorded with the video card. As you can imagine, there is a very good chance these won't sync up after time.

Well I am thinking that the IVTC is what has caused the problem here in your case. I mean 23.2603fps is a rather "odd" number. Maybe doing a "fix" somewhat similar to my old capture card "fix" will change the video enough to match the audio. Of course you probably don't want to adjust the audio as in my case but just the video. Tis a thought ...

Interesting. But with a digital stream, both are in there together. They aren't based on different timers.

I can certainly correct this, that's not really the issue--although I'd certainly like to avoid the decode, process audio, encode (I'd more likely try to cut a bit during the silence at each commercial break). If this were only one movie, I might call it academic to even wonder why there was a sync issue to begin with, fix it and go on. But given that I have a number of other movies I want to convert (now that I have bought a Z500), it seems silly to stab in the dark for each one. There has to be a reasonable explanation, and more importantly, a set procedure to correct for or avoid this.

I suppose this leads to another question ... is it out-of-sync playing the AviSynth file back in VirtualDub(Mod) before trying to convert or only after converting or ... ???

I'm very sorry, I forgot to mention this. The sync is the same (good at the start, a second or so out at the end) on the PC as well as the Z500.

Also can you post the AviSynth script you used? Is the script outputting at 23.2603fps or did you try to change it to 23.976fps etc.

The script (below) is outputting 23.2603. The problem with converting to 23.976 is that I'd have to repeat a film frame, that is show the same image for 2/24 second (or 150% of the normal time, or 83 ms). If I let the player handle this on playback, it will hold a frame for 3/60 second (more often, but only 150% of the normal time or 50 ms). The difference is huge in the perceived "bump" as things move.

#AudioDub(MPEG2Source("Toy Story (WFAA).d2v"), DirectShowSource("Toy Story (WFAA) PID 034 T01 3_2ch 384Kbps DELAY 32ms.ac3"))
MPEG2Source("Toy Story (WFAA).d2v")
AssumeBFF()
Crop(0, 0, 0, -8)
tfm() #mode=5
tdecimate(mode=2, rate=23.2603)
LanczosResize(1280, 720)

I also purposely left out the fact that my local ABC affiliate is owned by Belo, which has a corporate policy that all their TV stations use 1080i, even if they are ABC affiliates. So they convert the 720p to 1080i, and we get the worst of both worlds (lower vertical resolution, and interlace flicker)! As a result, there's no real need to encode 1080@24p, since the source has no more (and actually less due to scaling artifacts) than 720 lines.

Oddly, the stream isn't 1088i as some are (or maybe DGIndex/DGDecode now handles this?), so I don't technically have to crop the bottom 8 lines. But there is still an 8-line black border on the bottom of this particular movie (I haven't closely examined any others yet). I decided to take the aspect ratio hit and scale 1920x1072 to 1280x720 to eliminate it.

Anyway, I mention this here only to explain the applicable portions of the script.

Xesdeeni

[FulciLives]

Well you got me LOL

I've had limited real world experience in working with HDTV sources ... the few HDTV captures I have played with have all been from a Hauppauge WinTV HVR 1600 ... a device I got for free iotherwise I would be mighty pissed as it works for shit ... but that's another story.

The last thing I will "throw out there" is that maybe there is some sort of corruption in the original HDTV capture. Maybe it needs to be processed through something like ProjectX (the only way I can get decent stuff out of my Hauppauge WinTV HVR 1600 captures) before further processing (like using DGIndex).

I have an idea that might be a big waste of time but ... a new version of HCenc just came out that can encode at 1080i ... I wonder ... if you re-encode the video with your script BUT with HCenc in MPEG-2 and then mux it with the audio ... will it still be out-of-sync? What if you simply go from 1080i to 1080i without any IVTC or resize to 720p ... will it still be out of sync? It shouldn't unless the file is corrupted somehow or there is a "flaw" in your script.

I guess if it were me I would be curious if I simply re-encoded with HCenc as 1080i (with no IVTC) and then see if it is still in sync. If it isn't then you know something is up with the original file ... but if it is still in sync then you know it is something else ... perhaps the IVTC method used?

Of course god only knows how long it takes to encode at HDTV resolutions in MPEG-2 ... I've yet to try that myself LOL

If it makes you feel any better I have faith that if anyone can lick this it is you ... after all you are a god in my eyes with all your PAL to NTSC stuff 8)

- John "FulciLives" Coleman

[Xesdeeni]

The last thing I will "throw out there" is that maybe there is some sort of corruption in the original HDTV capture. Maybe it needs to be processed through something like ProjectX (the only way I can get decent stuff out of my Hauppauge WinTV HVR 1600 captures) before further processing (like using DGIndex).

I've been thinking about this, but I edited the commercials out with VideoRedo Plus. As I understand it, VRD+ does a good job of fixing stream issues. I also double-checked the original source (sanity check) and there is no sync issue there.

I have an idea that might be a big waste of time but ... a new version of HCenc just came out that can encode at 1080i ... I wonder ... if you re-encode the video with your script BUT with HCenc in MPEG-2 and then mux it with the audio ... will it still be out-of-sync? What if you simply go from 1080i to 1080i without any IVTC or resize to 720p ... will it still be out of sync? It shouldn't unless the file is corrupted somehow or there is a "flaw" in your script.

That's something I just thought of over the weekend. I guess I can try this. It doesn't really matter how good it looks, so I can cut lots of corners. Heck, i can scale it down really small to make encoding faster.

In the mean time, if there are any other suggestions, I'm all ears....

Xesdeeni

[vhelp]

Evening Xesdeeni,

FWIW, re-telecing a video source for fitting the time slot on TV is
a brilliant idea devised by the those (broadcasters) of today. And
I must admit, it took me a while to figure this all out on my own
and then realize that many of today's TV broadcast who's origin is
from Film, but is edited to *FIT* inside a given simeslot presented
me, many great challenges over the years. And I've learnt that you
have to analyise a troublesome video source that you know if Film
origin, to discover weather or not they re-telecined it to fit the
time slot for TV. And, when they do this, the Audio is adjusted to
match. I believe that area of this format is the "pitch". And that
is what has to be addressed, and possible a separate tool might have
to be developed just for this. More on this, below.

I could be wrong or a widdle off. But in any case, take this as a
theory, if anything, a clue..

In the early days of video processing, I've had my fair share of
audio de-sync issues. One of my biggest troublesome areas were
when I converted the video by some frame rate other than the video's
original frame rate.

Obviously, whenever a video frame rate is changed, the audio has to
match it back in sync. Course, you know this. I mean. I know you
know this. (fulci know's this, too) Anyways.

Audio Pitch..

But, there is this area in the audio that controls the timing or syncing
(pitch, I think its called) and this is not being addressed. You see,
when you change a video's frame rate, and the audio is adjusted to match,
usually, things work out correctly -- or should I say, in sync. Your
audio's "pitch" was internally edited to meet the *new* speed (fast/slow)
of the re-Telecine process applied by the broadcasters or whom'ever is
responsible in that area. So, in -your- unique cercumstance, your audio
has to be internally adjusted in its pitch. The audio was "sped" up to meet
the "time frame" duration of the edited for tv's time slot. That's why the
Telecine was adjusted in the unusual (or, nonstandard) way -- not the tipicle
[3-2 pulldown] pattern way.

Two things can be applied to Audio. Speeding up or Slowing down. And when
you attempt to change a framerate through nonstandard IVTC methods, the
audio has to be adjusted to match, pitch'wise. Then, you'll have both video
and audio in sync after -your- IVTC conversion.

Sometimes, even when they jimmy the audio (on account of the *new* telecine
they apply to the broadcast source) there are no sync issues. And other
times, there are. I guess you are experiencing those *other* times.

-vhelp 4218

[Xesdeeni]

Obviously, whenever a video frame rate is changed, the audio has to match it back in sync....You see, when you change a video's frame rate, and the audio is adjusted to match, usually, things work out correctly -- or should I say, in sync. Your
audio's "pitch" was internally edited to meet the *new* speed (fast/slow)
of the re-Telecine process applied by the broadcasters or whom'ever is
responsible in that area. So, in -your- unique cercumstance, your audio
has to be internally adjusted in its pitch. The audio was "sped" up to meet
the "time frame" duration of the edited for tv's time slot. That's why the
Telecine was adjusted in the unusual (or, nonstandard) way -- not the tipicle
[3-2 pulldown] pattern way.

What you have to remember is that the broadcaster already changed the speed of the video and audio at their end. They might have adjusted for the pitch shift, or they might not have. But either way, the resulting video and audio are in sync.

Now, when I convert the frame rate, regardless of what the destination is, so long as the video isn't actually sped up or slowed down, the exact same audio should sync to that as well. The acid test for whether the video speed has changed is the resulting duration of the video. As an extreme example, if I converted the video to 10 fps, so long as the resulting video had the same duration, the audio will sync without being adjusted. That's why I started above by pointing out that all the relevant durations were (basically) the same.

At least, that's what mathematics says. This particular case is an exception, but it can't really violate arithmetic. It's just a challenge to figure out why.

FulciLives' idea of skipping the inverse telecine is my next try (unfortunately, I didn't have time to do this test last night). I'm trying to convince myself that there is a glitch in the original TS that is causing my problem. As I mentioned, I used VideoRedo Plus to edit, but maybe I have to explicitly fix the D2V? Or maybe I should have run the TS through something like TSRepair before I edited it. But regardless, I still can't figure out how the sync can be off, but the durations be the same.

There is timing information in the TS that keeps the audio and video sync'd. So if there were a glitch in the video, afterwards, the player would use the timing info to get back in sync. But when I process the video, the decoder might not be as good, and might drop frames it can't decode properly. The problem is that if that were the case, the duration of the video would actually be shorter, since there would be fewer frames.

I just don't know.

Xesdeeni

[Xesdeeni]

OK guys, I've figured this out! Many thanks!!!

The culprit is.....tivtc.

I tried the experiment FulciLives suggested (thanks!). When I just encoded the video at 29.97 fps (field blending and all), the audio stayed in sync.

So I went back and looked at the end of my movie in VirtualDub. The 29.97 encode went all the way to the black frame on which I cut the end. But when I looked at the 23.2603 fps tivtc version, it stopped more than 30 frames early!

So I checked out the AVISynth script in VD and saw the same symptom (stopping before the end). And without tivtc, the AVISynth script showed all the frames in VD.

This explains why even though the video and audio were the same length, they were out of sync (and I should have checked this as soon as I noticed they were the same--shame on me).

I switched to using Telecide() from Decomb plus FDecimate() and when I looked at the AVISynth script in VD, all was well.

So tivtc has some kind of problem, at least with this movie.

Again, many thanks FulciLives!

Xesdeeni

[FulciLives]

I'm glad you got it to work out

My suggestion was based on two things:

1.) Rule out an issue with the source
2.) If the source was OK then it was probably the IVTC process

I guess in this case it was the IVTC process. Not that you were wrong per se but something wrong I guess with tivtc in this case ... as you said.

I really wish I could get my Hauppauge WinTV HR-1600 to work properly. I'd love to capture some HDTV stuff to play with.

- John "FulciLives" Coleman