Transport Stream PreProcessing for A/V Sync

[Turtleggjp]

Hi guys,

Thanks for the info so far on your method for processing HDTV. I ran into a little snag that I'm hoping you can help me with. I have a recorded program (1 hour long, plus a minute or two at either end) that I am trying to edit and recompress. It is in Transport Stream format (.tp extension) recorded from my FusionHDTV 5 Lite card. When I feed this into ProjectX, there is some glitch in the file that causes ProjectX to give me this error message several times:

!> incoming PES packet without payload or wrong header data (5/19/9/10/0)

It keeps going with its work, but it stops writing the AC3 stream. So I end up with a 1+ hour long .m2v file, and an AC3 file that is only about 21 minutes long:

summary of created media files:
.Video (m2v): 111590 Frames 01:02:03.380
Audio 0 (ac3): 39920 Frames 00:21:17.440 0/5/0/0

I have tried running the clip through the MPEG2Repair utility that was mentioned above, but that did not fix the problem. It's log file shows me these messages at the error point:

Sequence Frame 38310(26-P) / Time 0:21:18 :
AudioWarning: Timestamp gap of 0.016700 sec. ending at file offset 2624682156

Sequence Frame 38311(24-B) / Time 0:21:18 :
AudioWarning: Timestamp gap of 0.016700 sec. ending at file offset 2624836958

Sequence Frame 38313(2-I) / Time 0:21:18 :
AudioWarning: Timestamp gap of 0.022000 sec. ending at file offset 2624903592

Sequence Frame 38315(1-B) / Time 0:21:18 :
AudioWarning: Timestamp gap of 0.980800 sec. ending at file offset 2625268150
AudioWarning: Timestamp gap of 1.553878 sec. ending at file offset 2625297328

Sequence Frame 38327(10-B) / Time 0:21:22 :
AudioWarning: Timestamp gap of 0.992000 sec. ending at file offset 2626224704

Sequence Frame 38341(23-B) / Time 0:21:22 :
VideoWarning: TemporalRef gap of 1021. Timestamp gap of 3.586922 sec. ending at file offset 2625300031

If I watch the original file, there is definitely a noticeable glitch/freeze at this point (when the show came back from a commercial break). I have also tried using HDTV2MPEG to cut around this glitch, but it seems that everywhere I make a cut, even in good areas, ProjectX still stops writing the AC3 file at the cut point. I have been running ProjectX without any of the command line options. Is there an option that will help it get past this glitch? Or is there another program that might be able to fix this type of glitch? Thanks.

Matt

[GaveUp]

The first question is what version of ProjectX are you using? Versions previous to the .90 series had heap and memory overflow errors that would cause crashes. This doesn't sound like what it happening, but if you are using an older version it doesn't hurt to try the latest version.

As for an alternative method. You can try running the file through dgindex and then use a separate tool (the ac3 cutter mentioned will do it) to sync fix the ac3. This method has the downside that it's not able to correct bad errors because the PTS timestamps are gone, but if PX doesn't work there's really not much else to try ... at least not that I can think of.

[Turtleggjp]

The version of ProjectX I was using is the one that was mentioned at the beginning of the thread: 0.90.1. I see there is a version 0.90.2, but I didn't get a compiled version, so I haven't been able to try that one yet.

DGIndex was what I was originally going to use. I think this error did cause some sync problems, but I'll run it through one more time to see for sure.

Matt

[Soopafresh]

Run it through Mpeg2Repair first.

http://users.adelphia.net/~mwilczyn/mpeg2repair/mpeg2repair.zip

[sengsational]

...cut out the commercials

Informative thread! In most cases, I'm a bit too lazy to go through the "tool chain" to get the results, though. That's why I used HDTV2DVD and now SVCD2DVD; they provide nearly a one click DVD burn (no demuxing, remuxing). So far I've been happy with the audio synch, except for a recent case where audio from a commercial screws up the tool.

Being lazy, my idea of the perfect solution would be to right click on a transport stream and choose, "save to DVD". That would be good enough 90% of the time. The other 10% I'd want to cut commercials before going to DVD, which brings me to my next point.

If the audio changes between the real programming and the commercial, maybe a tool could be developed that would auto-cut that bad audio right out of the file, and leave a nice clean splice, without hand editing. That would solve two problems: 1) manual editing, and 2) junky splices that confuse our stream tools.

I downloaded VideoReDo on trial, and saw something that indicated it could find commercials automatically. Maybe that's what it's doing. That brings me to my final point: GUI madness. Having six tools, even if they are free, or $20, or $50, each with their own GUI, drives me nuts. It would be so cool if each of these operations on a stream could be initated as a filter, configured and called from an API. A GUI it great for doing a proof of concept, but after that, I need an API!! I know, I'm dreaming, but how about a tool that generates a script as you do the dragging and dropping, then submits that script to itself when you say 'run'. Next time, you have the choice: GUI or edit and submit the script.

But, I digress. I just wanted to keep in the loop on the latest HDTV to DVD methods.

--Dale--

[Soopafresh]

Absolutely. It IS a hassle to use so many tools to process this material. VideoRedo is the closest to an integrated solution, but you have to pay for the ease of use. But it might be $50 well spent. Still, the commercial detection feature is new and needs time to season.

What would be cool is a utility that could scan the Transport Stream file and automatically cut it whenever it sees a switch from 5.1 to 2.0 sound.
This could be an easy way to remove commercials.
PVAStrummento kinda does that, but not perfectly.

There's a commandline app called COMSKIP which is actively updated and I suspect this will end up being the panacea for commercial detection.

http://gbpvr.com/pmwiki/pmwiki.php/Manual/ComSkip

[Leehro]

the link for ac3 cutter is now giving me connection refused. Is there another place to get this tool or another tool that works as well?

[Soopafresh]

Ac3Cutter - http://ac3c.vidphiles.com/

[Soopafresh]

The latest version of precompiled ProjectX has been posted to VideoHelp.com. Uninstall your old version of the software before you install this one. http://download.videohelp.com/~download/ProjectX_09.3.0.zip

[GaveUp]

Ok so a few comments. First, in regards to AC3 Cutter, the website should be back up. Apache decided to bomb out on me and noone was nice enough to inform me of it.

As for commercial audio causing sync problems, I've only seen this happen in select cases. These being a FOX broadcast being put into an AVI container with AC3. On the main page of Vidphiles I posted a rather lengthy explanation what causes the problem, if interested. As a note, this sync issue is NOT a problem if encoding to MPEG2PS or similar.

The changing from 2.0 to 5.1 and vice-versa is not uncommon, and should not cause sync issues. Unfortunately, it doesn't serve as a reliable way to detect ads, at least not yet because often the audio was switch slightly early or slightly late similar to how it's not that uncommon, unfortunately, to have pullarboxing coming into and going out of ad breaks. Hopefully, this will change as HD becomes more popular and more people start complaining about these problems.

A good, fairly quick and reliable, way to detect commercials is still to do it manually in Virtualdub or such and use signals to see where to start looking. For example, the audio change is a good signal. You can use the ProjectX logs to find where the changes happen, then goto that time and only have to step a few frames to fine tune.

[DVWannaB]

thanks guys.

[Turtleggjp]

Speaking of FOX...

Now that 24 Season 5 has started, which was the main reason for me to get my HDTV card and capture TV shows with it, I've started to run into problems with my captured transport stream that seems to throw off ProjectX. I was able to get around this problem by cutting out most of the commercials with HDTV2MPEG, then running the result through ProjectX. I have a feeling that this log entry might be an indicator of the problem, since I never see this message on my edited versions:

-> skip sequence_end_code following GOP# 14479

Obviously, the GOP# is different each time this message occurs. Down futher in the log, I can see ProjectX reporting the changes between 5.1 and 2.0 channel audio, and it does report 1 frame of audio inserted. If I remember correctly, the audio was playing early by the end of the 2+ hour clip, so it sounds like more audio needed to be inserted. I had been using version 0.90.1 of ProjectX, but when this problem came up, I tried the newest version too. Unfortunately, it didn't help either. I did also try MPEG2Repair, but it didn't change anything either, although it did help clear up another problem a couple weeks ago that also was throwing off ProjectX.

I always knew that working with MPEG2 files was going to be a hassle, which is why I've always dealt only with uncompressed video and audio until now. However it is nice to see all these great tools available for us to play around with. Sooner or later we all find that perfect combination of tools that helps us achieve the results we want. Patience is a wonderful thing to have!

Matt

[GaveUp]

The skip_sequence_endcode errors are generally nothing to worry about and will not cause problems with the end product. You don't mention what your encoding process was. What tools did you use and what was your final format?

[Soopafresh]

GaveUp - Thank you for the great clarification on these sync problems, and thanks for your great AC3 software.

Matt - Yeah, every once in a while I'll have an especially problematic Mpeg2 HD file that ProjectX doesn't like. Mpeg2Cut2 will demux, and does a pretty good job of it, as long as the file isn't too messed up.

[Turtleggjp]

GaveUp,

After ProjectX gives me the .m2v and .ac3 files, I use DGIndex to index the .m2v file, and then use an AVI Synth script with DGDecode to decode the .m2v file into an uncompressed AVI file in YV12 format. I also reduce the size by a factor of 4 (1280x720 -> 320x180) so it doesn't take up nearly as much space. I then run the .AC3 file through BeSweet to decode it into a stereo .WAV file. I use these two files to figure out how to edit out the commercials. It is in this format, that I can tell if there are sync problems. This is where I could see that by the end of the 2 hour capture file, the audio had drifted out of sync.

The rest of my process goes like this:

Once I have determined my cut points, I save them as a VirtualDub .vcf file. I then convert the editing portions of the .vcf file into AVI Synth Trim() commands, and add them to my original decoding script. So now, instead of the script decoding the entire .m2v file, it only decodes the frames that I want to keep. I then apply any cropping needed, as well as IVTC stuff for 1080i, or Framerate decimation for 720p. The video is served into VirtualDub, and I have been using XviD 1.1.0 from there. The only trick left is to cut the .AC3 file accordingly as well. I currently have VirtualDubMod load up the uncompressed video clip and the AC3 file, which I then apply the .vcf settings to, and then just save the .AC3 clip as a .WAV file (still in untouched AC3 form though, NOT decoded). I would rather just use another program to cut based on just the .vcf file, but I have yet to find one that can do it. AC3 Cutter always crashes everytime I have it cut an AC3 file based on a .vcf file. Once the video is done compressing, VirtualDubMod simply puts the video and AC3 together for me.

Whew! If that sounds complex, yeah I know it is. But I've been really happy with the results so far. I get frame accurate editing, not GOP accurate editing, and with the exception of that first 2 hour clip of 24, the audio usually stays in sync very well.

Here's another message from the log:

-> Pics exp/cnt 9/9, inGOP PTS diff. 33ms, new Timecode 01:28:49.612

A few of these also showed up when I was having the sync problems. If it is not the skip sequence errors causing the problem, could it be this instead?

Matt

[Turtleggjp]

I've run into another problem lately, this time on NBC. I've been trying to record the reruns of Star Trek: Enterprise on Sunday nights, and the last two weeks in a row, there are errors in the captured file that causes ProjectX to give massive amounts of errors such as this:

!> PTS difference of 12012 (00:00:00.133) to last exported GOP detected 
!> dropping useless B-Frames @ GOP# 537 / new Timecode 00:04:17.724
!> PTS difference of 6006 (00:00:00.066) to last exported GOP detected 
!> dropping useless B-Frames @ GOP# 545 / new Timecode 00:04:21.361
!> PTS difference of 7507 (00:00:00.083) to last exported GOP detected 
!> dropping useless B-Frames @ GOP# 546 / new Timecode 00:04:21.795
!> PTS difference of 12012 (00:00:00.133) to last exported GOP detected 
!> dropping useless B-Frames @ GOP# 550 / new Timecode 00:04:23.730
!> PTS difference of 10511 (00:00:00.116) to last exported GOP detected 
!> dropping useless B-Frames @ GOP# 551 / new Timecode 00:04:24.164

This causes very noticeable breaks in the video stream, and definite skips in the audio stream. The audio almost sounds like it's in a Fast Forward mode sometimes. I tried using the Java version of ProjectX to mess around with some of the settings, but nothing has caused these errors to stop. Using DGIndex, I was able to create a .m2v file and .ac3 file for one of the areas with all the problems, and at first glance there appears to be no skipping at all. Also, when I play the original captured file in my HDTV card's software player, it seems to play perfectly. I'm guessing that whatever this error is, ProjectX is making a big fuss over it for no reason. I'm hoping that one of its settings can make it ignore these errors. Anybody have any ideas? Thanks.

Matt

EDIT: MPEG2Repair also did not fix this problem.

[DVWannaB]

All,

I am having problems converting a 1280x720p 59.94 file to 720x480p for DVD creation.

I used DGIndex 1.4.3 (honor all pulldown flags checked) to create *.d2v. I used this script from a previous thread:

LoadPlugin("C:\Program Files\avisynth 2.5\plugins\undot.dll")
LoadPlugin("C:\Program Files\avisynth 2.5\plugins\BT709ToBT601.dll")

MPEG2Source("f:\1workfolder\working\disc4\game.d2v ")

BT709ToBT601()

SelectEven()

UnDot()

LanczosResize(704,480)

addborders(8,0,8,0)

converttoyv12()

Encode using TMPGEnc 2.5 Plus as Top Field First, progressive, 16:9.

I burn to DVD-RW and the playback on the standalone stutters like an earthquake on any movement.

What can I do to fix this problem? Thanks.

[Soopafresh]

Sounds like a setting is wrong in TMPGenc, or the media you burned to is bad. It should be a full progressive encode, no need for TFF or BFF.

BTW, HcEnc v020 is a fantastic Mpeg2 encoder, and look at the nice feature on the right

[DVWannaB]

ok soop, I'll give HCenc a go. Thanks.

[Soopafresh]

If that doesn't work, you can always try to Tdecimate() down to 23.976 and choose "pulldown" in the HCenc settings.

[DVWannaB]

I dont think that will work soop. It is an ESPN-HD broadcast of a recent basketball game. So I believe it was shot in NTSC progressive.

[Soopafresh]

In that case, you're absolutely right.

[DVWannaB]

hey soop,

HCenc wouldnt open my *.avs file. It opened the *.ts file. Am I missing a *.dll or something. I just DL the and unzipped to a folder on my C drive.

[Soopafresh]

That's weird. Lemme check your AVS.

Load the Dgdecode.dll manually ?

You specified the output file name ?

[Soopafresh]

Found a clip of your source file. The quality is really sub-par. I think the networks actually record in 1080i, then dumb-bob the picture to 720p. There are so many artifacts, not to mention the lighting is bad, plus the straight lines are all aliased. Still, you can't polish a turd. Here's a heavily filtered attempt, but the encode time will be unacceptable.



bball_5.avi

[DVWannaB]

hey soop,

whats your *.avs script? That looks pretty good. Did you bob it?

[Soopafresh]

Here you go. But don't say I didn't warn you. You'll have to time the encode with a calendar. Your 2Ghz machine is going to cry out for mercy.

Needed Avisynth Plugins:

AGC - http://strony.aster.pl/paviko/Hdragc-1.8.7.zip
MVtools - http://avisynth.org.ru/mvtools/mvtools-v1.4.13.zip

1) Unzip the plugins above and copy .DLLs into C:\Program Files\AviSynth 2.5\plugins\


SetMemoryMax(400)
mpeg2source("E:\input.d2v")

converttoyv12()
selecteven()
spline36resize(720,480)

backward_vec2 = last.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = last.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = last.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = last.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
last.MVDegrain2(backward_vec1,forward_vec1,backwar d_vec2,forward_vec2,thSAD=800,idx=1)

HDRAGC(coef_gain=1.0,max_gain =2.0,min_gain=0.9,corrector=0.8,coef_sat=0.9,max_s at=1.4)



Because the scene has constant action, you'll also need to crank up the bitrate, which means your output file is going to be huge. I wouldn't put more than an hour of game on a DVD.

[DVWannaB]

yeah, I've seen a similar script on doom9. Slow is not the word I did a small test clip and stopped it after about 45 seconds of the video clip wa encoded

I finally have a smooth playing file now. Guys at doom9 said not to use SelectEven() because it decimates too much motion. Just resize and interlace. The interlace interpolates the original 59.94 and so the motion is as it should be.

Here is the final script that worked:

LoadPlugin("dgdecode.dll")
MPEG2Source("clip.d2v")
LanczosResize(704,480)
addborders(8,0,8,0)
assumetff()
separatefields()
selectevery(4,0,3)
weave()

So, lesson learned here. For future reference:

1280x720p (NTSC 59.94 fps)----> 720x480i (should use script above to get the standard 29.97 fps)
ESPN-HD, ESPN2-HD, UPN original live events (I believe ABC & Fox also)

1280x720p ----> 720x480p (SelectEven & Decimate to return to orginal 23.97 fps)
Hollywood movies on TV and many HDTV weekly series on ABC, Fox & UPN