field order wrong after AVStoDVD conversion

[4evrplan]

I just tried out AVStoDVD and played the resulting disc (DVD-5) in my standalone. First I'd like to say that I love this tool, especially the fact that it allows you to edit the avisynth script. The audio turned out perfectly synced when much other software I've tried messed this up (though it wasn't normalized ).

However, the motion in the video looked jittery or jumpy. I loaded up my source file in VirtualDubMod with the following script

AVISource("...")
SeparateFields()

and clicked through several frames one at a time. Sure enough, my field order is backwards. So, here's my question, should putting "AssumeTFF()" in the script fix it, or will it have to be "ComplementFields()", or am I just out of luck? I've just redone the whole thing (about a 5 hour process) with "AssumeTFF()" in the script, and I haven't had a chance to play it in my standalone. However, on initial inspection with VDM, it still seems to be the wrong field order.

Also, AVStoDVD seems to ignore some of the script commands, because I tried doing a "preview" with "SeparateFields()" in the script, and it didn't seem to change anything; I just got the full frame in the preview.

For those of you more knowledgeable about avisynth and AVStoDVD than me, I could sure use some help.

EDIT: I just tried the new version in my standalone, and it's still jittery. I should have mentioned that this video was a movie captured off of analog cable. I know there's got to be some experts on interlaced video floating around this forum .

[45tripp]

not familiar with avstodvd.

no need to re-encode.
you can use restream to reverse the field order on the elementary mpeg2 stream.

use
AVISource("...")
Assumetff().SeparateFields()

and
AVISource("...")
Assumebff().SeparateFields()

step through in vdub and you'll determine field order.
Set field order accordingly in mpeg2 encoder.

it seems obvious it's bff
as encoders will default to tff output.

gl

[manono]

Exactly right.

I should have mentioned that this video was a movie captured off of analog cable.

If a movie, it probably should be IVTC'd instead, and encoded as progressive.

[4evrplan]

I intend to watch it on my television via my standalone DVD player. Doesn't it need to be interlaced for that, or do standalones interlace progressive videos on the fly for display on television?

[manono]

If it is a movie, and if you do IVTC it, and if you do encode it for progressive 23.976fps together with applying pulldown either during or after the encoding, then the player will output 59.94 fields per second, as required for NTSC DVD. Most movies on NTSC DVD are encoded as progressive, with progressive 23.976fps stored on the DVD.

[4evrplan]

manono (or anyone): You seem to know what you're talking about, so I'd like to pose another quick question. Can editing a video (tv station inserting commercials or me taking them out) mess up the IVTC process, since removing or adding frames my change the telecine pattern? If so, how can I edit my videos in such a way that IVTC won't get confused?

[manono]

It may be a quick question, but there's no quick answer. Yes, the IVTC can get messed up. One thing you can do is to try and cut on only every 5th frame (5th before IVTC) to keep the cadence, or make sure the cadence coming out of the ad is the same as going in. But what happens when the source doesn't have a steady pattern? TV shows are usually shot on 24fps film, but often edited as already telecined interlaced 29.97fps video. And they don't care where they make the cuts or edits, as it's going to be broadcast in standard def (if indeed, that's the case), and will be interlaced 29.97fps (like your capture), and will play just fine. But if you want the best encoder efficiency, and if it was originally film, you'll want to return it to its original 23.976fps. The IVTCs will have a deinterlacer built in to catch those places where the cadence breaks and where interlaced frames might otherwise sneak through. With a good deinterlacer being used in a good IVTC (like AviSynth's TIVTC), you'll never know that a frame was deinterlaced around an advertisement or scene change. That's the theory, anyway.

[_MrC_]

Hi 4evrplan, I am the developer of AVStoDVD. I'll try to solve with you the problem.

I just tried out AVStoDVD and played the resulting disc (DVD-5) in my standalone. First I'd like to say that I love this tool, especially the fact that it allows you to edit the avisynth script.

Thanks. I think AVStoDVD strength is its flexibility in editing AVS script too.

The audio turned out perfectly synced when much other software I've tried messed this up (though it wasn't normalized

But you can easily add

audio = Normalize(audio, x.yz)

to your script :P

However, the motion in the video looked jittery or jumpy. I loaded up my source file in VirtualDubMod with the following script

AVISource("...")
SeparateFields()

and clicked through several frames one at a time. Sure enough, my field order is backwards.

Look at video stream info AVStoDVD gives to you. After fps what do you read? If "PPF", video stream is (encoded) progressive, if TFF/BFF is (encoded) interlaced top field first or bottom field first. In that case, try switching on "interlaced encoding" in MPEG2 Advanced Options: encoding time will be affected but quality should be improved.

Also, AVStoDVD seems to ignore some of the script commands, because I tried doing a "preview" with "SeparateFields()" in the script, and it didn't seem to change anything; I just got the full frame in the preview.

I assure that AVStoDVD previews exacly the script it finds when you press the preview button (thru VDM).

Hope these hints help. Let me know.



Bye

[4evrplan]

Thanks, MrC. That's another great thing about this forum, you actually get to hear from the developers.

I'm sorry to say I've given up on standalone playable DVDs for a while. I think I'll stick to XViDs for now, for two reasons. 1) I can fit more and longer videos on a single disk without compromising quality, and 2) AutoGK is such a comprehensive tool, that I never have to worry about the end result. It's output is always sufficient to my untrained eyes.

However, standard DVD format has the advantage of being able to play it pretty much anywhere I go, PC or standalone. So, assuming I do try again, and I'm sure to sooner or later, how would you recommend changing the script and/or program options to output progressive video instead of interlaced video? Quality is important, but I try to keep my encodes at about 4-5 hours on my computer (3.4 Ghz HT with 1 Gig of RAM and SATA hard drive).

Thanks again for your input and this great tool.

[_MrC_]

For best quality on progressive input stream, check "Very High Quality" and select HCenc as the default very high quality encoder. If the input stream is already good enough, don't do anything else. Otherwise it depends on the stream bad feature: I have no definitive recipes.

Hoping to hear your try soon



Bye

[4evrplan]

After a very long hiatus from working on this video, I've revisited it. I no longer have the original source, just the DVD I burned years ago. I will not inverse telecine it (it's 3:2), because it wasn't a clean source like a rip, rather it was a capture off of cable with dropped frames and breaks in the telecine pattern, and decimating it causes the audio sync to drift off. So, I'll just pretend it's progressive 29.97fps and treat it as such.

Here's the weird part though. I no longer believe the jumpiness was caused by wrong field order, or not just wrong field order anyway. Even when I know the field order is right (I've used SeparateFields() and watched at least a couple dozen frames in a row where the motion is correct), there's still an occasional sudden reversal of motion like there's one frame out of place. Ideally, I'd like to simply mask out these frames, but I don't know any way to identify them without stepping through the entire movie one frame at a time (not gonna happen). No idea what I messed up to cause this issue.

[jagabo]

After a very long hiatus from working on this video, I've revisited it. I no longer have the original source, just the DVD I burned years ago. I will not inverse telecine it (it's 3:2), because it wasn't a clean source like a rip, rather it was a capture off of cable with dropped frames and breaks in the telecine pattern, and decimating it causes the audio sync to drift off. So, I'll just pretend it's progressive 29.97fps and treat it as such.

A proper IVTC would not cause A/V sync problems.

Here's the weird part though. I no longer believe the jumpiness was caused by wrong field order, or not just wrong field order anyway. Even when I know the field order is right (I've used SeparateFields() and watched at least a couple dozen frames in a row where the motion is correct), there's still an occasional sudden reversal of motion like there's one frame out of place.

It's likely that the capture missed a frame so the previous frame was repeated. That causes a quick back and forth movement. You can easily check for this with VirtualDub. Apply the Bob doubler filter then step through the video with the left/right arrow keys. You'll see the input side shows the same interlaced frame for the duration of 4 fields, and the motion in the fields will jump back and forth in the output pane.

Ideally, I'd like to simply mask out these frames, but I don't know any way to identify them without stepping through the entire movie one frame at a time (not gonna happen). No idea what I messed up to cause this issue.

You might be able to search for duplicate frames with combing via a complex AviSynth script.

[4evrplan]

@jagabo

If it was just a duplicate frame here and there, it wouldn't bother me, but it actually seems to be a much earlier frame stuck in randomly. In fact, duplicating the previous frame over each of these earlier frames is the best way I can think of to mitigate the problem.

But, now that I'm thinking about it, I was seeing this problem after Telecide() and Decimate() (using the Decomb filter by Donald Graft). I couldn't swear to it that I was seeing this problem before the IVTC, so I'll have to go back and double check. Maybe Telecide() was detecting the pattern wrong and shifting frames around incorrectly?

But that brings me back to:

A proper IVTC would not cause A/V sync problems.

I've seen this same problem with what I consider a much cleaner source, a VHS. But, it also had dropped frames, 72 of them in a roughly 90 minute movie. I've since redone the capture on that one with no dropped frames, but for the sake of troubleshooting, it's useful to know. All I can figure is that Decimate() was removing the duplicate frames that were there due to being dropped and not just the ones left over from Telecide(). So how would I fix this?

EDIT: After going back and double checking without doing an IVTC, it still has the same problem, so I can't blame Telecide(). But, it's not consistently the same frame every time, so it must be the way the frames are being seeked and served. I had been using FFDShow via DirectShowSource(), but I switched to FFMPEGSource2(), and it's still doing the same thing.

[4evrplan]

See my edit above.

[jagabo]

If you're not already, I suggest you use DgIndex to index the source (honor pulldown flags), Mpeg2Source() to read it, and TFM().TDecimate() to IVTC.

As to the occasional out-of-place frames -- you'll have to post a short sample.

[4evrplan]

Yes! I just found some more information about processing mpeg2 sources, and they said the same thing. I just downloaded DGIndex and tried it out, and it has fixed the occasional random frame! It only took nearly five years* for me to figure this out (though most of that was me just ignoring it).

No word yet on whether this fixes the sync issue. I take it TFM() and TDecimate() are better than Telecide() and Decimate() for this task?

EDIT: I mean almost six years.

[jagabo]

No word yet on whether this fixes the sync issue. I take it TFM() and TDecimate() are better than Telecide() and Decimate() for this task?

I haven't used Telecide() and Decimate() in such a long time I don't remember how well they work or how flexible they are. I've been using TFM() and TDecimate() for years now. They work very well.

[4evrplan]

I won't see the final product for over ten hours (I'm running a motion compensated temporal smoothing filter as a last step, and it takes forever). But, my initial tests using jagabo's advice seem very promising, nice smooth progressive video at 23.976fps with synchronized audio. I had all but given up on this video completely.

Thanks jagabo! And thanks to the authors of DGIndex and TIVTC!

[manono]

And thanks to the authors of DGIndex and TIVTC!

The creator of DGIndex also developed Decomb (Telecide and Decimate). Much if its code was used in TIVTC (TFM and TDecimate), the results should be very similar, and Decomb's IVTC won't create any out-of-synch audio either. TIVTC's default post-processor is better, and the default settings may be slightly different, but if one works well, the other, perhaps with some settings tweaking, should work just as well. TIVTC may handle orphaned fields better but the results of both should be satisfactory.