4:2 pulldown?

[ADigitalMan]

I am attempting to convert video, but the results so far continue to be crappy. I did a frame by frame analysis and the interlacing pattern seems to be 4 normal, 2 jaggy, repeat. Just when I was getting comfortable with 3:2 pulldown, this has been sprung.

The source is a m2v file. I am opening in VirtualDub MPEG2, cropping, resizing, and capturing to AVI (converting FF to Anamorphic). I am then attempting to convert back to M2V using TMPGEnc 2.5. The 3:2 pulldown flag was checked, but the output still gets lots of jagged frame. I believe this is because of the strange interlacing pattern. Any ideas?

[shteeve]

i was just up against a similar problem and it was resolved quite nicely in this thread:

https://www.videohelp.com/forum/viewtopic.php?t=275109&highlight=

if you're new to avisynth, its not that intimidating, my script ended up being this simple:

loadplugin("PluginPath\Decomb521.dll")
avisource("FootagePath\footage.avi")
converttoyv12()
telecide(order=0, guide=0,post=2,vthresh=30,show=false)
decimate()

[manono]

Hi-

Yes, but yours had the standard 3 clean, 2 interlaced pattern, shteeve, while his is 4:2. Yours was standard 23.976fps, while his is 24.975fps. After a standard IVTC, yours could be handled by a standard Pulldown, while his can't.

As I see it, you have a three choices, ADigitalMan, assuming your analysis is correct.
1. Keep it interlaced 29.97fps
2. Restore it to 24.975fps, and run DGPulldown afterwards for 24.975->29.97.
3. Restore it to 24.975fps, and then slow it to 23.976fps, run regular Pulldown, and also slow the audio to match.

As far as I know, AviSynth is the only way to handle options 2 and 3. For options 2 and 3, you may be able to use a common:

Telecide(Order=1,Guide=2).Decimate(6)

That assumes it's not blended. Most of the ones like this I've seen have blended fields from being a lousy PAL to NTSC conversion. To be sure, you have to examine the fields by putting on a "Smart Bobber" such as:

KernelBob(Order=1)

or

TDeint(Mode=1)

If they are blended, then RePAL is your best bet to restore the progressive frames. If you need further help, then ask. If you have an unprocessed sample you can make available, I'd be glad to have a look at it.

I have no idea why you're first converting to AVI and then back to MPEG-2, but I guess you have your own reasons.

[junkmalle]

TMPGEnc can IVTC any consistent pulldown pattern. From the IVTC dialog right click on a frame and select "Deal after this frame according to selected pattern" and enter whatever pattern you want. For it to work well ther will have to be no breaks in the pattern.

The pattern you're seeing, 4 clear frames followed by two interlaced, is not 4:2, it's 3:2:3:2:2. It's used to take 25 fps PAL video directly to NTSC 29.97. I've used TMPGEnc to IVTC this type of material back to 25 fps progressive. The pattern you'll want to enter into TMPGEnc IVTC dialog is 101010010100 or some rotation thereof.

[ADigitalMan]

Thanks to all for your help so far. I'm glad it's not just my understanding that's messed up.

It's still not converting right. I did another frame-by-frame analysis in VirtualDub, and it seemed to be 3:3 for a while, then switched to 4:2.

I did a straight conversion in virtualdub to uncompressed avi, then back to M2V via TMPGEnc trying junkmalle's settings. While cleaner, it still wasn't clean.

The original source probably was a PAL video, though the disc I'm taking the M2V file is NTSC. Any ideas on how I can give you all a good analysis of the video beyond what my eyes are seeing? Given that they're not being terribly trustworthy?

Edit: I just analyzed the pattern and from where I want to start, the pattern seems to follow 010010101001 where 0=clean frame and 1=jagged frame. Does this mean anything to anybody else?

[junkmalle]

It's still not converting right. I did another frame-by-frame analysis in VirtualDub, and it seemed to be 3:3 for a while, then switched to 4:2.

You won't be able to perfectly IVTC the whole video in one chunk. You'll have to break it up into sections with unbroken telecine patterns.

I just analyzed the pattern and from where I want to start, the pattern seems to follow 010010101001 where 0=clean frame and 1=jagged frame. Does this mean anything to anybody else?

You have it backwards. In TMPGEnc 1 is a clean frame 0 is an interlaced frame. And that is the same pattern I gave, just rotated.

mine:      10101001 0100
yours: 010010101001 010010101001

[ADigitalMan]

Thanks for the clarification. Swapping the 1 and 0 does the trick for a while, but then it breaks again. Will this simply be a tedious process of marking each frame to IVTC manually?

[junkmalle]

Thanks for the clarification. Swapping the 1 and 0 does the trick for a while, but then it breaks again.

Then there's a break in the telecine pattern. I've IVTC'd a lot of material. Sometimes there are no breaks through the entire video. Sometimes there's a break at every camera change. A lot of made-for-tv material is shot on film, teleinced to video, then edited. When editing they don't pay any attention to the telecine pattern.

Will this simply be a tedious process of marking each frame to IVTC manually?

Could be. You're workflow will be something like this:

Get everything set up as you want in TMPGEnc. Set the compression mode to single pass CBR or CQ. Then:

1) Set the IVTC pattern for the first section.
2) Start saving.
3) When you start seeing interalced frames slip through stop TMPGEnc.
4) Open the new segment in an editor that doesn't reencode.
5) Find the first interlaced frame.
6) Save from the start of the file to there as FILE01.MPG.
7) Find the same frame in TMPGEnc.
8) Mark that as the starting frame.
9) Go back to #1, creating FILE02.MPG, FILE03.MPG,etc

When you are finished join all the segments together with the non-encoding editor.

[manono]

Come on, junkmalle, who actually uses TMPGEnc for this kind of thing anymore? It's obsolete and incredibly slow. You're asking him to work his way through the entire video, noting where the pattern changes, and then finding the new pattern. It's simple and automated with one of the AviSynth IVTCs:

LoadPlugin("C:\Path\To\DGDecode.dll")
LoadPlugin("C:\Path\To\Decomb.dll")
MPEG2Source("C:\Path\To\Movie.d2v").
Telecide(Order=1,Guide=2)
Decimate(6)

Or you can use SmartDecimate or TIVTC. He has only to install AviSynth, the Decomb.dll, and the DGMPGDec package with the DGDecode.dll and DGIndex. There are excellent help files for getting started in AviSynth inside the DGMPGDec package, and for the use of Telecide/Decimate inside the Decomb package.

[junkmalle]

In my experience none of the automated IVTC tools give perfect results.

[ADigitalMan]

Manono, I installed the two DLLs you referenced and created the script you mentioned. I'm a total AVISynth newbie, but when I try to run the script (starting in Nero Showtime, because that's what's available in my rightclick menu) I get errors where it doesn't like the DLLs. I think I've got the latest version of both, and certainly the latest AVIScript. Ideas?

Thanks for your patience, all of you. I'll get the hang of it.

edit: to be more specific, the two dlls produce the error message that they are not AVISynth 1.0 plugins. Curious, given that AVISynth 2.0.8 is installed.

[manono]

Hi-

Curious, given that AVISynth 2.0.8 is installed.

That's old, get 2.5.5 or 2.5.6:

http://sourceforge.net/project/showfiles.php?group_id=57023

Uninstall the 2.0.8 version. Decomb521 requires one of the 2.5.x versions. To later confirm that you're running 2.5.5, make an .avs with only this in it:

Version()

and open it in VDubMod. I think that the Decomb.dll is really called Decomb521.dll (if you got the most recent version). I just renamed it. To get real error messages, open the resulting .avs in VDubMod. I don't use Nero Showtime, so I don't know what it does. Always test your scripts in VDubMod before doing any encoding, like in CCE or whatever.

And did you make the Movie.d2v Project File using DGIndex (using Video->Field Operation->Honor Pulldown Flags)?

Another tip, by going the AviSynth route, rather than TMPGEnc, you can use Video->Fast Recompress in VDubMod, rather than Full Processing Mode. The encoding will go much faster (depending partly on the colorspace of your original M2V), and produce better quality because of no colorspace conversions. To find the colorspace of the original M2V (you hope it's YV12), add :

Info()

to your script temporarily.

Another tip, after you've got it running OK, you might try and turn off the deinterlacer which is on by default to catch any stray interlacing that may remain after the IVTC:

Telecide(Order=1,Guide=3,Post=0)#Note: I was wrong before about Guide=2 for PAL to NTSC
Decimate(6)

You'll have to check it pretty carefully, though, as doing that can be risky. But the benefit of doing that if possible is significantly faster encoding.

If you get serious about using AviSynth for frameserving, then you should go and study everything at:

http://www.avisynth.org/

It'll pay huge dividends in the end.

In my experience none of the automated IVTC tools give perfect results.

And what if you miss a pattern change when doing it through TMPGEnc, later find you've still got interlacing in some section(s), and have to do it all over again. My experience says that a consistent pattern all the way through is very rare. Sure, the IVTC's often have to be tweaked, and sometimes if one doesn't do the job right, you switch to another. But the benefits still make it miles ahead of the TMPGEnc manual method (in my opinion).

[vhelp]

junkmalle wrote:
>> want to enter into TMPGEnc IVTC dialog is 101010010100 or some rotation thereof.

I use this method for IVTC that are clean. (not the number you posted
though) and I want to point out something here, that is *key* to successful
pattern-matching through this method.

Always start at the first Progressive frame. Never in the middle, nor on
a Interlace frame. These numbers are sequences to the pattern.

Using the number from junkmalle 's example ...

If you start 101010010100 you will have pattern breakage, hence
interlace in your source, and you will not have every frame progressive.

-vhelp 3477

[junkmalle]

I want to point out something here, that is *key* to successful
pattern-matching through this method.

Always start at the first Progressive frame. Never in the middle, nor on
a Interlace frame. These numbers are sequences to the pattern.

I always start on the first frame and use whatever rotation of the pattern is appropriate. To IVTC 3:2:2:3:2 material:

101010010100
010101001010
001010100101
100101010010
010010101001
101001010100
010100101010
001010010101
100101001010
010010100101
101001010010
010100101001

for normal 3:2 material:

10010
01001
10100
01010
00101

[vhelp]

TMPGenc ...

The pattern I use for 3:2 telecine work is this one:

--> Deal with after.. 10100

This is the one that always works for me every time, (on the first
progressive frame) when the source Telecine is clean, with no breaks.

I don't bother with other pattern numbers because I don't really
use them in my projects. Just 3:2 onces' for Film source. I consider
anything else that is out of the ordinary in Film, to be too tained
to bother. There is where AVIsynth would come in handy through the
use of the many plugin's available. But, I agree that you can do
anything in TMPGenc (pattern 'wise) but it is slow in finding out if
all those unknown and strange breaks and patterns are, manually speaking.

However, I feel that I would probably give TMPG's method a go, if the
issue came my way, and I was feeling up to the challenge.

-vhelp 3478

[ADigitalMan]

Well, I've got the 2.5 AVISynth installed now (earlier one must have been from an outdated link or something) and tried at great length last night to get things running. Unfortunately, still a lot of errors trying a lot of different plugins. So far I've tried:

Decomb521
DGDecode
GreedyHMA
SmartDecimate
SmoothDeinterlacer
TCPDeliver
TIVTC

Your continued help is appreciated, as is your continued patience with my newbie status on AVISynth. Clearly I'm not a total novice, but with this program (and scripting in general) I'm a babe in the woods.

edit: I think my problem is MUCH more general ... I can't seem to open M2V files currently in AVISynth. I am able to open the files that have been converted to AVI, though the colorspace of the AVI is in RGB.

Should I do what was working and convert all the M2V files to AVI first, or should I be opening the AVS scripts in some program that is capable of opening M2V files? And if the latter, what program? The latest virtualDubMod crashes when I open AVS files.

[autodidact]

You mention opening M2V files with AVISynth. If you are processing using the example script provided by manono you should be opening a D2V file. This is the result of first running your M2V file through DGIndex. You then open the D2V file in AVISynth using the DGDecode plugin as shown in the example. The versions of the two programs, DGIndex and DGDecode, should match for compatibility.

[manono]

Thanks autodidact-

Yes, quoting from my previous (long) reply:

And did you make the Movie.d2v Project File using DGIndex (using Video->Field Operation->Honor Pulldown Flags)?

There was so much in there that maybe it was difficult to separate the wheat from the chaff. But that's an important step. It's the D2V file that DGIndex creates that indexes the M2V, and the DGDecode.dll that actually decodes and serves up the frames to the encoder. They work together, and as autodidact mentioned, you must have the same versions of each (they come together in the DGMPGDec package, along with some very worthwhile help files).

Try and get Decomb running first. If you get errors you can't figure out, break down the script file to it's simplest parts, and then add stuff back into it one-by-one until you pinpoint the problem. So, after you get the D2V created, and you try and open it, but get an error message you don't understand, comment out (put an "#" in front of) the added stuff, like so:

LoadPlugin("C:\Path\To\DGDecode.dll")
#LoadPlugin("C:\Path\To\Decomb521.dll")
MPEG2Source("C:\Path\To\Movie.d2v").
#Telecide(Order=1,Guide=3)
#Decimate(6)

Personally, I usually use SmartDecimate, although Decomb is way better known, and almost universally used. Also, SmartDecimate can be a bit tricky to get going.

Don't worry, you'll get the hang of it. By the way, if you've ever used Gordian Knot, it creates the AviSynth script files for you after you open the D2V in it. If you continue to have trouble, you might give it a try. You'll just have to edit the script it creates for your special case of Telecide/Decimate.

Should I do what was working and convert all the M2V files to AVI first?

No.