3:2 Pulldown--Do it in AVISynth, DVD2AVI or in Encoder?

[CubDukat]

Recently I have been experimenting with 3:2 pulldown with my HD downconversions in CCE SP.

I'm using the old approach of using an AVISynth script to load the video into the encoder, and both the 720p and 1080i scripts have sections for IVTC, but I have them commented out at present.

I was wondering which would handle pulldown better--CCE SP, AVISynth or DVD2AVI? I've heard good things about all of them.

Also, the 720p files are being converted to 480i in the AVISynth script. I'm able to see where the pulldown was done in the resulting interlaced file. If I reactivate the decimate (?) section of the script, will I have problems with jerky motion? Or would I be better off encoding both formats as progressive instead of interlaced?

Hopefully I'm making sense here.

[manono]

Hi-

Hopefully I'm making sense here.

Not really. If it's a movie, then certainly you want to return it to its original 23.976fps before encoding. With a 1080i stream, this will mean a full IVTC. With a 720p stream, this usually just means pulling out the dupe frames. Then run pulldown during the encode or afterwards to get the DVD to output 29.97fps. Since these streams are usually encoded as 29.97fps, then a Force Film can't be done in DGIndex (don't use DVD2AVI as it's been replaced by DGIndex). So, don"t Force Film in DGIndex and don't allow CCE to do the IVTC. Do it in AviSynth before sending it to the encoder.

If I reactivate the decimate (?) section of the script, will I have problems with jerky motion?

No, because if you remove the dupe frames, when output for your HDTV, they'll just be put back again. It should play on your progressive display with the same smoothness. They should play even more smoothly on an interlaced display. What you'll gain is compression efficiency, as you'll be encoding 20% fewer frames.

Or would I be better off encoding both formats as progressive instead of interlaced?

Why on earth would anyone purposely encode interlaced when it was progressive originally? Interlacing is much harder to compress, and you're back to encoding those 25% more frames. If it was originally created at 24fps progressive, then, in my opinion, you should always encode it at 23.976fps progressive, no matter how the transport stream was broadcast and captured.

[pijetro]

I used decomb.dll a long time ago, and would prefer going this route. You can see the results at the frame level within VirtualDub, and watch over any sort of irregular patterns. You can't do that through CCE.
Even at its' default settings, IVTC works awesome. This tool was especially usefull, since you could totally customize the settings. This suited many of the people who ran into hybrid videos that contained different patterns..

Have you ever read the Decomb.dll readme??

[Guest34343]

Since these streams are usually encoded as 29.97fps, then a Force Film can't be done in DGIndex

You lost me there Tom. What is "these streams"? If a stream has consistent soft 3:2 pulldown, then you can use Force Film in DGIndex. Maybe if you clarify your point, I'll get it. Thank you.

[manono]

Hi Don-

Aren't Hi-Def transport streams hard telecined and broadcast at 29.97fps or 59.94fps (he says having never captured anything in his life)? Am I assuming too much by thinking that CubDukat is capturing his Hi-Def streams?

If they're sometimes soft telecined, it's news to me and I'll happily stand corrected. Can 1080i and 720p TS ever be Force Filmed?

[CubDukat]

Yeah, I'm getting these streams from my HD tuner card.

At least in the case of the 720p streams, I'm converting them to interlaced mainly because when I was using the script without commenting out the IVTC sections, it would work fine for film-originated material, but video-originated material (Indy 500, the Oscars, etc.) had jerky motion and didn't look like they should.

I'll try experimenting with putting the IVTC section back in. I may end up having two scripts--one for film-based shows and another for video-based shows.

I already tried experimenting with using inverse 3:2 pulldown in CCE SP, but the results have been somewhat disappointing. I experimented with a clip from ER representing 1080i and one from Lost for 720p. The ER clip came out very nice and sharp, with natural motion, while the Lost clip was jerky and not quite as sharp.

Basically what I'm looking at is being able to put out near-pro quality discs.

So, just so that I have it right, since the end result will be a DVD (which is why I'm keeping everything interlaced), I should take out the commenting in my AVISynth scripts and let it do the IVTC?

[manono]

Hi-

I should take out the commenting in my AVISynth scripts and let it do the IVTC?

I'll defer to neuron2 if and when he comes back. First you have to determine what you have. Some (all?) of the 720p are 59.94fps, and a straight IVTC won't restore them to 23.976fps. Ordinarily you decimate by 2 (or do a SelectEven() or some such) before applying IVTC. You can't blindly IVTC anything. In any event, if you're not sure how to proceed, if you can upload a small unprocessed part of the video, I or neuron2 or someone else can have a look. If you don't know how or where to do that, then you can do it for free here:

http://rapidshare.de/

[Guest34343]

1080i can be soft telecined. You won't see much of it because if you capture from a component output, the pulldown is already applied and the capture device just uses interlaced encoding for everything. But if you capture transport streams, you sometimes see it. Remember, with HD it's even more important not to broadcast duplicated fields.

I'm just cautioning to be sure what you have.

[edDV]

720p as broadcast on ABC is 59.94 frames per second. The original 23.976 film frames are repeated in a 3:2 pattern to build up to 59.94*.

You can't use normal IVTC or decimate by 2. You need to detect and remove the 3:2 repeat frames.

Simple decimate by 2 will get you a studder

AAABBCCCDD
A_A_B_C_D_
or
AABBBCCDDD
A_B_B_C_D_

What you want is
A__B_C__D_
@23.976

Live 720p is 59.94 frames per second from the camera. If you need to compress and will accept 29.97 fps progressive flicker on a good TV, then you can decimate by 2 to 720/29.97 progressive or do a reverse bob to make true 720/29.97i without flicker.


* this is the preferred format used by ABC but they do accept 1080i material for conversion. This is often the case for commercials which are almost always shot on film and distributed 1080i. HDCAM VTR's can interpolate 720p/59.94 from a 1080i tape but reversing this for film can get complicated.

1080i is normally telecined just like 480i. Conversion to 720p is most likely accomplished by a bob (line double) to 1080p/59.94 followed by a downscale to 1280x720. Or it could be done by IVTC to 1080p/23.976 followed by a downscale to 1280x720/23.976 and then 3:2 frame repeated.

When local HDTV origination starts, expect alot of this cross conversion.

[CubDukat]

Hi-

I should take out the commenting in my AVISynth scripts and let it do the IVTC?

I'll defer to neuron2 if and when he comes back. First you have to determine what you have. Some (all?) of the 720p are 59.94fps, and a straight IVTC won't restore them to 23.976fps. Ordinarily you decimate by 2 (or do a SelectEven() or some such) before applying IVTC. You can't blindly IVTC anything. In any event, if you're not sure how to proceed, if you can upload a small unprocessed part of the video, I or neuron2 or someone else can have a look. If you don't know how or where to do that, then you can do it for free here:

http://rapidshare.de/

I'll try and do that.

I know for sure that the material is 23.976 because I've stepped frame-by-frame through the actual files by loading the .avs files into VDubMod, and I can see the 3:2 patterns.

[CubDukat]

Oh, yeah, one other thing. I'll also be posting the 1080i script I'm using. I did a brief test on the Lost segment with IVTC re-enabled, and it looked really good.

Unfortunately the ER test didn't happen. I couldn't even load it into VDubMod. It gave me a "Telecide does not have a named argument 'Reverse,'" or something to that effect. I think what I may end up doing is just doing the IVTC in either DVD2AVI (I don't use DGIndex because programs don't seem to like the .d2v files it puts out--or at least I don't have it set up so that it would work) or CCE SP.

[manono]

Hi-

I don't use DGIndex because programs don't seem to like the .d2v files it puts out

You have to use DGIndex together with the DGDecode.dll that comes in the same DGMPGDec package, and not any old DGDecode.dll or MPEG2Dec.dll you happen to have lying around. A basic sample script might be:

LoadPlugin("C:\Path\To\DGDecode.dll")
MPEG2Source("C:\Path\To\Video.d2v")

I can'thelp with the Decomb problem unless you're more specific about the error message. You might also post the script that causes the error message, and the version of Decomb you're using.

[CubDukat]

Here's a copy of the AVISynth script I'm using for 1080i. Other than the addition for BT709toBT601, this script is the same was the Inmatrix one:

# Modify the path so that it finds your decomb dll file #
LoadPlugin("c:\program files\AVISynth 2.5\plugins\decomb521.dll")

# Modify the path so that it finds your MPEG2DEC dll file #
LoadPlugin("C:\Program Files\AVISynth 2.5\plugins\MPEG2DEC3.dll")
LoadPlugin("C:\Program Files\AVISynth 2.5\plugins\BT709ToBT601.dll")

# Modify the path so that it finds the d2v file you created using dvd2avi #
MPEG2Source("E:\Video\RawCaptures\ER--Two Ships (Pt1).d2v")

BT709ToBT601()

# This is the IVTC line. Consult your decomb readme file for tweaking #
Telecide(reverse=false,swap=false,firstlast=false, post=true,threshold=15,dthreshold=9,blend=true,chr oma=false,y0=0,y1=0)

# Leave this alone. This will remove one out of every five frames, converting the fps to 24 fps #
#Decimate(cycle=5,mode=0,threshold=0)
LanczosResize(720, 480)
Sharpen(0.7)


The exact error message is "Telecide does not have a named argument "reverse."

I could live with doing the IVTC in CCE SP, but I'd like to do as much as possible outside the program so that I can speed up render times.

[Guest34343]

The parameters you gave for Telecide() are for the legacy 4.05 version. The current version has completely different options. Whatever guide you used is ancient and needs to be updated.

Just change the line to Telecide() without options and take it from there.

Also, you have commented out the Decimate() call. Why? It's not a complete IVTC without that.

Finally, mpeg2dec3.dll has been superceded by dgdecode.dll.

[CubDukat]

The parameters you gave for Telecide() are for the legacy 4.05 version. The current version has completely different options. Whatever guide you used is ancient and needs to be updated.

Just change the line to Telecide() without options and take it from there.

Also, you have commented out the Decimate() call. Why? It's not a complete IVTC without that.

Finally, mpeg2dec3.dll has been superceded by dgdecode.dll.

I tried using Telecide() with nothing in between the parentheses, but it gave me the following error:

"Telecide: You must specify the order parameter (0=bff, 1=tff)."

I just downloaded 5.2.2 as well as DGMPGDec and I'm going to try them out. Maybe that's why DGIndex didn't work, because I used mpeg2dec3 instead of the new one. After I set up my current method, I never check back to see if things got updated. I suppose that would be a good idea

As for having the Decimate command commented out, I guess I forgot to uncomment that. I've done that in the master script now. I may just end up rolling my own script in the end; it'll be my first.

As soon as I set up a rapidshare account, I'll upload the final result of my Lost encode.

[CubDukat]

Okar, I just modified my script to take into account the new plugins, but it still doesn't work.

This is the revised script:

# Modify the path so that it finds your decomb dll file #
LoadPlugin("c:\program files\AVISynth 2.5\plugins\decomb.dll")

# Modify the path so that it finds your MPEG2DEC dll file #
LoadPlugin("C:\Program Files\AVISynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AVISynth 2.5\plugins\BT709ToBT601.dll")

# Modify the path so that it finds the d2v file you created using dvd2avi #
MPEG2Source("E:\Video\RawCaptures\ER--Two Ships (Pt4--DMPG).d2v")

BT709ToBT601()
AssumeBFF()

# This is the IVTC line. Consult your decomb readme file for tweaking #
Telecide()

# Leave this alone. This will remove one out of every five frames, converting the fps to 24 fps #
Decimate(cycle=5,mode=0,threshold=0)
LanczosResize(720, 480)
Sharpen(0.7)

The problems started with loading it into DGIndex. It didn't detect the PIDs, but that was easily resolved. The next problem came when I saved out the .d2v file. When I attempted to open it up in VDumMod, it said "MPEG2Source: The input file is not a D2V project file."

I checked the locations of everything and the script, and everything was correct.

I had heard compatibility rumors with DGIndex, so until this gets figured out, I'm going back to the old way. i don't have the patience to deal with it right now; I have two 2-hour season finales to clear off my HD today.

[CubDukat]

Found the problem. When I changed DGDecode.dll back to the old MPEG2Dec3.fll, it loaded up again.

I'd like to use DGDecode.dll, but I can't see what's going wrong here.

[manono]

Hi-

Well, thanks to neuron2 you've been making progress since I last checked in.

When I changed DGDecode.dll back to the old MPEG2Dec3.fll, it loaded up again.

So, you must be using the original D2V made with DVD2AVI. When you use the DGDecode.dll, you have to have made the D2V using the DGIndex that comes along with DGDecode.dll. If you make a fresh D2V using DGIndex and load the DGDecode.dll again, you should be good to go.

This might seem pretty frustrating at the moment, but once you get squared away, it should be smooth sailing.

[CubDukat]

So, you must be using the original D2V made with DVD2AVI. When you use the DGDecode.dll, you have to have made the D2V using the DGIndex that comes along with DGDecode.dll. If you make a fresh D2V using DGIndex and load the DGDecode.dll again, you should be good to go.

Actually, it did that with the DGIndex-generated file. It must be something in the DG setup somewhere, but I can't figure out what.

I even tried using DGIndex to "fix" the d2v file, but that didn't work either.

I'll be fooling around with it later on. I burnt the Lost season finale not too long ago, and now I'm working on 24.

I'm also trying to find a way to shorten my workflow, since the individual parts go through at least five separate programs before they get burnt. I really want to maintain the highest quality possible.

Right now my workflow is as follows:

1. Record and rough-edit .tp file in FusionHDTV2
2. Demux in DVD2AVI
3a. Plug d2v file into AVISynth script for encoding in CCE SP/TMPGEnc Xpress (whichever one doesn't refuse to render it)
3b. Correct AC-3 2.0 issues in ProjectX and remove delay in Delaycut
4. Mux AC-3 file and video in TMPGenc Xpress.
5. Make DVD with TMPG DVD Author and burn.

I've tried using HDTVtoMPEG2 to turn it into an MPEG-2 file, but that doesn't work well, and neither does removing commercials in that program. I've tried taking the .tp files directly into Nero Vision, but it hangs up during rendering.

Maybe when HD-DVD and Blu-Ray burners become commonplace, there'll be an easier solution--if we aren't DRM'ed to hell and back by then.

[Guest34343]

[deleted]

[manono]

I wonder what neuron2 deleted...

It must be something in the DG setup somewhere, but I can't figure out what.

Try this. Make sure you have the latest DGMPGDec package. Make the D2V from the DGIndex that comes in that package. Use the basic script I gave above:

LoadPlugin("C:\Path\To\DGDecode.dll")
MPEG2Source("C:\Path\To\Video.d2v")

And check very carefully the paths. Make sure you're pointing to the DGDecode.dll that comes in the same package, and make sure the D2V is the one you made from the DGIndex in that same package. Open just those 2 lines in VDub(Mod). Once it opens and you can scroll around, then add back the other stuff, Telecide/Decimate/filters/resizing/whatever you had before. Also, have you read the docs that come with the DGMPGDec package? There are 3 of them, and they're all good reading, particularly the QuickStart Guide.

I'm also trying to find a way to shorten my workflow...

Well, although I don't use any of the Pegasus products, I'm almost positive steps 4 and 5 can be combined. That is, there's no point in muxing to MPG (?) before authoring, since one of the things that an authoring app does is to mux audio and video. Have you tried sending the elementary streams (separate MPV/M2V and AC3 streams) directly to TDA for authoring?

[CubDukat]

Yeah, I do that every so often when I don't feel like waiting for TMPGEnc XPress to mux it.

Usually I mux the file for the beginning and ending of shows so that if I want I can take it into another program like MediaStudio Pro to add transitions. I used to use Premiere Pro to import the two separate elements and edit them there, but it doesn't import AC-3 files (I don't know if that's still the case, now that Dolby is more or less giving away licenses to include encoders in editing software).

I had considered using ProjectX for the editing and demuxing, but I can't figure out how it works. I tried using the doom9 tutorial, but it hasn't been updated for the new version.

I'll try the script you gave me, but I wonder if it's not something in the DGDecode.dll file itself. Like I had said before, I heard that a lot of people had tried DGIndex for converting ATSC, but that they eventually went back to DVD2AVI 1.76tr, since that seems to be the version that works best with ATSC files.

[CubDukat]

Okay, i just tried your example. Before that I copied DGIndex and DGMPGDec to my GordianKnot folder, and the DGDecode.dll file to my AVISynth plugins folder, where the old MPEG2Dec3.dll was located.

When I did the same procedure and attempted to load the .avs file into VDubMod to verifiy it worked, it gave me the following error message:

"MPEG2Source: DGIndex/DGEncode mismatch. You are picking up a version of DGDecode, possibly from your plugins directory, that does not match the version of DGIndex used to make the d2v file."

Since I copied the newer version to the corresponding folder in GordianKnot, it should have overwritten the previous version, so all versions of DGIndex and DGDecode should be exactly the same.

I'm going to try changing the path to the same one that the files came out of, which I saved to my desktop. If that doesn't work, who knows?

[CubDukat]

Solved the problem. Apparently DGIndex wasn't copied over. I finally got it up and running perfectly now. Thanks for the help.
I'll be posting the results from my "Lost" and "ER" encodes later on today.

[manono]

Ah yeah, maybe partly a GKnot problem. If you ever use GKnot to give you the script, in the Program Paths Tab you can point to a different location for DGIndex. However, I just replace the entire DGMPGDec folder in GKnot from time to time, when neuron2 comes out with a new version, so I don't have to mess with the paths. If you have these things scattered around in different places, it can get very confusing very quickly.

Also, another thing I find handy is to set up some template .avs's for the common different kinds of videos with which I work. Then I just have to tweak a couple of things and I'm ready to encode. DGIndex can also generate the .avs's for you via similar templates, but I haven't tried that one out yet, as I use my own.

[MaxBlack]

I have not read this thread closely, but the "ER" aspect caught my eye, as I have had lots of trouble down-converting these from 1080i myself. I finally found a script that looked good, but I understand that what I'm doing is throwing away half the vertical resolution of the show:

MPEG2Source("D:\ER.d2v")
SeparateFields().SelectEven()
Decimate(5)
Crop(2,4,-2,-10)
BilinearResize(656,368)

But this works well enough for me, even when I downsize to 1280x720 it looks good.

Note too I found that unless I cropped AFTER the field ops, the result had some odd color ghosting--maybe I got the mods wrong for the resize filter, I dunno. The experts here will ding me for not working harder at this, but I'm mostly "flying blind" cuz I don't understand the filter innards very well (yet).

I will say that THE BEST way to analyze these things IMO is to make a project/d2v file with DGindex, then an "Inspect for Motion" script e.g.

MPEG2Source("D:\Texas Ranch House - The Cookie Crumbles.d2v")
AssumeTFF() #or whatever "parse d2v" tells you
SeparateFields()

and open it with VdubMod and step thru the video frame-by-frame. I found (though I read this a thousand times it took forever to sink-in) that this is the best way to see what's going on, but not only that, you can now add to your script, step-thru the video again, and see the operation of your script on the video (tho you have to be patient if the script is decimating or whatever).

All this is Magic to me; I have the utmost reverence & respect for neuron2 (DGindex, wow) and of course manono really knows his stuff...

[MaxBlack]

Right now my workflow is as follows:

1. Record and rough-edit .tp file in FusionHDTV2
2. Demux in DVD2AVI
3a. Plug d2v file into AVISynth script for encoding in CCE SP/TMPGEnc Xpress (whichever one doesn't refuse to render it)
3b. Correct AC-3 2.0 issues in ProjectX and remove delay in Delaycut
4. Mux AC-3 file and video in TMPGenc Xpress.
5. Make DVD with TMPG DVD Author and burn.

I, too, have had problems w/HDTV2MPEG2 and ended-up buying VideoReDo Plus (VRD+) instead--this is absolutely a must-have if you are using transport streams I think. My workflows:

Convert HDTV ts to DVD:

1. Open ts in VRD+ and edit-out commercials; save as transport stream
2. Open with HDTV2DVD and make a VIDEO_TS for burning to DVD

Convert SDTV ts (already DVD-compliant) to DVD

1. Open ts in VRD+, edit, and save as Elementary Streams
2. Import to MuxMan 0.15p and make a VIDEO_TS for burning to DVD

Convert Transport Streams to XviD:

1. Edit ts w/VRD+
2. Convert w/AutoGK, or StaxRip (using DGindex)

I've found ProjectX demuxes ABC programs OK, but for some reason doesn't do NBC, you gotta use DGindex to demux the audio. Works all day every day!

[CubDukat]

Thanks for the assistance.

I have a couple weeks to experiment on NBC streams because "ER" isn't on for at least two weeks. I might experiment on a few eps of "My Name is Earl" tonight.

As promised, I finally was able to post an example of one of my 720p encodes. This is the final part of last night's "Lost" episode, and it was done with Decimate enabled in the AVISynth script.

Note that it's in two parts in order to get it loaded onto rapidshare.de:

http://rapidshare.de/files/22565204/Lost--Adrift_Pt4_.part1.rar.html
http://rapidshare.de/files/22566467/Lost--Adrift_Pt4_.part2.rar.html

If you don't already have it, you'll need WinRAR to get them back together.

These were encoded in CCE SP using 1-pass VBR with the bitrate set at 0-8000 and a quantization level of 60. I was very surprised at the quality of the end result because the average bitrate doesn't even break 4000 for the majority of the show. Despite being rather cranky, CCE SP has become my renderer of choice.

Naturally I kept the Dolby 5.1 tracks, so if you don't have a way to decode Dolby Digital, you're out of luck.

Anyways, enjoy...And please let me know what you think. I'm always looking for ways to improve my encodes.

[manono]

I got one of the rars, but can't get the second one:

You have downloaded 20076 KB. Want to download more?
Get your own Premium-account now! Instant download-access! (Or wait 81 minutes)

If you had rar'd using store, I would have been able to get see it anyway, by renaming it, but I think you actually compressed them a little bit. One would have been plenty anyway to see what you did, if it had the right amount of movement. I'll try again later on to get the other one.

[manono]

Hi-

I went back later on and got the other rar. Nice looking video there. You got all the dupes out, as near as I can tell. One thing, though. You encoded it as progressive, and therefore you should be using zigzag scanning, rather than the alternate scanning you used. Alternate scanning is for interlaced encoding. I don't know if it's a big deal or not, but just thought you should know.

Also, personally, I'd use a much higher max bitrate than 8000. but there are plenty around here that would disagree.