HDTV to MPEG (with IVTC) - simplest option?

[Tiernan]

I've been doing video processing for a number of years now, first MPG captures to VCD, then to DVD, and for the past couple of years have been doing HD captures (.TS streams (1080i and 720p)) and converted them, first with TMPegEnc, then FFMpeg (command-line), and more recently with WinFF.

Given the volume of HD I've been capturing, I've started to dabble with moving some material that is not likely to see any kind of HD release (Flash Gordon, for one) to MKV form. In the process, I'm getting more used to the relatively complicated way that the MP4/AC3/Sub streams get resized/IVTC'd/processed and muxed into the MKV container, in my case via MeGUI. While that process is pretty cumbersome, i've been very impressed with the results, and it's made me think more about more properly processing the .TS HD files I've been capturing into MPEG2/DVD form. The 2-pass processing done in WinFF (via FFMpeg, of course) has been very impressive, but I'm keenly aware that I'm wasting quite a bit of space by having processed them as NTSC Video, and not having IVTC'd them back to film (which should be the source format for most TV i'm capturing).

Having said all this, here's my question: given that MeGUI doesn't handle MPEG2 conversion, what does everyone recommend as an equivalent GUI/App setup to handle .TS to 24fps MPEG2 (for DVD authoring)?

[manono]

Having said all this, here's my question: given that MeGUI doesn't handle MPEG2 conversion, what does everyone recommend as an equivalent GUI/App setup to handle .TS to 24fps MPEG2 (for DVD authoring)?

An AviSynth script with IVTC and resizing applied, frameserved into your MPEG-2 encoder (HCEnc being free and good). I can't speak for "everyone", though.

[PartingShot]

An AviSynth script with IVTC and resizing applied, frameserved into your MPEG-2 encoder (HCEnc being free and good). I can't speak for "everyone", though.

+1

I've done film-based HD (1080i and 720p) conversion to standard-def DVD @ 23.976fps, and Avisynth is the best way to achieve the necessary 3:2 pulldown removal step of the conversion process. Manono's suggestion is the most straightforward way to achieve your stated goals, though I myself would not frameserve to your MPEG encoder (1 step), but rather use Avisynth (in tandem with VirtualDub, as a front-end GUI) and output your conversion to an intermediate AVI... then apply the AVI into your MPEG encoder (2 steps). The only reason is because, at least for me, the 1 step frameserving method takes 4 or 5 times as long as the 2 step method. Maybe I don't know how to optimize my encoder for frameserving, but it's just easier for me to do it in 2 steps.

Good luck in your conversion endeavors

[manono]

The only reason is because, at least for me, the 1 step frameserving method takes 4 or 5 times as long as the 2 step method.

You have a fairly slow computer by current standards, if your computer specs are correct. Are you running multiple passes in CCE, perhaps? That resize to 720x480 is a very slow process. And if you IVTC the original Hi-Def source, I suppose that could take a while as well. You might try this (just as an example):

LanczosResize(720,Height)#or your favorite resizer
TFM().TDecimate()#or your favorite IVTC
LanczosResize(Width,480)

And if you're pretty certain it's an easy IVTC, you can disable the post-processor to speed things up even more:

TFM(PP=0).TDecimate()

And if you do create an intermediate lossless AVI (I use Lagarith), you should probably still frameserve using an AviSynth script, this time with AVISource.

[PartingShot]

I have to update my comp specs, sorry

I'll definitely try out your scripting idea... that's quite clever, actually (shrink horizontal -> IVTC 29.97i to 23.976p -> shrink vertical). I didn't "get it" until just now.

Could you just explain why an AVS script should be loaded into the MPEG encoder, rather than just inputting the AVI? Thanks!

You have a fairly slow computer by current standards, if your computer specs are correct. Are you running multiple passes in CCE, perhaps? That resize to 720x480 is a very slow process. And if you IVTC the original Hi-Def source, I suppose that could take a while as well. You might try this (just as an example):

LanczosResize(720,Height)#or your favorite resizer
TFM().TDecimate()#or your favorite IVTC
LanczosResize(Width,480)

And if you're pretty certain it's an easy IVTC, you can disable the post-processor to speed things up even more:

TFM(PP=0).TDecimate()

And if you do create an intermediate lossless AVI (I use Lagarith), you should probably still frameserve using an AviSynth script, this time with AVISource.

[manono]

Could you just explain why an AVS script should be loaded into the MPEG encoder, rather than just inputting the AVI?.

Maybe it's just me, but colorspace conversions usually have to be done (unless done properly when creating the lossless AVI), and I prefer to do them myself in the script rather than letting the encoder do them. You mentioned 3 different encoders and the only one I'm the least bit familiar with is TMPGEnc and it accepts RGB. I use CCE and do a conversion to YUY2 in the script before sending it to CCE. HCEnc requires YV12 input.

Also, a full IVTC is only needed with the 1080i sources. 720p just needs decimation (usually something like SelectEven().TDecimate()) and it can be resized safely to 720x480 before removing the duplicate frames. That should speed things up as well.

And I guess you already know that a colorimetry conversion is required when going from Hi-Def to Standard-Def, if you want the colors to remain 'true'. The ColorMatrix filter is one way to convert from BT.709 to BT.601. Or the BT709ToBT601 filter:

http://avisynth.org/warpenterprises/

[PartingShot]

And I guess you already know that a colorimetry conversion is required when going from Hi-Def to Standard-Def, if you want the colors to remain 'true'. The ColorMatrix filter is one way to convert from BT.709 to BT.601. Or the BT709ToBT601 filter:

http://avisynth.org/warpenterprises/

Unfortunately, you overestimated me... I didn't realize that the colorimetry conversion was needed. Thank you for pointing that out.

I have one final question before ending this (inadvertant) threadjacking: Where should I insert the colorimetry conversion within the following AVS script?


############
############
# These lines call the scripts required for IVTC
Import("C:\program files\avisynth 2.5\plugins\jdl_util.avs")
Import("C:\program files\avisynth 2.5\plugins\jdl_telecine.avs")

LoadPlugin("C:\program files\avisynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\program files\avisynth 2.5\plugins\deblock.dll")
############

# This line loads the MPEG source
MPEG2Source("mpeg2source.d2v")
############

# IVTC Section:
# If source has perfect 3:2 pulldown pattern, choose JDL_IVTCPattern
# Otherwise, choose telecide().decimate()
# Use only one, comment out the other!
JDL_IVTCPattern(2, false) #3:2 pulldown removal for this clip
#telecide().decimate()
############

# This line removes macroblocking
deblock(35,0,0)
############

# This line converts from MPEG2 YV12 to YUY2
converttoyuy2()
############

spline36resize(720,480)
############

sharpen(0.20)
############
############


Thank you very much, manono, for your informative responses

[manono]

That's a real risky way to IVTC, the JDL_IVTCPattern. As it clearly says. it needs a perfect pattern to work. If you're cutting out ads, there goes your perfect pattern. If the movie has been edited, perhaps to fit into a time slot, there goes your perfect pattern. Me, I'd surely go for a full IVTC. I'd hate to find out later, perhaps after having deleted the source, that it screwed up somewhere.

Where to stick the colorimetry change? I haven't the foggiest idea. I don't know if it even matters. But most scripts that I've seen have it at the beginning, right after the MPEG2Source line. It has to be that way if using the ColorMatrix filter and if you're using hints. Perhaps the docs for the 2 filters recommend where in the script to stick it.

Ordinarily you want the colorspace conversion at the end, if possible (not always possible, if some filter or other requires a certain colorspace). If the source is YV12, you want to do all the filtering in that colorspace and ConvertToYUY2() (for your encoder, I presume) at the very end.

[Tiernan]

Thanks for everyone's responses. Not really having done any AVS scripting in past, I'm tending to feel like I'm in way over my head, with most of the conversions I've done in past having basically ignored telecining, etc. The result was still uniformly very good, but some material (CHUCK (and JOURNEYMAN) captures from 1080i come to mind) still had the far-too-frequent judder in playback that was troubling.

So, since I didnt really know where to start, I went to MeGUI and pinched the AVS script that it created for the FLASH GORDON MKV encodes I'd been doing, and it coincided pretty closely with the code you'd suggested, manono. I installed HCenc, per your suggestion, and determined that, for my initial test, I had a old b-movie from AMC-HD to test (THE ATOMIC SUBMARINE). In this case, though the capture was 1920X1080i (59.94), AMC (like they do so often) had stretched the original 1.33 Academy-ratio film to fill the 16:9 screen. It's possible that this is a progressive Stretch-o-vision-type stretch that can't really be fixed by putting it back to 4:3, but I was going to be content with the results, regardless.

So, I created the .d2v, dropped it in HCEnc, and with 2-pass encoding, had the 77-minute film done in 3-4 hours. I THINK the result looks very good, though it's possible that the processing could have been quicker. I noted that a couple of other suggestions were made, specifically about macroblocking, and the different way to IVTC. In the case of your script, manono, would your suggestions be entered verbatim as you've suggested, or would the "Height" and "Width" be dependent on the original HD source? Additionally, with PartingShot having included the "deblock", would that be needed for most/all processes? Lastly, given that the HD captures tend to be sometimes less then perfect, I assume that the pattern is never perfect. Has it been your experience that AVS will handle the IVTC properly, even if it has to account for the occasional oddity?

Again, thanks for all your suggestions and directions. I have a feeling that I won't end up being able to use HCenc for ALL the HD I end up having to process, but it's been helpful to know that it seems to work out for the special projects I'd like to undertake.