1080i hdtv transcoding issues

[topsyturvy]

Hi guys. I have a number of 1080i hdtv caps that I would like to transcode using x264/MeGUI. I've been reading up on how to do this but the topic is ALOT more complex than I had expected, and with all the information out there I'm getting completely lost. I'm hoping some kind soul will answer my questions, some of which are bound to be newbie-ish.

My sources are PBS kids tv shows (anime) capped of HDTV (NTSC 1080i). I would like to transcode them and have the following questions:

1) Should I transcode them to 720p or 1080p? I don't have storage constrains and would prefer to keep them 1080p but I remember reading somewhere that 1080i -> 720p is "better" than 1080i ->1080p?

DGIndex (and MeGUI auto detect) are detecting the source as hybrid 29.97fps (more than 90% video, rest Film).

2)Does this mean that 90% of the source is 60i and 10% 30p? Or is it 90% 60i and the rest 24p with flags to play at 30p.

3) The interlaced parts are clearly 3:2 telecined. Is the original source ALL Film (24p)? If yes, why is it detected as Hybrid (i.e. 30fps video +24fps film)? I doubt the creators are making it hybrid since its anime .. or are the broadcasters making it hybrid during transmission? WHY would they ???

I tried IVTC using an aviSynth setting: tfm(order=-1).tdecimate(mode=1,hybrid=1) however the output isnt looking as smooth as the original mpeg2 source and some of the frames have ghosting that is missing in the source (I'm assuming this is thanks to the hybrid paramter).

4) Do I really need to tdecimate() after tfm()? What if I just do tfm() so that the output is at 29,97 fps? This seems to remove the ghosting (are there any disadvantages to consider?). The output is meant to be played on a TV through xbmc.

Please excuse me if the questions are confused, because thats how I'm feeling at this point. After alot of reading i'm sorry to say I haven't made as much progress as I would have liked

[manono]

1. Is there something wrong with keeping it at 1080i and saving yourself all the work?

2. I doubt any of it is 30p. But there's no way to tell what it really is without examining the frames yourself. The last thing I would do would be to trust MeGUI to tell me what I have and how I should deal with it.

3. If that 90% video is all hard telecine, then it was shot on film and (most of it) edited as video. And, if true, it can be IVTC'd back to 23.976fps. Also if true it's not hybrid and you can leave off that setting and see if the ghosting disappears and if it plays more smoothly.

4. If you make it progressive 29.97fps and if it's supposed to be 23.976fps, in every five-frame cycle there will be a duplicate frame. This will make for slightly jerky playback. So yes, you need the TDecimate line.

[topsyturvy]

Thanks for your reply! Please see my comments below...

1. Is there something wrong with keeping it at 1080i and saving yourself all the work?.

My playing system (xbmc on nvidia ion) isnt able to de-interlace properly during playback. On this point, I've wondered if software players only deinterlace or will they also ivtc during play back?

2. I doubt any of it is 30p. But there's no way to tell what it really is without examining the frames yourself. The last thing I would do would be to trust MeGUI to tell me what I have and how I should deal with it.

Any idea how I can check this? i.e see if the progressive sections are 30p or 24p?

3. If that 90% video is all hard telecine, then it was shot on film and (most of it) edited as video. And, if true, it can be IVTC'd back to 23.976fps. Also if true it's not hybrid and you can leave off that setting and see if the ghosting disappears and if it plays more smoothly..

Why would most of it be edited a video but not all of it? I don't know anything about video editing ...

4. If you make it progressive 29.97fps and if it's supposed to be 23.976fps, in every five-frame cycle there will be a duplicate frame. This will make for slightly jerky playback. So yes, you need the TDecimate line..

Slightly jerky but exactally as its shown on the TV during live transmission right?

[manono]

1. I know of no software players that can IVTC on the fly. I understand the NVidea one can, but I've never seen it in action. Better might be to play a script. But I know nothing about xbmc. Even most standalone DVD players can't IVTC, but only deinterlace.

2. You look at the frames. If video (30i) they'll all be interlaced. If 30p they'll all be progressive. If hard telecined, in every 5 frame sequence 3 will be progressive and 2 interlaced.

3. Maybe there are logos or credits that were handled differently. Maybe some scenes were handled differently than others. The reason doesn't matter. It's perfectly legal and it happens all the time.

4. No. The TV doesn't play it at 29.97fps, but at 59.94 fields/frames per second, depending on whether the broadcast is transmitting fields or frames. Rather than one repeated frame in 5, there will be 2 repeated fields/frames in 10, within the same period of time. In your 29.97fps example, you'll get 4 identical fields/frames in a row. In the usual 3:2 pulldown, they'll be spread out more with nowhere more than 2 identical fields/frames in a row. The net effect, while still slightly jerky, will be smoother than having the dupes piled up all at once like that. You've been watching 3:2 pulldown all your life. Does it seem noticeably jerky to you? Your one-in-five 29.97fps example will be much more noticeable. During certain kinds of movement like a slow pan, the movement will seem to stutter.

[topsyturvy]

Thanks again. Somethings are clearer, I've listed my left over questions below

1) Are there any advantages of going 1080i-> 720p vs 1080p in terms of playback smoothness/ivtc/deinterlacing etc?

2) I looked at the video already. Most of it is hard telecined (3:2), however there are groups of frames (~1 seconds duration) that are progressive (I can tell because there is movement in those scenes but no combing effect). However stepping through frame by frame I cant tell if these are 30p or 24p (how do I tell this), but they are definately progressive. I assume this mixture is what is called "Hybrid". I can't understand why digitally created anime might be hybrid...but I'm trying to ignore this and just work with what I can see.

IF the progressive portion in the hybrid video is 24p, and I've set to IVTC for 24p output, then will decimate still drop 1 in 5 in the 24p progressive section?

3) I'm not sure I understand your comment #4 above. Creating a progressive video from telecined will give 1 duplicate frame every 5 progressive frames. Worst case the scenario in 10 frames, the 2 different sets of identical frames will be together (I'd assumed the duplicates would be at the same location in the 5, so we'd never have 2 sets of duplicates together). Is this want you meant?

[manono]

1. No.

2. I don't know how you looked at the frames. For example, some (most?) players will ignore the RFF/TFF flags and show 24p the same way they'd show 30p. Me, I make AviSynth scripts and have a look that way. That way I'm sure I'm seeing it correctly. It may or may not be hybrid. That partly depends on how you define hybrid. You could always make available a few seconds of it for us to have a look.

3. A TV broadcast would never transmit (where U=Unique frame and R=Repeated frame) UUUUUURRRR. Instead they'd go UUURRUUURR. That second one can also be 3:2 pulldown, like on soft telecined DVDs. Notice the 3 2 3 2 pattern? Your 29.97fps with one frame in every five being a repeat is the first pattern The TV broadcast of films and also DVDs, whether soft or hard telecined, are the second pattern. You can understand how the first pattern would play with more stutter than the second.

[topsyturvy]

An excellent suggestion. I've attached a short sample in which the video switches from interlaced to progressive to interlaced to progressive.

In dgindexer if I click enable pull down flags the whole sequence appears interlaced, but if I say ignore pull down then the progressive scenes appear progressive. This makes me think that the progressive scenes are 24p.

On this point I'm wondering, with soft pull down enabled in dgindexer, why does the megui AviSynth script creator (which analizes the d2v file) even detect it as a hybrid video? Visually in the associated video preview the progressive scenes have been telecined.

i.e. Enabling soft pull down in dvindexer it should make the whole thing 60i and a tfm(order=-1).tdecimate(mode=1) should be recomended instead of a tfm(order=-1).tdecimate(mode=1,hybrid=1)

Lastly I thought the hybrid=1 option was only for 30p??

[manono]

OK, you have a mix of hard (the telecine encoded into the video) and soft (the telecine done by the flags set in the video) telecine. None of that sample is 30p. It's all from film. Make the D2V using 'Honor Pulldown Flags' (as I do anytime the statistics window doesn't show it as 100% film), and use a script similar to this:

TFM(D2V="C:\Path\To\Video.d2v")
TDecimate()

That was a good sample. Thanks.

I don't use MeGUI and certainly wouldn't use its video analyzer. I suspect (but don't really know) that anything with a mix of hard and soft telecine is labeled as 'hybrid', as well as 'real' hybrids (a mix of real 30fps (either p or i) and 24p). And, as you discovered, using that hybrid setting can lead to problems unless it's really a hybrid.

And, as I'm sure you've noticed, what you call 60i I (and many others) call 30i. 30p is 30 progressive frames per second. 30i is 30 interlaced frames per second. I have no idea why many think 30i should all of a sudden get a new and potentially confusing naming convention, one meaning 60 unique fields per second, when 60 fields per second is exactly the same as 30 interlaced frames per second. And, since fields by definition aren't interlaced, 60i doesn't even make any sense to me.

[jagabo]

That was a good sample.

Yes, I'm adding that file to my folder of odds video files.

[topsyturvy]

OK, you have a mix of hard (the telecine encoded into the video) and soft (the telecine done by the flags set in the video) telecine. None of that sample is 30p. It's all from film. Make the D2V using 'Honor Pulldown Flags' (as I do anytime the statistics window doesn't show it as 100% film), and use a script similar to this:

TFM(D2V="C:\Path\To\Video.d2v")
TDecimate()

That was a good sample. Thanks.

I don't use MeGUI and certainly wouldn't use its video analyzer. I suspect (but don't really know) that anything with a mix of hard and soft telecine is labeled as 'hybrid', as well as 'real' hybrids (a mix of real 30fps (either p or i) and 24p). And, as you discovered, using that hybrid setting can lead to problems unless it's really a hybrid.

And, as I'm sure you've noticed, what you call 60i I (and many others) call 30i. 30p is 30 progressive frames per second. 30i is 30 interlaced frames per second. I have no idea why many think 30i should all of a sudden get a new and potentially confusing naming convention, one meaning 60 unique fields per second, when 60 fields per second is exactly the same as 30 interlaced frames per second. And, since fields by definition aren't interlaced, 60i doesn't even make any sense to me.

Sorry about the 60i confusion. I'm new to much of this and may have used 60i instead of 30i because it made more sense to me.

You are welcome about the video sample, glad i could contribute whatever little back.

I only use megui because it allows me to queue jobs e.g. batch indexing of ts streams.

Lastly, i noticed you didn't rcomend a decimate(mode=1). The manual seems to recomend decimate with mode=1 for anime decimation?