To Decimate or Not to Decimate ?

[Soopafresh]

I have a bunch of material (progressive, film) that's at 29.97. I'd like to make NTSC DVDs out of this material. Which makes better sense? :

1) Decimate the 29.97 to 23.976, then run resulting mpeg2 through DGPulldown before authoring.

2) Keep the fps at 29.97 and don't sweat it.

TIA

[adam]

Keep it 29.97fps progressive. With a progressive 29.97fps source you will gain very little in encoding in ntscfilm, and really there's no way to get there with the source you have anyway. Anything you'd do would still result in jerky playback.

[manono]

I disagree. He says it's film based. That means every 5th frame is a duplicate frame. So there will be a couple of benefits to removing the dupe frames and applying pulldown after reencoding. One is that it will play more smoothly. The other is that he'll be encoding 20% fewer frames when encoding at 23.976fps as compared to encoding at 29.97fps, and will achieve much better quality/compression if encoding for a fixed file size (such as a DVD5).

[mattso]

So there will be a couple of benefits to removing the dupe frames and applying pulldown after reencoding. One is that it will play more smoothly.

I know it will compress better, but I didn't know it would play "smoother"? I thought your DVD player would just add those extra frames back in to pull it back "up" to 29.97 anyway?

(I'm from PAL-land, I've never really given this "IVTC" stuff any thought...just encode my NTSC captures as they are!)

[hech54]

One entry found for decimate:
Pronunciation: 'de-s&-"mAt
1 : to select by lot and kill every tenth man of
2 : to exact a tax of 10 percent from <poor as a decimated Cavalier -- John Dryden>
3 a : to reduce drastically especially in number <cholera decimated the population> b : to destroy a large part of <firebombs decimated large sections of the city>

http://www.merriam-webster.com/dictionary/decimate

[adam]

manono I guess we are reading his post differently. It seems to me that his source is 29.97fps progressive. Maybe it was film at one point but not anymore. If it originated as film it was telecined and then deinterlaced, there's no good way to reverse that.

Also the main benefit to IVTC is not the lower framerate IMO. Those ~20% fewer frames are almost entirely redundant since they are duplicates, so they use nowhere near as much as ~20% of your bitrate anyway. The main benefit is that you are encoding frame based pictures as opposed to field based ones, which require more bitrate. Since his source is progressive to begin with he won't gain much by decimating to NTSCfilm and in his case I do not think it is possible to do so and maintain smooth proper playback.

mattso the only way the IVTC and pulldown would make playback smoother would be if the telecine was done poorly, or maybe using an odd pattern for that source. If it was done using 2:3 pulldown, which is pretty standard, than the hard telecined and the soft telecined version should playback identically but there will be benefits when played back on a progressive scan device via a progressive scan DVD player.

[sammie]

I agree with Adam. Decimate or inverse telecine were great back when we were all doing bit starved encodes such as VCD. But for a regular DVD or a DiVX/XVID where you aren't trying to cram more than say an hour and a half, two hours on a disk, the artifacts you introduce during the process aren't worth the saved space.

I (and a bunch of others) did a lot of extensive testing on this a while back and the problem is that most encodes are more of a mixed bag. The inverse process has to assume some sort of regularity and many if not most files switch in and out of telecinied and more importantly have lots of pattern shifts and breaks. These result it a flicker or stumble in your new "film" encode. It isn't usually bad, certainly very watchable. But when you are being critical about your work (and aren't we all? ), it is usually enough to annoy you even if it your uninformed audience never notices.

Bottom line is try it and see if you like it. Disks are cheap enough these days. My bet though is unless you are trying to squeeze too much onto one disk, you'll prefer the one you just encode with out restoring to 24 fps.

[manono]

Hi-

I guess we'll have to wait for Soopafresh to come back and confirm what he's got. If it's 29.97fps (progressive, film) as he says, then it almost has to be 1 2 3 4 4 5 6 7 8 8. Assuming Soopafresh knows a little something by now, he'd know better than to ask about decimating hard telecine that's been deinterlaced. Notice that nowhere did he mention IVTC, but only Decimate. It's all progressive; no field reconstruction is necessary. He's only wondering if he should pull out the dupe frames. I say yes.

If it was done using 2:3 pulldown, which is pretty standard, than the hard telecined and the soft telecined version should playback identically

Not hardly. If it's as I'm guessing it is, what he has is 6 duplicate frames every second. As you know adam, that's not really hard telecine. But your answer to mattso's question ignored what he was asking. If he Decimates away the dupe frames and then applies pulldown afterwards, what he gets is 12 extra fields a second. Big difference there, and the version with the dupe frames will play noticeably more jerky, with noticeably more stutter. Most NTSC people don't notice 3:2 pulldown. Duplicated frames are much more obvious and annoying. I've seen both DVDs and captures like this. It's not all that uncommon.

[adam]

I tend to assume people's sources aren't utterly butchered unless they mention it. Its speculation at this point what he's got. If its as you presume than its not about maximizing quality its about fixing a mistake.

But your answer to mattso's question ignored what he was asking.

I think its more likely that mattso and I are talking about something completely different than you. I'm pretty sure mattso was talking about IVTC'ing which assumes an actual hard telecined source. Removing duplicated frames is not an IVTC as you say. Like I said, you are talking about repairing an improper conversion. Naturally that will make playback smoother but only because the source is fubar.

[manono]

Funny, eh, how 2 seemingly intelligent people can read the same few sentences and draw such different conclusions about their content?

Oh Soopafresh, where are you?

[vhelp]

I waited all day to post, but others just kept adding to what I wanted
to say anyways.. oh well.

I still believe that his particular source is all progressive, and
that there is no dup frames. Manono makes a good point about it may
be a posible dup in the video pattern throughout, but is probably
unlikely for this clip.

For some reason, I can't see how a broadcaster would make this kind
of mistake. I see no benefit, except for *IF** the source was based
on a strange or incompetant conversion on their part, now resorting to
this "dup" frame theory here.

** My gut tells me that this would seem to be the case here


fwiw, I'm not correcting anyone here.., just pointint out a few things..


adam, by hard Telecine, I take it you mean those sources that have the
two interlaced frames (fields) already incorporated into the video.

These are from those sources that are from broadcasters and captured
by a given capture card - guess that includes hdtv pci cards that are
capable of pushing .TS sources onto one's HDD. Which would also lead
me to believe that they are not "flagging" them as Commercial DVD's do.

But, there is not reason to dismiss the fact that the broadcasters can
actually add Telecine/Pulldown "flags" to these sources (.TS) if they
were to choose so, because after all, hdtv sources are "digital", right ?

Last, of course Commercial DVD's can have "hard telecine" imbeded in
them as well. In fact, even when the source to be captured is Analog,
they (broadcasters) can still imbed "flags" into *their* streams, but
our capture cards will not even see them, since the Analog source is
not digital once received/read by our capture cards.

( course, I know you know this. I was just pointing it out because it was
on my mind, and I felt like expressing it [while we wait for Sooperfresh's
closing response] )

-vhelp 3810

[Soopafresh]

Thanks for the replies so far. Attached are two clips. Original Mpeg2 Source is 720p. Problem is, I don't want to transcode from the original files - it's too time consuming. Way back when, I made high bitrate transcodes of this stuff, but at 29.97 fps. At the time, I couldn't figure out how to decimate and get smooth movement (although with Tdecimate and AviSynth, it is now pretty easy)

BTW, I'll be the first to admit that I have holes in my knowledge. Thank goodness for sites like Videohelp for the valuable information.





29970.m2v

23976.m2v

[vhelp]

Hi Soopafresh

First, I would like to admit that I was wrong.. Your source
was *just as Manono sumized* !!

This is how I did it, to receive smooth playback..

using vdubMOD, if you import your *.m2v mpeg file and:

** file\video\frame_rate..
** (o) Reconstruct from fields - manaul, and enter [3 ]
** followed by, [x] Invert Ploraty
** then, export out to another .AVI or other (can't frameserve this)

The above will give you 23.976fps, aka IVTC video.

You should be able to perform the above on your videos.
But I'm sure that you don't want to go this route. But its there
for record

I know you can do this inside an AVIsynth script because I have
done this very same video scenario, but I can't remember the exact
script line - I used a one-line function, SelectEvery() to accomplish
this. It was the fastest, when I used it on such sources.

-vhelp 3812

[edDV]

I've been following this and was equally confused.

"progressive film" @29.97 -- Where would a mortal get such stuff? Maybe he meant normal telecine.

Then the magic word of clarification "720P"

720P is handled just like 480P.

24fps (23.976) film is frame repeated 3:2 to 59.94 fps like this:

abcd (23.976)
to
aaabbcccdd (59.94)

if one chops every other frame in an attempt to get to 29.97fps, you get this

aaabbcccdd
a_a_b_c_d_

or
_a_b_c_c_d

your first file follows this pattern displaying unnatural pause motion.

To get back to 23.976 fps, you need to delete one of the repeated frames (every 5th frame) to get back to

abcd (23.976) for progressive DVD authoring.

[vhelp]

fwiw, here is a divX version of the conversion via vdubMOD..

(note, be sure you start on the first-pattern-Progressive frame)

--> lost.rar

-vhelp 3813

[edDV]

If ABC and Fox would just transmit their series and movies (including commercials) as 720p/23.976 and let the tuner do the 3:2 frame repeat, they could fit two 720p subchannels into the same bitrate and we would have an equal quality subchannel stream at 8-9Mb/s.

This is all legal under ATSC.

[vhelp]

Ok. Wahooo!!

@ Soopafresh

I got it. I knew I had it somewhere on my HDD.
So here's the AVS script (just hate avisynth) but this gave me nice
fluid encodes for this type of source, and you should have no trouble
with these type sources.

Please note, you should start at the first position of the progressive
pattern set of frames (use the trim() command) like this in your source,
and note the 'P' position below:

PPPPd PPPPd PPPPd PPPPd PPPPd PPPPd ...

Where 'd' is that dup frame.

  separateFields()
  selectEvery( 10, 0,1,2,3, 4,5, 6,7   ) 
  weave()

-vhelp 3814

[Soopafresh]

Right on, Vhelp, EdTV. Mucho thanks. Vhelp, the file you enclosed looks good. Plays back smoothly.