Does de-interlaced video = progressive?

[yoda313]

OK I'm finally getting to ************** I'm recording through media center. I am using tmpgenc dvd author to make a program stream file then I edit out the commercials and*********
I have several action movies I am working with.

My question is this. Would it be worth my time to de-interlace these? They are off analog cable at 480i obviously. These are off of SPI** and unfortunately all seem to be fullscreen so far.

Does de-interlacing in effect make them progressive? Is there any appreciable quality gain for the work involved? I have a dual core amd so it shouldn't take too long to deinteralce a movie. I'm just wondering if its worth while considering the source.

Also what's the best method to go about this? I imagine using a filter in virtualdubmod or virtualdubmpeg2 would be the best approach. But what would you do to keep it in the mpeg world? Would I frame serve to tmpgenc 2.5? Or would I just use tmpgenc 2.5 in the beginnning? What settings?

I woiuld expect some opinions to come from vhelp who seems to be a keen deinterlacer given the recent string of olympics postings.

What about other users? Would I just be wasting my time?
Advice welcome. Thanks.

[Soopafresh]

Movies (like things that play at the theater) don't need deinterlacing, they need a process called Inverse Telecine and Decimation, to bring them back to their original progressive 23.976fps frame rate.


Live TV shows (like Leno, Conan, The Olympics) on NBC and CBS are interlaced. ABC is 720P, so it is progressive to begin with.

[jagabo]

What are your source file? AVI? MPG? TS? Something else? Standard def or hi def?

Yes, after deinterlacing you have progressive video. But if you simply deinterlace film movie broadcast as 29.97 fps interlaced, you'll get jerky video and lose resolution. As was pointed out, what you really want to do in inverse telecine back to 23.976 film frames.

[vhelp]

I just wanted to first note here with you that while you can deinterlace Analog and Digital type sources, the digital type sources do come out much better than those of analog because they have too much noise that confuse the deinterlacer algorithms. Those games that I did post about originated from my analog source and were a bit noisy but still they hung on to give an overal good deinterlaced look. Those were based off my analog captures prior to my finally getting NBC over hdtv using my pinnacle pctv pro stick thanks to the chance I took on getting a roof antenna amplifier. I've even made some add'l discoveries in that endeavor but I'll hold off for another post/topic for that. Anyway.

And now that I have my hdtv card recieving the NBC games I was able to capture (record) the program streams from those sources and do some deinterlacing work with them. I am still comming up with various deinterlacing ideas and working them out for this source medium. Those games are All-Interlace and originate from PAL and have gone through a pal->ntsc frame rate conversion. But I have found that it is not a good idea to apply a ntsc->pal frame rate conversion back to pal's 25 fps because that source went through some exhaustive conversion that ultimately removed too many fields (through field-blending aspects of the pal->ntsc conversion) that restoring back to 25 (24.975 fps) would produce jerky to studdery motion, and it was better to just remove as much of the field-blending as possible while staying at 29.970 fps.

More to say, but this is what I have so far because I am doing a lot of muti-tasking at the moment. So its hard for me to consentrate.. but i'll be back.. anyway.

The one thing I (and some of you) might have to watch out for in hdtv reception is the random intermittent activities that seem to be a part of this source medium. I find that if I leave it alone (while recording) it is better in the long run because the source is still recorded. The pctv seems to stop the recording (like a pause) and then continue if the picture strength gets better. But my receiption isn't too bad.

I'm watching beach vollyey ball.. China 5, USA 7

-vhelp 4833

[yoda313]

Movies (like things that play at the theater) don't need deinterlacing, they need a process called Inverse Telecine and Decimation, to bring them back to their original progressive 23.976fps frame rate.

Ok silly question here - ************************************************** ************

What are your source file? AVI? MPG? TS? Something else? Standard def or hi def?

As mentioned in my post its analog cable at 480i and captured with windows media center. That is the dvr-ms file which is an encapsulated mpeg2 file that I let tda 2.0 remultiplex into a program stream file for further authoring

Yes, after deinterlacing you have progressive video. But if you simply deinterlace film movie broadcast as 29.97 fps interlaced, you'll get jerky video and lose resolution. As was pointed out, what you really want to do in inverse telecine back to 23.976 film frames.

So would I do that in tmpgenc 2.5 then? All I need to is load the program stream mpeg2 file after conversion and then use ivtc? Are there settings involved in this or is it all or nothing in terms of calibration?

Again I'm curious about making a fully compatible dvd after the result. Well as compatible as home made dvdrs are you know, obviously not pressed quality when your talking about a home burn...

I just wanted to first note here with you that while you can deinterlace Analog and Digital type sources, the digital type sources do come out much better than those of analog because they have too much noise that confuse the deinterlacer algorithms

Those games that I did post about originated from my analog source and were a bit noisy but still they hung on to give an overal good deinterlaced look.

Are you saying you did deinterlace analog video? Is that just for tv type broadcasts that weren't originally film sources?

Thanks everyone. I'm interested in experimenting since I finally have multiple harddrvies and a speedy dual core processor for faster, better results. (its a 2.7 dual core amd with a 80gb os drive, one 500gb internal drive and a 320gb firewire drive - I do have a 160gb usb drive I can press into service but I am still trying to get around to installing it as a permanent backup drive for my xp machine, just haven't gotten around to it yet).

[jagabo]

Most NTSC movie DVDs contain 23.976 fps frames with 3:2 pulldown flags that tell the DVD player how to produce 59.94 fields per second for transmission to standard definition TV. If your video is already DVD compatible 29.97 fps interlaced MPEG2, and will fit on a DVD, you won't really gain anything by IVTC. Otherwise IVTC and authoring progressive with pulldown flag will get the best results. Encoding 24 progressive fps requires less bitrate than encoding 29.97 interlaced frames per second. And progressive frames give better color resolution.

The automated IVTC tools in VirtualDub and TMPGEnc Plus are not especially good. Several AviSynth filters (Yadif plus TDecimate for example) are better. But all of those automated methods sometimes let interlaced frames through, and sometimes skip or duplicate a frame.

The very best results can be had by more manual methods. When video has gone through 3:2 pulldown and is later cut there are 5 possible starting positions. Given source film frames A, B, C, etc., with each letter representing a field pulled from those frames (grouped into fives for clarity):

AAABB...CCCDD...EEEFF...
AABB...CCCDD...EEEFF...
ABB...CCCDD...EEEFF...
BB...CCCDD...EEEFF...
B...CCCDD...EEEFF...

To inverse telecine this "manually" you have to determine where the first recorded frame starts in the pulldown sequence and select the corresponding IVTC pattern. This works until there is a break in the pattern. A break can be caused by a dropped frame while recording, edits made at the broadcaster (after telecine), commercial breaks, etc.

Breaks in the pattern mean you have to work with the movie in sections. You identify where the breaks are, then use the correct IVTC pattern for each section.

In VirtualDub you use the Reconstruct From Fields - Manual option in the Frame Rate dialog. You set the offset with a value from 0 to 4 depending on where in the sequence the recording started.

In AviSynth you can use a sequence like:

SeparateFields().SelectEvery(10, 0,1, 2,3, 4,5, 7,8).Weave()

Again, there are five different SelectEvery() patterns to choose from. Actually, there are only 4 and you have to use a little cheat for the 5th.

As you can see, if there are no, or very few, breaks in the pulldown pattern this isn't too much work. But if there are hundreds of breaks you'll want to resort to the automated methods.

Here's a sample automated AviSynth IVTC with Yadif

Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
MPEG2Source("VTS_02.d2v")
Yadif(mode=0, order=1)
TDecimate()

Here's a sample manual IVTC of a video with 3 pattern breaks (four sections):

src = MPEG2Source("VTS_02.d2v")

v1 = trim(src, 0,  26802).SeparateFields().SelectEvery(10, 0,1, 2,3, 4,5, 7,8).Weave()
v2 = trim(src, 26803, 99770).SeparateFields().SelectEvery(10, 1,2, 4,5, 6,7, 8,9).Weave()
v3 = trim(src, 99771, 126069).SeparateFields().SelectEvery(10, 0,1, 2,3, 4,5, 7,8).Weave()
v4 = trim(src, 126070, 0).SeparateFields().SelectEvery(10, 0,1, 2,3, 5,6, 8,9).Weave()

return(v1 + v2 + v3 + v4)

[jman98]

Movies (like things that play at the theater) don't need deinterlacing, they need a process called Inverse Telecine and Decimation, to bring them back to their original progressive 23.976fps frame rate.

That is a sweeping generalization if I've ever heard one. jagabo's follow up statement that "most NTSC movie DVDs" are progressive is correct, but it's still possible to find interlaced movies, yes, movies on DVD. There are quite a large number of foreign movie DVDs that are interlaced. Here in the USA there is some pressure to put stuff out as progressive as the public may perceive it as inferior quality otherwise, but many foreign studios happily put out interlaced DVDs with no thought to whether or not the public will view such as inferior product.

[yoda313]

@jagaboo - thanks but that is a lot more involved then I was looking to get with this.

I'd be happy with a link for a basic ivtc for tmpgenc plus 2.5.

@jman 98 - thanks but I'm not really into foreign dvds except possibly some anime. And this project is meant for tv caps off analog cable for mostly action movies and scifi stuff that is US based.

I did a brief test in tmpgenc plus but I'm not really sure what I'm doing yet to know if did anything.

Also I must be honest I've never messed with avisynth so I don't know if this is a good thing to tackle with for the first time.

[vhelp]

I'm sorry I missed all the fun, but I was busy watching the games -- womens diving, a very memorable experience, and also Womens Gymnastics, etc.

I'll touch on the restoration of 30i -> 24p process from Film sources, or in yoda313's perception, deinterlacing Later on or so, I'll touch on other aspects of deinterlacing: de-interlacing, and then maybe some pal->ntsc / ntsc->pal restoration attempts, though I'm still new to that area.

yoda313, at your request, here is a basic TMPGenc ivtc process you can try.

If you know for sure your captured videos originate from Film:

1 -- Settings\Advanced\[x]source range, and set your starting point of the "PPPii" mark.
2 -- Settings\Advanced\Inverse telecine
2a - click the [clear] button first, always do this to remove any random patters from cache
2b - select that first frame (above step 1) by clicking it then, right-click and select 'deal after this frame..'
2c - now, in the box, enter the value, 10100

In step 2c, you can alter the series position, (though that can lead to confusion) so I like to be consistant and set up my pattern to always start at the PPPii group, and use the first P as the starting point for the 10100 value. But prior to this you have to know where in the source the films 3:2 pattern begins, and work with that as your starting point. Every {P} is a progressive frame, and every {i} is an interlace frame.

When you feed a source in some editors or encoder gui's you have to know in advance when and where the position of the PPPii starts in your movie conversion routines. That's why you have to watch for the signs of the PPPii pattern. You do that in your editor (ie, virtualdub) and watch for 3:2 frame groups that are 3 progressive frames followed by two interlaced frames. That is your PPPii pattern. Then, the source has to be consistant in this step so that when you feed it into your ivtc routines they get processed correctly, else you things like: bobbing; studdering; jerky motion; or a mixture of progressive/interlace. So its criticle that you understand the PPPii aspects in your videos.

But be warned. Todays videos seems to be nearing the end of the old generation usage of the 3:2 pattern layout. It has been changed in favor of time expansion/compression telecining. So, the 3:2 pattern, though still there is more or less eluding and consists of manipulated fields (ie, reversed or pushed ahead of other fields, etc) and this confuses the old logic handling of 3:2 processing as used in tried and true older method in step 2c of TMPGenc pattern guidance and even in some AVIsynth scripts that include the built-in function of: SelectEvery() where the usage syntax was SelectEvery(5, 0,1,2,3) or SelectEvery(5, 1,2,3,4) which they all do the same thing, restore back to 24p but are no longer working in many of todays captured videos -- at least in mine they don't.

So, moving out the old 3:2 pattern decode -- selectevery(5,0,1,2,3) -- method, and replacing it is the newer generation (time expansion/compression) method, selectevery(10, 0,1,2,3,4, 5,6, 9) in avisynth.

Here is the AVIsynth method for restoring 30i -> 24p frame rate.

old method, may still work of some broadcasted Film sources:

x = "c:\videos\filename.avi"
trim( 0,0 )
AVIsource( x ) 
separatefields()
 #selectevery(5, 0,1,2,3) # or use selectevery(5, 1,2,3,4)
weave()

newer method, for broadcasted Film sources:

x = "c:\videos\filename.avi"
trim( 0,0 )
AVIsource( x ) 
separatefields()
 selectevery(10, 0,1,2,3,4, 5,6, 9) 
weave()

I used the script above to restore all the Star Wars episodes on SpikeTV, All the Indys, The Mummy, and The Mummy Returns, and many more.

* I use the avisynth trim() for marking my staring position of the PPPii pattern.

I have not had the time to test the TMPGenc alternative for its pattern guide function. I only know the 10100 for the older 3:2 pattern layout. Perhaps someone with more (TMPGenc) knowledge of its pattern guidance setup has the answer. Otherwise, I'll try and figure it out when I find some spare time.

{EDIT}
* 1, I justed wanted to note here that older movies, such as those on laserdisc and vhs can use the older avisynth method, selectevery(5, 0,1,2,3) or the TMPGenc pattern guidance method, 10100 on these.
* 2, But since the change in broadcast air time, a new era has begun with respect to the telecine of film based sources in the form of time expansion/compression. So the 3:2 pattern is manipulated at the field level, and making it more difficult to restore.
{/EDIT}

-vhelp 4835

[FulciLives]

You are better off leaving them interlaced if the final output is to be a DVD Video.

As I understand it ... you captured in MPEG-2 format and want to edit out the commercials and then put on a DVD Video. Assuming the original capture is DVD compliant MPEG-2 then no re-encoding is needed.

Just cut out the commercials and use TDA to author it. Hell you can use TDA to do the editing as well.

- John "FulciLives" Coleman

[yoda313]

As I understand it ... you captured in MPEG-2 format and want to edit out the commercials and then put on a DVD Video. Assuming the original capture is DVD compliant MPEG-2 then no re-encoding is needed.

Just cut out the commercials and use TDA to author it. Hell you can use TDA to do the editing as w

EDIT - Thanks thats a plan.

yoda313, at your request, here is a basic TMPGenc ivtc process you can try.

If you know for sure your captured videos originate from Film:

1 -- Settings\Advanced\[x]source range, and set your starting point of the "PPPii" mark.
2 -- Settings\Advanced\Inverse telecine
2a - click the [clear] button first, always do this to remove any random patters from cache
2b - select that first frame (above step 1) by clicking it then, right-click and select 'deal after this frame..'
2c - now, in the box, enter the value, 10100

Thanks. I'm not sure I understand all that right now. I'll have to experiment when I get a little more time to get involved in this.

So is broadcast 480i top field or bottom field?

[jagabo]

yoda313, at your request, here is a basic TMPGenc ivtc process you can try.

If you know for sure your captured videos originate from Film:

1 -- Settings\Advanced\[x]source range, and set your starting point of the "PPPii" mark.
2 -- Settings\Advanced\Inverse telecine
2a - click the [clear] button first, always do this to remove any random patters from cache
2b - select that first frame (above step 1) by clicking it then, right-click and select 'deal after this frame..'
2c - now, in the box, enter the value, 10100

Thanks. I'm not sure I understand all that right now. I'll have to experiment when I get a little more time to get involved in this.

That will only work properly if you start on the right frame and there are no telecine pattern breaks.

So is broadcast 480i top field or bottom field?

Broadcast video is neither top nor bottom field first. It is simply an alternating sequence of top and bottom fields (which came first the chicken or the egg?). The capture device captures fields and combines them into frames. If it starts with a top field then adds the next field (which will always be a bottom field) to complete the frame the frame is TFF. If it starts with a bottom field then adds the next field (which will always be a top field) to complete the frame the frame is BFF. So the capture device determines the field order.

[yoda313]

@jagaboo - thanks.

Regarding the capture device that could be a problem. I am using two tuners with windows media center and as far as I can tell there is no marker to determine which capture device it came from.

I have a fusion hdtv 3 pci card and a pinnacle hd pro usb stick. I am not sure how I would determine which it came from since they all get recorded to a dvr-ms file.

Would gspot be able to analzye that info for me? What settings would I lok for?

[jagabo]

GSpot can tell you the field order of MPEG files but it's not totally reliable.

Open your files in VirtualDub and apply the Bob Doubler filter (pick TFF or BFF). Step through the video and watch the output frame. If the motion results in a "2 steps forward, 1 step back" appearance the field order is wrong. Ie, if you pick TFF in the Bob Doubler filter and the motions jerk back and forth your video is BFF. Note that telecined film will show repeats but with the right field order will not jerk back and forth.

AviSynth's Bob() filter can be used for this too:

MPEG2Source("filename.d2v")
AssumeTFF()
Bob()

If you get back-and-forth motion with this script your video is BFF.