Deinterlace for progressive?

[junkmalle]

For MPEG-2 video, we support three types of conversion; Blending, Direct Waving and Interpolating. We use the Blending method by default which gives good results on still images and slow moving images.

This is similar to VirtualDub's Blend deinterlace. It simply blurs the two fields together looking like a double exposure when there is fast motion. I've seen ConvertX samples with this method.

For fast moving images, the Interpolation method provides best results but this method is not great for still images.

It's not perfectly clear what this means but from the few ConvertX stills I've seen I suspect they drop a field and generate the second field by interpolating from the scanlines of the first. This leads to jagged edges on sharp edged objects. You can simulate this in VirtualDub by using the Discard Field deinterlace filter followed by the resize filter (bilinear, bicubic, lanczos?) to restore the picture height. You can verify this by capturing a telecined source. Step through the frames. You will see a sequence of 4 different images followed by a duplicate, 4 different frames and a duplicate... Or you can capture a test image where one field is all one color and the other field is all another color. You will get a cap that's all one color. Here's a sample image I posted a while back:

https://www.videohelp.com/forum/viewtopic.php?p=1277021#1277021

The Direct Waving method is great for still images but bad for slow and fast moving images.

I think they meant "weave" not "wave". This would be the interlaced mode. Progressive 29.97 fps would end up progressive (not much of this in the NTSC SD broadcast world), telecined movies would remain telecined (60/40 mixture of progressive and interlaced frames), a fully interlaced video source would remain fully interlaced. I don't know how you reconcile this with the claim that they always deinterlace. Maybe they meant that only for the MPEG 4 mode.

[edDV]

OK commence deinterlace rant:

Case 1: Let's start with normal NTSC or PAL interlaced TV:
Assume our goal is to capture an interlaced TV show from a tuner or composite/S-video input. Lets assume we want to edit out the commercials and store several such shows to a standard interlaced DVD. Lets assume horizontal resolutions of 720, 704 or 352 can be used.

I contend that best image quality will result from maintaining interlace and full vertical resolution from capture to DVD at 480i/576i. Any deinterlacing at the capture card or by filtering will degrade the 1x size image. The DVD will play directly on an interlace TV. An HDTV will offer internal options to upscale and/or progressive convert the 480i/576i input to its native display.

Case 2: Same as Case1 but the show is a movie.
ref: http://www.dvdfile.com/news/special_report/production_a_z/3_2_pulldown.htm
ref: http://hometheater.about.com/od/beforeyoubuy/a/progressivescan_2.htm
ref: http://www.hometheaterhifi.com/volume_7_4/dvd-benchmark-part-5-progressive-10-2000.html
Should we try to create a progressive DVD or let the Progressive DVD player or HDTV do the conversion. The idea is to software detect and inverse the 3:2 pulldown sequence to create a 24 fps progressive image MPeg2, then flag it as progressive and let the DVD player try to play out the progressive or interlace stream. The player will need to recreate 3:2 pulldown for the interlace output or repeating frame sequences for 60 frame per second progressive playout.

I say let the DVD player or ED/HDTV do it all. The hardware deinterlacers are getting better and more rubust. This will only improve with time. Your home brew deinterlace is frozen in pixels on the DVD for all time.

Other reasons not to software deinterlace yourself? Most available deinterlacers do more harm than good to the 1x size image. Some of the common techniques.

- drop odd or even fields.
This gets you a progressive image but with half the vertical resolution and half the temporal resolution. (i.e. 1/30 vs 1/60 second motion resolution)

- weave
The idea is to combine odd and even fields for a progressive frame every 1/30 or 1/60th second. If there is no motion, this appears to work but during motion the 1/60 sec time discontinuity between alternating lines makes the image tear and break into a double vision effect.

- "blend" or "blur"
This is an attempt to hide the line tear and double image by mixing or blurring motion areas. The results are not natural.

- bob
This is a progression of 60 fields per second with the missing lines filled in with interpolated values from prior and subsequent fields. The advantage is a more contained image during motion, but at the cost of overall picture blur.

- motion adaptive deinterlace
Here is where sophisticated motion analysis alogithms are applied to separate motion from still areas of the image. Low motion areas get a weave (higher detail), motion areas get a bob or single field. Intemediate motion areas may get a combination of bob and weave.

- motion prediction and compensation
This adds more analysis to apply directional filtering along motion paths to make a more natural transition between high and low motion areas.

Software deinterlacers don't do the last two, or at best do them crudely because the processing overhead is high and there are patent complications. Hardware deinterlacers in broadcast equipment produce excellent results, but cost upwards from $10,000. This more sophisticated hardware is gradually working down into DVD player and HDTV internal deinterlacers along with 3:2 pulldown detectors and compensators.

Here is my final point. If you keep the DVD data in original interlace format, you are going to get a good interlace playback on a conventional TV. If you export your software generated progressive concoction to the DVD authoring program, the DVD player is going to have to recompute an interlace output for a normal TV that most likely will seriously diverge from the original. Not to mention the problems your future HDTV will have with that DVD.

/rant

comments and opinions welcome.

[manono]

Hi edDV-

comments and opinions welcome.

Ok, I'll take that at face value.

Case 1. I can't argue with keeping interlaced material interlaced for standard interlaced displays, but for progressive displays (computer monitors, HDTVs and the like), I think a good case can be made for using good AviSynth deinterlacers, which you evidently aren't aware of.

As you well know, encoding interlacing takes a considerably higher bitrate for the same quality when compared to deinterlacing it and encoding it as progressive. I've read estimates of upwards of 50% higher bitrate for the same quality when encoding the interlacing/combing. When trying to compress a DVD9 at 7-8 GB original size down to a DVDR, this can become a major factor.

An HDTV will offer internal options to upscale and/or progressive convert the 480i/576i input to its native display.

Meaning that the progressive display is going to deinterlace it anyway. So, it then comes down to the quality of the internal deinterlacer of the DVD player or the display. Also as you well know, the quality of these deinterlacers varies wildly, and many are crap. For example, there's no software player that deinterlaces as well as does a good AviSynth deinterlacer (although I've read the new NVidia player is pretty good). Now, if your player and/or display has Faroudja DCDi or something similar, then, of course, it'll do a good job. But how many people with all progressive setups have such quality deinterlacers? I would guess very few. I think a good case can be made for deinterlacing interlaced material and then encoding it as progressive. One reason is for the bitrate savings, and the other is that when using a quality deinterlacer, you are often using a better deinterlacer than you might have in your player/display. People with progressive setups should be aware of just how their interlaced material is being deinterlaced, and make their decision on whether or not to encode interlaced based on that.

Case 2. I say let the DVD player or ED/HDTV do it all.

You aren't really suggesting that hard telecined material should remain that way, without being IVTC'd are you? That is to say, are you saying that 29.97fps material shouldn't be IVTC'd when possible to restore the progressive frames so that 23.976fps progressive frames can be encoded? If so, in my opinion you couldn't be more wrong. And let me count the ways. First, when hard telecined, up to 40% of the frames exhibit combing, which for the same reasons as earlier require a higher bitrate to maintain quality. Second, when encoding hard telecined material, you're encoding 29.97fps, as opposed to encoding 23.976fps, and you're encoding 25% more frames than necessary, and the precious bits are being spread around 25% more frames, and you are taking a major quality hit. Again, this becomes a major consideration when compressing a DVD9 down to a DVDR with only 4.37 GB of data available. And lastly, for those with progressive setups, the vast majority of the players/displays out there are pure flag readers. Hard telecined material is always encoded as interlaced without the flags that might signal progressive material, and such players will deinterlace that material, giving you major quality degradation upon playback. Only cadence readers (such as those with the Faroudja chipset or similar) can spot the 3:2 pattern, realign the progressive frames, and give you clean output.

Other reasons not to software deinterlace yourself? Most available deinterlacers do more harm than good to the 1x size image.

The same can be said for the deinterlacers in the software players, and most Progressive Scan DVD players, and most HDTVs.

Software deinterlacers don't do the last two

I know of at least two AviSynth deinterlacers that do motion adaptive deinterlacing. As for motion prediction and compensation; if by that you mean something like what Faroudja's DCDi does (and I'm sure you do), then there's already a VDub Deinterlacer available that emulates DCDi, and there will soon be one available as an AviSynth deinterlacing filter.

Sure, except for the bitrate savings, encoding as interlaced will play fine on standard interlaced TV sets. But for those already with progressive setups, including HTPCs, where the material must be displayed progressive, or for those planning on getting such a setup in the future, I don't think it's nearly as cut-and-dried as you make it out to be.

Back to the hard telecined material that you don't think should be IVTC'd. Although most Hollywood movies are encoded as progressive these days, there is a wealth of DVDs from overseas companies that are hard telecined. In addition, the extras on Hollywood DVDs are as a matter of course hard telecined and encoded as interlaced. When you want to keep the extras when making a backup, but give them a lower bitrate, IVTCing them and encoding as progressive can often make the difference between something that looks like crap and something that looks decent.

Comments and opinions welcome.

[lordsmurf]

Leave interlaced alone. It's really that easy. Unless you have equipment that cost about $10,000 or more, don't even think about touching it.

[edDV]

Case 1. I can't argue with keeping interlaced material interlaced for standard interlaced displays, but for progressive displays (computer monitors, HDTVs and the like), I think a good case can be made for using good AviSynth deinterlacers, which you evidently aren't aware of.

I'll answer in parts because it's late.
First as stated earlier in the thread, I'm not talking about computer display or high compression file sharing goals. I'm limiting these comments to playback on TV sets (interlaced and HDTV) from standard spec DVD players.

If you think you can make a case for AviSynth deinterlacers vs hardware for this purpose, make it. I think you will find the processing time and bother excessive. However I'm all ears.

As you well know, encoding interlacing takes a considerably higher bitrate for the same quality when compared to deinterlacing it and encoding it as progressive. I've read estimates of upwards of 50% higher bitrate for the same quality when encoding the interlacing/combing. When trying to compress a DVD9 at 7-8 GB original size down to a DVDR, this can become a major factor.

Are you refering to removal of repeated fields for film sources? Granted there is 25% 3:2 padding to get 23.976 fps to 29.97 fps. This is for film sources only. Regular TV has no such padding.

An HDTV will offer internal options to upscale and/or progressive convert the 480i/576i input to its native display.
Meaning that the progressive display is going to deinterlace it anyway. So, it then comes down to the quality of the internal deinterlacer of the DVD player or the display. Also as you well know, the quality of these deinterlacers varies wildly, and many are crap.

That is exactly what we are talking about. We are talking about the quality tradeoffs and effort of software deinterlacing vs. hardware algorithms in mid to upper range progressive DVD players and HDTV.

For example, there's no software player that deinterlaces as well as does a good AviSynth deinterlacer (although I've read the new NVidia player is pretty good).

We aren't talking about playback on computer displays.

Now, if your player and/or display has Faroudja DCDi or something similar, then, of course, it'll do a good job. But how many people with all progressive setups have such quality deinterlacers? I would guess very few. I think a good case can be made for deinterlacing interlaced material and then encoding it as progressive. One reason is for the bitrate savings, and the other is that when using a quality deinterlacer, you are often using a better deinterlacer than you might have in your player/display. People with progressive setups should be aware of just how their interlaced material is being deinterlaced, and make their decision on whether or not to encode interlaced based on that.

First, for interlaced TV display, an interlaced DVD will play with greatest quality. Your only point so far is that film source progressive conversion can increase disc utilization. My point is progressive conversion (other that precise 3:2 sequence removal) can destroy image quality.

When it comes to progressive playback to ED/HDTV, newer progressive DVD players with "cinema" chipsets and 3:2 sequence removal are getting pretty good with detection and progressive playback of film sources. Soon this process will be fully automatic in both the player and HDTV itself.

Progressive conversion of non film 480i/576i inputs inside the HDTV is rapidly improving as well. High end hardware motion adaptive/predictive deinterlacers are just around the corner and will be common as HDTV gains market share.

You aren't really suggesting that hard telecined material should remain that way, without being IVTC'd are you? That is to say, are you saying that 29.97fps material shouldn't be IVTC'd when possible to restore the progressive frames so that 23.976fps progressive frames can be encoded?

Film source is a gray area. I'm saying that it is becoming 50/50 whether this is done before encoding or automatically in the progressive DVD player or HDTV.

If so, in my opinion you couldn't be more wrong. And let me count the ways. First, when hard telecined, up to 40% of the frames exhibit combing, which for the same reasons as earlier require a higher bitrate to maintain quality.

Their is no combling. Film progressive reconstruction is a simple reverse process and is being automated in the player and the TV. I've agreed that interlace carries 25% DVD capacity overhead, but no quality disadvantage.

Second, when encoding hard telecined material, you're encoding 29.97fps, as opposed to encoding 23.976fps, and you're encoding 25% more frames than necessary, and the precious bits are being spread around 25% more frames, and you are taking a major quality hit. Again, this becomes a major consideration when compressing a DVD9 down to a DVDR with only 4.37 GB of data available.

Again, 25% overhead yes, quality hit no. And only for film source.

And lastly, for those with progressive setups, the vast majority of the players/displays out there are pure flag readers. Hard telecined material is always encoded as interlaced without the flags that might signal progressive material, and such players will deinterlace that material, giving you major quality degradation upon playback. Only cadence readers (such as those with the Faroudja chipset or similar) can spot the 3:2 pattern, realign the progressive frames, and give you clean output.

Once you get above the $40 K-mart players, automatic 3:2 detection and removal is becoming standard. Anyone with a progressive TV should be using such a progressive DVD player. For interlace TV viewers all of this is irrelevant.
So We have talked the 3:2 film subject to death and concluded HDTV viewers should be using a "Cinema" quality progressive DVD player with 3:2 detection and the interlace DVD carries a 25% capacity penalty.

Other reasons not to software deinterlace yourself? Most available deinterlacers do more harm than good to the 1x size image.

The same can be said for the deinterlacers in the software players, and most Progressive Scan DVD players, and most HDTVs.

We aren't talking about software players.
Progressive Scan DVD player and HDTV hardware deinterlacers are getting much better and will continue to do so.
HDTV deinterlacers are engineered for maximum quality for that particular native display.
The question is mute for the film material above which is 480p/576p native.

Software deinterlacers don't do the last two

I know of at least two AviSynth deinterlacers that do motion adaptive deinterlacing. As for motion prediction and compensation; if by that you mean something like what Faroudja's DCDi does (and I'm sure you do), then there's already a VDub Deinterlacer available that emulates DCDi, and there will soon be one available as an AviSynth deinterlacing filter.

Realtime hardware deinterlacers are getting very good as high end algorithms are being included. Software can't compete. Processing times are excessive.
DCDi technology is already in $1200 projectors after being introduced at $50K. This and other hardware technology is landing in mass market players and HDTV processors.

Sure, except for the bitrate savings, encoding as interlaced will play fine on standard interlaced TV sets. But for those already with progressive setups, including HTPCs, where the material must be displayed progressive, or for those planning on getting such a setup in the future, I don't think it's nearly as cut-and-dried as you make it out to be.

HTPCs, like HDTVs should have automatic 3:2 detection and removal for normal TV viewing and they will.

Back to the hard telecined material that you don't think should be IVTC'd. Although most Hollywood movies are encoded as progressive these days, there is a wealth of DVDs from overseas companies that are hard telecined. In addition, the extras on Hollywood DVDs are as a matter of course hard telecined and encoded as interlaced. When you want to keep the extras when making a backup, but give them a lower bitrate, IVTCing them and encoding as progressive can often make the difference between something that looks like crap and something that looks decent.
Comments and opinions welcome.

Well, we are back to disk capacity as the only arguement. Maybe if and when dual layer DVD media prices get reasonable and HD DVD is introduced, that will become mute.

I think the hardware trends fall to my side of the arguement.

The inverse 3:2 (IVTC) path to a progressive DVD was not my main target. That is a straight forward process that does not damage the picture and has disc capacity advantages.

My target was the knee jerk rush to deinterlace standard 29.97 fps video and apply filters that will destroy quality on an interlaced TV and/or HDTV. I still say keep it interlaced with the 50/50 exception of film sources that can be simply converted to progressive. .

[manono]

Hi-

I said that I can't argue with keeping interlaced material interlaced for watching on an interlaced display. I can and will argue against leaving film telecined prior to encoding for broadcast or for DVD alone, whether for an interlaced or a progressive display.

First as stated earlier in the thread, I'm not talking about computer display or high compression file sharing goals.

I don't know why not. There are plenty of people out there sending their HTPC to a progressive display, just as there are plenty watching on a computer. No one said anything about file sharing.

If you think you can make a case for AviSynth deinterlacers vs hardware for this purpose, make it. I think you will find the processing time and bother excessive.

It's my hobby. It takes time, but the time and "bother" is worth it if the end result is good. Since you linked to one of the Secrets Of Home Theater pages earlier, then I'm sure you've seen this one before:
http://www.hometheaterhifi.com/cgi-bin/shootout.cgi?function=search&articles=all#Lexic...0Disc%20Player

Start at the top, and find the Motion Adaptive column, and then follow it down. You'll see that as you go down the column, more and more of that column is in the red, until you get to the bottom and realize that the majority of these Progressive Scan DVD players don't have Motion Adaptive deinterlacing, and none of the software players have it. As I said last time, whether or not you choose to deinterlace depends at least partly on your player's deinterlacing capabilities. This page rates and compares the best NTSC progressive scan players, many costing over a thousand dollars. And price is by itself no indication of quality at all. So, if by "mid to upper range" you mean the most expensive ones, well that's no guarantee. They also rate the popular ones, and a lot of lousy ones. The Secrets people say elsewhere that, in general, the players are better at this than are the displays. I stand by my point that there are AviSynth deinterlacers that produce better output for progressive displays than do the majority of the players and displays themselves. Sure, you can talk all you want about the lousy Cyberhome and Toshiba players, and how they shouldn't be counted, but the fact remains that they are very popular, with many thousands of people having bought them.

Are you refering to removal of repeated fields for film sources?

I wasn't there, but I will now. As I said in the last post, there is a lot of film sourced material where 29.97fps is actually stored on the DVD (so-called hard telecine). The extras on DVDs are perhaps the easiest place to find it, but there are many thousands of DVDs where whole movies are stored that way. Then there are the many, many cases where the film drops to video from time to time, during opening logos, at chapter breaks, or where bad edits have been made. The kind where the DVD shows in DVD2AVI (DGIndex) as less than 100% FILM (or as 0.00% FILM or pure NTSC in the case of hard telecine). These can and should be IVTC'd, both for the bitrate savings, and because many progressive scan players will screw up the playback, deinterlacing such material. Again, the majority of Progressive Scan DVD players are pure flag readers. Go back to that player comparison link, and this time follow the 3-2 Cadence, Video Flags, and 3-2 Cadence, Mixed Flags down (3rd and 4th columns over from the left). The majority of the players fail on such material.

Film source is a gray area. I'm saying that it is becoming 50/50 whether this is done before encoding or automatically in the progressive DVD player or HDTV.

In my opinion it's not a gray area at all. Again, what's the point in encoding 25% more frames than you have to, when space on a DVDR is at a premium, and the chances are very good that your progressive scan DVD player doesn't have 3:2 Pulldown detection. Sure, in the future more and more will have it. I'm talking about what's available now, not in the future.

Again, 25% overhead yes, quality hit no. And only for film source.

Quality hit, yes, in terms of the higher average quant that will result from encoding 25% more frames for a fixed file size (in the case of DVDR, a max of 4.37 GB).

Once you get above the $40 K-mart players, automatic 3:2 detection and removal is becoming standard. Anyone with a progressive TV should be using such a progressive DVD player. For interlace TV viewers all of this is irrelevant.
So We have talked the 3:2 film subject to death and concluded HDTV viewers should be using a "Cinema" quality progressive DVD player with 3:2 detection and the interlace DVD carries a 25% capacity penalty.

Don't patronize me. Check that link and you'll see thousand dollar players that can't do it. 3:2 Pulldown detection is far from the standard just yet. In addition, I believe that it can refer to only progressively encoded material, with the proper flags set, and doesn't necessarily handle hard telecine well. There are even the newest chipsets that can't do it. What people should be using and what they are using are often 2 different things.

DCDi technology is already in $1200 projectors ...

Heck, it's even in $200 DVD players now. I own one, the second on the list, beat in the ratings only by a $3,500 Denon player.

My target was the knee jerk rush to deinterlace standard 29.97 fps video and apply filters that will destroy quality on an interlaced TV...

Sure, I agree with you. Where I disagree is when you extrapolate to saying that the same thing applies to all progressive displays all the time. I do not agree that the way it is on the DVD is the way it should always be kept. Heck, I even reencode DVD5s when they have encoded a film source as interlaced, even though my player and TV can both handle it OK. It just irks me when I see bad mastering and bad encoding practices. You're a pretty sharp fellow, and I've learned much from reading your posts, edDV. I wouldn't expect you to change your ways, but I just hope to let others know that there are differing views on the subject, and that sometimes you can improve on the source, particularly when compressing down from DVD9 to DVD5. Nice talking to you, edDV. I apologize for the long posts.

[jeanl]

I'm really enjoying this exhange of points of view guys! (I'm not being sarcastic here!). There's a ton of info in everything you guys mention. Really useful stuff. Thanks a bunch to both of you!
jeanl

[pfh]

I concur with jeanl- I've saved this and other threads for this reason. Great discussion.

[edDV]

I'm really enjoying this exhange of points of view guys! (I'm not being sarcastic here!). There's a ton of info in everything you guys mention. Really useful stuff. Thanks a bunch to both of you!
jeanl

That is the fun of debate.

[edDV]

I said that I can't argue with keeping interlaced material interlaced for watching on an interlaced display. I can and will argue against leaving film telecined prior to encoding for broadcast or for DVD alone, whether for an interlaced or a progressive display.

First as stated earlier in the thread, I'm not talking about computer display or high compression file sharing goals.

I don't know why not. There are plenty of people out there sending their HTPC to a progressive display, just as there are plenty watching on a computer. No one said anything about file sharing.

We agree that most people today are driving interlace TV sets and have never used the progressive features of their DVD player. They are recording with DVR, Tivo or computer tuner/capture cards. Many now are moving to DVD standalone recorders.

As for tuner/capture cards, most people capture to some form of MPeg and most have settled into a Low Quality/High Quality strategy for recording to HDD and DVD.

For low quality recording, they have a choice. Will it just play on the computer? or Must it play on a DVD player? If the latter, they choose a format from VCD, SVCD, DVD-MPeg1, DVD-MPeg2 (low rate) or attempt player specific MPeg4 based formats (wmv,DivX,XviD,etc.). For this use, only extreme compression junkies care about progressive conversion. They just play these utility recordings from the HDD or DVD and accept the video quality.

High Quality shows or movies that they want to save to DVD are are collected to large hard disks usually in MPeg2 (~2hr mode in something like 5-7Kbps VBR) in interlaced NTSC or PAL format and most will be simply copied/authored to DVD.

Most people will just author this material to DVD interlaced and be done with it. It looks good enough even on a HDTV after upscaling the 480i material.

Purists may take extreme measures attempting to capture a TV film broadcast at higher quality (uncompressed, Hufyuv, DV etc.) filtering and authoring it into a progressive DVD approaching the quality of a store bought commercial DVD. They will capture to huge disk space, do the inverse 3:2 pulldown, filter and then author the DVD. The process will take many hours per movie.

Most others would rather purchase the DVD (a superior product) for the few movies that they care to keep in highest quality form. For the rest of the library, 480i is good enough.

That is my attempt to describe the current environment. I agree with you that this will change in the future. My vision of the future focuses on automation through hardware. I see the Avisynth world as a purist monk like cult that will continue to exist but in more isolation. OK I overstated that last sentance. Sorry.

If you think you can make a case for AviSynth deinterlacers vs hardware for this purpose, make it. I think you will find the processing time and bother excessive.

It's my hobby. It takes time, but the time and "bother" is worth it if the end result is good. Since you linked to one of the Secrets Of Home Theater pages earlier, then I'm sure you've seen this one before:
http://www.hometheaterhifi.com/cgi-bin/shootout.cgi?function=search&articles=all#Lexic...0Disc%20Player

Start at the top, and find the Motion Adaptive column, and then follow it down. You'll see that as you go down the column, more and more of that column is in the red, until you get to the bottom and realize that the majority of these Progressive Scan DVD players don't have Motion Adaptive deinterlacing, and none of the software players have it. As I said last time, whether or not you choose to deinterlace depends at least partly on your player's deinterlacing capabilities. This page rates and compares the best NTSC progressive scan players, many costing over a thousand dollars. And price is by itself no indication of quality at all. So, if by "mid to upper range" you mean the most expensive ones, well that's no guarantee. They also rate the popular ones, and a lot of lousy ones. The Secrets people say elsewhere that, in general, the players are better at this than are the displays. I stand by my point that there are AviSynth deinterlacers that produce better output for progressive displays than do the majority of the players and displays themselves. Sure, you can talk all you want about the lousy Cyberhome and Toshiba players, and how they shouldn't be counted, but the fact remains that they are very popular, with many thousands of people having bought them.

I will grant you that for the present situation. We are in transition. I'm looking at the next generation chipsets that are coming in with the SD and HD DVD players, set top boxes (STB) and internal HDTV deinterlacers. As the average TV set moves from interlaced CRT to LCD and Plasma (and yes HTPC) the progressive issue needs to be solved in a fundamental way.

Unfortunaly the broadcast and cable community are dragging their heels on progressive broadcasting. Most of what we get is 1080i because cheaper mass market HD ready sets are CRT tubes or DLP projectors optimized for 1080i.

Next wave sets (1-3yrs) will be progressive 480p/576p EDTV, 720p or 1080p (1440x1080 to 1920x1080). Broadcasters will still be mostly feeding us 480i and 1080i. The deinterlacing problem needs to be fundamentally solved in hardware in these sets and/or in the HD DVD player and STB. I expect the internal deinterlacers to get very sophisticated and expect that they will first focus on making 480i look good because that is what most people will be playing into their new HDTV. Remember that all those millions of DVD standalone recorders are producing 480i DVDs.

Are you refering to removal of repeated fields for film sources?

I wasn't there, but I will now. As I said in the last post, there is a lot of film sourced material where 29.97fps is actually stored on the DVD (so-called hard telecine). The extras on DVDs are perhaps the easiest place to find it, but there are many thousands of DVDs where whole movies are stored that way. Then there are the many, many cases where the film drops to video from time to time, during opening logos, at chapter breaks, or where bad edits have been made. The kind where the DVD shows in DVD2AVI (DGIndex) as less than 100% FILM (or as 0.00% FILM or pure NTSC in the case of hard telecine). These can and should be IVTC'd, both for the bitrate savings, and because many progressive scan players will screw up the playback, deinterlacing such material. Again, the majority of Progressive Scan DVD players are pure flag readers. Go back to that player comparison link, and this time follow the 3-2 Cadence, Video Flags, and 3-2 Cadence, Mixed Flags down (3rd and 4th columns over from the left). The majority of the players fail on such material.

Film source is a gray area. I'm saying that it is becoming 50/50 whether this is done before encoding or automatically in the progressive DVD player or HDTV.

In my opinion it's not a gray area at all. Again, what's the point in encoding 25% more frames than you have to, when space on a DVDR is at a premium, and the chances are very good that your progressive scan DVD player doesn't have 3:2 Pulldown detection. Sure, in the future more and more will have it. I'm talking about what's available now, not in the future.

Again, 25% overhead yes, quality hit no. And only for film source.

Quality hit, yes, in terms of the higher average quant that will result from encoding 25% more frames for a fixed file size (in the case of DVDR, a max of 4.37 GB).

Once you get above the $40 K-mart players, automatic 3:2 detection and removal is becoming standard. Anyone with a progressive TV should be using such a progressive DVD player. For interlace TV viewers all of this is irrelevant.
So We have talked the 3:2 film subject to death and concluded HDTV viewers should be using a "Cinema" quality progressive DVD player with 3:2 detection and the interlace DVD carries a 25% capacity penalty.

Don't patronize me. Check that link and you'll see thousand dollar players that can't do it. 3:2 Pulldown detection is far from the standard just yet. In addition, I believe that it can refer to only progressively encoded material, with the proper flags set, and doesn't necessarily handle hard telecine well. There are even the newest chipsets that can't do it. What people should be using and what they are using are often 2 different things.

DCDi technology is already in $1200 projectors ...

Heck, it's even in $200 DVD players now. I own one, the second on the list, beat in the ratings only by a $3,500 Denon player.

OK, I grant you that it isn't all there yet but rapidly approaching. Doing the inverse 3:2 prior to encoding will be a personal choice by the user on a DVD to DVD basis. It is a simple procedure but currently needs manual hands on and time to do. I would like to see this process automated in DVD authoring programs.

I don't really associate reversing 3:2 to deinterlacing. It is a reordering of fields that restores a natively progressive source. It should be automated.

By deinterlacing, I think in terms of dealing with camera generated interlace material. I still believe that that sort of material should be left interlaced on the DVD by most users except in extreme cases. Future HDTV deinterlacers will be able to deal with 480i DVDs better than today's software deinterlacing filters.

My target was the knee jerk rush to deinterlace standard 29.97 fps video and apply filters that will destroy quality on an interlaced TV...

Sure, I agree with you. Where I disagree is when you extrapolate to saying that the same thing applies to all progressive displays all the time. I do not agree that the way it is on the DVD is the way it should always be kept. Heck, I even reencode DVD5s when they have encoded a film source as interlaced, even though my player and TV can both handle it OK. It just irks me when I see bad mastering and bad encoding practices. You're a pretty sharp fellow, and I've learned much from reading your posts, edDV. I wouldn't expect you to change your ways, but I just hope to let others know that there are differing views on the subject, and that sometimes you can improve on the source, particularly when compressing down from DVD9 to DVD5. Nice talking to you, edDV. I apologize for the long posts.

I don't think we are in disagreement on anything other than the labor required to get an acceptable quality 720x480 DVD. I'm more lazy than you ... that and I probably author 10x the DVD's you do and I have it down to a fast realtime encoding process for 480i DVDs. Most of my recordings are not movies. I'm a Discovery-History-Sci-Wings-PBS-InHD junkie and I'm time shifting mostly those shows.

As said above, my opinion is progressive TV and HTPC environments need high quality automated internal deinterlacing as a fundamental requirement.

[jeanl]

manono, the link you provided was absolutely phenomenal. Not just the comparison of players, but the technical notes on DVD flags, inverse pulldown (heck, even the chroma bug) were absolutely first class. I've learned more in the last couple hours than in the last couple months!

1GThanks!

jeanl

[edDV]

manono, the link you provided was absolutely phenomenal. Not just the comparison of players, but the technical notes on DVD flags, inverse pulldown (heck, even the chroma bug) were absolutely first class. I've learned more in the last couple hours than in the last couple months!

1GThanks!

jeanl

Isn't it cool how a little debate can draw out the best links?

And remember it is the duty of the wealthy to buy these machines in order to stimulate the state-of-the-art and grow the economy. AC-3 went through a similar debug process before reaching $40 K-Mart DVD players.

[jeanl]

Oh, not just the links man! The whole discussion is very enjoyable. MPEG video specifications are fairly complicated to the newcomer, and it's great to hear the same thing expressed many different ways. Also great to hear different things expressed different ways!
Keep it up guys!
Jeanl

[pfh]

You are so right jeanl.
I'm on the line of many of edDV's comments. I especially like the idea of automating the inverce 3:2 in an authoring stage.
I'd think everyone would agree that the near future looks pretty good for NLE, dvd, displays, HT, etc.
Huh, I would have never thought I would enjoy learning and messing with video.

[lordsmurf]

The #1 issue people forget, regarding IVTC, is that takes an enormous BLIND LEAP OF FAITH on your part that the telecine was done perfectly, and that it can be undone with a standard method. In many many cases, it cannot.

Even a "proper" IVTC often leaves you with chroma interlace trails. "Red" lines and other linear "color" artifacts in the video.

Your filesize is not all that affected by retaining interlace. I don't really think it approachs 25%, probably closer to 10% in practice. At least on good hardware/software.

[jeanl]

The #1 issue people forget, regarding IVTC, is that takes an enormous BLIND LEAP OF FAITH on your part that the telecine was done perfectly, and that it can be undone with a standard method. In many many cases, it cannot.

Can you expand on that? Apart from sequence breaks at edit points (etc) I don't see what can go wrong in doing the telecine...
jeanl

[jeanl]

The discussion prompted one question: I read here (and I've read elsewhere) that interlaced material is inherently harder to encode than progressive source. I'm puzzled. If the source frame rate is the same, say 30fps, then I'm not sure I understand why a progressive source would be easier to encode than an interlaced one, as the encoder has the liberty to encode fields as a full frame, or independently (and even parts of the fields, at the macroblock level).
Of course, if the source is true film at 24fps, then I can see why that would be easier to encode than true interlaced (or progressive for that matter) 30fps... But other than that, what would be a technical argument for justifying the above assertion?

jeanl

[edDV]

For inverse telecine (aka 3:2, IVTC) pick your source carefully. It's one thing to have a TV screw up the display, its another to encode the mistake into your only DVD copy.

Solid sources would be your major movie channels (HBO, Starz, TBS, Network prime time movies, etc.). They are probably playing back from D1 (or better 10 bit) derived masters from a Rank flying spot scanner film transport.

Joe's UHF channel may be projecting a 10th generation 16mm print to the wall and shooting it with a home camcorder. No deinterlace possible there More likely it's a 10year old Betacam tape of unknown history.

Mileage will vary.

[edDV]

The discussion prompted one question: I read here (and I've read elsewhere) that interlaced material is inherently harder to encode than progressive source. I'm puzzled. If the source frame rate is the same, say 30fps, then I'm not sure I understand why a progressive source would be easier to encode than an interlaced one, as the encoder has the liberty to encode fields as a full frame, or independently (and even parts of the fields, at the macroblock level).
Of course, if the source is true film at 24fps, then I can see why that would be easier to encode than true interlaced (or progressive for that matter) 30fps... But other than that, what would be a technical argument for justifying the above assertion?

jeanl

It's a bias from computer science majors who are only taught 1kx1k (sometimes 640x480) progressive raster frames. Real world video operates in interlace and has for 50+ years.

There are graphics gurus and there are video gurus. They seldom find reason to speak to each other.

Graphics guys are usually found in internet forums demanding that interlaced video be banned as an environmental hazzard. Video guys are found designing motion adaptive deinterlacer chips for next generation HDTV receivers.

Of course, if the source is true film at 24fps, then I can see why that would be easier to encode than true interlaced (or progressive for that matter) 30fps... But other than that, what would be a technical argument for justifying the above assertion?

jeanl

We've been talking about recording the 3:2 (telecined) 29.97 TV signal and reverse processing it to a 23.9 fps progressive frame sequence matching the original film.

recall this link.
http://www.dvdfile.com/news/special_report/production_a_z/3_2_pulldown.htm

[gadgetguy]

Can you expand on that? Apart from sequence breaks at edit points (etc) I don't see what can go wrong in doing the telecine...
jeanl

I have noticed on several TV shows that the telecine process is applied before editing. This causes irregular telecine patterns from scene to scene. Therefore an inverse telecine filter cannot be applied to the whole video.

[edDV]

Can you expand on that? Apart from sequence breaks at edit points (etc) I don't see what can go wrong in doing the telecine...
jeanl

I have noticed on several TV shows that the telecine process is applied before editing. This causes irregular telecine patterns from scene to scene. Therefore an inverse telecine filter cannot be applied to the whole video.

This happens to TV movie tapes as someone edits out scenes to add more commericals late night.

[lordsmurf]

Can you expand on that? Apart from sequence breaks at edit points (etc) I don't see what can go wrong in doing the telecine...
jeanl

I have noticed on several TV shows that the telecine process is applied before editing. This causes irregular telecine patterns from scene to scene. Therefore an inverse telecine filter cannot be applied to the whole video.

Yeah, this is what I am referring to. I find most non-"movie channel" playback to be severely lacking and full of errors. Cable stations, satellite stations, network tv .. LEAVE IT INTERLACED!

Satellite streams are by far one of the worst. You get a MIX of video type through the SAME MOVIE or show. I've OFTEN seen it switch interlace fields in the middle of a programs, or a mix of telecine and true interlace. Commercial breaks are the enemy, the reason most "reverse" processes are impossible. As well as "edited for content" or "edited for time". Some shows have been known to be "sped up" by about 2 minutes.

Directly editing DVB streams or attempting to IVTC it (or a capture of it) is major voodoo. It requires patience, aspirin, and an empty house (so you can scream at the computer and not bother anybody). Even then, you may fail.

[vhelp]

Late evening all.

First, I wanted to say, that I waited all day to say what I
wanted to say now

So, do bare with me, even if things were worked out in the end

@ edDV

Have you tried the various IVTC (and de-interlacing) techniques
in your video endeavors ?? ..and if so, how long have you spent
on each (shall I say) algorithem of methods ??

I'm curious, because even though I do agree with most of your
comments, I *am* curious if you really invested enough time into
these varouis processes, or did you just give up ??

Anyways.

I agree with you that you should best leave Interlace content
alone and encode to DVD specs as such. But, I believe that the
*method* used needs to be revised (to some degree) because of
the lack of bitrate for (shall I say) "rougher" materials to
encode (that being) interlace. Because every frame is interlaced,
this is seen (by encoders) as noise. Noise, as most people know,
are bitrate driven. They need lots of bitrate.
.
But, I see lots of reference to VBR in most of these types of
projects.. though they are not specifically stated (interlaced)
they are most likely so.
.
I've stated in various thread topics around this FORUM over time,
that for Interlace source (content, or however way you refer to it
as) ..that for interlace sources, the best method is CBR and high
bitrate. Most TV shows (for instance) are 1 hr long. What I can't
understand, (from most of what I read - though quickly) is how
(or why) users speak of VBR in their attempts to nail down the
quality (or best quality) in their process. "How do I get the best.."
or "what's the best way/method.." yada yada.. And, my answer to those
questions is simply, "..raise the bitrate and incorporate CBR instead"
.
Why incorporate CBR (vs. VBR) in interlace source materials ??
.
Well, (from my experience) Interlace and VBR just don't mix very
well. At least, with the Encoders that I have used over the years.
Because interlace is in every frame, VBR in high mition is not enough
for the MPEG algorithem to reproduce those frames w/ the quality
required - in other words, for reduced pixelation. Pixelation will
undoubtably show its nasty self when users choose VBR (multi-pass or
what-have-you) and low bitrates. With VBR (the way I see it) is that
it is more space (then) quality driven. First, we must calculate the
amount of space for my project, then, within that algorithem, produce
the best quality *that* given encoder can produce. I've not seen
the level of quality from Interlace sources come out of anything better
than TMPGenc, (and CBR/high bitrate) with the exception of Procoder
and CBR/high bitrate. I still will not use VBR, no matter how good
the encoder is, because in all the years I've ben doing this, I've
not seen anything mathching closely enought to the original source,
which is the goal to begin with.
.
But, most users are per-disk minded. I want to fit 2 hrs program on
one DVD disk, but I still want (so called) DVD quality. We are still
talking about Captured Sources here. Anyways.

Anyways.

But, I do agree with you (didn't I say that already) .. with your
comment(s) above about keeping source interlaced during Encoding.
.
But, perhaps its time now, to revise some of the rules. Because, like
it or not (and you seem to be ready'ing yourself already, if I read
your comments above correctly) that HDTV and other (high bitrate) format
content will undoubtably require a shift in gears (modes) and VBR will
replaced with CBR, because of the high noise level in Interlace content.
.
In order to keep safe, we just have to adjust our modes by changing
gears from VBR, over to CBR and high bitrate. 1 hr should get us a
good DVD quality, and very close to the original source, if not, then
our source is dirty, like my source - Antenna.
.
In my source content endeavors, when the source is interlace, I will
encode CBR and high bitrate. I use 9000 bitrate as my minimum because
1 hr of program material can fit on one dvd disk. Plus, because of the
noise level of my source (Antenna) I have no choice *but* to use such
high bitrate.

Film (aka, Telecined) -> IVTC ...

I think that maybe your attempts were based on bad Telecined (or EFC)
sources were what causes you disapointment. ECF (or, Edited For Content)
sources are those that are agree by a consortium, to cut various scenes
out of an already processed video. These are what have been giving
many users here (and other FORUMS) great tributations.
.
Its these users that just never quite put 2n2 together, to add up the
reality of what is to become of there sources when run through a
particular IVTC method. I believe that you may have been one of those
who experienced this, and then gave up, and are now arguing that IVTC
is nonsense (you didnt' say that - I did) but that one should just
encode their sources (weather Interlace or Telecined) straight through.
.
It is here that I disagree with you. Not to argue though. Just disagree
with you.
.
The reasons have been stated above already, but I'm still wondering
(after reading these post early this morning thoughout the day, till now)
why you still feel that IVTC should be skipped, specially for other
platforms (ie, HDTV; etc) and for such devices that incorporate better
de-interlace or other algorithem, when dealing w/ Interlace materails.
.
The mixup here, (I beleive) is that of the source.
.
When a given source is Film, and transferd via Telecined, and then that source
is aired on TV, (and user knows it's Film, [not interlace]) ..that the
source will be 29.970 fps (thanks to Telecine) because a few frames were
added (blended) to make 24F into 30F. Why suggest to encode with the
30F (fps) then ?
.
Here's the catch ...
.
Remember I memtion above, EFC sources. This is the secret to a succesful
IVTC project.
.
Know your source. Become familiar with what is, and what is not, ECF 'ed.
ECF sources will undoubtably have distorted 3,2 patterns. Broken, chopped,
or added frames to make a given content viewable to the public within
a certain Audiance range.. (ie, PG; PG-13; R; etc)
When the source is Film (Telecined already) and the consortium gets their
hands on it, and if those responsible do not take into consideration, the
nature of the source during editing, you can expect the source to be cut
anywheres inside the 3,2 pattern within the whole contents time-line.
.
Note the u_Film vs. e_Film scenarios.
.
When the source is:

u_Film - PPP II PPP II PPP II PPP II PPP

And an IVTC is applied, your end result will be a perfect 23.976 fps
source (w/ flags added in to re-produce the same captured TV sources
29.970 fps) but with 20% less video area to encode. That means greater
quality. Those using VBR, can expect better returns in there DVD bitrate
investments, as Manono indicated.
.
But, when it goes through an ECF process, the above u_Film may become:

e_Film - PPP II II PPP IP PPP II PPP II PPP PII IPP PIP II PPP ...

Try IVTC 'ing that. You won't be able to. Even thought the source is
still Film (in nature) it no longer holds it's 3,2 pattern consistantly.
.
I recall some two plus years ago, when I was capturing Star Wars, using
my DV cam as the VCR unit (I called my mini-tivo) and remembered using
Decomb for this, because it was rated the best IVTC engine around.
but, no IVTC method could undo what was ECF 'ed. But at the time, I
did not realize this then. I only knew, that I did not drop any frames
and I applied the IVTC method properly. It was that e_Film 's end
result (above) that I was dealing with.
.
Try doing Friends from DVD. This is a twist. They edited it *AFER*
the Telecine, and authrored onto DVD. I bought a few of these. And,
let me tell you, you can't IVTC friends, no matter how hard one tries.
so, I gave up on buy them, and made up a new rule. Anything that I
know if EFC 'ed, I will not by on DVD, because it would be a waist
of monies, ..when I can just capture it, with the same EFC 'ed video.
.
I know you know this. You are not dumb. But, for others reading here, I
bring this to light for there benefit.
.
This is the *light* they have to see from this day forward. Once they
realize this, they will learn to take new action.

Speaking again on IVTC and technque ...

There is a techique to use when one wants to deteremine if there source
(after capture) is IVTC 'worthy. That is, will it pass throughout
the whole movie source
.
I've ben using it for some time, and I'll share it here with you all.

* You start by opening up your source (assuming either
.. vdub or AVIsynth) and an IVTC method in these editors.

* Turn on, or apply your IVTC method.

* Then, frameserve that into TMPG.

* Inside TMPG, press Ctrl+Shift+P and drag your [ ] button
.. (or chicklet) through your video time-line, and observe
.. quickly, your source 3,2 pattern stability.  The quick
.. and dirty way, is like this.
.. 
.. * Start at the beginning, and scroll through your
.. .. sources time-line. and observe for any Interlace
.. .. frames.  If none shows, bring to mid section of
.. .. the video, and observe some more frames, if no
.. .. Interlace shows, scroll through, near the end of
.. .. your time-line, and observes again, for any signs
.. .. of Interlace.
.. .. .
.. .. If you do not see any Interlace throughout your
.. .. source's time-line, then its safe to assume that
.. .. you'll have a perfect IVTC project.
.. .. .
.. .. The above is the quick and dirty way of determining
.. .. IVTC stability.
.. .. .
.. .. IF however, you do see any Interlace, then you have
.. .. some digging to do.  You have to find out where the
.. .. 3,2 pattern broke. This is why I don't like to
.. .. capture 1 hr (or longer) straight through the
.. .. project. (ie, VHS movie; 2 hr Movie on Cable; etc)
.. .. I like to capture in segments, because I have better
.. .. control of this mechanism with IVTC debuggin - it's
.. .. quicker, because I can load in any given area/
.. .. section/segment, and find the broken 3,2 pattern,
.. .. and deal with it there, or recapture it, etc.

Finding illregularities in IVTC sources is really quite easy. I do it
all the time. But, I prefer to capture in small AVI segments so that
I can spot the broken 3,2 patter quicker. If I capture in one 13gig Gulp
AVI file, then I have to go through the whole AVI file to find it.
My VHELP rule of thumb says, work in segments for better control, and
then some

-->

As for de-interlace techniques for transfering to DVD ...

There still are various techniques to obtain maximum quality.
One only has to do the research and find the one that fits there
comprimise.
.
If de-interace proves too much loose in quaility, then what I have
commented on (above) will serve you better in the long run, because
nothing is destroyed, other than the final encode to MPEG. And, that
will be determined on your knowledge/skills/techniques etc.

When I'm lazy, and don't want to bother with all the above nonsense
and/or don't have the time, I too will use my dvd recorder
Sometimes, I just don't need quality. Its not always as important as
I once felt.

-vhelp 3325

[jeanl]

Your post is very long, but this part at least is, I think, inaccurate:

I agree with you that you should best leave Interlace content
alone and encode to DVD specs as such. But, I believe that the
*method* used needs to be revised (to some degree) because of
the lack of bitrate for (shall I say) "rougher" materials to
encode (that being) interlace. Because every frame is interlaced,
this is seen (by encoders) as noise. Noise, as most people know,
are bitrate driven. They need lots of bitrate.

Any good encoder will not attempt to encode 2 very different interlaced fields as 1 frame picture. That would be a really bad choice and a good encoder knows to encode them as 2 separate field pictures. So your saying that "every frame is interlaced, this is seen by the encoders as noise" is highly inaccurate. Read up on how encoders can encode interlaced frames, the flexibility is amazing. They can encode 2 fields as 1 frame picture (for example for a very still image), they can encode it as 1 frame picture, but with interlaced macroblocks (i.e. parts of the images are encoded as fields, other parts as a complete frame), or they can encode the 2 fields as 2 separate field pictures.
I can't see any good reason why interlaced video would be harder to encode than progressive, except maybe because encoding 2 half-images might be less efficient in some cases than encoding 1 full image... But in any case, your argument does not convince me. As for the superiority of CBR over VBR, I don't buy it either! Very high CBR over very low VBR sure, but with an overall equal average bitrate, I would think that VBR is always going to be better than CBR (if VBR can go as high as CBR that is)...

jeanl
P.S. A great link on interlace/progressive/3-2 pulldown/inverse telecine and the difficulty of it is here (thanks manono!) . It explains everything from the ground up, including the flags in the MPEG sequence and how they can be helpful or detrimental to the inverse telecine process. It does not explain how interlaced frames can be encoded in the MPEG stream though... It's fascinating reading!

[edDV]

@ edDV

Have you tried the various IVTC (and de-interlacing) techniques
in your video endeavors ?? ..and if so, how long have you spent
on each (shall I say) algorithem of methods ??

I'm curious, because even though I do agree with most of your
comments, I *am* curious if you really invested enough time into
these varouis processes, or did you just give up ??

I'm here to better understand consumer compression tools and process techniques. It's for personal interest but I find experience gained here is generally useful. My background is from the other end of the industry, analog and digital video production infrastructure equipment. The kind of equipment found in post houses and broadcast operations.

I understand deinterlacing from algorithm and hardware implementation perspective in video special effects equipment and transmission processors. In that equipment it's all done realtime so forgive my impatience with multi-hour software processes. That said, I'm highly impressed that this technology is available to all of us in our home computer labs and home theaters. I would have never imagined even 10 years ago that I could have a fully functioning digital video editing suite on my desk at home and 3CCD digital acquisition equipment.

I'm fairly new to DVD authoring hence my presence here to learn the tricks available. I'm comfortable with interlace video processing and recording to DVD since it is directly analogous to magnetic video recording. Progressive movie DVDs are great. I have a progressive HDTV and enjoy its progressive and interlace modes.

I guess the main culture clash I have here is my bias for tigher process control and objective measurement of video performance. I am cursed with an engineers eye for poor video levels and image defects. At the same time, I seem to have a more casual standard for compression atrifacts for home entertainment recording that seeems to light peoples fire here. Hence, I can live with with interlaced DVD recordings but incorrect black levels will keep me up late.

As for IVTC 3:2 pulldown, its on my optional extra list when I cap a good movie.

Anyways.

I agree with you that you should best leave Interlace content
alone and encode to DVD specs as such. But, I believe that the
*method* used needs to be revised (to some degree) because of
the lack of bitrate for (shall I say) "rougher" materials to
encode (that being) interlace. Because every frame is interlaced,
this is seen (by encoders) as noise. Noise, as most people know,
are bitrate driven. They need lots of bitrate.
.
But, I see lots of reference to VBR in most of these types of
projects.. though they are not specifically stated (interlaced)
they are most likely so.
.
I've stated in various thread topics around this FORUM over time,
that for Interlace source (content, or however way you refer to it
as) ..that for interlace sources, the best method is CBR and high
bitrate. Most TV shows (for instance) are 1 hr long. What I can't
understand, (from most of what I read - though quickly) is how
(or why) users speak of VBR in their attempts to nail down the
quality (or best quality) in their process. "How do I get the best.."
or "what's the best way/method.." yada yada.. And, my answer to those
questions is simply, "..raise the bitrate and incorporate CBR instead"
.
Why incorporate CBR (vs. VBR) in interlace source materials ??
.
Well, (from my experience) Interlace and VBR just don't mix very
well. At least, with the Encoders that I have used over the years.
Because interlace is in every frame, VBR in high mition is not enough
for the MPEG algorithem to reproduce those frames w/ the quality
required - in other words, for reduced pixelation. Pixelation will
undoubtably show its nasty self when users choose VBR (multi-pass or
what-have-you) and low bitrates. With VBR (the way I see it) is that
it is more space (then) quality driven. First, we must calculate the
amount of space for my project, then, within that algorithem, produce
the best quality *that* given encoder can produce. I've not seen
the level of quality from Interlace sources come out of anything better
than TMPGenc, (and CBR/high bitrate) with the exception of Procoder
and CBR/high bitrate. I still will not use VBR, no matter how good
the encoder is, because in all the years I've ben doing this, I've
not seen anything mathching closely enought to the original source,
which is the goal to begin with.
.
But, most users are per-disk minded. I want to fit 2 hrs program on
one DVD disk, but I still want (so called) DVD quality. We are still
talking about Captured Sources here. Anyways.

But, I do agree with you (didn't I say that already) .. with your
comment(s) above about keeping source interlaced during Encoding.

I use CBR and 8Kbps for DV source material and sometimes for important TV captures.

Are you sure encoders see interlace as noise? I don't understand why that should be the case.

My recent encoder tweaking has been focused on realtime software only MPeg2 performance using an "average" 2.4 GHz machine. I have it working with VBR and CBR. The main interest so far has been to keep the CPU load under control for various types of picture action. Initial quality seems good enough.

But, perhaps its time now, to revise some of the rules. Because, like
it or not (and you seem to be ready'ing yourself already, if I read
your comments above correctly) that HDTV and other (high bitrate) format content will undoubtably require a shift in gears (modes) and VBR will replaced with CBR, because of the high noise level in Interlace content.
.
In order to keep safe, we just have to adjust our modes by changing
gears from VBR, over to CBR and high bitrate. 1 hr should get us a
good DVD quality, and very close to the original source, if not, then
our source is dirty, like my source - Antenna.
.
In my source content endeavors, when the source is interlace, I will
encode CBR and high bitrate. I use 9000 bitrate as my minimum because
1 hr of program material can fit on one dvd disk. Plus, because of the
noise level of my source (Antenna) I have no choice *but* to use such
high bitrate.

I'm "playing" with various encoders for HDTV. I've also been editing HDTV in Vegas. Currently I'm evaluating Mainconcept's new H.264 encoder, Microsoft's VC-1 and WMV-HD. I've also tried MPeg2 and DivX HD. They all work and they are all slow.

Film (aka, Telecined) -> IVTC ...

I think that maybe your attempts were based on bad Telecined (or EFC)
sources were what causes you disapointment. ECF (or, Edited For Content) sources are those that are agree by a consortium, to cut various scenes out of an already processed video. These are what have been giving many users here (and other FORUMS) great tributations.
.
Its these users that just never quite put 2n2 together, to add up the
reality of what is to become of there sources when run through a
particular IVTC method. I believe that you may have been one of those
who experienced this, and then gave up, and are now arguing that IVTC
is nonsense (you didnt' say that - I did) but that one should just
encode their sources (weather Interlace or Telecined) straight through.
.
It is here that I disagree with you. Not to argue though. Just disagree
with you.
.
The reasons have been stated above already, but I'm still wondering
(after reading these post early this morning thoughout the day, till now)
why you still feel that IVTC should be skipped, specially for other
platforms (ie, HDTV; etc) and for such devices that incorporate better
de-interlace or other algorithem, when dealing w/ Interlace materails.
.
The mixup here, (I beleive) is that of the source.
.
When a given source is Film, and transferd via Telecined, and then that source
is aired on TV, (and user knows it's Film, [not interlace]) ..that the
source will be 29.970 fps (thanks to Telecine) because a few frames were
added (blended) to make 24F into 30F. Why suggest to encode with the
30F (fps) then ?
.
Here's the catch ...
.
Remember I memtion above, EFC sources. This is the secret to a succesful
IVTC project.
.
Know your source. Become familiar with what is, and what is not, ECF 'ed.
ECF sources will undoubtably have distorted 3,2 patterns. Broken, chopped,
or added frames to make a given content viewable to the public within
a certain Audiance range.. (ie, PG; PG-13; R; etc)
When the source is Film (Telecined already) and the consortium gets their
hands on it, and if those responsible do not take into consideration, the
nature of the source during editing, you can expect the source to be cut
anywheres inside the 3,2 pattern within the whole contents time-line.
.
Note the u_Film vs. e_Film scenarios.
.
When the source is:

u_Film - PPP II PPP II PPP II PPP II PPP

And an IVTC is applied, your end result will be a perfect 23.976 fps
source (w/ flags added in to re-produce the same captured TV sources
29.970 fps) but with 20% less video area to encode. That means greater
quality. Those using VBR, can expect better returns in there DVD bitrate
investments, as Manono indicated.
.
But, when it goes through an ECF process, the above u_Film may become:

e_Film - PPP II II PPP IP PPP II PPP II PPP PII IPP PIP II PPP ...

Try IVTC 'ing that. You won't be able to. Even thought the source is
still Film (in nature) it no longer holds it's 3,2 pattern consistantly.
.
I recall some two plus years ago, when I was capturing Star Wars, using
my DV cam as the VCR unit (I called my mini-tivo) and remembered using
Decomb for this, because it was rated the best IVTC engine around.
but, no IVTC method could undo what was ECF 'ed. But at the time, I
did not realize this then. I only knew, that I did not drop any frames
and I applied the IVTC method properly. It was that e_Film 's end
result (above) that I was dealing with.
.
Try doing Friends from DVD. This is a twist. They edited it *AFER*
the Telecine, and authrored onto DVD. I bought a few of these. And,
let me tell you, you can't IVTC friends, no matter how hard one tries.
so, I gave up on buy them, and made up a new rule. Anything that I
know if EFC 'ed, I will not by on DVD, because it would be a waist
of monies, ..when I can just capture it, with the same EFC 'ed video.
.
I know you know this. You are not dumb. But, for others reading here, I
bring this to light for there benefit.
.
This is the *light* they have to see from this day forward. Once they
realize this, they will learn to take new action.

Speaking again on IVTC and technque ...

There is a techique to use when one wants to deteremine if there source
(after capture) is IVTC 'worthy. That is, will it pass throughout
the whole movie source
.
I've ben using it for some time, and I'll share it here with you all.

* You start by opening up your source (assuming either
.. vdub or AVIsynth) and an IVTC method in these editors.

* Turn on, or apply your IVTC method.

* Then, frameserve that into TMPG.

* Inside TMPG, press Ctrl+Shift+P and drag your [ ] button
.. (or chicklet) through your video time-line, and observe
.. quickly, your source 3,2 pattern stability.  The quick
.. and dirty way, is like this.
.. 
.. * Start at the beginning, and scroll through your
.. .. sources time-line. and observe for any Interlace
.. .. frames.  If none shows, bring to mid section of
.. .. the video, and observe some more frames, if no
.. .. Interlace shows, scroll through, near the end of
.. .. your time-line, and observes again, for any signs
.. .. of Interlace.
.. .. .
.. .. If you do not see any Interlace throughout your
.. .. source's time-line, then its safe to assume that
.. .. you'll have a perfect IVTC project.
.. .. .
.. .. The above is the quick and dirty way of determining
.. .. IVTC stability.
.. .. .
.. .. IF however, you do see any Interlace, then you have
.. .. some digging to do.  You have to find out where the
.. .. 3,2 pattern broke. This is why I don't like to
.. .. capture 1 hr (or longer) straight through the
.. .. project. (ie, VHS movie; 2 hr Movie on Cable; etc)
.. .. I like to capture in segments, because I have better
.. .. control of this mechanism with IVTC debuggin - it's
.. .. quicker, because I can load in any given area/
.. .. section/segment, and find the broken 3,2 pattern,
.. .. and deal with it there, or recapture it, etc.

Finding illregularities in IVTC sources is really quite easy. I do it
all the time. But, I prefer to capture in small AVI segments so that
I can spot the broken 3,2 patter quicker. If I capture in one 13gig Gulp
AVI file, then I have to go through the whole AVI file to find it.
My VHELP rule of thumb says, work in segments for better control, and
then some

-->

I never said IVTC shouldn't be done,or that it doesn't work. I feel that it is an optional extra for special material and is too time consuming to do all the time. I also said that I hope the process will be automated into authoring programs so it could be used without all the hand holding and previewing before encode.

As for de-interlace techniques for transfering to DVD ...

There still are various techniques to obtain maximum quality.
One only has to do the research and find the one that fits there
comprimise.
.
If de-interace proves too much loose in quaility, then what I have
commented on (above) will serve you better in the long run, because
nothing is destroyed, other than the final encode to MPEG. And, that
will be determined on your knowledge/skills/techniques etc.

When I'm lazy, and don't want to bother with all the above nonsense
and/or don't have the time, I too will use my dvd recorder
Sometimes, I just don't need quality. Its not always as important as
I once felt.

-vhelp 3325

I don't consider IVTC a true deinterlace. The fields are just reordered to restore a native progressive stream.

I consider deinterlacing camera video shot 29.97 fps a true deinterlace. I've looked over varous deinterlace software and have been appauled at what they are doing. There may be better stuff out there but I'll pass for now thank you.

[akrako1]

New project....
I've got some Justice League DVD's I'm looking to re-encode. They're interlaced, but with a scheme I haven't seen mentioned here. When I scroll thru the frames under TMPGEnc's deinterlace option, I see 4 full frames and then 1 interlaced frame. Is this a standard interlacing-type that could be de-interlaced successfully to a progressive file? What program (hopefully free) contains the proper de-interlacing scheme to fix it?

Since this is a properly authored dvd that I'm using as a source, I'm hoping the lack of post production editing will eliminate any inconsistencies in the interlacing scheme, and it will de-interlace smoothly.

Thanks

[junkmalle]

New project....
I've got some Justice League DVD's I'm looking to re-encode. They're interlaced, but with a scheme I haven't seen mentioned here. When I scroll thru the frames under TMPGEnc's deinterlace option, I see 4 full frames and then 1 interlaced frame. Is this a standard interlacing-type that could be de-interlaced successfully to a progressive file? What program (hopefully free) contains the proper de-interlacing scheme to fix it?

Since this is a properly authored dvd that I'm using as a source, I'm hoping the lack of post production editing will eliminate any inconsistencies in the interlacing scheme, and it will de-interlace smoothly.

Thanks

That's a variation on 3:2 pulldown, 2:2:3:3:

ABCD -> AA BB CC CD DD

4 film frames become 10 fields (24 frames become 60 fields).

I don't know of any VirtualDub filters that can IVTC that pattern. You can probably write an AVISynth script but I'm not familiar enough with those tools to say for sure. TMPGEnc's "deal after this frame according to selected pattern" can IVTC any consistent pattern.

[akrako1]

TMPGEnc''s "deal after this frame according to selected pattern" can IVTC any consistent pattern.

Can you explain exactly how to setup TMPGEnc to do this? or point me to a guide?

I've just been playing with some of the settings and can't seem to create a proper progressive file... I'm using the 'inverse telecine' setting, not the 'de-interlace' setting. Do you need to use both? For 'inverse telecine' I'm using the 'auto-setting' feature (I think the other choice is to select each frame individually!) Under the 'auto-setting' window, there are a few options I'm confused about.... For 'setting method', I've tried both 24fps and 30fps settings, and also tried both 'flicker prioritized' and 'motion prioritized'. There's also a 'de-interlace' window that is set to 'none' by default. Do I need to set this? Aren't we already deinterlacing? When I do click 'start', and it reads the whole file, it does seem to select the proper, full-frames. But then after I convert, the result file is shown as interlaced if I load it back up in TMPGEnc as a source file to convert.

Sorry if I'm a bit of a noob as far as de-interlacing is concerned... it seems to be pretty complicated business...

Thanks

[junkmalle]

TMPGEnc''s "deal after this frame according to selected pattern" can IVTC any consistent pattern.

Can you explain exactly how to setup TMPGEnc to do this? or point me to a guide?

Heheh, I knew this question was coming next! Here's the short answer:

Set up TMPGEnc as you normally would for an inverse telecine. Before encoding go to the inverse telecine dialog (Settings, Advanced, Inverse Telecine). Do not use AUTO and its related settings (it doesn't work very well with normal 3:2 pulldown, let alone anything unusual). At the top of the dialog are the thumbnails of each possible pairing of the fields. If you right click on one (use the first one) you will get a menu. The second selection down is "Deal after this frame according to selected pattern" -- select it. You will now have a small "pattern setting" dialog. Here is where you select the pattern to use. The basic pattern you will use for a 2:2:3:3 inverse pulldown is "1010100100".

Unfortunately you don't know exactly where in the pulldown sequence your video starts. I could go into a long description of how to determine it but the easiest thing to do is the the above pattern then step through all the variations:

1010100100
0101001001
1010010010
0100100101
1001001010
0010010101
0100101010
1001010100
0010101001
0101010010

(Note I've just rotated the digits, sucking one off the front and moving it to the back at each line.)

Using the first thumbnail as the starting point, use the first pattern. Step through the hilited thumbnails using CTRL+RightArrow. If all the hilited frames are free of interlace artifacts you have the right pattern. If not, go back to the first frame and use the next pattern. Repeat until you find the right one.

Some Caveats:

This technique requires that there be no breaks in the telecine pattern. If there are breaks from an edit or a dropped frame while capturing, frames after the break will start to show errors. You can find the break and repeat the selection process at the break, but if there are a lot of breaks it will take you forever.

The pattern given is to undo 2:2:3:3 pulldown. For a normal 2:3 IVTC the pattern to use is 10100 and it's variations.

If you want a full description of what's going on, and the ability to figure future patterns out for yourself, let me know and see if I can write one up. It's not entirely trivial though!

[akrako1]

Thanks for the wealth of info junkmalle... I worked with the Inverse Telecine window and got my frames selected ok. There was just one break, after the initial credits, and then all good.

Are there any additional settings I need to change to output a progressive file? I'm starting with the standard DVD template, and modifying it as I go... I notice under the 'Video' tab, theres an 'Encode Mode' option. It's currently set to 'Interlace'. The other options are 'Non-Interlace' and 'Inverse 3:2 pulldown'. Any ideas which one to use? Since I'm already doing the 'inverse telecine', is it necessary to set it to 'Inverse 3:2 pulldown'? Or just use 'Non-Interlace'?

I'll try encoding with both and if I don't hear back first, I'll let you know what I find...
[/img]