Deinterlace the right way, a theory that needs implementing?

[n0lamers]

What's the best way to deinterlace with freeware app's? What filters? This is what has been plaguing me for the past month or so.

I was thinking that the best possible way for practically all interlaced material (telecined or not), would be if the source(29.97fps in this case for NTSC) was 'bobbed' into fields(59.94fps)--then somehow turn the fields into frames--blended the SEPARATE field/frames (now don't get crazy when I say blend, if I was blending a two field composite frame, it would be blurry, but this MIGHT eliminate the blur by doing each field separate)--turn the frames back into fields and 'unbob' back to 29.97fps for standalone DVD playback.

I use DVD2AVI, VirtualDub, (could use avisynth if needed), aviutl, TMPGEnc--to encode. Also, if it could be done in only a few steps, that would be great for reducing the color quality loss from going to and from YUV 2 RGB and back again, ya know? Any insight is appreciated....thanx

[Betamax]

If your source is interlaced and your intension is to create dvd in the final place, why the h*** should you go to de-interlace?
You just will keep the best quality using interlaced.

Only VCD in non-interlaced, SVCD and DVD are interlaced : non-interlaced avi's which should become SVCD/DVD will have less quality (not noticeble at at computer screen, but sure at a TV)!
So save your effort and keep the interlaced.

All my sources are interlaced. Whenever I create a VCD, I let TMPGenc do the de-interlacing internally (not even using a special filter for it) - the result is very much acceptable, but of course DVD (720x576/480) to VCD (352x288) always is less quality.

[SaSi]

That's interesting.

I have been converting 8mm camcorder video into DVD and de-interlacing is a separate step I do.

One of the reasons I do it is decreasing file size. Deinterlaced video compresses into MPEG-2 much better (more than 30% better as I have found).

Also, I never noticed lower picture quality. To the contrary, interlaced video showed bad on TV (but not so noticeably on the computer).

Obviously, different people are doing different things in different ways. I am certain we are both correct, but am afraid we are doing different things.

But you gave me "inspiration" to try this out.

[sammie]

I'd like to see a good guide as well. DVD2svcd gives exellent results, but it seems to use a very complicated process that I haven't tried to reverse engineer. It works why bother.

But the simple solutions, just run this or that filter under VDub or AVIsynth and encode using 23.976 and 3:2 pulldown just result in twitchy video (on both standalone and computer) or an audio track that is way too long or typically both.

I do mostly high bitrate DVDs so the extra 30% isn't strictly necessary. But I realize it would be better and I'd like to understand more about how to really do it correctly.

[SLK001]

Once again, we are confusing INTERLACED with TELECINED video. All video that is shown on a non-HDTV television, is sent as interlaced. That even includes VCDs.

VCDs are, however, only encoded as progressive frames. SVCD and DVDs can be encoded as either progressive frames (at 23.976 fps, to which a 2:3 pulldown is applied), or as fully telecined streams (at 29.97 fps).

You should endeavour to encode at the original frame rate (if FILM, it is 24 fps), then apply the 2:3 pulldown flags. By doing this, you can either increase the length of the video by ~ 25%, or you can increase the encoding bitrate by 25%. (You pick the extra up by NOT encoding the telecined frames, since they are only duplicates of frames you have already encoded, and there is one telecined frame for every four "unduplicated" frames.)

"Twitchy" video and audio length problems are problems that lie elsewhere.

[sammie]

Umm, no they aren't, notice I didn't say interlaced. I understand telecined vs. interlaced just fine. I was probably should have been clearer about that this is a telecine/ inverse telecine thing that I am questioning. Doing it wrong clearly does result in mismatched audio and twitchy video. What I'm saying is that I haven't come accross a good guide that says how to convert 29.97 back to a 23.976 with the pulldown flag set properly. I understand that part of the problem is that many sources aren't consistent all the way through as well. But, as I mentioned, using DVD2SVCD does a consistently good job. They use several steps, including the program pulldown.exe. I'd like to see a clear guide outlining the process they use and explaining what the goal is of each step. I've tried a few of the "simple" methods, filter then set the flag type. I have always ended up with either the audio too long (by the expected amount) or most typically video that isn't particularly smooth. Not awful, 2:3, vs 3:2 vs no flag bad, but just noticeable. DVD2SVCD is just as smoth as the 29.97.

[SLK001]

Twitchy video usually indicates a field order problem during your encode. The audio problem doesn't make sense. The actual video length should remain unchanged. For instance, one hour of FILM at 24 fps is 86,400 frames. Applying a 2:3 pulldown to the 86,400 frames gives you 108,000 frames, but at 30 fps the length of the video is still one hour. It is possible that you have applied the pulldown and failed to set the new frame rate. This will cause the film to run in slow motion (4/5 speed) and will not properly sync with the audio.

[n0lamers]

The post that I made was a theory that I wanted to check out by using freeware apps/filters to deinterlace all interlaced material, whether it be telecined or not.

@Betamax
I don't have a DVD burner, therefore when I said this:

for standalone DVD playback

it was only in relation to vcd/svcd played back on a standalone DVD player.

Maybe I should've been clearer as to what I wanted for the final product. If its source was originally 23.976fps(3:2 pulldowned to 29.97fps for DVD) and I wanted to make a vcd, I could just use telecide/decimate after 'unbobbing', which would effectively IVTC(or reverse 3:2 pulldown). The implemented theory in my original post would take care of pure interlaced(non-telecined) material as well(without the need for IVTC obviously).

The main point I was trying to get at is if blending seperate fields, then re-composing into a 2 field frame, INSTEAD of blending a composite 2 field frame as-is(like the current blending methods used by prog's such as virtualdub/avisynth) would produce better results(ie: not blurry or unnoticable blur).
But, I don't know what combo of free prog's/settings to use that will get the the least amount of color quality loss from rip to author.

The intention of my orginal post wasn't to start a huge debate about telecining or whether to leave it interlaced or not. I posted with the assumption that anyone who would respond to my post would also be interested or have ideas about deinterlacing, for what it is--not for what should be done instead.

[sammie]

Field order is what I thought too. But it isn't (tried that), plus it isn't near as noticeable as a field order problem, much more subtle. To me field order isn't watchable, this is. In fact, I only see the twitching when I'm being critical of my work. Not usually in normal viewing.

The audio isn't as common. The odd part is that like I said the neither the video or audio speed seems particularly off. Yes, I know that one of them has to be. I suspect it's the video knowing what I am doing to things. I think one of the solutions there is to demux, encode the video with pulldown, then remux the audio.

I was about there when I figured out that DVD2svcd seemed to do it right. I don't really need to know how now except for a few things where I really want to apply some filters, or make some adjustments to the encode that aren't available from DVD2svcd. So I haven't pursued it and just used DVD2svcd when I really needed it. I would LIKE to know how. It is something that will improve the quality of what I (and a lot of folks here) do. It's just that the guides I've seen quickly devolve into a discussion of deinterlacing, which is as you pointed out not (typically) what I'm after. Or they are very incomplete and either wrong or sufficiently unclear that I'm not doing it right.

Yes, a lot of people use the two terms sloppily, and/or incorrectly. I'm guilty I admit. But we still need a good guide for setting pulldown and framerate up correctly, not another lesson in what the diffenece between telecine and interlace is.

[sammie]

deferring back to n0lamer Didn't mean to hijack your thread. But agree, no one answers the questions. Only gives another lecture on deinterlacing vs. IVTC. or why you should or shouldn't deinterlace.

[n0lamers]

I see what you mean sammie, maybe if my topic gets enough bumps, then someone who is actually concerned with making deinterlacing better will actually try what I'm suggesting. Maybe also they could clue me in as to what process they used and if it looked better than the automatic methods given in virtualdub/avisynth/TMPGEnc.

The more I think about it, my theory could go either way--either alot worse because of compounding the blends during the 'unbob' process, or better because you aren't blending the final product, only the seperate fields of a frame.

It is important to further the deinterlacing concept, because it is ultimately what will be used for all interlaced material that doesn't stay interlaced (as in vcd), AND for svcd authorers who want their svcd to look just as good on TV as it does on a computer monitor. A more complex method (which may or may not be simplified later) of deinterlacing is what authorers use now in their decodings if they want the best quality. It has overcome the hurdle of looking worse than IVTC'd material due to the types of algorithms in filters that have been written for virtualdub/avisynth, like smart bob, telecide, and decimate.

Now all that is needed is an all-in-one filter that combines all those functions in an adaptive mode and that can be executed as the video is being frameserved. But, until then I'd like to know how to do it using the existing filters and not lose color from YUV/RGB/YUV conversions, as well as not creating a redundancy of time wasted because these types of filters cannot be executed in the frameserving mode of virtualdub. The only way I can think of to use some or all of these filters effectively is an avisynth script within virtualdub, but it is over my head as far as ordering of the filters and settings to combine all of those functions the right way...

Filters such as these, in combination with possibly other filters (anti-aliasing maybe) in the future--will work together in a frameserve mode of a freeware decoder to be done 'on-the-fly' during another program's encoding process. This is a part of the direction that video processing is going so it will be as close to the original as possible, getting over the problems of blur, mice teeth, combing, and artificing.

All-in-one auto IVTC is crude, (like the one provided in TMPGEnc) and has been known to drop non-duplicate frames and produce objectional artificing/blurring. Deinterlacing using a 'bob' method is the only way to ensure that you have the most available frame info. As far as removing a broadcaster/company's 3:2 pulldown, there is no freely available way to preserve the most info other than with a 'bob' method deinterlace. I'm not going so far as to say that the methods at, for example:100fps.com that involve 'bob' deinterlacing are the best, but they are going in the right direction, it just appears that they don't live in an NTSC country, so their info lessens the impact on furthering deinterlacing as a whole. Also, their processes are aged a bit with respect to what filters/version upgrades are out there now.

[bbb]

You'd be surprised on what stuff you can IVTC and have it looking 95-100% smooth on digital and analog TVs. My video sources include TV commercials, DV material, and TV shows. Only scenes with panning CG might exhibit a slight loss of smoothness (but these ocassions are rare, at least for me).

IVTC can be used to reduce heavily interlaced frames from any (film or interlaced) source. This provides two advantages: (1) reduce bitrate requirements (which will eliminate/reduce macroblocks) and (2) to eliminate/lessen the amount of ghosting from the later blending in SmartDeinterlace.

The result is interlaced video transformed into progressive video with very little loss of visual quality (no noticeable blurring or ghosting) and motion smoothness. And the advantages of having progressive video are enormous: enabling unrestrained ways to noise reduce/edit videos, allowing for encoding into DIVX format, allowing for beautiful playback on computer monitors, HDTVs, and analog TVs future-proofing your videos, and to never again see annoying interlace lines pop up in Windvd/Powerdvd.

Procedure:
1. IVTC in Tmpgenc 24fps Flicker Prioritized, Threshold 1 (you can adjust threshold).
2. Save as *.tpr project.
3. Convert tpr to *.avi with VFAPIConv.exe.
4. Open *.avi in VirtualDub.
5. Use SmartDeinterlace filter. Check: Field Only Differencing, Blend, Cubic, Motion Map Denoising. (Check Compare Color for animated shows).
* If lines interlaced lines remain (quickly scan through the clip), apply Blur and Sharpen filters.
6. Use Resize filter in Lanzcos mode to 480x480 (NTSC) and save as an AVI file using PicVideo MJPEG 20 setting (Subsampling 4/2/2). Saving into an AVI file will save time when using multiplass encodings.
7. Feed the AVI file into Tmpgenc or CCE and use Progressive mode encoding settings to create a MPEG file.
8. Use Pulldown.exe to change framerate of the MPEG file from 23.97 to 29.97. This step is very important as it will make sure that the video will play as smooth as possible after IVTC.

------------------

BTW, 100fps.com is an interesting read but the data is hardly applicable to MPEG creation. The site advocates upping the framerate to double the fps and then Bobbing fields. This may work alright for DIVX, XVID but won't work in MPEG as doubling the framerate (e.g., to 50 or 60 fps) is not allowed for valid MPEGs. Besides, I never found bobbing to be an acceptable method of deinterlacing (too jumpy).

As for experimenting in upping the framerates+field bobbing and then reducing the framerates to MPEG framerates, I think that would cause a reduction in quality (due to the field bobbing-->causing jumpy video and/or the blurring caused by the anti-bob filter) and probably be a big waste of time. The reason is because the 100fps.com site emphasizes upping the framerates to maintain motion smoothness. When you reduce the framerates again, you do not gain the motion quality for doubled framerates. Also, the anti-bob filter causes some blurring so you might as well use a traditional MPEG deinterlacing method or filter. Of course, if one has the time to try it out, one can try it to remove all such doubts. And if one does do so, please post the results (clips and pics).

[n0lamers]

@bbb
The stuff I encode is mainly ripped anime, so I do have to worry about deinterlacing more than the average joe, but as I stated before--using telecide, decimate does the same thing as TMPGEnc's IVTC, only much better. TMPGEnc's IVTC has been known to drop non-duplicate frames and creates blur.

One way any significant amount of blur can enter the picture is if blending is used on a 2 field composite frame, which is what you are doing with your example:

5. Use SmartDeinterlace filter. Check: Field Only Differencing, Blend, Cubic, Motion Map Denoising. (Check Compare Color for animated shows).

Field only differencing works best with interpolation, cubic, and motion map denoising.

Also, why would I want to blur again(or at all for that matter)?

* If lines interlaced lines remain (quickly scan through the clip), apply Blur and Sharpen filters.

I want to avoid blur to begin with, you lose video quality by blurring, then trying to compensate with sharpening. This also gives anime characters a false 'white aura' around their black borders when utilizing blur/sharpen.

Fortunately, Donald Graft's Telecide is a true adaptive 3:2 pulldown that adjusts to variable telecining patterns within the same clip, which is more than I can say for TMPGEnc's IVTC. All of those 'adaptive' settings are fooling everyone. Take any anime for example. All they have to do is perform an adaptive IVTC in TMPGEnc, encode, then drop the mpeg into virtualdub and step through the video. It has a mixture that includes many interlaced frames, blur, and ghosting. Also it excludes many frames that have unique footage because it doesn't adapt to different telecining patterns, therefore it deletes frames to follow whatever pattern it happens to be using! This defeats the purpose of deinterlacing afterward if you've already lost important footage.

BTW, 100fps.com is an interesting read but the data is hardly applicable to MPEG creation. The site advocates upping the framerate to double the fps and then Bobbing fields.

The 100fps site has excellent examples of deinterlacing which are perfectly applicable to mpeg creation. While their methods don't specifically cover IVTC, this step can be done using decomb(avisynth), or telecide/decimate(virtualdub).

'Bobbing' is the only way to get all the info that an interlaced video clip provides, therefore this process in and of itself is not the cause of blur. Also, you are being redundant when you mention 'upping the framerate' and 'bobbing fields'--the framerate gets upped via the bobbing process. This is because each frame is made up of two fields, all bobbing does is seperate the lines of a frame and stacks them into two seperate fields which makes two frames for every previous one(and these frames are exactly half the size of the frame they once made up). The 100fps site's main point is that they are trying to take out some of the guess work as to which deinterlace methods do what during decoding, then the user is supposed to encode however they choose after using elements of those decoding methods. Its not meant to supply an encoder with an every-step-of-the-way decoding/encoding for all formats. They mention Divx at the beginning, but since their examples are thorough and in abundance for different framerates/filters, etc., they expect at least some leg work on the part of the user.

Lastly, anti-bob only reverses the 'bobbing' after the field/frames have been manipulated back to the original framerate, therefore this step in itself cannot create blur. The manipulation in between bob and anti-bob steps is the aspect of deinterlacing that I want to incorporate decomb(telecide/decimate) with, but I'm not entirely sure of combining an avisynth script like this with virtualdub's frameserver, or which settings/thresholds to use with this combination due to double the available frames during the manipulation stage.

I suggest that for the type of video material you encode, the decomb filter package for avisynth used in conjunction with smart deinterlacer (interpolation instead of blend), should yield a lot less blurry picture. Keep your other settings as is. This should get you much better results for any media (including cg).

[sammie]

Yes, now we are getting some good discussion. I too suspected that TMPEGEnc's IVTC was part of the problem. I didn't try too many of the others. I was still learning avisynth and I was limited to what I could do with VDUB. I also felt that some de-interlacing clean up was needed after even a good IVTC. BOB seemed the way to go, but I wasn't sure about the best way to deal with the doubling, then the audio syc....... well you all know. Hope this discussion gives us some good results.

[dphirschler]

Not sure if I am following, but I will go ahead and tell you about one of my favorite techniques for cleaning up very noisy interlaced video:

1. Deinterlace: unfold

This just puts each field side by side in the same frame

2. Use some filters here for cleanup, including a heavy dose of temporal cleaner

3. Deinterlace: fold

This has the ability of cleaning up some major noise, but preserving the crispness between the individual fields. In other words, the fields don't get blended together.

4. Decomb

If my understanding of your method is correct, then my method is similar, only it does not change the frame rate.


Darryl

[n0lamers]

@dphirschler
Yes, this is the direction I'm trying to go. Could you post what programs/filters/settings/etc. that you use to accomplish this? I noticed that you unfold (bob), use temporal cleaning filters, then fold back up the frames (anti-bob), then decomb. I fully understand the logic of cleaning up the seperate fields, my original thought on this was of blending seperate fields before 'folding' them back up, but I wasn't sure if this would improve things or not.

Also, have you found it better to decomb after anti-bobbing? One method that I proposed had the intention of bringing back down the frame rate with the inclusion of a decomb before anti-bob. I wouldn't know how to process a successful decomb though, on a doubled frame rate (and accomplishing reverse 3:2 pulldown if needed for telecined material) so that the correct framerate is created (ie: 59.94 to 47.94), before anti-bobbing. Anti-bobbing would then bring 47.94 down to 23.97 NTSCfilm progressive frames. Could there be any significant advantage to this? I think using decomb in this way is not necessarily what the author intended, but would it work? What settings for all of this? Thanks for any help.

[dphirschler]

My method has several advantages:

1. Unfolding (using VirtualDub's internal 'Deinterlace' filter), keeps the frame rate at 29.97. It just changes the frame size.

2. Heavy use of temporal cleaners after unfolding but before Decombing (or IVTC) is essentially blending the frame (which is two fields side by side) with surrounding frames.

3. It also makes use of the extra doubled frame that will be removed in the Decomb process.

4. Folding (again using VirtualDub's internal 'Deinterlace' filter) after the heavy use of filters (most of which would blend the frame slightly) preserves the fields. It does not blend the fields together.

5. Decomb (an AVIsynth filter) does an amazing job at inverse telecining and decimating. I would unfold/filter/fold then save the file. Then I would use AVIsynth to decomb it and serve it up to VirtualDub for final filters (if necessary). Then it gets served to TMPGenc.


Darryl

[n0lamers]

Thanks for the extra info Darryl. I don't know if this might interest anyone, but I just found a way you can do all of what Darryl mentioned in the same VirtualDub session. The program's called VirtualDubMod. I had to hunt for a while to figure it all out, but this is a great start in the right direction for me. As of last night, I could drop a *.vob ripped straight from the DVD directly into VirtualDubMod and edit the video with any avisynth or virtualdub plugin/filter I wanted. This is great because it eliminates the need for DVD2AVI all together, not to mention saving the color quality some more by video footage not having to pass through near as many programs.

To open a vob for avisynth and virtualdub editing, use the option 'open it via avisynth'. Within that open window, you have to use the 'mpeg2source' template to open it or it won't open. When you initially try to open the *.vob via this method, be patient because you'll have to wait for VirtualDubMod's Avisynth connection to create the .d2v file first, which may take a few seconds or a few minutes depending on how big the vob is. Don't fret when this happens, VirtualDubMod hasn't crashed!

VirtualDubMod works with Avisynth ver. 2.05 or later excluding 2.5 versions (I have 2.07 and it works fine), and you'll need to get the lastest versions of both programs from sourceforge. In order to use the importing of vob files directly with this program, I got this version of mpeg2dec.dll here:http://nic.dnsalias.com/Misc.html. You'll need to put it in the plugins folder of avisynth and make sure that it is exactly named "mpeg2dec.dll" or it won't work. There may be other mpeg2dec.dll's that work just as well if not better than the one I have listed (in fact, the one that comes with either Avisynth 2.07 or VirtualDubMod's dll package may work, but it didn't for me--probably because I didn't realize about the name discrepancies from the orginal 'mpeg2dec.dll' until later), so feel free to experiment around with them. As a side note, I've had other mpeg2dec.dll's that were named a little different and it didn't work with VirtualDubMod unless I changed the mpeg2source template's avisynth script to reflect the name change or just changed the name back to mpeg2dec.dll from whatever hybrid name it was previously.

I've just started to use this program, but I did transform a script that I found to make it compatible with MPEG2Source that will separate the fields (bob the fields) of the vob:

clip=MPEG2Source("Vts_??_??.vob")
clip.SeparateFields
***notice that you don't need to repeat the entire path of the vob, its already listed in the script the template made

To add this script in VirtualDubMod, just go to 'Tools' / 'Script Editor'. This will show you what avisynth script the VirtualDubMod MPEG2Source template created upon opening of the vob. After you type in the 'bob' script (listed above) under what the template created, just go to 'File' / 'Save and Refresh' then exit out of the script editor and you should see the same video as before but the vertical has been shrunk to half the height due to only half of the former full frame's lines being present. Your framerate has also doubled at this point(if NTSC DVD source, then its now 59.94fps instead of 29.97fps). You can do a number of things to it now before anti-bobbing, including anything that Avisynth plugins or VirtualDub filters can do.

I've only gotten as far as the steps illustrated above as of this posting, but if anyone cares I can continue to post what else I find out about decombing/cleaning after 'bobbing' and then reassembling fields to ultimately make progressive VCD output.

[bbb]

nolamers,

The stuff I encode is mainly ripped anime, so I do have to worry about deinterlacing more than the average joe, but as I stated before--using telecide, decimate does the same thing as TMPGEnc's IVTC, only much better. TMPGEnc's IVTC has been known to drop non-duplicate frames and creates blur.

Since you also deinterlace and encode anime, you realize that anime is not perfectly telecined, anime titles are a mix of interlaced-progressive-cg video sources that have been interlaced, meaning whatever method you use to deinterlace will leave in some interlace frames in and take out some non-duplicate frames. This is why most people give up and advocate encoding in interlace mode.

Of course, I realize the current impossibility of perfectly deinterlacing typical anime titles but have found ways to minimize the disadvantages (loss of non-duplicate frames) and yet reduce the number of interlaced frames, all the while, retaining clarity (despite your assumption of my videos being blurry--they are not). The fact is, my method using IVTC first and SmartDeinterlace second works very well for the anime titles I've dealt with.

One way any significant amount of blur can enter the picture is if blending is used on a 2 field composite frame, which is what you are doing with your example

As I've explained, IVTC is used to filter out the most interlaced frames. And the remaining few interlaced lines are blended out.

The result is very clear--You will not notice excessive blending--It will appear very close to the source video.

For a while I strayed away from blending but I found out that using IVTC first and SmartDeinterlace blend second creates clear progressive video. It may also work with your telecide technique--I suggest you try it before disparaging it.

Repeat: this is not a quick and dirty blend deinterlace job. IVTC significantly reduces the interlace frames and hence significantly reduces the amount of blend/blur to the point that it's unnoticeable from the source.

Also, why would I want to blur again(or at all for that matter? ... I want to avoid blur to begin with, you lose video quality by blurring, then trying to compensate with sharpening. This also gives anime characters a false 'white aura' around their black borders when utilizing blur/sharpen.

This aura I haven't noticed. When I need to blur (e.g., for noise reduction purposes--yes even retail anime dvds have macroblocks and mosquito noise) and then sharpen, I fine tune my sharpening to maintain as much closeness to the source as possible. Blur+fine-tuned-sharpening works pretty well ... it's not soo "blurry" as you seem to presume. The key is fine tuning.

Also, anime titles are not perfectly telecined. You seem to be overly confident about your "perfect" deinterlacing technique.

I have utilized SmartDeinterlace interpolation before. And for a while, I used to swear by it. Until I found out that it inserts specks of random dot noise in certain scenes. These specks are especially clear when there are horizontal white edges (not real interlace lines) like underlined text, subtitles, etc. It is very unsightly. Even the author of the SmartDeinterlace filter acknowledges this problem. SmartDeinterlace interpolation mode also incorrectly interpolates small details (e.g., eyes, mouths that are moving)--the results can be quite ghastly--although you may not notice right away (I was encoding several videos using SmartDeinterlace interpolation before I thoroughly checked and was surprised to find these flaws).

Also, Area Based deinterlacing similarly creates ghastly artifacts.

No amount of settings can completely eliminate these interpolation artifacts.

This why I started using IVTC+SmartDeinterlace blend.

Fortunately, Donald Graft's Telecide is a true adaptive 3:2 pulldown that adjusts to variable telecining patterns within the same clip, which is more than I can say for TMPGEnc's IVTC. All of those 'adaptive' settings are fooling everyone. Take any anime for example. All they have to do is perform an adaptive IVTC in TMPGEnc, encode, then drop the mpeg into virtualdub and step through the video. It has a mixture that includes many interlaced frames, blur, and ghosting. Also it excludes many frames that have unique footage because it doesn't adapt to different telecining patterns, therefore it deletes frames to follow whatever pattern it happens to be using! This defeats the purpose of deinterlacing afterward if you've already lost important footage.

Again, you have too much faith in your deinterlace filters in deinterlacing anime. It would take a 24th century artificial AI deinterlacing filter with human-like visual senses, an anime artist's mentality, and perfect interpolation abilities to deinterlace the typical anime title--which is a mix of interlace/progressive/cg video sources that have been interlaced. I've learned that it takes some compromise (e.g., minimize the interlace frames as much as possible and blend as little as necessary) to deinterlace these titles but my result is still very very good. However, unlike some, I dare not call my deinterlacing method "perfect" ... as the typical anime cannot be perfectly deinterlaced regardless of the type of filters.

Also, I suggest to look over your own videos that have used the SmartDeinterlace filter using interpolation settings. You might start noticing unsightly specks/dots near horizontal lines. You see this SmartDeinterlace is smart but not perfect. It will try to deinterlace all horizontal lines and sometimes distort them. Also, it tends to interpolate interlaced frames inaccurately resulting in really wierd looking moving "eyes" and "mouths." I've tried changing the SmartDeinterlacing settings to try to reduce these random noise and have had some success but, it will still inaccurately reproduce smaller details in some of my videos.

As such, I moved on to using IVTC+SmartDeinterlace Blend. It works pretty damn well (again I dare not claim it is perfectly deinterlaced) and is very clear. This is definetely preferable to the random wierd artifacts that SmartDeinterlace Interpolate produces.

The 100fps site has excellent examples of deinterlacing which are perfectly applicable to mpeg creation. While their methods don't specifically cover IVTC, this step can be done using decomb(avisynth), or telecide/decimate(virtualdub).

On the contrary. The site is aimed entirely at DIVX creation. It downplays if not disparages other deinterlacing techniques as being flawed. Most notably, it discourages you from deinterlacing non-film based interlaced sources (i.e., your typical anime titles). Of course, we both don't agree with this that is why we have found our own methods of deinterlacing our videos, animes, etc.

'Bobbing' is the only way to get all the info that an interlaced video clip provides, therefore this process in and of itself is not the cause of blur.

Bobbing is far from perfect--never was impressed with the results. Also, whenever you resize, you are interpolating data. Your assumption that you can retain "all the info" with one method or with using bobbing is not only overconfident but untrue and unnecessary. When we convert from DVDs to SVCDs (down resizing, deinterlacing, frame reduction, etc) size and motion interpolation is occurring. Interpolation is not a bad thing if the results closely mirror the source.

The manipulation in between bob and anti-bob steps is the aspect of deinterlacing that I want to incorporate decomb(telecide/decimate) with, but I'm not entirely sure of combining an avisynth script like this with virtualdub's frameserver, or which settings/thresholds to use with this combination due to double the available frames during the manipulation stage.

Although I have my doubts with the practicality of this endeavor, I wish you good luck. And hope you can report or better yet show samples of the results versus current deinterlacing techniques.

Most people on this site are afraid to experiment with deinterlacing and stick with interlaced mode only. To that I applaud your industrious attitude.

I suggest that for the type of video material you encode, the decomb filter package for avisynth used in conjunction with smart deinterlacer (interpolation instead of blend), should yield a lot less blurry picture. Keep your other settings as is. This should get you much better results for any media (including cg).

My encodes are not blurry. I spend ample time using VD to match the clarity in my resultant video with the source; also, I compare my final SVCD with the DVD to ensure that accurate clarity and smoothness is maintained). In other words, I do quality control my SVCDs and would not accept a blurry or distorted creation.

However, I would love to compare deinterlacing techniques. Name some anime titles on dvd (I've got a few titles) and a scene. We can both encode a short clip (of the same scene) and post them on a website. I have 50 megs of space on Streamload.com (free web based media storage).

[n0lamers]

First of all, bbb, I never once called my proposed method perfect, its not even verified yet! I'm well on my way to finally being able to produce a sample of what I'm proposing though, so its not impractical at all, its just that few have tried using some of these filters in a double available framerate before.
The proof will be 'in the pudding' so to speak. I may take you up on uploading comparative anime clips to your site once I've produced some. I work and go to school full time, so it may be a few days before I can make some finished product.

As far as using TMPGEnc's IVTC, there are plenty of people who believe that it is worth disparaging, not just me. Smart Deinterlacer is a good filter, and if you really used to swear by interpolation and experimented with settings to compensate for artificing to no avail using interpolation, then perhaps you should try using 'frame and field differencing' instead of just field differencing, and also change the threshold of 'motion map denoising' to something in between 10-15, lower will reduce more combs, higher will reduce less combs--BUT produce less sparkling/artificing. As of ver. 2.6 of smart deinterlacer, Donald Graft has improved motion map denoising.[/quote]

[bbb]

I've thoroughly experimented with SmartDeinterlace and its settings. I've been able to eliminate most of sparkling dots using low thresholds values.

But, one thing that can never be eliminated is the inaccurate interpolation of smaller moving details (e.g., a quick moving mouth) using SmartDeinterlace interpolation mode, regardless of the field/frame/fieldandframe and using motion map denoising with low values for motion threshold and scene change threshold.

Another thing I noticed is in SmartDeinterlace Interpolate mode is that in some scenes, it creates a line doubled appearance in straight lines or diagonals. This line doubled appearance does not occur when using blend mode in the same scene which is ironically clearer in these cases.

This doesn't mean SmartDeinterlace interpolation should never be used due to the occassional but glaring interpolation of small moving detail flaw or the occasional line doubled appearance.

In fact, in one occcasion, it helped me create clear progressive frames of many single non-repeating details that existed between 2 interlaced frames in an anime clip (e.g., the Captain Tylor commercial break video that rapidly flashes through all the main characters of the show). There was no other deinterlacing method (I've tried them all in this case) to create a progressive frame of this clip without losing some of the characters or blending in two characters without using the settings, 29.97 fps and SmartDeinterlace interpolation mode. Because of this, I was very appreciative towards SmartDeinterlace interpolate mode.

However, for general full length anime shows and movies (without rapid flashing non-repeating vitally important detail scenes that exist for only 1 frame--which is pretty much everything else I've encountered other than the Captain Tylor commercial break clips) I use SmartDeinterlace blend mode as it eliminates the possibility of creating small detail motion interpolation distortions (e.g., ghastly distorted quick moving mouths). Again, I use IVTC before these videos to greatly minimize any blurring. And the result is very clear.

Another thing to note is that not all blending is bad (especially when constrained to the heaviest motion scenes). In fact, motion blur (achieved by blending) should be maintained in certain scenes. Whereas heavy motion interpolation to create clear progressive frames may create overly clear/unnatural motion scenes that should have contained extended motion blurs for the intended artistic effect. This is certainly the case with certain live action martial arts/fighting scenes and animal documentaries.

Since you prefer telecide/decimate to IVTC, and when you start to notice SmartDeinterlace motion interpolation's distortions, you should switch to SmartDeinterlace blend mode. You might start to prefer it as I have. If you use low threshold settings (<10 for motion, <10 for scene change) you may notice an ever so slight blur (only noticeable if you compare with the original video side by side). But by simply increasing the sharpness between 8-32, the original sharpness is restored without introducing any noticeable noise or distortion (that almost assuredly eventually will occur when using SmartDeinterlace interpolation mode).

In any case, I hope we can eventually compare deinterlaced anime clips to achieve improvements in our techniques, perhaps in the summertime when both of us have more free time. The reason is because it is rare to find one who is interested in high quality deinterlacing of all sources. Everyone seems rely on using interlace encode mode which I find very unsatisfactory.

[n0lamers]

I don't really plan on using smart deinterlace at all anymore, since I have other deinterlace methods in the works.

@bbb
I'm also not entirely opposed to blending, and you'd know that if you read some of my earlier posts in this thread--one suggestion I was contemplating was SeparatingFields and blending the field/frames before recomposing. But, I'll never blend a 2 field composite frame as is.

I think that I may have been misunderstood with my use of the word(s) 'bob' and 'bobbing' as well. I don't intend to confuse the act of bobbing frames into fields, with the D.Graft filter 'Smart Bob'. His filter works on frames that are bobbed using the SeparateFields command of avisynth. The Smart Bob filter is practically the same as the Smart Deinterlace filter except that it works on double frames. So there is room to be speculative on this filter, but the actual act of bobbing frames (which is separating the 2 fields of a frame) allows for no criticism whatsoever, it is was it is--but it puts the video in a mode that is the most exposed, so that even the finest detail is revealed to the user and to filters that look at such detail. I personally feel that a Smart Bob deinterlace is better than a Smart Deinterlace for this reason. However, I think that the Bob filter could use more of the options that have been put in Smart Deinterlacer to become even better. So I'm still in limbo as to whether I want to include this filter as a part of my plan for an all-in-one solution. I'll have to return here with some results...