problem with proper deinterlacing of NTSC DVD

[LaFonda]

Hello, it's my first post here. Since many days I'm trying properly convert music videos (to h264) from some of my japanese DVDs but with no success. My poblem is I can't properly remove interlacing.

I'm using MeGUI and AviSynth. Most of MVs is recognized by MeGUI as "partially film/varying field order (sometimes "top field first")". I tried deinterlace using filters MeGUI propose and else but there's always the same problem. For example, when I'm using TIVTC with tdecimate(hybrid=1) or tdecimate(hybrid=3) - it blend frames and result in ghost effect. When I'm using tdecimate(hybrid=0) it looks good but one per few frames is always deinterlaced wrong (sharpen, not smooth lines). I tried using other filters and change their settings but result is always the same (I got blend or that "bad" frames).

Here's screen how good and bad deinterlaced frames looks:
http://img827.imageshack.us/img827/3596/goodframe.jpg
http://img510.imageshack.us/img510/7029/badframe.jpg

I'll be grateful for any help and suggestions.

[jagabo]

Qtgmc() http://avisynth.org/mediawiki/QTGMC

[LaFonda]

I tried Qtgmc() with some basic settings but it gave me the same effect + much bigger file size. It's not that easy to learn how to use advanced settings. I would be grateful for any suggestion what settings I could try or other - easier - plugin to solve the problem.

[poisondeathray]

Post an unprocessed video sample

[jagabo]

QTGMC() doubles the frame rate from 30i to 60p, or 25i to 60p. That doesn't change the file size unless you are using uncompressed or lossless compression.

[LaFonda]

Post an unprocessed video sample

Here it is:
http://www.avacom.pl/GND.rar

It's dynamic part so problem is very easy to seen (but it's the same as in almost every music video I'm trying to compress).

QTGMC() doubles the frame rate from 30i to 60p, or 25i to 60p. That doesn't change the file size unless you are using uncompressed or lossless compression.

I used the same codec settings as in tests with other plugins/filters.

[jagabo]

QTGMC() doubles the frame rate from 30i to 60p, or 25i to 60p. That doesn't change the file size unless you are using uncompressed or lossless compression.

I used the same codec settings as in tests with other plugins/filters.

Yes, but what codec? Different codes will behave differently. A lossless codec uses whatever bitrate is necessary. Doubling the number of frames will roughly double the file size. But with lossy codecs where you specify the bitrate doubling the number of frames will not change the file size. It will just use fewer bits per frame, resulting in lower quality.

Your clip is a mixture of 30p and hard telecined 24p. All encoded interlaced, top field first. What is your intention with this video? Make a DVD? An MP4 or MKV file?

[LaFonda]

I'm compressing using h264. I know it's dedicated more to HD materials than SD, but it gave me better quality with smaller file size than Xvid do. Output file will be MKV container.

Almost every file I'm trying to convert is interlaced the same way as uploaded preview. I tried various settings of TIVTC and Yadif but I'm not satisfied from results. No matter what I tried some parts remaining badly interlaced (not smooth picture/straight lines) and I'm able to see it while playing final video.

[jagabo]

You won't be able to get a perfect IVTC of sources like that. They are shot on film, telecined, then edited (and effects added). There are many breaks in the telecine pattern, orphaned fields, and a mix of 30p and 24p with pulldown. TFM().TDecimate() did a fairly good job on the sample. I saw only one or two frames with artifacts, and a few duplicate frames at cuts. And the 30p section became jerky because one of every 5 frames had to be thrown out. For a better result you would have to handle the 30p sections differently and create a variable frame rate MKV. There is no perfect fix for the duplicate frames near cuts and some of the badly aliased frames. These occur because of the orphaned fields (fields for which the other half of the original film frame are not present). Some people will manually IVTC each shot then choose where to place a duplicate or produce a blended frame at the cuts.

Or you could encode interlaced and let the player handle the deinterlacing, just like the DVD.

[LaFonda]

Thank you very much for explanation. I'd rather deinterlace than leave it interlaced (because I would play files also on my other pc and laptop - where player have some problems with good looking deinterlacing).

For a better result you would have to handle the 30p sections differently and create a variable frame rate MKV.

I'd like to try it to rate what I would gain this way. I read about handling of hybrid source but there was a lot of things I don't understand. I tried something like "vfr via mkv" (from examples of TIVIC documentation) but I guess I did something wrong (got errors and avisynth hang out). Could you please tell me how could I create vfr mkv? You seems to know this topic very well and I'm still learning...

[poisondeathray]

Post the exact error message

There are examples in the TIVTC package, you just copy & paste it (change paths & filenames) , run 1st pass, run 2nd pass, mux timecodes in mkv with mkvtoolnix

But that's only part of the problem in your sample. The other problem jagabo mentioned is bad fields around the edit points. Some of the fields have aliasing and moire, so you might have to manually identify & filter/fix them, try antialias, or try to generate new ones from interpolation

[LaFonda]

I tried One Pass before and it always hang Avisynth while script loading. Following by your advice I tried with Two Pass. 1st pass passed well (created "matches.txt" and "metrics.txt" files) but 2nd pass gave me "TDecimate: input error (mode 5 and 6, all frames must be entries". I'm not sure what I'm doing wrong.

[poisondeathray]

post your avs scripts

[jagabo]

But that's only part of the problem in your sample. The other problem jagabo mentioned is bad fields around the edit points.

There is also the problem of effect being added after the hard telecine. For example, at about 2 seconds into the sample the video fades in from black. But that fade was added after the video was telecined. There is one pair of fields, both from the same film frame, in the middle of that fade where one field is darker than the other. When TFM() pairs the light and dark fields together it looks like a worse match than pairing with the other nearby fields. So it pairs with the other fields instead. That leads to comb artifacts (which the post processing tries to eliminate) and moire artifacts in the thin lines on the walls.

[poisondeathray]

Yes, in that section you can't manually match a combination that is clean, you have to do a manual repair

You can get better post processing in some sections by specifying clip2 , something like qtgmc or even nnedi3 - so those frames that it detects as combed get better deinterlacing instead of bicubic resized fields

"Automatic" VFR script also produces some errors, even if you tweak the long list of settings. For example at default settings, some dupes remain in the 24p section. You have to manually do it or specify overrides file manually, and then fix the bad frames later with photoshop or interpolation... A lot of work to get it "perfect".

For LaFonda, that error message occurs when you don't complete the 1st pass properly. For example, run the 1st pass .avs in vdub (file=>video analysis pass), close it, then open the 2nd pass .avs. If you ever open the 1st pass avs to preview, it will overwrite only a section, so the framecount won't match (you have to re-do the analysis pass then)

[LaFonda]

Well, I did two pass encoding without errors this time. I received file which Media Info see as 24.508 fps. Unfortunately there's no difference how TIVTC removed interlacing earlier and now - bad interlaced frames left.

Till now the best I could get was when I changed TFM to mode=5 and specified clip2. I'll try to do something with "ovr" parameter but I'm not sure how to use it yet.

[jagabo]

When I did a simple TFM().TDecimate() on the short sample I didn't get any interlaced frames. As noted earlier there were a few badly deinterlaced frames during the fade from black, missing frames in the 30p section, and a few duplicate frames near shot changes.

Mpeg2Source("GND - 02 - Drive away (Special Version).demuxed.d2v", CPU=2)
AssumeTFF()
TFM(d2v="GND - 02 - Drive away (Special Version).demuxed.d2v")
TDecimate()

Be sure "Honor Pulldown Flags" is set as the field mode in DgIndex.

[LaFonda]

Oh... seems that I used DGindex with default settings. Instead of "Honor Pulldown Flags" I had marked "Forced film". I changed it, indexed file again and tried but there's no difference. In file attached by you there are still that badly deinterlaced frames during fade from black.

[jagabo]

In file attached by you there are still that badly deinterlaced frames during fade from black.

If you read what I wrote earlier you would understand why. The only way to fix that is to manually brighten one field of the frame back to its original intensity. No automated IVTC will be able to handle it.

[LaFonda]

How can I do it manually? "Ovr" parameter have something common with it?

[jagabo]

The brightness and colors aren't quite perfect, but...

Mpeg2Source("GND - 02 - Drive away (Special Version).demuxed.d2v", CPU=2)
AssumeTFF()
SeparateFields()
p0=Trim(0,123)
p1=Trim(124,124).ColorYUV(gain_y=-115).ColorYUV(off_y=7) #darken
p2=Trim(125,125).ColorYUV(gain_y=100).ColorYUV(off _y=-6) #brighten
p3=Trim(126,126)
p4=Trim(127,127).ColorYUV(gain_y=30).ColorYUV(off_ y=-2) #brighten
p5=Trim(128,0)
Weave(p0+p1+p2+p3+p4+p5)
TFM(d2v="GND - 02 - Drive away (Special Version).demuxed.d2v")
TDecimate()

That fixes the fade in from black. The other problems remain.

[poisondeathray]

How can I do it manually? "Ovr" parameter have something common with it?

You can do it manually with yatta, but that fade in section is impossible to get a clean match, even manually. Apparently jagabo's script above fixed that section, but I can't get it working. The way I used was to replace / reconstruct the frame
EDIT: it's a copy & paste error, the message board inserts spaces where it shouldn't. Jagabo's script works well for the fade in section

You can use display=true to tweak the scripts for tfm and tdecimate to see what the algorithm "thinks", which matches are being made, and where it "thinks" combing occurs. Then you can use an manual override file for either tfm and/or tdecimate, and adjust cthresh and MI to adjust where it detects combing

Also , highly recommend using "clip2" parameter, something like qtgmc.selecteven because the deinterlacing used by TFM by default when combing is detected is a simple bicubic resize, which leaves aliasing

In this small example, the first section is clearly 30p, and the part after the fade in is mostly 24p, so you can specify that in the tdecimate ovr file.

If you were doing vfr without the overide, the default settings messes up the 24p section (it leaves some duplicates)

As jagabo described earlier, the fade in section needs to be handled differently. I would just manually decimate 1 of the bad frames with a ovr file, and interpolate the remaining bad frame with mvtools

For example, the ovr file for tdecimate I used was

 
0,58 v
59,770 f
62 -

This tells tdecimate in the 2nd pass to treat frames 0-58 inclusive as video, 59-770 as film, and delete frame 62. The frame numbering is based on what was provided by TFM previously . This leaves the remaining bad frame (its now #62 with the new numbering), so you can try mvtools2 to "fix" that. In this example I used a helper function based on mflowinter

How you approach a longer clip depends on how many sections are 24p or 30p. If it's very messed up, it might even be better using yatta (very tedious)

[LaFonda]

Thank you guys for explanation. I'll try to follow by your advice and examples.