Help with artifacting on solid color text

[duelle]

Original from DVD:
https://drive.google.com/open?id=0B0ObrFpgaHE8UUpoeW9aeE03TGs

My encode:
https://drive.google.com/open?id=0B0ObrFpgaHE8NTRXQ28yRi1DcXc
AVS script in MeGUI:

<input>
ConvertToYV12(interlaced=true)
AssumeTFF()
QTGMC(Preset="Placebo", SourceMatch=3, TR2=3, Lossless=2)

iTunes clip (hoping this counts as fair use):
https://drive.google.com/open?id=0B0ObrFpgaHE8TG1oMTY1LTNuZkk

So the source is very aliased to begin with and so far I'm just running it through QTGMC to deinterlace. I feel like the iTunes version is oversmoothed and loses too many details (which is why I'm trying to do my own encode), but the text about 5 seconds into the clip looks better (fewer artifacts, at least). Basically, I'm wondering if there's a way to reduce the aliasing without losing as much detail as the iTunes version using an AviSynth filter. I'm very new to AviSynth scripting and don't really know where to start for this sort of thing.

[jagabo]

NTSC DV 4:1:1 was badly converted to 4:2:0 by duplicating chroma samples rather than interpolating. To fix the chroma, try something like:

MergeChroma(PointResize(width/2,height).TurnLeft().nnedi3(dh=true).TurnRight().Spline36Resize(width,height))

By the way, the video is encoded interlaced but none of the frames are interlaced. If the whole video is like this there's no need to deinterlace. You may still want to use QTGMC's other processing though.

[duelle]

The video definitely has parts that are interlaced, this is just a seperate thing I want to try and mess with.

NTSC DV 4:1:1 was badly converted to 4:2:0 by duplicating chroma samples rather than interpolating. To fix the chroma, try something like:

MergeChroma(PointResize(width/2,height).TurnLeft().nnedi3(dh=true).TurnRight().Spline36Resize(width,height))

Wow, that is much better. So how would I know for next time when to use something like this? Is there some sort of analysis tool I can use? Also, would it increase the quality to use something other than Spline36Resize (like LanczosResize)?

[LMotlow]

the video is encoded interlaced but none of the frames are interlaced. If the whole video is like this there's no need to deinterlace. You may still want to use QTGMC's other processing though.

The m2v is progressive mixed with interlaced starting with frame 570.

[duelle]

Another question:
Where should I put the MergeChroma() line in my AVS script? Before or after QTGMC() and (if before QTGMC) before or after ConvertToYV12()?

[jagabo]

Ah, I missed those interlaced frames near the end. The earlier code seems two work ok on the interlaced parts but, to be sure you would probably want to SeparateFields() before the fix, then Weave() again after.

Another question:
Where should I put the MergeChroma() line in my AVS script? Before or after QTGMC() and (if before QTGMC) before or after ConvertToYV12()?

After playing around with the video a bit more, and trying the fix both ways (before or after QTGMC) I thought the best result was from this:

QTGMC()
MergeChroma(Spline36Resize(width/2,height).TurnLeft().nnedi3(dh=true).TurnRight().Spline36Resize(width,height))

Note I also changed the initial Point resize to Spline36 -- I thought that looked better (and has less potential to cause problems with other parts of the video). You can try Lanczos but I find it often creates oversharpening halos. You can always view the chroma planes directly with UtoY() or VtoY(). That converts them to greyscale and you can see the sharpness or lack thereof or look for other artifacts.

If you want to try the fix before QTGMC do it this way:

SeparateFeilds()
MergeChroma(Spline36Resize(width/2,height).TurnLeft().nnedi3(dh=true).TurnRight().Spline36Resize(width,height))
Weave()
QTGMC()

Note that nnedi3 shifts the chroma a bit (actually, it shifts the chroma and the luma, but in this case we're only using the chroma). With this video that aligns the chroma better with the luma. If you reverse the order of the TurnLeft() and TurnRight() filters the chroma will shift the other way and the alignment will get worse.

[duelle]

Okay, another small issue that QTGMC is causing.

The speaker grill on the TV in this clip gets very distorted near the end while the original doesn't have this issue.

Original:
https://drive.google.com/open?id=0B0ObrFpgaHE8Nk8taHM4eV9zMFE

Encode:
https://drive.google.com/open?id=0B0ObrFpgaHE8U21jVmdIQXpGVm8

I tried doing the encode without MergeChroma() and it's the same. QTGMC is the only other filter in the script so it must be doing something to cause this. I tried messing with some options but couldn't get it to make difference (for good or worse), so I defer to the experts. I should also mention this is visible in the script preview, too, so it's not the x264 encoding settings, either.

[jagabo]

Thin grills like that confuse most deinterlacers. If you can't find a deinterlacer that works without distotring the frames you can use two different filter paths and ReplaceFramesSimple() to select which of the two paths is used it the final mix:

Mpeg2Source("VTS_01_1.demuxed.d2v", CPU2="ooooxx", Info=3) 
q = QTGMC()
v = ConvertFPS(q.framerate)
ReplaceFramesSimple(q, v, Mappings="[0 105]")

Frames 0 to 105 will come from v, the rest from q.