Severely telecined frames in PAL DVD, can't seem to fix it

[GlitchBob]

So usually I'd use TFM() or even QTGMC(FPSDivisor=2), and that'd take care of it, but neither yield satisfactory results.

Raw:

[Attachment 61847 - Click to enlarge]


TFM: (note the duplicate frame afterwards which still has issues)

[Attachment 61849 - Click to enlarge]

[Attachment 61850 - Click to enlarge]


QTGMC (again a duplicate with issues):

[Attachment 61851 - Click to enlarge]

[Attachment 61852 - Click to enlarge]


There are also a few very aliased frames as well. Almost the entire scene is just one big mess.

[Attachment 61848 - Click to enlarge]


Looking to avoid duplicate frames and maintain the 25FPS if at all possible.

Many thanks in advance!

[jagabo]

This worked pretty well:

QTGMC()
TDecimate(mode=2, rate=25)

But the clip was too short. And next time include a medium speed panning shot that's a few seconds long.

[GlitchBob]

Thanks jagabo! I hadn't considered that combination. As for the clip length, sorry will keep that in mind next time.

[Brad]

There are only two frames in this clip that have combing, and they're caused by "dropped fields" that I suppose happened when the video was ingested for the DVD transfer. If you step through with SeparateFields, the pattern goes from 2-2 to 2-3-1-2-3-1. These repeated top fields that replace the original top fields are duplicates of the previous bottom field, just shifted down by one line.

Then before that is the aliased frame, where a single field was essentially duplicated with no shifting.

IMO you should just manually replace frames 9, 12, and 15 with QTGMC-filtered versions rather than running QTGMC over the whole thing.

[jagabo]

There are only two frames in this clip that have combing, and they're caused by "dropped fields" that I suppose happened when the video was ingested for the DVD transfer. If you step through with SeparateFields, the pattern goes from 2-2 to 2-3-1-2-3-1. These repeated top fields that replace the original top fields are duplicates of the previous bottom field, just shifted down by one line.

Then before that is the aliased frame, where a single field was essentially duplicated with no shifting.

IMO you should just manually replace frames 9, 12, and 15 with QTGMC-filtered versions rather than running QTGMC over the whole thing.

I was lazy at the time and just gave the first thing I tried. Manually replacing frames will be very tedious if there are many occurrences of such irregular pulldown patterns. I worked out an automated method that works for this clip:

##########################################################################
#
#	Abs(v1-v2), Y channel only
#
##########################################################################

function AbsSubtractY(clip v1, clip v2)
{
    Subtract(v1, v2).ColorYUV(off_y=2)
    Overlay(last.ColorYUV(off_y=-128), last.Invert().ColorYUV(off_y=-127), mode="add")
    GreyScale()
}

##########################################################################


Mpeg2Source("REBOOT.Title9.chapter05.demuxed.d2v", CPU2="ooooxx", Info=3) 

q = QTGMC().SelectOdd()

sep = SeparateFields()
eodiff = AbsSubtractY(sep.SelectEven(), sep.SelectOdd()) # difference between even and odd files of each frame

ConditionalFilter(last, q.Subtitle("combed"), last, "IsCombedTIVTC(cthresh=9)") # substitute q if frame is combed
ConditionalFilter(eodiff, q.Subtitle("eodiff", y=20), last, "AverageLuma()", "lessthan", "1.2") # substitute q if fields are the same

ConditionalFilter() is used to substitute a QTGMC() frame whenever combing is detected via IsCombedTIVTC(). Another ConditionalFilter() is used to substitute a QTGMC() frame whenever the even and odd fields are nearly identical.

One may need to adjust the AverageLuma() threshold if you get too many false positives or false negatives. You may need to change the threshold in IsCombedIVTC(cthres=9) to compensate for false positives/negatives too. QTGMC().SelectOdd() may need to be changed QTGMC().SelectEven() to prevent duplicates frames in some cases. Deciding on how to determine which is a project for another day (and a clip that needs it). And remove the Subtitle() calls for the final production.

[coolgit]

################################################## ########################
#
# Abs(v1-v2), Y channel only
#
################################################## ########################

function AbsSubtractY(clip v1, clip v2)
{
Subtract(v1, v2).ColorYUV(off_y=2)
Overlay(last.ColorYUV(off_y=-128), last.Invert().ColorYUV(off_y=-127), mode="add")
GreyScale()
}

################################################## ########################


src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).kil laudio()

q=QTGMC(src).SelectOdd()

sep=SeparateFields(src)
eodiff=AbsSubtractY(sep.SelectEven(), sep.SelectOdd()) # difference between even and odd files of each frame

cc=ConditionalFilter(last, q, last, "IsCombedTIVTC(cthresh=9)") # substitute q if frame is combed
cs=ConditionalFilter(eodiff, q, last, "AverageLuma()", "lessthan", "1.2") # substitute q if fields are the same

S1=StackHorizontal(\
subtitle(src, "Source", size=24, align=4),\
subtitle(q, "qtgmc", size=24, align=4)\
)
S2=StackHorizontal(\
subtitle(cc, "combed fields", size=24, align=4),\
subtitle(cs, "same fields", size=24, align=4)\
)
output=StackVertical(S1,S2)

Return output

I got the "invalid arguments to function conditionalfilter" line 24 which is cc=blah blah.
Any solution?

How exactly are you identifying differences in each frames?

I was thinking of using IsCombedTIVTC(mi=?) in both conditionalfilter.

I had some dvd extras that i wanted to incorporate in the main video. The extras interlaced vid was diabolical and used TFM to identify the correct fields. Sometimes is was same frame aka clean frames or matching previous field or next field or using one field then interpolate. The MI value were very reliable in detecting combed frames. Perhaps i am wrong but i suspect more reliable in using colours.

[jagabo]

I got the "invalid arguments to function conditionalfilter" line 24 which is cc=blah blah.
Any solution?

I couldn't duplicate the error. I cut/pasted the script from your post, just changing the source filename. It ran with no problems.

How exactly are you identifying differences in each frames?

I'm not sure what you mean. Combing is detected with IsCombedIVTC. Duplicate fields within a frame are detected by subtracting one field from the other.

I was thinking of using IsCombedTIVTC(mi=?) in both conditionalfilter.

That won't work for a frame where the two fields are identical (the second conditional filter). There's no combing in those frames.

I had some dvd extras that i wanted to incorporate in the main video. The extras interlaced vid was diabolical and used TFM to identify the correct fields. Sometimes is was same frame aka clean frames or matching previous field or next field or using one field then interpolate. The MI value were very reliable in detecting combed frames. Perhaps i am wrong but i suspect more reliable in using colours.

Use whatever works best for you.

[coolgit]

I got the "invalid arguments to function conditionalfilter" line 24 which is cc=blah blah.
Any solution?

I couldn't duplicate the error. I cut/pasted the script from your post, just changing the source filename. It ran with no problems.

Did you use vdub2?
Ok what dll files do i need for avisynth plugins folders as perhaps i am missing some files. Most are internal except for qtgmc and tivtc which i already have.

How exactly are you identifying differences in each frames?

I'm not sure what you mean. Combing is detected with IsCombedIVTC. Duplicate fields within a frame are detected by subtracting one field from the other.[/QUOTE]

I referring to your function.

[jagabo]

Yes, I used AviSynth+ and VirtualDub2 to preview the script, x264 CLI to encode -- all 64 bit. But I just tried 32 bit versions and they ran fine too. Try changing the "cc=ConditionalFilter(...)" line to just "cc=last". That will eliminate IsCombedTIVTC from the equation. The script won't perform its intended function but it may help you narrow down the possibilities.

AbsSubtractY() uses AviSynth's built in subtract filter to subtract one image from the other (in this case the top and bottom fields) and return the absolute value of the differences (just the Y channel). Essentially abs(a-b) on a pixel by pixel basis. The result is a greyscale image where identical pixels are black, non-identical pixels are not black, the bigger the difference the brighter the pixel. For example, if the pixel at X,Y in the first image a has a Y value of 100 and the pixel at the same location has a 90, the difference is 10. If the first image is 90 and the second image is 100 the difference also 10. You can always look at the resulting image to see what it looks like. eg "return(eodiff)" any time after it's calculated. You may want to brighten it because most of it will be very dark, "return(eodiff.ColorYUV(gain_y=1000))"

[coolgit]

cc=last - tried that and the error moved to the next line 25 cs=blah blah.

I'm going to restart my pc later and see if that will fix it.

[jagabo]

Oh wait! I see what's wrong with your script. I accidentally fixed it when I substituted my source line for yours. Change your source line:

src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).kil laudio()

to

src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).kil laudio()
last=src

That will get your script working. But your script doesn't accumulate all the fixes it just shows which frame gets which fix, if any.

[coolgit]

Oh wait! I see what's wrong with your script. I accidentally fixed it when I substituted my source line for yours. Change your source line:

src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).killaudio()

to

src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).killaudio()
last=src

That will get your script working. But your script doesn't accumulate all the fixes it just shows which frame gets which fix, if any.

I got it now. I mistakenly assume 'last' would be the previous line. Changed 'last' to 'src'. It is your script - all i did was change the output presentation to 2x2 clips.

[jagabo]

Most AviSynth filters take an input clip and produce an output clip. If you don't specify a clip by name AviSynth assumes the name "last".

In my script the first conditional filter takes the original video and replaces all the interlaced frames with a QTGMC frame. Then the output of that filter has the frames with duplicate fields replaced with a QTGMC frame. The end result is single clip where all the interlaced frames and all the dup-field frames have been replaced by a QTGMC frame.

Your script produces two clips, cc with only the interlaced frames replaced, and cs with only the dup-field frames replaced. It never produces a clip with both types of frames replaced.

[coolgit]

Does this work the same as yours?

################################################## ########################
#
# Abs(v1-v2), Y channel only
#
################################################## ########################

function AbsSubtractY(clip v1, clip v2)
{
Subtract(v1, v2).ColorYUV(off_y=2)
Overlay(last.ColorYUV(off_y=-128), last.Invert().ColorYUV(off_y=-127), mode="add")
GreyScale()
}

################################################## ########################

src=Mpeg2Source("F:\BABYLON5\Season 4\4x03 - The Summoning.d2v").ConvertToYV12(interlaced=true).kil laudio()

q=QTGMC(src).SelectOdd()

sep=SeparateFields(src)
eodiff=AbsSubtractY(sep.SelectEven(), sep.SelectOdd()) # difference between even and odd files of each frame

cc=ConditionalFilter(src, q, src, "IsCombedTIVTC(cthresh=9)", show=true) # substitute q if frame is combed
cs=ConditionalFilter(eodiff, q, cc, "AverageLuma()", "lessthan", "1.2", show=true) # substitute q if fields are the same

S1=StackHorizontal(\
subtitle(src, "Source", size=24, align=4),\
subtitle(q, "qtgmc", size=24, align=4)\
)
S2=StackHorizontal(\
subtitle(cc, "combed fields", size=24, align=4),\
subtitle(cs, "same fields", size=24, align=4)\
)
output=StackVertical(S1,S2)

Return output
#return(eodiff.ColorYUV(gain_y=1000))
#Return q

When I use show=true the 'same fields' clip have two set of info overlapping each other which makes it impossible to read the first info.

[jagabo]

That should give the same result. Regarding the show=true problem, use only one of them at a time.

[coolgit]

Sorted... thanks for your help.

[killerteengohan]

It's an easy fix for that series. I worked on that show before. No matter what deinterlacer or decomb filter I used, those would not go away. A solution was suggested to me, and it worked perfectly.

Just use this.

santiag(strv=0, nns=4, nsize=5)