inverse telecine weird NTSC DVD back to film introduces problems

[skh]

hello,
I usually always inverse-telecine 29,97 NTSC DVD's which are a mix of progressive and interlaced frames back to 23,976 by using the classic
TFM().TDECIMATE() from avs.

However in this special case of a TV show which was probably edited on Video but was converted back to film along the chain or whatever happened here on this particular DVD this method introduces strange errors probably due to some field problems but as this is not fully interlaced it doesnt do much to just decide between even or odd, in fact in this case both even and odd look much more terrible than an inverse telecine. Here is a sample: https://workupload.com/file/UCXaT7kWvDj
maybe some of the experts have a tip on how to deal with such cases.
thanks in advance for looking into it
regards
-skh

[Alex_ander]

Just tested this fragment and it works OK for with the 'Honor pulldown flags' option of DGIndex and TFM+Tdecimate defaults. I extracted the mpg from mkv first, of course. It has soft telecining, maybe combined with encoded one in some parts (since it says like just 80% Film), but with 'Honor pulldown flags' it worked OK.

[hello_hello]

It's because the blinds are so fine they're virtually impossible to de-interlace, and probably partly due to the noise TFM is detecting combing and attempting to de-interlace. Raising the threshold for combing detection seems to fix it. I remuxed your MKV as a TS file so I could open it with DGIndex/DGDecode. If you're opening the MKV with ffms2 or lsmash be sure the option for honoring repeat flags is enabled.

Try either
TFM(CThresh=12, slow=2).TDecimate()
or
TFM(MI=90, slow=2).TDecimate()

If it prevents TFM from repairing combing in other places, you might want to use Trim to separate the problem area, but make sure each Trim section is a multiple of 5 frames. You'll need to work out the frame numbers to use for Trim before adding TFM and TDecimate. For your sample:

Trim(0,194).TFM().TDecimate() + \
Trim(195,284).TFM(CThresh=12, slow=2).TDecimate() + \
Trim(285,0).TFM().TDecimate()

[jagabo]

Or use TFM(pp=0) to disable the post-field-matching deinterlacer.

LWLibavVideoSource("VTS_01_4.mkv") 
TFM(pp=0)
TDecimate()
Crop(12,0,-4,-0)

[skh]

thanks a lot you guys for the info on how to configure the combing threshold and how to disable the post-field-matching order. I never thought about such settings before. The only thing I ever did if something wasn't detected right was a AssumeTFF() but other than that the standard TFM setting always did the job great.
Both of your suggested methods worked equally fine. However there are still plenty of interlaced leftover frames in the source. Here is a sample of a scene where with TFM(pp=0) already applied where you see it if you look at the knife and his mouth https://workupload.com/file/nqg7s8Gua4N
I tried to fix it with qtgmc progressive repair QTGMC(Preset="Fast", NoiseProcess=0, SMode=0, TR2=0,InputType=1)
but it didn't help much so I guess those are baked in?

thanks a lot for your further help

[hello_hello]

I set the combing threshold to -1 to test it, which means TFM will de-interlace the entire frame, rather than only where there's combing, but it looks like it can be fixed by de-interlacing if you let TFM do it.

TFM using QTGMC to deinterlace.

A = QTGMC().SelectEven()
TFM(metric=1, CThresh=-1, Clip2=A)
CropResize(0,0, 18,0,-12,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)
ConvertBits(8)

I changed TFM's own de-interlacing to "blend" as the default created some jagged edges, but that mightn't be needed when working with the original source. These samples are both slightly blurred as the entire frame is being de-interlaced, but it appears de-interlacing is all that's required, although you might need to lower the combing detection threshold. The default for CThresh is 9.
And maybe try Metric=1 for TFM as sometimes it can be better at detecting fine combing.

TFM(metric=1, CThresh=-1, pp=5)
CropResize(0,0, 18,0,-12,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)
ConvertBits(8)

[skh]

Thanks a lot for your further help.
I usually use the lossless mode for the standard cases QTGMC(Preset="fast", NoiseProcess=0, SMode=0, TR2=0, lossless=2, sourcematch=3)
this way you don't lose any sharpness. The standard settings for QTGMC often appear to blurr/unsharp. Not for all sources of course but I noticed it
on some DVD's which had good sharpness before. However this mode doesn't do any good in this special case it was probably only meant for the fully interlaced sources wheras for the progressive repair jobs the standard settings are better suited.

Regarding your samples:
the qtgmc method appears to be a bit sharper than the tfm only method but both methods have their issues on specific scenes.
For example if I use TFM(metric=1, CThresh=-1, pp=5) or TFM(metric=1, CThresh=12, pp=5) I'm getting weird things such as in frame frame 112 (looks even more weird when played in real time with mpc)
https://workupload.com/file/F3kbhy4W9hJ
and if I turn it off with 255 (no pixel will be detected as combed) I'm getting the weird grey pixels again as seen in 00:00:18 - 00:00:20 when applying TFM() TDECIMATE() without special settings on the sample of the main post

[jagabo]

The video in post #5 looks like it was resized vertically before TFM -- causing the two fields to be co-mingled. Can you upload the source?

[skh]

I didn't apply any resize to that video but didn't set the display aspect ratio to 4:3 when I encoded it as it was just a test so it automatically changed to 3:2 as it always does when doing that with 720,480 which only looks right with 4:3 dar.
I made another export with hellohellos suggested resize CropResize(0,0, 18,0,-12,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0) but the problems remained. If you check out sample1 between 00:00:02 - 00:00:07 you'll notice the bug and it is expected that this will happen every time when that weird unstable stair pattern occours of this "film which was edited on video converted back to film + pulldown for DVD" mess...

Here are all 3 samples extracted from the original DVD with mkv toolnix which should be a lossless sample extract of the original DVD source to work with: https://workupload.com/archive/pjqNBpCjP9
thanks

[hello_hello]

It's one of those sources where if you want the best possible result, you'll need to adjust TFM (and maybe use other filters) for different sections accordingly.
The first sample has the stair pattern baked in. It's not TFM's fault, but you can at least settle it down a bit with an anti-aliasing filter. I just threw one at it for the attached encode and didn't try adjusting it.


[Attachment 79133 - Click to enlarge]

For the record, an easy way to see where TFM is detecting combing is to do this. Anywhere you see red, it's detecting combing. It can help when adjusting it's settings.

A = Last
Red = BlankClip(A, Color=color_red)
TFM(Clip2=Red)

[Attachment 79134 - Click to enlarge]

The cropping and resizing is only added due to my OCD. It's not supposed to improve the quality in any way.

Sample1 = DGDecode_mpeg2source("M:\NVME2\sample1.d2v")
Sample2 = DGDecode_mpeg2source("M:\NVME2\sample2.d2v")
Sample3 = DGDecode_mpeg2source("M:\NVME2\sample3.d2v")

Sample1 = Sample1.TFM().TDecimate().MAA2()

Sample2 = \
Sample2.Trim(0,194).TFM().TDecimate() + \
Sample2.Trim(195,284).TFM(CThresh=12, slow=2).TDecimate() + \
Sample2.Trim(285,0).TFM().TDecimate() 

Sample3 = Sample3.TFM(pp=5, MI=40, Metric=1, chroma=true).TDecimate()

Sample1 + Sample2 + Sample3
CropResize(0,0, 18,0,-12,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)

Maybe someone else will come up with something more clever.

[jagabo]

There's definitely something wrong with sample3.mkv. You can't cleanly IVTC it. vinverse() can blur away the residual combing without damaging the rest of the picture much.

By the way, vinverse() does mess up the horizontal blinds in the sample2.mkv video.

[Selur]

Another suggestion, not clever, but lazy: (used Vapoursynth, but all these filters are available for Avisynth too)

# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip, PP=0)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=0)# progressive
# Denoising using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", InputType=1, TR2=2, SourceMatch=3, Lossless=0, opencl=True)
# contrast sharpening using CAS
clip = core.cas.CAS(clip=clip, sharpness=0.630)
# applying anti aliasing using santiag
clip = havsfunc.santiag(c=clip, nns=2, qual=2, pscrn=2, opencl=True)

I seem to work okay for all the samples.

Cu Selur

[skh]

thanks you guys for working on this so much it means a lot.

@Selur that worked awesome, very impressive. I used AssumeFieldBased() to replace the Vaporsynth SetFrameProps command as SeparateFields seemed wrong to me.
For santiag the options "nns=2, qual=2, pscrn=2" don't seem to be available in AVS or maybe they are there under different names?
Is there a santiag vaporsynth documentation somewhere so I could compare their setting to the ones from the avs wiki? Couldnt find anything and am curious if that would improve the results even more
thanks so much

[Selur]

I used santiag from https://github.com/Selur/VapoursynthScriptsInHybrid/blob/41f5e24261340a384525067be0473...vsfunc.py#L159 don't think there's any documentation for it.

For santiag the options "nns=2, qual=2, pscrn=2" don't seem to be available in AVS

nns is listed under http://avisynth.nl/index.php/Santiag
qual and pscrn are option for NNEDI3. (2 is the default)
=> id you want different options for those you would have to modify the santiag script in Avisynth.

[skh]

alright, great thanks a lot

[skh]

the project just turned into a nightmare. I just realized that it is actually a composite cut out of telecined film and fully interlaced video.
Before making hellohello's suggestion with the Trim section based filter work a reality I kindly ask if there is a way to detect the scene parts which are fully interlaced? So as an alternative to the "detect combing when everything is red" a detect the fully interlaced sections?

[Selur]

Sadly, there isn't even a reliable method that detects if a source is interlaced,...

[hello_hello]

As your video is problematic it mightn't be extremely accurate, but you can run TIVTC in variable frame rate mode.
It requires two passes. The first (analysis pass) looks like this:

TFM(Output="TFM.txt")
TDecimate(Mode=4, Hybrid=2, Output="TDecimate.txt")

This would be for the second (encoding pass), but when you open the script it'll create Timecodes and OriginalFrames text files.

TFM(Input="TFM.txt")
TDecimate(Mode=5, Hybrid=2, Input="TDecimate.txt", tfmIn="TFM.txt", mkvOut="Timecodes.txt", orgOut="OriginalFrames.txt")

This is the contents of the timecodes file for sample2.

# timecode format v1
Assume 29.970030
# TDecimate v1.0.11 by tritical
# Mode 5 - Auto-generated mkv timecodes file
0,219,23.976024
225,912,23.976024
# vfr stats:  99.47% film  00.53% video
# vfr stats:  1135 - film  6 - video  1141 - total
# vfr stats:  longest vid section - 5 frames
# vfr stats:  # of detected vid sections - 1

TDecimate decided there was a single section of video with a duration of five frames. After decimation that video section begins at frame 200.
If you then open the OriginalFrames text file with a text editor that displays line numbers, you can scroll down to line 200, add one more because Avisynth starts counting at zero, and before any decimation, and therefore using the original frame numbers, the video section begins at frame 275.
In this case the blinds in the background probably caused TFM to de-interlace enough successive frames for TDecimate to decide they must be video, so it's probably not really video in this case, but over-all the above method should help you find any substantial sections that are.


[Attachment 79174 - Click to enlarge]

[Alex_ander]

... if there is a way to detect the scene parts which are fully interlaced?

You can visualize them in AVSpMod for scenes with motion, by turning on/of e.g. the TFM filter. If a video fragment is full interlaced, it will still look interlaced throughout after turning on TFM(pp=0), while if it's just field-shifted progressive or telecined, it will become progressive after field-matching with TFM. So the interlaced scene will stand out.

... it is actually a composite cut out of telecined film and fully interlaced video.

Can you post a more lengthy sample containing both telecined and full interlaced parts?

The 3 samples you posted don't show this. As for the scene with blinds in sample2 (=renamed first upload), it's not full interlaced either. Due to specific type of content (thin horizontal lines) TFM filter (with its default setting pp=6) takes it for line structure and tries to modify their internal areas after field matching (frames 245, 274, 276, 277 & 278 - I didn't notice it before doing my older post, sorry for that). This issue completely goes away with disabled post-processing - TFM(pp=0), as jagabo has already shown, which results in standard IVTC pattern (4+1) for further decimation. Here's the difference before/after disabling the post-processing (pp=0):
frame 245:

[Attachment 79179 - Click to enlarge]
[Attachment 79180 - Click to enlarge]
frame 274:

[Attachment 79181 - Click to enlarge]
[Attachment 79182 - Click to enlarge]

You can apply the pp=0 setting selectively (to that short fragment only), by using Trim.

What source filter do you use for importing the video into script?
I think DGIndex applied directly to MPEG2 is more reliable here than some DS filter with mkv. I can't see residual combing in the other two samples, other than in the very beginning (frame 1 with pp=0 setting only, otherwise frames 0-2 are the same), which might come from VOB cutting within a GOP, etc.

Here are samples 1, 3 encoded directly from AVSpMod, using the simple script below and muxed with audio (AR set in mkv):
https://workupload.com/file/rsqAVakSY4E
https://workupload.com/file/59dRcDugTaj
MPEG2Source("D:\1-Temp\sampleX.d2v", cpu=0)
TFM()#TFM(pp=0) for sample2 (maybe partly, with using Trim)
TDecimate()

[skh]

thanks hellohello and alex_ander for the input on how to deal with such cases.
The mkv samples I created with mkvtooolnix from the original dvd_vob source because I wouldnt know how else to make a lossless sample from a vob format DVD
Your suggestion that the broken parts of the samples jagabo mentioned are a result of a wrong placed cut by toolnix is probably right.
I'm working with the original VOBs here of course and I index them with MPEG2Source just as you describe.

I havent posted any of the actual telecine/video mix scenes yet but will do so in a couple of hours. thanks for your help and later

[skh]

here is the sample of the telecine/interlace mix: https://workupload.com/file/gxYsu4xAczq

[Alex_ander]

here is the sample of the telecine/interlace mix: https://workupload.com/file/gxYsu4xAczq

Yes, it's really a sequence of eleven different pieces: 5 telecined, 4 progressive (deinterlaced 29.97) and 2 interlaced (2580-2748 & 2819-2970 - looks like overlayed graphics).
In case it happens through all the film, the easiest would be to just import it and encode interlaced. If progressive result is important, another opportunity is to IVTC the whole thing (some frames will be dropped in progressive inserts), with TFM's post-processing for the interlaced parts (it works OK in this sample) and without it (pp=0) for scenes like the blinds we discussed.

[Selur]

for simple handling: maybe use same frame rate deinterlacing and replace duplicates with interpolations,...

[skh]

it has to be progressive. yea my first idea was to interpolate the 23,976 back to 29,97 to match to the deinterlaced 29,97 parts but that of course only leads to stuttery playback as it can only duplicate frames in order to achieve the 29,97. So I guess my only chance is to either live with all the dropped frames in the ntsc video parts which of course are mostly noticeable in fast movements (the frame drops are the same with and without (pp=0) ) or to interpolate the ntsc video parts to 23,976 with the most clever interpolation around what ever that is as I never did that before and have a hard time understanding select every frame patterns and after that cut them into the film

[jagabo]

You can convert to 59.94 fps progressive. Playback on a 60p monitor or TV will look the same as playing the original DVD. Many 120 Hz TVs will recognize the 24p sections and play them with 5:5 repeats, and the 30p sections with 4:4 repeats.

[skh]

You will laugh but that is the idea which came to me while asleep. I just played around with it and it looks pretty good regarding smoothness. However of course the stair pattern which we so greatly eliminated becomes more obvious again as we don't remaster any frames.
Doing some SeparateFields() and throw away the odd keep the even or throw away the even and keep the odd experiements only lead to worse results.
So if you have a clever idea how to fix the stair pattern (obvious in sample1: car and sample 2 rolling shutter) that would be awesome

[Sharc]

FWIW trying sample2.mkv using DGTools/NVidia seems to cope reasonably well with the blinds

LWLibavVideoSource("D:\Temp\skh\sample2.mkv")
assumeTFF()
DGTelecide(mode=0,show=false)
DGDecimate(cycle=5,keep=4,show=false)

[jagabo]

An antialias filter like Santiag(2,0) will reduce the aliasing in sample1. A second call will reduce it even more. Here's an example using a mask to limit Santiag() to strong edges:

LWLibavVideoSource("sample1.mkv") 
TFM()
TDecimate()

aa = Santiag(2,0)
edgemask = mt_edge(mode="hprewitt", thy1=254, thy2=254).mt_expand().Blur(1.0).Blur(1.0).Blur(1.0)
Overlay(last, aa, mask=edgemask)

[hello_hello]

If it pleases the court, I would like to offer an alternative hypothesis for the "mixed" sample.
I was really only concerned with making the shots of the flying gizmo look smooth. In the second shot, if you look closely, it does make the motion of the cars in the background a little jerky in a couple of places as it's a combination of different frame rates, but it's barely noticeable. I was only worrying about the two overlaid shots of the flying gizmo and not about applying the best filters.

A = QTGMC().SelectEven()
TFM(metric=1, clip2=A, CThresh=8, chroma=true)

Trim(0, 2579).TDecimate(denoise=true, conCycle=2) + \
Trim(2580, 2749).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2750, 2814).TDecimate(denoise=true, conCycle=2) + \
Trim(2815, 2969).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2980, 0).TDecimate(denoise=true, conCycle=2)

MAA2()
CropResize(0,0, 16,0,-16,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)

[hello_hello]

Apparently I don't possesses enough smarts to realize the mixed sample contained a couple of slabs of video when I first looked at it, and I accidentally skipped ten frames, so here's two more attempts:

# TelecineInterlace-Mix (take 2)

A = QTGMC().SelectEven()
TFM(metric=1, clip2=A, CThresh=8, chroma=true)

Trim(0, 144).TDecimate(denoise=true) + \
Trim(145, 549).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(550, 599).TDecimate(denoise=true) + \
Trim(600, 1244).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(1245, 2579).TDecimate(denoise=true) + \
Trim(2580, 2749).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2750, 2814).TDecimate(denoise=true) + \
Trim(2815, 2979).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2980, 0).TDecimate(denoise=true)

MAA2()
CropResize(0,0, 16,0,-16,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)

# TelecineInterlace-Mix (take 3) - easier, but with more obvious blending

A = QTGMC().SelectEven()
TFM(metric=1, clip2=A, CThresh=8, chroma=true)

Trim(0, 2579).TDecimate(denoise=true, hybrid=1) + \
Trim(2580, 2749).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2750, 2814).TDecimate(denoise=true) + \
Trim(2815, 2979).TDecimate(denoise=true, conCycle=2, m2PA=true, mode=7, rate=20000.0/1001.0).FrameRateConverter(NewNum=24000, NewDen=1001) + \
Trim(2980, 0).TDecimate(denoise=true)

MAA2()
CropResize(0,0, 16,0,-16,0, ResizeWO=true, CropDAR=4.0/3.0, InDAR=15.0/11.0)