Help Restoring 24p Cadence of Video

[csc0709]

I've have been working on restoring/remastering a DVD release of TV series back to the original 24p. The series, Matt Houston, was shot on film and edited on film from the early to mid 80s. It was released on DVD by VEI in the US and Canada. The third season somehow used PAL masters for the DVD release. They were badly converted to PAL and then back to NTSC for this release and are bad transfers to begin with. This will be very obvious from the telecine pattern. Most of the episodes I was able to restore except a couple. I used the following script in avisynth:

tfm()
tdecimate(cycle=6, cycleR=1)

This script fails on this particular episode. When I run it, I get duplicated frames about every 33 seconds or so (I think the pattern shifts). I've tried different combos with not much success. So I'm reaching out for help for some other more advanced scripting suggestions. I'm beginning to think its a lost cause.

Below is a link to a direct rip from the DVD of the episode:

https://f002.backblazeb2.com/file/ChrisShares/Video+Other/Deadly+Games.m2ts

I am just interested in the episode itself, no need to worry about the intro.

[Sharc]

Use

TFM()
TDecimate(cycle=5,cycleR=1)

it is the usual/standard/default IVTC for 2:3 telecined stuff.

[csc0709]

Use

TFM()
TDecimate(cycle=5,cycleR=1)

it is the usual/standard/default IVTC for 2:3 telecined stuff.

No it isn't as I've already tried that, when I do that I get a combo of dropped and duplicated frames. You can clearly see this in the panning shot at the beginning after the intro.

[Sharc]

I don't get it. What source filter are you using? Frame number? Maybe I am missing something.
TFM().TDecimate(cycle=6,cycleR=1) is sometimes used for 25fps sources. In your case it returns 24.975 fps with occasional repeats here. Cycle=5,CycleR=1 gives 23.976fps original progressive film frames. No glitches..... or I missed it.

[csc0709]

This is my script:

FFmpegSource2("G:\Matt Houston\Rip\Ep11\Deadly Games.m2ts")
tfm()
tdecimate(Cycle=5, CycleR=1)

I get my first dropped frame at frame 80 and again at 160. I don't get any dropped frames using the TDecimate(cycle=6,cycleR=1), but I do get duplicated frames about every 33 seconds. In the telecine pattern, sometimes its two interlaced, sometimes 3, sometimes as many as 4.

[Sharc]

Yes, got it now. There are pattern breaks.
Maybe increasing the cycle helps, something like

TFM()
TDecimate(cycle=25,cycleR=5)

Use the display=true options for TFM() or TDecimate() to see what happens

Maybe someone else has a better proposal.

[jagabo]

I used DeDup() to analyze a 1000 frame section with no still shots. It looks like there were 820 unique frames out of that 1000. 30000 / 1001 * 820 / 1000 gives a frame rate of about 24.5754245754. Using TDecimate(mode=2, rate=24.5754245754) worked pretty well.

# after a remux to MPG and using DgIndex to build an index file
Mpeg2Source("Deadly+Games.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Deadly+Games.d2v", Clip2=QTGMC(FPSDivisor=2)) 
TDecimate(Mode=2, rate=24.57542457542)

[Sharc]

I used DeDup() to analyze a 1000 frame section with no still shots. It looks like there were 820 unique frames out of that 1000. 30000 / 1001 * 820 / 1000 gives a frame rate of about 24.5754245754. Using TDecimate(mode=2, rate=24.5754245754) worked pretty well.

# after a remux to MPG and using DgIndex to build an index file
Mpeg2Source("Deadly+Games.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Deadly+Games.d2v", Clip2=QTGMC(FPSDivisor=2)) 
TDecimate(Mode=2, rate=24.57542457542)

Great analysis spanning 1000 frames

[csc0709]

I used DeDup() to analyze a 1000 frame section with no still shots. It looks like there were 820 unique frames out of that 1000. 30000 / 1001 * 820 / 1000 gives a frame rate of about 24.5754245754. Using TDecimate(mode=2, rate=24.5754245754) worked pretty well.

# after a remux to MPG and using DgIndex to build an index file
Mpeg2Source("Deadly+Games.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Deadly+Games.d2v", Clip2=QTGMC(FPSDivisor=2)) 
TDecimate(Mode=2, rate=24.57542457542)

Thank you, I'll give this a shot and see what I get.

[csc0709]

I used DeDup() to analyze a 1000 frame section with no still shots. It looks like there were 820 unique frames out of that 1000. 30000 / 1001 * 820 / 1000 gives a frame rate of about 24.5754245754. Using TDecimate(mode=2, rate=24.5754245754) worked pretty well.

# after a remux to MPG and using DgIndex to build an index file
Mpeg2Source("Deadly+Games.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Deadly+Games.d2v", Clip2=QTGMC(FPSDivisor=2)) 
TDecimate(Mode=2, rate=24.57542457542)

This worked pretty good. Thanks again for your help.

[jagabo]

I noticed a few drops/dups in some parts of the sample after the above script. Adding "m2PA=true" to the TDecimate filter fixed some of those.

TDecimate(Mode=2, rate=24.57542457542, m2PA=true)

[csc0709]

I noticed a few drops/dups in some parts of the sample after the above script. Adding "m2PA=true" to the TDecimate filter fixed some of those.

TDecimate(Mode=2, rate=24.57542457542, m2PA=true)

I'll give that a try, though without that setting, it was a lot better than what I was getting before. What does the m2PA setting do? Also, in this case what is the QTGMC doing?

[jagabo]

What does the m2PA setting do?

It allows for a longer lookahead to match the desired frame rate.

Also, in this case what is the QTGMC doing?

When TFM finds residual comb artifacts after frame matching it deinterlaces those areas. If supplied, it uses clip2 to deinterlace those areas.

Oh, I also added vinverse() after TFM() to reduce the residual comb-like artifacts caused by compression of interlaced frames.

[csc0709]

What does the m2PA setting do?

It allows for a longer lookahead to match the desired frame rate.

Also, in this case what is the QTGMC doing?

When TFM finds residual comb artifacts after frame matching it deinterlaces those areas. If supplied, it uses clip2 to deinterlace those areas.

Oh, I also added vinverse() after TFM() to reduce the residual comb-like artifacts caused by compression of interlaced frames.

I think this method worked great for the first part of the video, but I was getting a lot of dropped frames on the back part of the video, say the last 15 to 20 minutes. Any thoughts on how to approach that?

[jagabo]

Can you give some specific frame numbers so I don't have spend all day looking for drops? And the avisynth script you used.

You'll probably never get perfect results because of the variable pattern of the dups and compression artifacts.

[csc0709]

Can you give some specific frame numbers so I don't have spend all day looking for drops? And the avisynth script you used.

You'll probably never get perfect results because of the variable pattern of the dups and compression artifacts.

Ok makes sense. This is the script I used:

FFmpegSource2("G:\Matt Houston\Rip\Ep11\Deadly Games.m2ts")
tfm(Clip2=QTGMC(FPSDivisor=2))
tdecimate(mode=2, rate=24.57542457542, m2PA=true)

I will watch the whole thing a little more closely, but the area I noticed the drop frames was the last epilogue scene before the close credits. I'm not sure what frame numbers, but during the building pan at the start of the scene there was a dropped frame and while the two were walking down the stairs there were two drops. The fade to blacks for the commercial breaks I think mess up the cadence too which might be affecting the IVTC.

[jagabo]

I was able to verify there are some missing frames in those areas. I didn't notice any duplicates nearby though. Which means the only way you get those missing frames back is to use a higher frame rate in TDecimate(). But then you'll be back to having duplicates in the earlier parts of the video.

[Selur]

How about TFM().TDecimate(mode=7, rate=25) some denoising and then something like FillDuplicateFrames/FillDrops ?