Deinterlacing from a TS

[naoto89]

There's an anime I want to encode from TS captures, but I'm having a problem with frames. I want to clarify that I'm not good at identifying patterns.

For some reason, decimate to 23.976 fps results in lost frames that are only visible at 29.97 fps or doubled to 59.94 fps. I've been thinking that maybe it's VFR footage and not CFR, however, the old fansub encodes have the video at 23.976 fps. If the anime series had actually been VFR footage, fansubs would have encoded it as VFR or 120.000 fps.

On streaming platforms from Japan (as AMZN JP), the video for that series is 29.97 fps, but on NF from Hong Kong the video was encoded at 23.976 fps and consequently has skipped frames.

What is the correct Avisynth code for this case? Here are two (1, 2) video clips for your check.

[Selur]

normal IVTC seems to work fine,...

TFM()
TDecimate()

didn't really spot missing frames,...
You could also try

TFM()
TDecimate(mode=7, rate=23.976)

Cu Selur

Ps.: if you want to go vfr, look at http://avisynth.nl/index.php/VFR

[naoto89]

normal IVTC seems to work fine,...

TFM()
TDecimate()

Maybe I forgot to say or was not very clear when I said that a normal IVTC has not been enough and in fact, the normal IVTC is what causes frames to be skipped that are only seen at 29.97 fps. A few years ago I read from someone say that this usually happens because when the video is returned at 23.976 fps, the filter skips odd frames or something like that, and to avoid this you have to modify certain parameters.

[Selur]

did you try "TDecimate(mode=7, rate=23.976)" alternatively use QTGMC and sRestore or go vfr,...

[naoto89]

did you try "TDecimate(mode=7, rate=23.976)" alternatively use QTGMC and sRestore or go vfr,...

With TDecimate the result is the same regardless of the mode set, I don't know how to use QTGMC and I definitely want to avoid VFR, that's why I left 2 sample clips, so that someone with more knowledge can check it and say what exactly can be make.

[Selur]

Personally I would simply stick with normal TIVTC, but here's an example how to call QTGMC and sRestore within a portable Avisynth setup:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\DGDecode.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\MedianBlur2.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\grunt-x64.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\Average.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\TIVTC.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\nnedi3.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\mvtools2.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\masktools2.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\RgTools.dll")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\Srestore.avsi")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\Zs_RF_Shared.avsi")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\QTGMC.avsi")
# loading source: C:\Users\Selur\Desktop\IE OP.ts
# color sampling YV12@8, matrix: bt709, scantyp: top field first, luminance scale: limited
MPEG2Source(d2v="J:\tmp\ts_a7438651b176a13c6dd7bdc9c4a18bd1_853323747.d2v",fieldop=0)
# current resolution: 1440x1080
# deinterlacing
AssumeTFF()
QTGMC(Preset="Fast", ediThreads=2)
SelectEven()
# removing ghosting
srestore(frate=23.976,omode=6)
Preroll(Int(last.FrameRate()))
# Converting from 8 to 10bit for encoder
ConvertBits(10)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
#  output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last

Maybe there is someone else who will suggest something else,...

Cu Selur

[naoto89]

Personally I would simply stick with normal TIVTC, but here's an example how to call QTGMC

That's a lot of parameters to get to 23,976, and even using QTGMC, which is per se slow to process, it skips frames just like TDecimate does.

An example of frames being dropped when the video goes to 23.976 fps is in the OP clip halfway through, there's a scene where all characters zoom out and before change to the next scene, the last frame of that sequence doesn't appear unless you watch the video at 29.97 fps.

[poisondeathray]

An example of frames being dropped when the video goes to 23.976 fps is in the OP clip halfway through, there's a scene where all characters zoom out and before change to the next scene, the last frame of that sequence doesn't appear unless you watch the video at 29.97 fps.

It's there for me using normal TIVTC @ 1:01.436, or frame 1473

What source filter are you using ? Post your script

[jagabo]

The main animation is 24p (telecined to 30i) but the crossfades and title/subtitle fades were created at 30i (ie every field is different).

Oh, I only looked at the first clip, "IE OP.ts" And I used LWlibavVideoSource() as the source filter in AviSynth.

[naoto89]

It's there for me using normal TIVTC @ 1:01.436, or frame 1473

I wasn't referring to that scene. I'm talking about frame 4231, or 1:10.587 when the video is duplicated at 59.94 fps with QTGMC. When I use only TFM at 29.97 fps, the frame 2115, or 1:10.570 It shows the previous frame, not the last frame that only appears with QTGMC, the zoom out of that scene is noticeable until the last frame before changing to the next scene. When I use TDecimate at 23.976 fps, the frame 1691, or 1:10.529 It's the same frame that is seen when using only TFM.

What source filter are you using ? Post your script

This is the code I started with.

LWLibavVideoSource("EP.ts", cachefile="EP.ts.lwi")
#Trims
TFM()
TDecimate()
Spline36Resize(1280,720, 0,0,-0,-0)

[davexnet]

I saw those interlaced frames also. Here's an attempt rendered at 24 fps using Qtgmc() and Srestore()

[poisondeathray]

It's there for me using normal TIVTC @ 1:01.436, or frame 1473

I wasn't referring to that scene. I'm talking about frame 4231, or 1:10.587 when the video is duplicated at 59.94 fps with QTGMC. When I use only TFM at 29.97 fps, the frame 2115, or 1:10.570 It shows the previous frame, not the last frame that only appears with QTGMC, the zoom out of that scene is noticeable until the last frame before changing to the next scene. When I use TDecimate at 23.976 fps, the frame 1691, or 1:10.529 It's the same frame that is seen when using only TFM.

What source filter are you using ? Post your script

This is the code I started with.

LWLibavVideoSource("EP.ts", cachefile="EP.ts.lwi")
#Trims
TFM()
TDecimate()
Spline36Resize(1280,720, 0,0,-0,-0)

Are we looking at the same clip? "IE EP 1 clip.ts" ?

What is "EP.ts ?" Is that the same as "IE OP.ts"??

I'm assuming you're referring to IE OP.ts, there is an orphaned field when bobbed at 4231 - also the content is higher frame rate than 23.976. eg. there are things like fades and transitions as mentioned above by jagabo

ie. If you use 23.976 - you're going to drop real frames. It's impossible not to drop real frames at 23.976. That's why people use VFR , at least for the intro.

I doubt main episode has higher content framerate , but you have to check. For some types of animation, sometimes there are effects or overlays at a higher content framerate

[naoto89]

Are we looking at the same clip? "IE EP 1 clip.ts" ?

What is "EP.ts ?" Is that the same as "IE OP.ts"??

This is my AVS script with the full episode. The two clips I provided were cut with Smart Cutter, so in theory it should be the same.

I'm assuming you're referring to IE OP.ts, there is an orphaned field when bobbed at 4231 - also the content is higher frame rate than 23.976. eg. there are things like fades and transitions as mentioned above by jagabo

ie. If you use 23.976 - you're going to drop real frames. It's impossible not to drop real frames at 23.976. That's why people use VFR , at least for the intro.

I doubt main episode has higher content framerate , but you have to check. For some types of animation, sometimes there are effects or overlays at a higher content framerate

As I said in the first post, the frame that only appears when doubling at 59.94 fps, does appear in WEBdl at 29.97 fps from Japanese streaming platforms but I'm surprised that it doesn't appear when I use TFM. Why? And If I decide to go with VFR, how will it affect what I do with that encode in the future? For example, fansubs and other edits.

[davexnet]

Can you put up a screen shot showing this frame, also the time stamp where it appears

[poisondeathray]

As I said in the first post, the frame that only appears when doubling at 59.94 fps, does appear in WEBdl at 29.97 fps from Japanese streaming platforms but I'm surprised that it doesn't appear when I use TFM. Why? And If I decide to go with VFR, how will it affect what I do with that encode in the future? For example, fansubs and other edits.

Yes, I was looking at the main episode

bobbed frame 2115 for the intro doesn't appear because it's an orphaned field (it's a half frame, missing it's partner field) due to a bad interlaced edit, and that breaks the cadence. That frame will never get properly matched (it's missing 1/2 the data, so the quality will be worse compared to a full frame with both partner fields), it will only appear if deinterlaced (e.g. TFM using PP, or deinterlaced separately)

You can try fiddling with the settings (but that might break the matches in other scenes, because that intro was not edited correctly, and has variable cadences), or override it within tfm with ovr file

TFM(ovr="ovr.txt")

where ovr.txt is

2115 f 0 
2115 c

VFR is tricky to edit, difficult to do properly - but subs should work . Many anime groups use VFR with subs

Another option is to leave it at 59.94 - it's "easy" and you won't miss anything - but deinterlacing slightly degrades the quality compared to field matching on progressive content. Also you have many duplicates, and it's slightly less efficient in terms of encoding

If you decide to go ovr, or adjust TFM settings, what other fields are you missing or producing incorrectly? Are you going to double check everything field by field ? That intro has a high chance of other issues, because of the way it was produced

Pros/cons

[jagabo]

the frame that only appears when doubling at 59.94 fps, does appear in WEBdl at 29.97 fps from Japanese streaming platforms but I'm surprised that it doesn't appear when I use TFM. Why?

TFM() starts with one field then it looks at the field before and the field after for a match. My guess is it is starting with the field before the orphaned field -- so the orphaned field is being excluded. If you start with the other field you will get that orphaned field -- use the "field" setting in TFM -- either 0 or 1, I forget which is the default.

[poisondeathray]

Another similar example orphaned field is bobbed frame 4385, which TFM decides on a p-match at frame 2192, thus dropping

Also bobbed 4433, TFM 2216. Ctelecine gets that one too. All those are end of scene, orphaned fields

Ctelecine gets those using default settings (keeps combed frame, which can be post processed), but there might be other problems in other sections

LWLibavVideoSource("IE OP.ts")
clip2=bwdif(field=3)
CTELECINE
CPOSTPROCESSING(edeint2=clip2)

Maybe replace with Ctelecine . I haven't used it much so I haven't looked in detail at the potential problems . But it's newer than TIVTC. It's newer than what the old fansubbers used - they must have used different settings or overrides or something else (maybe other field matchers like decomb)

[naoto89]

Another similar example orphaned field is bobbed frame 4385, which TFM decides on a p-match at frame 2192, thus dropping

Also bobbed 4433, TFM 2216. Ctelecine gets that one too. All those are end of scene, orphaned fields

Ctelecine gets those using default settings (keeps combed frame, which can be post processed), but there might be other problems in other sections

LWLibavVideoSource("IE OP.ts")
clip2=bwdif(field=3)
CTELECINE
CPOSTPROCESSING(edeint2=clip2)

Maybe replace with Ctelecine . I haven't used it much so I haven't looked in detail at the potential problems . But it's newer than TIVTC. It's newer than what the old fansubbers used - they must have used different settings or overrides or something else (maybe other field matchers like decomb)

I assumed I didn't have that filter. I searched for the CTools package but the repository on Github disappeared and I only found a Web Archive saved from 2023. I guess since then, there have been no forks to continue the project. Unlike when I use TFM, with this other filter I have noticed some slowness when passing frames, but it becomes slower if I add TDecimate.

[poisondeathray]

The author looks inactive since late 2023 , and repository deleted . It might be archived on wayback machine

Here is the last version I have

I'd be careful because this is not tested as thoroughly as TIVTC. For example I noticed some of the matches chose a lower quality field with more blocking on other matches. There might be other issues

[naoto89]

I'd be careful because this is not tested as thoroughly as TIVTC. For example I noticed some of the matches chose a lower quality field with more blocking on other matches. There might be other issues

So, it's the only filter that can recover orphaned fields? I must say I don't know much about the advanced uses of the other filters either, but from what I see, everyone ignores orphan fields and it seems that the risk of having an encode with artifacts is high, that makes a bit sad.

[poisondeathray]

So, it's the only filter that can recover orphaned fields?

No, when you double rate deinterlace the orphaned field will become a frame

The problem is the source - it's not produced correctly

TFM can include orphaned fields, it's just that the settings for the current metric calculations cause it to use a p (match previous field) for the matching, hence the use of overrides . If you use display=true , you can see information on the metrics

You might be able to adjust some settings for TFM to automatically include those last fields - but that might produce problems in other sections, such as other dropped frames in the middle of a scene, jerkiness. You'd have to check and see which is preferrable

Of course, the best way is to go through it manually check everything and use overrides if needed to - but it's tedious. Possibly, some fan has already done this and posted the overrides, you'd have to check - especially fan forums

A proper source wouldn't have any of those issues

[jagabo]

LWLibavVideoSource("IE OP.ts") 
TFM(field=0, scthresh=100)

This gets those orphaned fields. It may miss some others though. The high scthresh may cause other problems though. Try lower values:

# bob using TFM()
function TFMBob(clip v, int "pp", int "cthresh", float "scthresh", int "MI")
{
    pp = default(pp, 6)
    cthresh=default(cthresh, 9)
    scthresh=default(scthresh, 12.0)
    MI = default(MI, 80)

    v.GetParity() ? Interleave(TFM(v, field=1, pp=pp, cthresh=cthresh, scthresh=scthresh, MI=MI), TFM(v, field=0, pp=pp, cthresh=cthresh, scthresh=scthresh, MI=MI)) : Interleave(TFM(v, field=0, pp=pp, cthresh=cthresh, scthresh=scthresh, MI=MI), TFM(v, field=1, pp=pp, cthresh=cthresh, scthresh=scthresh, MI=MI))
}


LWLibavVideoSource("IE OP.ts") 
TFMBob(scthresh=35.0) # adjust threshold as necessary
TDecimate(Cycle=10, CycleR=6)