Deinterlacing anime with VFR and AnimeIVTC

[starplayer]

Hey guys.
I'm a bit ignorant on this, so please bear with me.
I'm trying to deinterlace an old anime with staxrip. It has a weird framerate - 28.2438fps (probably because it has cgi scenes).
I'm using AnimeIVTC, but it refuses, but it always gives me the message

"What is your input clip?? It's not 30fps NTSC nor 25 fps PAL... maybe you're using the function for a purpose it was not meant to...."

I though it was supposed to support VFR?

Any better ideas in how to deinterlace this?
Here is an excerpt of the file
https://mega.nz/file/lexgzIhA#s7MoUQytyvuug1Qw2n7-iP35SrV8Y0zgB-3K0DRSEKw

Many thanks!

[Selur]

The stream says it's 29.976 and it's from a DVD, so it's not VFR. -> drop the time codes
Content isn't soft telecined or progressive.
Looking at the content with same (or with double) frame rate deinterlacing you see that the actual frame rate is nowhere 29.976.
Even in the cgi scenes or only sometimes 23.976.
=> I would:
1. deinterlace (same rate qith QTGMC) reduce the frame rate&count to 23.976 (using sRestore) or use TIVTC in mode 7 with target 23.976fps and QTGMC as deinterlacer.
AnimeIVTC can probably do the same , if you remux the stream, drop the time code, set the frame rate to 29.97 and then feed it to AnimeIVTC.
2. either stick with 23.976fps cfr and live with the duplicate frames
or
a. replace duplicate frames with interpolated frames and create a more fluid 23.976fps content, with motion interpolation artifacts
b. drop the duplicates using DeDup and create a vfr stream.

Cu Selur

[starplayer]

Thank you for your input, that indeed seems to work great!
Regarding point 2., what's the most close to original? What is the pratical difference between leaving the duplicate frames or using Dedup with a vfr stream? Is there any advantage/disadvantage to either?

Edit: Unfortunately I've noticed that QTGMC seems to be introducing some heavy aliasing. Notice the dude's hairline, his nostril, or the left chair. Bottom image is the result of:
TFM().TDecimate(mode=7)
QTGMC(preset="Medium", InputType=0, sourceMatch=3, sharpness=0.2, tr2=2, ediThreads=8, fpsdivisor=2)


[Attachment 70224 - Click to enlarge]

[Attachment 70225 - Click to enlarge]

Here is the excerpt, at about 3 seconds in.
https://mega.nz/file/hbQmARiS#AbblPji71h9P7NhZFpB3QfmJF2IzLDfv-frZUhiEu2w

What's weird is that these artifacts don't even appear with VLC's default deinterlacer for example.

[jagabo]

The video does have a variable frame rate. Some frames are pulled down and encoded interlaced. Other frames use soft pulldown. Some frames have a duration of 1/30 second, some 1/20. Try a simple TFM().TDecimate(). The result is probably close enough and will avoid all the problems that QTGMC creates.

LWLibavVideoSource("Episode 1 - ''encounter'' (1).mkv", FPSNum=30000, fpsDen=1001) 
TFM()
TDecimate()

[Selur]

I meant something like:

A2 = last
A2 = A2.QTGMC(Preset="fast", ediThreads=2, FPSDivisor=2)
TFM(clip2 = A2)
TDecimate(mode=7)

but if TFM().TDecimate() already fixes the issue, thats fine.

[starplayer]

Thank you for your input guys, I really appreciate it.
I've tried both options, and combinations of them, and tweaking tfm parameters, but there's always some issues, visible in the panning shots:
First, some frames don't seem to be detected as interlaced, and so aren't cleaned up. Like this one:

[Attachment 70242 - Click to enlarge]

Second, and more important, running it that way seems to introduce jitter. In fact, frame by frame I can see (in the initial panning shots), that every few frames there is a jump in the movement, like if a frame has been skipped. If I run a simple yadif the motion is smooth, but everything looks like crap of course. If I leave out TDecimate though there is no jitter. Should I leave it off after all? What's the catch?

[poisondeathray]

Frame rate mode : Variable
Frame rate : 27.742 FPS
Original frame rate : 29.970 (30000/1001) FPS

It's a bad rip that messed up the timestamps (makemkv ?) . Or maybe from the way it was cut. The content might be mixed in sections, but the timestamps should always be CFR from DVD. The first part looks like 29.97p content , but most of the content is going to be 23.976p

No, that screenshot is more like it has been deinterlaced badly, instead of field matched. It's a false positive from TFM where it wrongly "sees" the horizontal lines in the picture as "combing" . If there aren't any ophaned fields, you can use TFM(pp=0) to prevent it from deinterlacing , or you can adjust the thresholds cthresh or mi to be more selective. On the scenechange TFM gets chroma artifacts, but it matches it correctly with TFM(pp=0, chroma=true)

The benefit of VFR is each section will play smoothly at it's proper frame rate, and a minor benefit is fewer frames encoded or more bitrate allocated to remaining frames at a given bitrate (higher quality) . But if you're watching on a 60Hz screen the 23.976p sections will still have "regular" judder . Another benefit of unique frames is if you're applying any temporal filtering, it will be more effective

[Selur]

Alternatively to TFM+TDecimate you could try QTGMC+sRestore.

[jagabo]

DVD is always CFR after pulldown. But before pulldown the frame rate for NTSC DVDs can be anywhere between 20 and 30 fps (I'm using rounded frame rates here) -- and that rate can change at any time. If you look at the timestamps from ffprobe you'll see a mix of 30 fps (interlaced after hard pulldown) frames and 20 fps soft (3:3) pulldown frames. The only way to match every frame change precisely is to encode at 60 fps, ie, DVD's field rate after pulldown.

[rrats]

Do not use MakeMKV for DVD ripping! Try ripping the same disc with mpeg2cut2

[Selur]

I don't really a problem using MakeMKV, you just need to be aware of what it does to the mkv container.

[starplayer]

Again, thank you very much for your input guys. Unfortunately I can't rerip the dvd, so I'm stuck with these mkvs.
After much experimentation, this seems to be giving the best results, setting chroma=true solved the aliasing issue in those frames indeed, according to TFM docs it should've be enabled by default, idk.

TFM(mode=3, slow=2, cthresh=9, MI=40, pp=5, chroma=true, mmsco = true, hint = true, display=true, ovr="c:\recuva\lau.txt",
clip2=QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1, FPSDivisor=2 ))

(although there is still some shimmering in the decombed frames with QTGMC


However, since we're not decimating (which in this case causes stutters) or similar, the framerate is staying variable right?. Is there any issue with keeping it this way, or should I force it to 60 fps? What's the best way to do that? Is AssumeFPS(60000, 1001) appropriate in this case?

[Selur]

although there is still some shimmering in the decombed frames with QTGMC

Throwing QTGMC in mode=2 or mode=3 at it afterward might help with the shimmering.

the framerate is staying variable right?

if you used:

LWLibavVideoSource("Episode 1 - ''encounter'' (1).mkv", FPSNum=30000, fpsDen=1001)

you forced the decoding to 30fps cfr.

[starplayer]

But won't forcing the decoding to 30fps cause judder in the 24 fps parts?
Is there an issue with opening as VFR and running TFM and nothing else, hence staying VFR? I didn't notice anything scandalous in this particular case, but maybe I'm not looking at the right places.

[Selur]

You only stay vfr if you add the time codes back, adding time codes back would require mean that the frame count does not change.
What does your script look like atm. ?

[starplayer]

I'm using staxrip, but currently I have these lines. These seem to be working ok tbh. I notice that with ffprobe, the reported framerate of the original file is 30000/1001, but after encoding in vfr with its timestamps it becomes 1000/33.
However, there doesn't seem to be any audio desynching or playback issues. The framerate is moot anyway as this is VFR. I believe I can't achieve anything better than this, I've spent too much time than I care to admit. I learned many new things though.

In case this is useful for someone, here's the code. I had to finetune the cthresh and the MI value. The MI is low to detect mouth interlacing that only happens in a small part of the frame. The cthresh is high because this animation likes to show some images of screens with scanlines (including the credits), which were incorrectly being detected as interlacing.

tcFile = "%temp_file%_timestamps.txt" # timestamps file path
Exist(tcFile) ? FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", timecodes=tcFile) : FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex")

Crop(%crop_left%, %crop_top%, -%crop_right%, -%crop_bottom%)

TFM(mode=5, slow=2, cthresh=30, MI=40, pp=2, chroma=true, mmsco = true, hint = true, ovr="c:\recuva\lau.txt",clip2=QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1, FPSDivisor=2 ))

Thank you very much for your help guys, specially you Selur, who stuck with me.
Best wishes

[Selur]

Okay,... a few questions, since I have no clue about StaxRip.
a. did you create the override file for TFM ? (no clue where StaxRip would get it from otherwise)
b. does StaxRip have an option to ignore time codes?
c. does StaxRip itself move the crop before the field handling? (personally, I usually handle the fields first and then crop&filter)

Also 'pp=2'

     PP -

        Sets the post-processing mode.  This controls how TFM should handle (or not handle)
        any combed frames that come out of the field matching process.  Possible options
        are:

            0 - nothing (don't even look for combed frames)
            1 - find/hint combed frames but don't deinterlace
            2 - dumb blend deinterlacing
            3 - dumb cubic interpolation deinterlacing
            4 - dumb modified-ela deinterlacing
            5 - motion-adaptive blend deinterlacing
            6 - motion-adaptive cubic interpolation deinterlacing
            7 - motion-adaptive modified-ela deinterlacing

seems like a bad choice since it might introduce blending,...

About the shimmering, try something like:

A2 = last
A2 = A2.QTGMC(Preset="fast", ediThreads=2, FPSDivisor=2, PrevGlobals="Replace")# added PrevGlobals="Replace" to handle multiple QTGMC or TemporalDegrain instances
TFM(chroma=true,MI=40,clip2 = A2)
TDecimate(mode=7,dupThresh=0.04,vidThresh=3.50,sceneThresh=15.00)
AssumeFrameBased()
# cropping
Crop(0,58,0,-58)# 720x364
# filtering
# denoising filtering using QTGMC
QTGMC(Preset="Medium", InputType=3, TR2=1, SourceMatch=0, Lossless=0, EZDenoise=5.00, NoisePreset="Fast", ediThreads=2, PrevGlobals="Replace")# added PrevGlobals="Replace" to handle multiple QTGMC or TemporalDegrain instances

the last QTGMC call should help with the shimmering, but you probably need to tweak the settings for best results.

Cu Selur

[starplayer]

a) Yes, I have created a ovr file. I'm using it for the end credits, which have a scanline effect on them, and previously it was being deinterlaced when it shouldn't be. Changing the cthresh and MI values has solved that issue (as there are other portions with that effect), so it's redudant, I'll remove it for the other episodes.
b) It does, in the container "box", in the container options, one can erase the timestamps path that staxrip automatically adds.
c) Staxrip by default comes with a few filters added but disabled by default. One is a crop filter, which is indeed before QTGMC, so I left it in that order. The crop I'm making is only on the black bars up and down, I'm not cutting any real image. If I was a crop after would probably be better yes.

Yeah, the pp=2 I believe is not used as I'm using clip2 using QTGMC as per your suggestion. I should remove it for clarity

The issue with TDecimate mode 7 is I'm getting 23.79 fps, which is not what this anime uses in the panning shots, and so it skips some frames.
The simmering was also solved by adjusting the MI and cthresh values. As this is anime, the wrong deinterlacing because of some panning shots on far buildings (hence sharp horizontal window lines) was causing the shimmering.
The only issue I still have is the ugly color bleeding on scene changes, on deinterlaced frames. Also the deformed lines, hence why I'm still tempted to use pp=2, compare the deinterlaced with the frame after, look at the keyboard right at the center

[Attachment 70308 - Click to enlarge]

[Attachment 70309 - Click to enlarge]

[Selur]

What does your script now look like? I don't get the chroma shift here.

[poisondeathray]

The only issue I still have is the ugly color bleeding on scene changes, on deinterlaced frames. Also the deformed lines, hence why I'm still tempted to use pp=2, compare the deinterlaced with the frame after, look at the keyboard right at the center

There is no bleeding in your sample on that scene change when you use all the fields to field match with TFM(chroma=true). It's being deinterlaced in your case because you're using FFVideoSource, but rffmode = 0 by default (ignore all flags), so there are no repeat fields . ie. instead of ignoring pulldown, you should output all 29.97 frames/sec or 59.94 fields/s to field match

ffms2

##### int rffmode = 0
Controls how RFF flags in the video stream are treated; in other words it's equivalent to the "field operation" mode switch in DVD2AVI/DGIndex.
Valid modes are:

- **0**: Ignore all flags (the default mode).
- **1**: Honor all pulldown flags.
- **2**: Equivalent to DVD2AVI's "force film" mode.

rffmode=1 would give you all the fields for TFM to field match, but FFVideoSource can be inconsistent with mpeg2/DVD sources in my experience. Certain versions of ffms2 are known to be very bad too. I would stick with DGSource or Mpeg2Source - the are much more consistent for DVD sources.

[jagabo]

Remux MKV to VOB:

ffmpeg -i %1 -vcodec copy -acodec copy "%~dpn1.vob"

Build Index from VOB with DGIndex GUI. Be sure to use Video -> Honor PulldownFlags mode. Or use command line:

DGIndex -i "%~dpnx1" -o "%~dpn1" -fo 0 -om 2 -exit

AviSynth script:

Mpeg2Source("Episode 1 - ''encounter'' (122).d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Episode 1 - ''encounter'' (122).d2v", field=0) 
TDecimate()

24p, smooth pans, no blending, no loss of small mouth movements, no aliasing of keyboard. Of course, other parts of the video could be different...

[lollo]

24p, smooth pans, no blending, no loss of small mouth movements, no aliasing of keyboard...

So, finally a simple TFM().TDecimate() in AviSynth was enough. No deinterlace, no staxrip (useless) GUI, nothing else

[jagabo]

24p, smooth pans, no blending, no loss of small mouth movements, no aliasing of keyboard...

So, finally a simple TFM().TDecimate() in AviSynth was enough. No deinterlace, no staxrip (useless) GUI, nothing else

Well, yes, and no. If you use "field=1" in the TFM() call the blending seen in post #18 is still there. There may be other transitions where field=0 doesn't work but field=1 does. So blindly using my script may not work for other parts of the series. Reducing the video to 24p may at times result in slight differences in timing of character motion (compared to 60p, derived from the source's 30i) -- if the OP cares about that. And the erroneous deinterlacing of progressive frames by TFM could still be an issue with other parts of the video. The OP knows how to deal with that though (via cthresh, MI, pp).

[lollo]

Sure, I was referring to the processed part, before your appropriate editing

I am not too familiar with anime and experienced myself how difficult is a proper restoration with non simple fields/frames architecture

[Selur]

As a side note:

LWLibavVideoSource("C:\Users\Selur\Desktop\EPISOD~1.MKV",cache=false,dr=true,format="YUV420P8", prefer_hw=0)
# current resolution: 720x480
# deinterlacing
TFM(scthresh=15.00,metric=1,chroma=true,MI=120)
TDecimate(mode=7,dupThresh=0.04,vidThresh=3.50,sceneThresh=15.00)
AssumeFrameBased()

seems to work fine even without the remuxing.