Seeking help on my journey to better understand avisynth

[krykmoon]

There's no point in doubling the frame rate with QTGMC() after decimation. If you want to use it for its other cleanup properties use InputType=2, or FPSDivisor=2, or SelectEven() after.

You are absolutely right: there's no need for that QTGMC() after decimation.. I missed it at all.

You always give very nice tips and suggestion.. jagaboo!

[davexnet]

What about simplifying the script and reducing it to

Mpeg2source("C:\Users\davex\Desktop\avs_test\digimonc.d2v",cpu2="ooooxx")
assumetff()
qtgmc()
tdecimate(cycle=5,cycler=3)

I could make a case that it's slightly cleaner.

[jagabo]

I like to use MPEG2Source() because of the de-ringing filter. Its deblocking filters are too strong -- they remove too much detail. But they only work in the 32 bit version.

[LighthouseonaCliff]

My sincere apologies to everyone here. My health went downhill, and I was in the hospital for a few days.

I really like your enthusiasm .. and exchanging infos and opinions is funny and useful since we always learn new things ..

Thank you for your kind words. Learning something you never thought possible is incredibly rewarding and fun

A few things...

ConvertToYV12() should be ConvertToYV12(interlaced=true) -- since the video is interlaced at this point. Without the interlaced argument it will blend the chroma of the two fields together. Fortunately, video is already YV12 so the filter isn't doing anything here.

Checkmate() works best while the video is still interlaced. So it should be used before TFM().

The Checkmate() settings are too high. It is causing ghosting of previous/next frames. Weaker settings will reduce that ghosting.

There's no point in doubling the frame rate with QTGMC() after decimation. If you want to use it for its other cleanup properties use InputType=2, or FPSDivisor=2, or SelectEven() after.

I learned a lot from this, thank you

I modified the script to:

a = LWlibavAudioSource("H:\Avisynth\Digimon Digital Monsters S01E01.mkv")
v = LWlibavVideoSource("H:\Avisynth\Digimon Digital Monsters S01E01.mkv")
AudioDub(v,a)

CheckMate(thr=10, max=20, tthr2=0) # tthr2 up to 10 might be ok, might produce more ghosting
TFM()
Santiag(strh=2, strv=2) # reduce the residual dot crawl
TDecimate(mode=2, rate=23.96)
#QTGMC( Preset="Slow", TR2=2 ).SelectEven()
aWarpSharp2(depth=5) # resharpen after Santiag()
Crop(10, 0, -10, -0) #Left, top, right, bottom.
#ChubbyRain2(th=2, radius=2, show=false, sft=2, interlaced=false)

Santiag() reduces some more of the residual dot crawl. QTGMC() helps with some problems (aliased lines, minor noise reduction), but messes up others (ghosting, blurring of small details). I decided not to use it. You might decide to keep it in since the things it fixes are more visible than most of the problems it causes. I didn't notice ChubbyRain2() helping much, and it caused some ghosting and chroma blending, so I removed it. Santiag() blurs some edges -- aWarpSharp2() was used to restore the sharpness.

I tried your modified script out and the result is stunning. Well done. Thank you.

What about simplifying the script and reducing it to

Mpeg2source("C:\Users\davex\Desktop\avs_test\digimonc.d2v",cpu2="ooooxx")
assumetff()
qtgmc()
tdecimate(cycle=5,cycler=3)

I could make a case that it's slightly cleaner.

Thank you for the suggestion. I've never used MPEG2Source before, same with assumetff. Creating a d2v file doesn't really want to work. It sticks at 33%. No idea why Will look into it further.

----------------------------------------------------------------------

I tried a test with batch processing with VirtualDub2. Seems to work perfectly if I create a script for every file and add them to the queue. Awesome!

[davexnet]

I always use Avidemux to re-wrap the MKV to an mpeg program stream, then DGIndex works without issues

[krykmoon]

Thank you for the suggestion. I've never used MPEG2Source before, same with assumetff. Creating a d2v file doesn't really want to work. It sticks at 33%. No idea why Will look into it further.

It's because DGIndex doesn't work well with the MKV container (MKV is not a container meant for MPEG encoded material).. so you have to put the file back into an mpeg compliant container before indexing it and create the d2v file..

I use this tool, tsMuxerR : it's easy to use, fast (since it doesn't re-encode anything, it only changes the container type) and with lot of options.

https://www.videohelp.com/software/tsMuxeR

Drag the MKV into tsMuxer, then choose either MUX to TS or to M2TS (they're pretty the same) and import the resulting Mpeg compliant file into DGIndex.. this time it will complete the process and you'll have your d2v file.


Yes.. learning new things is really very rewarding .. and exchanging experiences and informations with other people is a lot of fun, too!

[manono]

Creating a d2v file doesn't really want to work. It sticks at 33%. No idea why Will look into it further.

It's from a DVD originally, right? Then it never should have been turned into an MKV to begin with. That's why.

[LighthouseonaCliff]

Rewrapping the mkv worked like a charm. Also tried it with the DVD and that also worked well.

I remux the files for convenience. Just a personal preference.

The result was good, but the settings jagabo recommended do look better (to my amateur eyes ). There is still a lot of rainbowing with these settings.

Thank you for all the great help.

I wish everyone a great weekend

Yes.. learning new things is really very rewarding .. and exchanging experiences and informations with other people is a lot of fun, too!

It's hard finding patient and kind people who want to teach. I'm so glad and thankful that you all want to help.

[LighthouseonaCliff]

I've been looking into what assumetff() means.

If I understand correctly, framed-based means progressive and is best for computer viewing, and field-based means interlaced which is best for TV viewing.

AssumeFrameBased = Progressive
AssumeFieldBased = Interlaced

Top field first and bottom first field relate to interlaced encoding. It means which field gets displayed first.

AssumeTFF = Top field first
AssumeBFF = Bottom field first

I have used Mediainfo and DGIndex, and they both show if the video is frame or field-based. They also both show which field is shown first.

Are these good ways to check, and am I understanding this correctly?


The tdecimate settings is confusing to me:

tdecimate(cycle=5,cycler=3)

I thought cycle=5 meant a cycle of 5 frames, while cycleR=3 meant that 3 frames out of those 5 will be dropped. Can someone please explain this to me further? I read the following page, but it's confusing me:

TIVTC/TDecimate

Thank you.

[davexnet]

I've been looking into what assumetff() means.

If I understand correctly, framed-based means progressive and is best for computer viewing, and field-based means interlaced which is best for TV viewing.

AssumeFrameBased = Progressive
AssumeFieldBased = Interlaced

Top field first and bottom first field relate to interlaced encoding. It means which field gets displayed first.

AssumeTFF = Top field first
AssumeBFF = Bottom field first

I have used Mediainfo and DGIndex, and they both show if the video is frame or field-based. They also both show which field is shown first.

Are these good ways to check, and am I understanding this correctly?


The tdecimate settings is confusing to me:

tdecimate(cycle=5,cycler=3)

I thought cycle=5 meant a cycle of 5 frames, while cycleR=3 meant that 3 frames out of those 5 will be dropped. Can someone please explain this to me further? I read the following page, but it's confusing me:

TIVTC/TDecimate

Thank you.

Your definition of cycle and cycler are correct. It's the same ratio as 60/36, drop 36 out of 60, leaving 24 (which we want)

Certain operations can be messed up if the field order as used by Avisynth is wrong. You can always check it by something like

AsssumeTFF()
SeparateFields()

and advance frame by frame in Virtualdub. If you see a forwards/backwards jumping, frame order is wrong

[LighthouseonaCliff]

Your definition of cycle and cycler are correct. It's the same ratio as 60/36, drop 36 out of 60, leaving 24 (which we want)

Certain operations can be messed up if the field order as used by Avisynth is wrong. You can always check it by something like

AsssumeTFF()
SeparateFields()

and advance frame by frame in Virtualdub. If you see a forwards/backwards jumping, frame order is wrong

Okay, I think I'm understanding correctly now. Thank you for explaining this to me. It's fascinating how all this works.

I wish you a wonderful day. Thanks again

[jagabo]

In case it's not clear: each frame of interlaced video contains two separate half-images, called fields. They are intended to be viewed separately and sequentially. The player, or in this case QTGMC(), has to know what order the two fields are supposed to be displayed. If you specify the wrong field order (or AviSynth assumes the wrong field order) you will get 2-steps-forward-1-step-backward motion. Ie, instead of fields being seen in the order 0,1,2,3,4,5... they will be seen as 1,0,3,2,5,4...

[LighthouseonaCliff]

Thank you for explaining it further. I'm going to experiment to see what that looks like. Am quite curious.

[avurgesser]

Stumbled here looking for any documented projects on the source material.

LighthouseonaCliff did you get through the series? How did you handle the hybrid material. All cel animation appears to be standard pull-down, but there are a lot of (sock footage) CG elements that run 30 fps progressive. I am trying to decide between the following:

• Bob (QTMGC) the pulldown and just rely on b-frames to reclaim efficency
• Bob the pulldown and drop duplicates (doesn't seem to have material advantages over the first option, QTMGC can introduce artifacts in progressive material, and a Bob + dedup pulldown does not restore an accurate 24 fps.)
• Manually cut hybrid material and reconstruct timestamp files with a script I wrote and remux after filtering. I've only done this for one fps transition an episode before so I expect this to be a PITA even if I know it will work)

Thanks for finding the dot crawl. Any other artifacts you needed to correct for?

krykmoon thanks for the tip on tsMuxer. Ive been using DVD Decryper for VOBs but still needed to also rip with MakeMKV for good quality VOB/Subs. If I can just remux the mkv into an indexable container that will greatly improve my ripping workflow!

[hello_hello]

avurgesser,
I've read none of this thread, but TIVTC has a variable frame rate mode. It creates a timecodes file and the output is effectively a combination of 23.976 and 29.97 frame rates. TFM and TDecimate have lots of options, so check the TIVTC help files.

It takes 2 passes. The first pass is generally pretty fast because you'd disable any filtering that'd normally follow TFM/TDecimate. Comment out/uncomment the appropriate lines.

# 1st pass

TFM(Output="tfm.txt")
TDecimate(Mode=4, Hybrid=2, Output="tdec.txt")

# 2nd/encoding pass

# TFM(Input="tfm.txt")
# TDecimate(Mode=5, Hybrid=2, Input="tdec.txt", TFMIn="tfm.txt, mkvOut="timecodes.txt")

When you have lots of encodes to do, below is a function that makes it easier by automatically creating and naming the metrics and timecodes files, based on the name of each script. You'd use it like this:

# 1st pass

TFM(Output=FTFM())
TDecimate(Mode=4, Hybrid=2, Output=FTDec())

# 2nd/encoding pass

# TFM(Input=FTFM())
# TDecimate(Mode=5, Hybrid=2, Input=FTDec(), TFMIn=FTFM(), mkvOut=FTC())

# ===============================================================================
#            Create metrics & timecodes files in the script folder
# ===============================================================================

function FTFM()  {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TFM.txt" }
catch(err){ FileName = "D:\_TFM.txt" }

return FileName  }

# -------------------------------------------------------------------------------

function FTDec()  {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TDecimate.txt" }
catch(err){ FileName = "D:\_TDecimate.txt" }

return FileName  }

# -------------------------------------------------------------------------------

function FTC()  {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TIMES.txt" }
catch(err){ FileName = "D:\_TIMES.txt" }

return FileName  }

# ===============================================================================

If you'd prefer to do any de-interlacing with QTGMC (you might be able to get away with TFM's de-interlacing for the 1st pass for a speedup as it only creates the metrics files).

A = last
DeintClip = A.QTGMC(FPSDivisor=2)

# 1st pass

TFM(Output=FTFM(), clip2=DeintClip)
TDecimate(Mode=4, Hybrid=2, Output=FTDec())

# 2nd/encoding pass

# TFM(Input=FTFM(), clip2=DeintClip)
# TDecimate(Mode=5, Hybrid=2, Input=FTDec(), TFMIn=FTFM(), mkvOut=FTC())

[avurgesser]

Thanks hello_hello. Further review of the source after my post made me realize that the source is not a standard 24/30 hybrid source but something different I suspect the animation is actually sub-24 fps. Regardless the most obvious path to me now is as you suggested a two pass IVTC.

Not sure I understand the advantage of generating your metric filenames by function though. I settled for algorithmic name generation based on input file and try/catch to control which pass is run:

template:

src_directory = LeftStr(d2v_src_path, 1 + StrLen(d2v_src_path) - FindStr( RevStr(d2v_src_path), "\") )
match_path = src_directory + "matches.txt"
metrics_path = src_directory + "metrics.txt"
timestamp_path = src_directory + "mkv-timecodesfile.txt"

try { # second pass
MPEG2Source(d2v_src_path, CPU2="0000xx", Info=3) #use dering, but defer deblocking
checkmate(20)
Bifrost(interlaced=true)
tfm(d2v=d2v_src_path,input=match_path)
tdecimate(mode=5,hybrid=2,vfrDec=0,input=metrics_path,tfmIn=match_path,mkvOut="mkv-timecodesfile.txt")
Deblock_QED()
daa3().daa3().daa3()
FFT3DFILTER(sigma=2, plane=4, bw=32, bh=32, ow=16, oh=16, bt=3, ncpu=3, dehalo=0, hr=1, sharpen=0.0)
HaloBuster()
flash3kyuu_deband()
a = last
aWarpSharp2(depth=4, blur=3, type=1)
b = last
Crop(8, 0, -8, -0)
#subtitle("Pass 2 (of 2)", x=16, y=8, size=24)
}
catch (x) { # first pass
mpeg2source(d2v_src_path, CPU2="xxxxxx", Info=3)
checkmate(20)
Bifrost(interlaced=true)
tfm(d2v=d2v_src_path , output=match_path)
tdecimate(mode=4, output=metrics_path)
subtitle("Pass 1 (of 2)", x=16, y=8, size=24)
}

episode script (I use a quick script to generate these for a serries/season):

d2v_src_path = "B:\Rip\Digimon - Digital Monsters\S01E01\VTS_01_1.d2v"
Import("B:\Rip\Digimon - Digital Monsters\template (Digimon).avs")

[hello_hello]

The try/catch idea is one I hadn't thought of. It'd save a bit of commenting and uncommenting of lines in scripts.

Creating file names with functions and doing it the way you do it, are really just different ways of achieving the same thing.
I created functions to add the files to the same directory as the script (they don't even look at file paths), but as they rely on script names for the file names, they use a try/catch to allow specifying a default location and/or file name when a script doesn't actually exist (such as when you create a script with a GUI and preview it before it's saved somewhere). They also handle script names without an avs extension (when you create a script from scratch in AvsPmod it has a name without an extension until it's saved, so the files are still created and end up in the same folder as AvsPmod instead of annoying you with an error).

I went with a different naming scheme so a given location could contain multiple scripts without their corresponding files bothering each other, because I tend to work that way a bit.

SomeScript.avs
SomeScript_TFM.txt
SomeScript_TDecimate.txt
SomeScript_TIMES.txt

AnotherScript.avs
AnotherScript_TFM.txt
AnotherScript_TDecimate.txt
AnotherScript_TIMES.txt

I couldn't use Import to import a script for every encode. I'm too OCD about clean cropping and resizing to matching dimensions, so I'd have to check each one manually anyway. Other than that, it's tackling the same thing from different angles. I do like the try/catch 2nd/1st pass idea though.

Edit: Thinking about it, there's probably no reason why the timecodes file needs a unique name, as long as I don't run more than one script in a folder simultaneously. It's re-created each time a script is open, so having the same name each time means I could use the same x264 command line each time without having worry about the name of the timecodes file. Maybe I'll change that....