Avisynth Filters Help Needed (DVD MPEG2 Arifacts + Cleanup)

[chinobino]

Hi all, I would like some input to cleanup some video I have ripped from my 16 DVD Transformers Boxset before I convert to MP4.

Here is an example clip (DVD MPEG-2, NTSC, 29.97 fps, 720x480, 4:3 DAR, Interlaced, 57 MB) which was originally sourced from 35mm film.

I had planned to de-interlace to 59.97 fps progressive (as I am never going to watch the videos on a CRT) and generally just clean up the image but on closer inspection the video has a lot of problems.

I have written over 70 avs scripts and I am not really getting anywhere.

There is a lot of visible compression artifacts/mosquito noise, DCT blocking, aliasing, dot crawl and haloing/shadowing.

If someone with some experience could have a look at the sample and tell me if they think the source can be 'improved' I would be very appreciative.

Here are some examples of the problems you can see in the clip;

Compression Noise @ frames 2342-2358

Mosquito Noise @ frames 748, 2704

Blocking @ frames 125-126, 591, 2664-2668

Aliasing @ frames 1484-1868

Dot-crawl 386-388 (red area on right wing) also on the 'Transformers' lettering @ frames (176-198)

Haloing/shadowing @ frames 316, 369, 384, 392, 394

I am using Avisynth+ 32-bit (2.6.0.5 Alpha) and previewing avs in MPC-HC 32-bit and VirtualDub 32-bit on Windows 8.1.

I was going to leave a sample script but I am unsure of my workflow, maybe like this?

Colour Balance?
Deblock
De-interlace
Anti-alias (Upscale + Super Sample -> Downsample?)
Denoise/Dehalo
Undot/Despot

[LMotlow]

Some video transfer and other problems aside:

Your video is progressive with pulldown (telecine) applied. Deinterlacing is not appropriate.
35mm film doesn't run at 29.97 or 59.94 fps without adding extra frames or telecine.
If telecine is remove from your sample, the video runs at 23.976fps progressive film speed, with duplicate frames.
If you remove all the duplicate frames the movie runs at somewhere between 10 and 15 fps (I didn't have time to go thru all that pruning process). Remember, this is anime, not real-world movies.

If you insist on deinterlace/de-telicine with every commercial movie you own or buy, you'll be working on your collection for a very long time.

[jagabo]

Too smooth?

[chinobino]

Wow, some fast replies!

@ LMotlow, some very useful info there thanks.

I had thought that it was unlikely for hand drawn animation from the 80's to have so many 'frames' - so the telecining makes much more sense.

I could not work that out for myself, even after stepping through each frame, but then I have zero experience at this so thanks for clarifying.

If you insist on deinterlace/de-telicine with every commercial movie you own or buy, you'll be working on your collection for a very long time.

Haha! This is my first large encoding project (but hopefully not my last).

The complete series (98 episodes) could take a while, but I am in no major hurry - if it takes a few months thats ok, I only need to do it once properly.

Too smooth?

You've already processed and encoded it - I am impressed!

Actually the flatness suits my taste, I am not too concerned about preserving film grain (and the random hair on the film).

You have done an excellent job, removing the compression artifacts/aliasing, duplicate frames, cropping and encoding - I was no where near getting this level of quality even at preview level...

Would you mind sharing your avs script and AVC encoding settings?

I should hopefully be able to replicate it and start processing the video - then I can move onto my area of expertise, the audio.

Thanks!

[poisondeathray]

Sort of a hijack, but does anyone have a high quality audio version of the original classic "transform" sound effect from this series ? All I could find was low quality versions

[jagabo]

Here's the script I used:

Mpeg2Source("01 More Than Meets The Eye (Part 1) Clip1.d2v", CPU2="xxxxxx", Info=3)  # deblock and dering
CheckMate(thr=10, max=20, tthr2=20) # reduces dot crawl, sometimes leave other artifacts, reduce tthr2 if so
TFM(d2v="01 More Than Meets The Eye (Part 1) Clip1.d2v")  # field match
TDecimate()  # remove 1 in 5 dup
Crop(6,0,-10,-0) # remove left/right black rec.601 border
vInverse() # remove some residual comb-like artifacts

ConvertToRGB() # a little white balance
RGBAdjust(r=0.96, b=1.08)
ConvertToYV12()

McTemporalDenoise(settings="medium") # noise reduction 
MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=15)) # sharpen luma and chroma

nnedi3_rpow2(2, cshift="Spline64Resize", fwidth=960, fheight=720) # upscale to 720p
aWarpSharp(depth=5) # sharpen
Sharpen(0.5) # sharpen more

BicubicResize(704,480) # downscale to SD

If you don't need it at SD resolution you can leave out the final resize. A FastLineDarken() might help a little.

I left deblocking up to Mpeg2Source(). It's responsible for some of the detail loss. You could try using DeBlock_QED() instead. It's tunable so you can get a better balance between deblocking and detail loss. I also fell back on MCTemporalDenoise() for noise reduction. That's also responsible for some of the detail loss. You can try some other noise reduction filters like TTempSmooth() or Deen() instead.

Regarding the frame rate, most character animation is draw at 12 fps (to cut the number of cells that need to be painted in half) but the panning shots will be at full 24 fps (because they just pan over a big background cell). So it's best to leave the animation at 24 fps unless you want to go to variable frame rates so save a little bitrate.

I just noticed that Checkmate created some nasty artifacts in the panning shot that starts around 56 seconds into the clip. I take it out. Or try another dot crawl remover.

[chinobino]

Sort of a hijack, but does anyone have a high quality audio version of the original classic "transform" sound effect from this series ? All I could find was low quality versions

Hi PDR, I have not looked at the audio much yet but here are 2 transform sounds (one ascending in pitch, the other descending) saved to wav.

They are probably not much better than what you have - the original audio track does not have many high frequencies above ~18 KHz and the analog tape noise is quite loud (in the 8-12 KHz range).

A FastLineDarken() might help a little.

I have applied FastLineDarkenMod with:

FastLineDarkenMOD(strength=100, luma_cap=191, threshold=4, thinning=0)

I hope 100 strength is not considered blasphemous. I think I need to do a quick encode to see how it looks.

I left deblocking up to Mpeg2Source(). It's responsible for some of the detail loss. You could try using DeBlock_QED() instead. It's tunable so you can get a better balance between deblocking and detail loss.

I have tried Deblock_QED_MT2Mod previously but I think your BlindPP setting looks better than anything I did with it.

I also fell back on MCTemporalDenoise() for noise reduction. That's also responsible for some of the detail loss. You can try some other noise reduction filters like TTempSmooth() or Deen() instead.

I have MCTD 'installed' and found medium to be the best in terms of processing time (high really slows things down).

I was experimenting with both Deen and TTempsmooth before I posted this thread and was toying with;

Deen("c3d",1,thrY=4,thrUV=6)

and

TTempSmooth(maxr=6)

but I think MCTD Medium is a bit stronger and flattens it more to my liking.

Regarding the frame rate, most character animation is draw at 12 fps (to cut the number of cells that need to be painted in half) but the panning shots will be at full 24 fps (because they just pan over a big background cell). So it's best to leave the animation at 24 fps unless you want to go to variable frame rates so save a little bitrate.

Completely agree.

I just noticed that Checkmate created some nasty artifacts in the panning shot that starts around 56 seconds into the clip. I take it out. Or try another dot crawl remover.

Yep, I can see what you mean about the artifacts (and the more I look the more dot crawl I find!)

I tried Decrawl (no go, memory access violation) and LUTDeCrawl but I couldn't work out which parameters to change.

The dot crawl is a frustrating beast isn't it?

I also tried frfun7 but it seemed to effect everything except the dot crawl.

After a little research I discovered,

Dot crawl is caused by chroma interference with luma

First you want to check which planes Y or (UV) are affected.

How can I determine this?

[Edit] Well I have had a bit more success with Tcomb after following the directions. I have ended up with;

TComb(mode=0, fthreshL=9, fthreshC=9, othreshL=6, othreshC=6, map=false, scthresh=12.0, debug=false, opt=4)

[jagabo]

I have applied FastLineDarkenMod with:

FastLineDarkenMOD(strength=100, luma_cap=191, threshold=4, thinning=0)

I hope 100 strength is not considered blasphemous. I think I need to do a quick encode to see how it looks.

That's on the dark side. I usually don't like to darken lines because the lighter lines don't get darkened and you end up with something that is obviously "processed".

I have had a bit more success with Tcomb after following the directions. I have ended up with;

TComb(mode=0, fthreshL=9, fthreshC=9, othreshL=6, othreshC=6, map=false, scthresh=12.0, debug=false, opt=4)

That leaves artifacts similar checkmate's in the shot mentioned earlier. Not as bad though. One surefire way of reducing dot crawl is to reduce the horizontal resolution by a factor of 2 and then restore it:

BicubicResize(width/2, height).BicubicResize(width,height)

With VHS that works pretty well because VHS doesn't have much horizontal resolution. But it will noticeably blur your video since it's reasonably sharp. You would need to use a mask to protect parts of the picture that don't have dot crawl.

[poisondeathray]

Sort of a hijack, but does anyone have a high quality audio version of the original classic "transform" sound effect from this series ? All I could find was low quality versions

Hi PDR, I have not looked at the audio much yet but here are 2 transform sounds (one ascending in pitch, the other descending) saved to wav.

They are probably not much better than what you have - the original audio track does not have many high frequencies above ~18 KHz and the analog tape noise is quite loud (in the 8-12 KHz range).

Thanks,

I guess I'll have to try and re-create it. I saw some instructions somewhere

Sorry for the hijack

[chinobino]

That's on the dark side. I usually don't like to darken lines because the lighter lines don't get darkened and you end up with something that is obviously "processed".

I agree, I have backed it off to 75 and I am still deciding if it should be less.

That leaves artifacts similar checkmate's in the shot mentioned earlier. Not as bad though. One surefire way of reducing dot crawl is to reduce the horizontal resolution by a factor of 2 and then restore it:

BicubicResize(width/2, height).BicubicResize(width,height)

With VHS that works pretty well because VHS doesn't have much horizontal resolution. But it will noticeably blur your video since it's reasonably sharp. You would need to use a mask to protect parts of the picture that don't have dot crawl.

Ooh masking! Too advanced for me I think.

So I would need to set a protected region around the majority of the dot crawl and try to just process the really bad parts? No idea how to do that - I only started using Avisynth 2 weeks ago and I'm still wrapping my head around all the terminology.

When I render to 480p I can't even see the dot crawl (using AVC @ Profile: High Level: 3.0, 2,000 kbps) - it's only when I upscale to 720p that I can see it (and the blocking).

As much as I like the idea of upscaling I guess I have to be realistic - if I can't get it to look better than the source then I am not being practical.

Thanks for all of your invaluable input thus far.

Thanks,

I guess I'll have to try and re-create it. I saw some instructions somewhere

Sorry for the hijack

You're welcome, hijack anytime .

[jagabo]

Ooh masking! Too advanced for me I think.

So I would need to set a protected region around the majority of the dot crawl and try to just process the really bad parts? No idea how to do that - I only started using Avisynth 2 weeks ago and I'm still wrapping my head around all the terminology.

I'll see if I can come up with an example over the next few days. Here's one I did a while back:

https://forum.videohelp.com/threads/368611-Edge-noise-when-TBC-is-active?p=2359187&view...=1#post2359187

If you search through the forums for for "dot crawl" you'll find more examples.

[chinobino]

I'll have a read and see if I can get the gist of it, thanks.

[jagabo]

Here's what I've come up with for Dot Crawl reduction on your sample:

#################################################
# absolute value of chroma after UtoY() or VtoY()
#################################################

function cABS(clip c)
{
    Overlay(c.ColorYUV(off_y=-128), c.Invert().ColorYUV(off_y=-127), mode="add")
}

#################################################


Mpeg2Source("01 More Than Meets The Eye (Part 1) Clip1.d2v", CPU2="xxxxxx", Info=3) 

ColorMatrix("rec.601->rec.709") # convert to rec.709 for HD upscale

#######################
# Dot crawl reduction
#######################

# we're working with interlaced video so separate the fields
SeparateFields()

# Create a filtered version of the video with less dot crawl
filtered=LanczosResize(width/2, height).LanczosResize(width,height)

# Create a mask of the more saturated colors
cmask=Overlay(UtoY().cABS(), VtoY().cABS(), mode="add").ColorYUV(gain_y=512).LanczosResize(last.width,last.height)

# replace pixels of the source with the filtered version, only were colors were saturated
Overlay(last, filtered, mask=cmask)

# Return(Interleave(filtered, cmask)) # un-comment this line to see the filtered field and mask

# Back to interlaced video
AssumeFieldBased()
AssumeTFF()
Weave()

##########################
# dot crawl reduction done
##########################

TFM(d2v="01 More Than Meets The Eye (Part 1) Clip1.d2v")  # field match
TDecimate() # remove 1 in 5 dups
Crop(6,0,-10,-0) # remove black borders leave rec.601 704x480 4:3 DAR frame
vInverse() # blur away any residual comb-like artifacts

# small white balance adjustment
ConvertToRGB()
RGBAdjust(r=0.96, b=1.08)
ConvertToYV12()

McTemporalDenoise(settings="medium") # denoise
#TemporalDegrain(sigma=4, SAD1=100, SAD2=75) # alternate denoise
MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=15)) # sharpen chroma more than luma

# if you want darker lines:
#FastLineDarkenMOD(strength=100, luma_cap=191, threshold=4, thinning=0)

# upscale to 960x720
nnedi3_rpow2(2, cshift="Spline64Resize", fwidth=960, fheight=720)
aWarpSharp(depth=5)
Sharpen(0.5)

I included some comments that may help you understand what's going on.

Dot crawl artifacts are most prominent where colors are highly saturated. In YUV video, Y contains the greyscale image, U and V contain the colors that are added or subtracted from that greyscale image to produce a full color image. U and V are normalized to 128 -- ie, a grey pixel has U and V values of 128. A color pixel will have U and V above or below 128. The more saturated the color the further U and V will be from 128. My cABS() function returns 0 (black) where the picture is grey non-zero where there is color. The more saturated the color the brighter the value.

cABS() is used to create a mask (of U and V) for Overaly(), replacing pixels of highly saturated color with pixels from the filtered version of the video. Where the mask is black the original pixel is used. Where it's white the filtered pixel is used. In between is a weighted average of the two.

I included an alternate noise reduction method using TemporalDegrain(). It leaves a little more detail but gives less noise reduction. Try it if you want -- remove MCTD and enable TD.

[chinobino]

Hi again, I used your earlier sample in the other thread and it *seemed* to remove most of the dot crawl.

LanczosResize(width/2, height).LanczosResize(width,height)

# build a mask of chroma edges
 SeparateFields()
 Uedges = UtoY().mt_edge(thY2=4)
 Vedges = VtoY().mt_edge(thY2=4)
 edges = Overlay(Uedges, Vedges, mode="add").BilinearResize(width,height).Weave()
 Weave()

# overlay the original video with the filtered video only where there were chroma edges
Overlay(last)

I struggled with the overlay as I wasn't sure what I was returning to with the 'Overlay(last)' command, but it still looked better than Tcomb.

I will definitely give the cABS function a go - thanks for spending your time on it, I appreciate it.

I was also hoping for an alternative to MCTD as it increases the processing time so much, so thanks again for the TemporalDegrain suggestion!

Skimming through the script i can see;

# Return(Interleave(filtered, cmask)) # un-comment this line to see the filtered field and mask

This kind of thing is extremely helpful to a noob as I will be able to see directly what the effect is.

I will need to study YUV and try to commit that info to memory.

Each component (Y,U,V) has 8 bits of info for each pixel - learn something new every day! lol

From Microsoft:

However, most YUV formats use fewer than 24 bits per pixel on average, because they contain fewer samples of U and V than of Y.

I'm still learning...

Going to give it all a go now - will report back once I have tested it all .

[Edit] There appears to be a script error on this line of code;

cmask=Overlay(UtoY().cABS(), VtoY().cABS(), mode="add").ColorYUV(gain_y=512).LanczosResize(last.width,last.height)

I will try to figure it out - it may take me a while though.

[Edit] Think I may have found the error:

ColorMatrix("rec.601->rec.709# convert to rec.709 for HD upscale

Should be;

ColorMatrix("rec.601->rec.709") # convert to rec.709 for HD upscale

What is going on!?!

I pasted the corrected code correctly and the forum is showing it differently than what I entered.

I just went to edit it and it comes up correctly when I click edit - how is that possible???

Without code tags;

ColorMatrix( "rec.601->rec.709" ) # convert to rec.709 for HD upscale

[LMotlow]

ColorMatrix("rec.601->rec.709") # convert to rec.709 for HD upscale

A slight error, Should be fully enclosed in quotes and closed parens:

ColorMatrix("Rec.601->Rec.709") # convert to rec.709 for HD upscale

I'm guessing it was a typo.

[jagabo]

Hi again, I used your earlier sample in the other thread and it *seemed* to remove most of the dot crawl.

LanczosResize(width/2, height).LanczosResize(width,height)

# build a mask of chroma edges
 SeparateFields()
 Uedges = UtoY().mt_edge(thY2=4)
 Vedges = VtoY().mt_edge(thY2=4)
 edges = Overlay(Uedges, Vedges, mode="add").BilinearResize(width,height).Weave()
 Weave()

# overlay the original video with the filtered video only where there were chroma edges
Overlay(last)

I struggled with the overlay as I wasn't sure what I was returning to with the 'Overlay(last)' command, but it still looked better than Tcomb.

The video in that post had dot crawl only at sharp chroma edges. So mt_edge() was used to locate those edges. The overly then only fixed the parts of the picture that corresponded to those edges.

I will definitely give the cABS function a go - thanks for spending your time on it, I appreciate it.

Your video has dot crawl not only at chroma edges but also in the middle of some small objects. So I used a mask that marked all high saturation areas for replacement.

I was also hoping for an alternative to MCTD as it increases the processing time so much, so thanks again for the TemporalDegrain suggestion!

TD isn't much faster! The settings I used were a lot more conservative than MCTD(medium). So there's less noise reduction and more twittering edges.

Skimming through the script i can see;

# Return(Interleave(filtered, cmask)) # un-comment this line to see the filtered field and mask

This kind of thing is extremely helpful to a noob as I will be able to see directly what the effect is.

I use that technique a lot. You can also use StackHorizontal() or StackVertical() to see the images side by side (top/bottom) instead. I also use Windows' built in screen magnifier to zoom in to see details.

I will need to study YUV and try to commit that info to memory.

Each component (Y,U,V) has 8 bits of info for each pixel - learn something new every day! lol

From Microsoft:

However, most YUV formats use fewer than 24 bits per pixel on average, because they contain fewer samples of U and V than of Y.

Your source is YV12. Each Y, U and V component is 8 bits, a range of 0 to 255 each. But with Y should be within the range of 16 (defined as full black) and 235 (defined as full white). U and V are limited to 16 to 240 (not all combinations of Y, U and V, even if all three fall withing their respective ranges, are valid. Also with YV12 the chroma channels (U and V) are half (each dimension) the resolution of the luma channel (Y). So your 720x480 video has a 720x480 luma channel and two 360x240 chroma channels. So each pixel of the 720x480 frame has 12 bits of YUV on average. You can view the separate channels with:

Y = GreyScale().Subtitle("Y")
U = UtoY().Subtitle("U")
V = VtoY().Subtitle("V")
StackHorizontal(Y, StackVertical(U, V))

Or all in one line:

StackHorizontal(GreyScale().Subtitle("Y"), StackVertical(UtoY().Subtitle("U"), VtoY().Subtitle("V")))

Or with some attempt to show the colors of the chroma channels:

Y = GreyScale().Subtitle("Y")
U = ColorYUV(cont_v=-256).BilinearResize(width/2,height/2).Subtitle("U") # remove V, leaving only Y and U
V =  ColorYUV(cont_u=-256).BilinearResize(width/2,height/2).Subtitle("V") # remove U, leaving only Y and V
StackHorizontal(Y, StackVertical(U, V))

[jagabo]

There appears to be a script error...

I added all the comments after pasting the script into the forum's edit box. So it's possible I screwed something up. But I just copied and pasted from the forum to a new script and got no errors.

[chinobino]

I am using AvsPmod 2.5.1 and it is showing python library errors and a memory access violation in Windows.

Traceback (most recent call last):
  File "avsp.pyo", line 9379, in OnMenuVideoToggle
  File "avsp.pyo", line 14234, in ShowVideoFrame
  File "avisynth.pyo", line 464, in GetFrame
WindowsError: exception: access violation reading 0x00000067

I have emptied out my plugins folder, added them back in as required and narrowed it down to nnedi3.dll.

I exchanged the 'Vit mod 2011' version for the 'Vit mod 2012' version and now I am getting;

nnedi3_rpow2: arg 0 must be a clip!

When I add the line back in for final resize;

nnedi3_rpow2(2, cshift="Spline64Resize", fwidth=960, fheight=720)

There must be a broken reference somewhere?

[Edit] If I comment out the nnedi3_rpow2 resize I get the python errors and Windows access violation error.

I changed SetMemoryMax(600) to SetMemoryMax(2048) and received the following errors;



Exception WindowsError: 'exception: access violation reading 0x74736C9E' in <bound method PClip.__del__ of <avisynth.PClip instance at 0x020F8468>> ignored
Traceback (most recent call last):
File "avsp.pyo", line 9379, in OnMenuVideoToggle
File "avsp.pyo", line 14234, in ShowVideoFrame
File "avisynth.pyo", line 464, in GetFrame
WindowsError: exception: access violation reading 0x00000001

Which version of Avisynth are you running btw?

[Edit] Ok figured it out, nnedi3.dll was the culprit - I had to download this version and now the script works.

[jagabo]

Do you have anything other than comments between MergeChroma() and nnedi3_rpow2() lines?

Are you using a multithreaded build of AviSynth? AviSynth 2.5 or 2.6? I'm using a multithreaded build of 2.6.

[chinobino]

Do you have anything other than comments between MergeChroma() and nnedi3_rpow2() lines?

Are you using a multithreaded build of AviSynth? AviSynth 2.5 or 2.6? I'm using a multithreaded build of 2.6.

There was nothing between MergeChroma() and nnedi3_rpow2() lines as I commented it all out.

I have managed to get the script to work by replacing nnedi3.dll.

I am using AviSynthPlus-r1576 from this page.

I found a problem with the script that I think I have fixed.

 ColorMatrix( mode="rec.601->rec.709" )

Produces black lines intermittently, like bad de-interlacing.

I changed the line to;

 ColorMatrix( mode="rec.601->rec.709", interlaced=true )

and now the black lines are gone .

I am just looking at TemporalDegrain now, thanks for the script!

I have also found that I have to put spaces near any curved brackets, otherwise the forum thinks I am trying to emote...

[jagabo]

 ColorMatrix( mode="rec.601->rec.709" )

Produces black lines intermittently, like bad de-interlacing.

I changed the line to;

 ColorMatrix( mode="rec.601->rec.709", interlaced=true )

and now the black lines are gone .

Sorry. I originally had that line near the bottom of the script (where the video was progressive). I moved it up near the top after pasting it into the post here. I forgot to add the interlace option.

[chinobino]

Well all goes well - encoding the first full episode at HD 960x720 (using AVC1 High Profile, Level 3.1, Faster Preset @ 2700 kbps), I am halfway through and VirtualDub is telling me it should be about 3 hours encoding time and approx. 450 MB without audio.

I might try encoding on a slower preset for a slightly higher quality and see if I can run multiple encodes concurrently on my i7 3770k, but I'll have to make sure I set the pagefile big enough to accommodate...

Anyone tried running 8 encodes at once with an Intel i7 with HT?

[chinobino]

Onto the final encode now, using the 'Placebo' preset in VirtualDub now and still maintaining 3-6 fps, so it will take ~2-5 hours per episode, depending the amount of fast action scenes I guess.

I put 50 videos into VirtualDub's job queue and 'Background I/O' keeps enabling itself after each video - is there a way to permanently disable it?

I would like to thank both jagabo and LMotlow for their help, the end result has far exceeded my expectations and I have learnt a lot.

Here is the final script for anyone that is interested;

#################################################
# absolute value of chroma after UtoY() or VtoY()
#################################################

function cABS(clip c)
{
    Overlay(c.ColorYUV (off_y=-128),c.Invert().ColorYUV(off_y=-127), mode="add")
}

#################################################

SetMTMode(5)
Mpeg2Source("01 More Than Meets The Eye (Part 1).d2v", CPU2="xxxxxx", Info=3)

ColorMatrix(mode="rec.601->rec.709", interlaced=true) # convert to rec.709 for HD upscale

threads=8
SetMTMode(2,threads)

#######################
# Dot crawl reduction
#######################

# we're working with interlaced video so separate the fields
SeparateFields()

# Create a filtered version of the video with less dot crawl
filtered=LanczosResize(width/2, height).LanczosResize(width,height)

# Create a mask of the more saturated colors
cmask=Overlay(UtoY().cABS(), VtoY().cABS(), mode="add").ColorYUV(gain_y=512).LanczosResize(last.width,last.height)

# replace pixels of the source with the filtered version, only were colors were saturated
Overlay(last, filtered, mask=cmask)

# Return(Interleave(filtered, cmask)) # un-comment this line to see the filtered field and mask

# Back to interlaced video
AssumeFieldBased()
AssumeTFF()
Weave()

##########################
# dot crawl reduction done
##########################

TFM(d2v="01 More Than Meets The Eye (Part 1).d2v")  # field match
TDecimate() # remove 1 in 5 dups
Crop(8,0,-8,-0) # remove black borders leave rec.601 704x480 4:3 DAR frame
# vInverse() # blur away any residual comb-like artifacts

# small white balance adjustment
ConvertToRGB()
RGBAdjust(r=0.96, b=1.08)
ConvertToYV12()

MCTemporalDenoise(settings="high") # denoise
MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=15)) # sharpen chroma more than luma

# darker lines:
FastLineDarkenMOD(strength=24, luma_cap=191, threshold=4, thinning=0)

# upscale to 960x720
nnedi3_rpow2(2 , cshift="Spline64Resize", fwidth=960, fheight=720)
aWarpSharp(depth=5)
Sharpen(0.5)

[jagabo]

You're probably losing a fair amount of detail with MCTD(high). I played around with deblock_qed() (interlaced version) instead of Mpeg2Source() and that also preserved a little detail. Of course, with more detail comes more noise so you probably won't like it. Watch the lines on the yellow background at ~6s into the sample -- some of them disappear with heavy filtering (you don't really notice in realtime playback though). Some small details in the terrain 14s to 17s into the video too.

I also played around with upscaling to 1080p but didn't think the results were worth the added processing and file size.

x264 placebo is probably overkill. Slow or veryslow is probably enough.

[chinobino]

Hi jagabo, how are you today?

I originally thought MCTD 'High' was overkill but 'Medium' doesn't remove the MPEG2 blocking (and the stylised/processed flat image appeals to me).

I messed around with deblock_qed() but it didn't seem to be capable of removing as much as Mpeg2Source() set to 'xxxxxx'.

I think my biggest gripe is the macroblocking. I really despise MPEG2 (and all of its shortcomings) - I actually stopped buying DVD's because they look so blurry and blocky compared to Blu Ray.

If I could have bought the Transformers Boxset on Blu Ray I would have (and I probably wouldn't have felt compelled to re-encode it).

I know I am removing a reasonable amount of fine detail by blurring everything but as long as I am not being constantly distracted by artifacts then I am happy.

I also messed around with 1080p and found the same as you - massive bit rate/size file required for not much (if any) visual gain.

I tested the script in AVS-Monitor and was averaging 4-7 fps as a result of the heavy processing from MCTD (High).

Setting the x264 encoding to placebo does not slow the encode at all, so I figured 'why not?'.

I am editing the audio now while VirtulDub encodes in the background.

VirtualDub can encode 7-8 episodes every 24 hours so I can do all 98 episodes in about 12 days, not bad for 100 GB of MPEG-2 video.

It should only take me a couple of hours to remux it all with chapters and then I can move onto my 6 DVD Japanese Transformers Headmasters Boxset (project 2)...

[chinobino]

You're probably losing a fair amount of detail with MCTD(high).

After encoding 20 episodes with the same script I came to the same conclusion, after analysing each episode I could see that it was overprocessed.

I had to make a custom "Medium-High" configuration for MCTD in order to retain some more detail.

MCTemporalDenoise(settings="High",radius=1,sigma=40,twopass=true,limit2=0,strength=200,Tovershoot=0,AAthr=16,method=0,useQED=false,ECrad=4,ECthr=16,ECmode="Removegrain(4,4)",stabilize=true,TTstr=2,GFthr=1.6,AGstr=1.8,thSAD=450,thSAD2=450,thSCD1=450,thscd2=115,MVsharp=1)

I played around with deblock_qed() (interlaced version) instead of Mpeg2Source() and that also preserved a little detail. Of course, with more detail comes more noise so you probably won't like it. Watch the lines on the yellow background at ~6s into the sample -- some of them disappear with heavy filtering (you don't really notice in realtime playback though). Some small details in the terrain 14s to 17s into the video too.

I disabled deblocking altogether as I discovered MCTD was filtering it out anyway, which means a lot more of the fine detail was retained.

x264 placebo is probably overkill. Slow or veryslow is probably enough.

I have changed x264 to very slow and it does decrease the encoding time significantly now that I am not deblocking.

One other major change I made was to stop some chroma bleeding for oversaturated reds;

FixChromaBleedingMod (last,0,3.0,0.8,true,1)

I only discovered the problem after watching several episodes that had more colour saturation than the sample I gave you.

Thanks again for all your excellent advice, here is the finalised script;

#################################################
# absolute value of chroma after UtoY() or VtoY()
#################################################

function cABS(clip c)
{
    Overlay(c.ColorYUV(off_y=-128), c.Invert().ColorYUV(off_y=-127), mode="add")
}

#################################################

SetMTMode(5)
Mpeg2Source("01 More Than Meets The Eye (Part 1).d2v")

ColorMatrix(mode="rec.601->rec.709", interlaced=true) # convert to rec.709 for HD upscale

threads=8
SetMTMode(2,threads)

#######################
# Dot crawl reduction
#######################

# we're working with interlaced video so separate the fields
SeparateFields()

# Create a filtered version of the video with less dot crawl
filtered=LanczosResize(width/2, height).LanczosResize(width,height)

# Create a mask of the more saturated colors
cmask=Overlay(UtoY().cABS(), VtoY().cABS(), mode="add").ColorYUV(gain_y=512).LanczosResize(last.width,last.height)

# replace pixels of the source with the filtered version, only were colors were saturated
Overlay(last, filtered, mask=cmask)

# Return(Interleave(filtered, cmask)) # un-comment this line to see the filtered field and mask

# Back to interlaced video
AssumeFieldBased()
AssumeTFF()
Weave()

##########################
# dot crawl reduction done
##########################

FixChromaBleedingMod (last,0,3.0,0.8,true,1)

TFM(d2v="01 More Than Meets The Eye (Part 1).d2v")  # field match
vInverse() # blur away any residual comb-like artifacts
TDecimate() # remove 1 in 5 dups
Crop(8,0,-8,-0) # remove black borders leave rec.601 704x480 4:3 DAR frame

# small white balance adjustment
ConvertToRGB()
RGBAdjust(r=0.96, b=1.08)
ConvertToYV12()

MCTemporalDenoise(settings="High",radius=1,sigma=40,twopass=true,limit2=0,strength=200,Tovershoot=0,AAthr=16,method=0,useQED=false,ECrad=4,ECthr=16,ECmode="Removegrain(4,4)",stabilize=true,TTstr=2,GFthr=1.6,AGstr=1.8,thSAD=450,thSAD2=450,thSCD1=450,thscd2=115,MVsharp=1)
MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=15)) # sharpen chroma more than luma

# if you want darker lines:
FastLineDarkenMOD(strength=24, luma_cap=191, threshold=4, thinning=0)

# upscale to 960x720
nnedi3_rpow2(2 , cshift="Spline64Resize", fwidth=960, fheight=720)
aWarpSharp(depth=5)
Sharpen(0.5)

[jagabo]

It looks like you have things well in hand.

MCTD() has an explicit deblocking function if you enable it. Deblocking after TFM().TDecimate() isn't as effective. MCTD also has support for interlaced video. I've never used that so I don't know how well it works. Here's what i did with Deblock_QED before IVTC:

Deblock_QED_i(quant1=36, quant2=52)

I didn't play with the numbers much (just the defaults, which left some blocks, and the above values -- so you may get better results with different numbers.

Here's the Deblock_QED_i function (Deblock_QED() for interlaced video):

# Deblock_QED for interlaced video

function Deblock_QED_i ( clip clp, int "quant1", int "quant2", 
 \                     int "aOff1", int "bOff1", int "aOff2", int "bOff2", int "uv" )
{

quant1 = default( quant1, 24 ) # Strength of block edge deblocking
quant2 = default( quant2, 26 ) # Strength of block internal deblocking

aOff1 = default( aOff1, 1 ) # halfway "sensitivity" and halfway a strength modifier for borders
aOff2 = default( aOff2, 1 ) # halfway "sensitivity" and halfway a strength modifier for block interiors
bOff1 = default( bOff1, 2 ) # "sensitivity to detect blocking" for borders
bOff2 = default( bOff2, 2 ) # "sensitivity to detect blocking" for block interiors

uv    = default( uv, 3 )    # u=3 -> use proposed method for chroma deblocking
                            # u=2 -> no chroma deblocking at all (fastest method)
                            # u=1|-1 -> directly use chroma debl. from the normal|strong deblock()

last=clp
par=getparity()
SeparateFields().PointResize(width,height)
Deblock_QED(last, quant1, quant2, aOff1, aOff2, bOff1, bOff2, uv)
AssumeFrameBased()
SeparateFields()
Merge(SelectEven(),SelectOdd())
par ? AssumeTFF() : AssumeBFF()
Weave() 
}

I also tried removing the film bounce after IVTC and MCTD, before upscaling:

Stab(dxmax=2, dymax=2)
Crop(2,2,-2,-2) # remove borders created by stab

In addition to removing the film bounce it reduced some of the twittering edges caused by aWarpSharp() sharpening edges differently in different frames. But I thought it might make some of the panning shots less smooth so I didn't mention it before.

[chinobino]

It looks like you have things well in hand.

MCTD() has an explicit deblocking function if you enable it. Deblocking after TFM().TDecimate() isn't as effective. MCTD also has support for interlaced video. I've never used that so I don't know how well it works. Here's what i did with Deblock_QED before IVTC:

Deblock_QED_i(quant1=36, quant2=52)

I didn't play with the numbers much (just the defaults, which left some blocks, and the above values -- so you may get better results with different numbers.

Here's the Deblock_QED_i function (Deblock_QED() for interlaced video): [snip]

I messed around with the variables and it seems that your settings do a pretty nice job on the occasional frames that are severely blocked, which leaves me in a bit of a predicament...

How can I quickly add the function into the 98 scripts I have made for each episode? Is there an automated way to do this?

Adding it in manually will take me a long time (as well as being mind numbingly dull and tedious).

I also tried removing the film bounce after IVTC and MCTD, before upscaling:

Stab(dxmax=2, dymax=2)
Crop(2,2,-2,-2) # remove borders created by stab

In addition to removing the film bounce it reduced some of the twittering edges caused by aWarpSharp() sharpening edges differently in different frames. But I thought it might make some of the panning shots less smooth so I didn't mention it before.

Doing some testing with stab now, thanks.

[Edit] Also, I changed my crop to be equal on left and right i.e. Crop(8,0,-8,-0), which cuts off the waviness on the left side.

Only the introduction video has more black on the right edge, compared to each actual episode which has fairly equal black borders on the left and right.

I suspect they took the intro from a specific 35mm reel and used it for all of the season 1 episodes.

[jagabo]

How can I quickly add the function into the 98 scripts I have made for each episode? Is there an automated way to do this?

Maybe you can find a text editor with batch processing and macros. If all your scripts are the same except the filename you can write a batch file that inserts the filename into a prototype script.

[chinobino]

Hi jagabo, I ended up not bothering with the deblocking function.

I can use batch files to edit parameters already in the script but I can't easily add new lines in where I want them.

I have been encoding to x264 and noticed that flat dark areas with a slight colour gradient are quite blocky.

The only way I could reduce the blocking was to increase the data rate by a lot, which made the file size larger than the original files.

I tried using 10-bit x264 encoding to help with the rounding errors but in the end found that encoding to 10-bit x265 HEVC looks considerably better.

The x265 codec doesn't seem to retain as much 'texture' detail but I can live with some flattening as long as I am not introducing artifacts.

With x265 I can use CRF 22 with 10 reference frames and get each episode down to ~550 MB, which is approximately half of the original file size and is pretty much transparent.

Now I just need to run all avs projects as a batch, using very fast each episode takes 1 hour - so ~96 hours of encoding and I should be finished...

Cheers for all your help!