Cant get rid of dot crawl?

[killerteengohan]

I have a DVD with a fairly poor looking source.

It has alot of moving artifacts that are usually Dot Crawl and easily fixed with Checkmate() but I cant seem to get rid of it from this source.
Checkmate() makes it less visible but it doesn't get rid of it like usual and its blurring the details and image as well.

Here is 2 small unfiltered sample video's to look at. (Only resized to 640x480)

https://mega.co.nz/#!sw4nkYIJ!vI5tqHd5F5zHQHNdRDxTZPeFxtA-vZIYuX9BS_HU6U8

https://mega.co.nz/#!54hgUJZK!zxIR9uSkfMGXUDbYgpqa76V_2gokF4aAj8J_VRUlt9o



Im having a hard time getting this to look better. Any suggestions are welcome.

[ndjamena]

You could try QTGMC in progressive mode:

qtgmc(preset="slower", inputtype=2, TR2=3)

(the fade was never deinterlaced properly and even qtgmc doesn't fix it properly.)

[killerteengohan]

You could try QTGMC in progressive mode:

qtgmc(preset="slower", inputtype=2, TR2=3)

(the fade was never deinterlaced properly and even qtgmc doesn't fix it properly.)

Sadly it doesn't remove the moving artifacts resembling dot crawl.

[LMotlow]

Here is 2 small unfiltered sample video's to look at. (Only resized to 640x480)

And pretty much rendered it useless as a sample we could help you with.

Really weird.

[jagabo]

Here is 2 small unfiltered sample video's to look at. (Only resized to 640x480)

And pretty much rendered it useless as a sample we could help you with.

I was going to say that too. And don't deinterlace or IVTC.

[killerteengohan]

Here is 2 small unfiltered sample video's to look at. (Only resized to 640x480)

And pretty much rendered it useless as a sample we could help you with.

Really weird.

I don't see how it's useless. All that's different is it's 640x480 instead of 720x480. That hardly makes the moving artifacts I am talking about more or less visible in any way and should be just as easy to see what Im asking for help with. I've given several samples like that in the past and got more than adequate help.

If it's really a must, I guess I can get you the VOB file or something.

[killerteengohan]

Here is 2 small unfiltered sample video's to look at. (Only resized to 640x480)

And pretty much rendered it useless as a sample we could help you with.

I was going to say that too. And don't deinterlace or IVTC.

I didn't deinterlace or use IVTC in the samples, nor did I plan to in the final output.

I just wish to know of a way to remove those annoying moving dots all over certain areas of the video. You cant miss them in the second much smaller video.

Checkmate() did not remove them like it usually does.

[ndjamena]

QTGMC does get rid of most of the dot crawl in the second sample, it's only during the initial fade in that some of it is still visible, but that would be a failure in motion detection. No dot crawl filter handles moving scenes very well because there's nothing to latch onto that separates dot crawl from detail and the only way around that would be to effectively blur the entire screen.

[ndjamena]

This works:

InterLeave(SelectEven().BiLinearResize(last.width, last.height/2, 0, 0, 0, -1), SelectOdd().BiLinearResize(last.width, last.height/2, 0, 1, 0, 0))
AssumeFieldBased()
AssumeTFF()
Weave()
qtgmc(preset="slower", TR2=3)
LSFmod(maa2(ss=3.00), source=last, ss_x=1.0, ss_y=1.0, strength=100, overshoot=0, undershoot=0, soft=0, edgemode=0)

Although I doubt it's an improvement on CheckMate.

[killerteengohan]

I don't think using a deinterlacer on a source that doesn't really need to be deinterlaced is a good thing is it? My scan says its progressive or mostly film.

[ndjamena]

It's QTGMC, I was just using it to clean up the image a bit.


Courtesy of the "Similar Threads" list:
https://forum.videohelp.com/threads/370883-Dot-Crawl-Artifacts-from-Composite-Source?p=...=1#post2381080

Just downscaling and upscaling does the trick.

[ndjamena]

Actually, if you're willing to put the time into it, you may be better off just recreating the title card. The fade really wasn't deinterlaced properly and I can't see any way of removing the dot crawl from it.

Use these to recreate the fades at the beginning and end:
http://avisynth.nl/index.php/Fade

[jagabo]

Actually, if you're willing to put the time into it, you may be better off just recreating the title card. The fade really wasn't deinterlaced properly and I can't see any way of removing the dot crawl from it.

You can merge pairs of frames together. Something like this works pretty well:

Merge(SelectEven(),SelectOdd()) # merge pairs together
vInverse() # remove residual combing from bad IVTC
ConvertFPS(framerate*2) # new blended frames to restore the frame rate

Of course, you can't do this for the body of the video.

[jagabo]

This worked pretty well with the first video and didn't hurt the rest of the picture much:

#################################################
# Chroma edges
#################################################

function cEdges(clip c)
{
    // play around with the expand and inpand modes and edge threshold
    mt_edge(c).mt_expand(mode="vertical").mt_inpand(mode="horizontal").GreyScale()
}

#################################################

ffVideoSource("Sample 1.mp4") 

# Create a filtered version of the video with less dot crawl
filtered=Spline36Resize(width/2,height).Spline36Resize(width,height)

# Create a mask of chroma edges
cmask=Overlay(UtoY().cEdges(), VtoY().cEdges(), mode="add").BilinearResize(last.width,last.height)

# replace pixels of the source with the filtered version, only at chroma edges
Overlay(last, filtered, mask=cmask)

[killerteengohan]

It's QTGMC, I was just using it to clean up the image a bit.


Courtesy of the "Similar Threads" list:
https://forum.videohelp.com/threads/370883-Dot-Crawl-Artifacts-from-Composite-Source?p=...=1#post2381080

Just downscaling and upscaling does the trick.

Well QTGMC is a bob deinterlacer if I'm not mistaken.

I used it as you said and it did not get rid of it for me. Instead of being obvious moving dots, they were still there but now they faded in and out over and over instead when QTGMC was used.

So they werent as obvious because the movement stopped but they were still there and visibly fading in and out for me.


Perhaps I will see what the scaling manages to do to it.

Thanks!

[killerteengohan]

This worked pretty well with the first video and didn't hurt the rest of the picture much:

#################################################
# Chroma edges
#################################################

function cEdges(clip c)
{
    // play around with the expand and inpand modes and edge threshold
    mt_edge(c).mt_expand(mode="vertical").mt_inpand(mode="horizontal").GreyScale()
}

#################################################

ffVideoSource("Sample 1.mp4") 

# Create a filtered version of the video with less dot crawl
filtered=Spline36Resize(width/2,height).Spline36Resize(width,height)

# Create a mask of chroma edges
cmask=Overlay(UtoY().cEdges(), VtoY().cEdges(), mode="add").BilinearResize(last.width,last.height)

# replace pixels of the source with the filtered version, only at chroma edges
Overlay(last, filtered, mask=cmask)

I must be missing something I need in plugins, because I get a (there is no function named "cEdges") error when trying to use it.

I have the newest masktools, what could I be missing?

[jagabo]

cEdges() is right there at the top of the script.

[killerteengohan]

cEdges() is right there at the top of the script.

hmmm I couldn't get it to work for me even when in the script.

Ill try making that into a .avsi file and see if it works then be back in a few minutes.

[killerteengohan]

Well pasting this into an avsi file just crashes my program the second I try to open it.

#################################################
# Chroma edges
#################################################

function cEdges(clip c)
{
    // play around with the expand and inpand modes and edge threshold
    mt_edge(c).mt_expand(mode="vertical").mt_inpand(mode="horizontal").GreyScale()
}

#################################################

and if I try it in my script it says

Script error: Syntax error
((null), Line 7, column 5)


Im assuming I should take this line out of the script?

// play around with the expand and inpand modes and edge threshold

[jagabo]

Sorry, I added the comment when I posted and used C syntax, not AviSynth syntax. Change the // to #.

[killerteengohan]

Okay it works now. Wouldnt work in the script but it did when made into an AVSI file and that line corrected.

Seems to do a decent job without too much damage. I do notice quite the overall blur but it did reduce it a little better than checkmate. It seems to be pretty much checkmate with a bit more blur to cover it but also get rids of the hard ones that wouldn't reduce even with checkmate alone.

Thanks!

[killerteengohan]

Would using Bilinear or Lanczos resizers instead of Spline36 help with retaining details any?

Im not sure which is the most accurate downscaler and upscaler.

[jagabo]

It's nothing at all like checkmate. It has no temporal component. It blurs at chroma edges.

[ndjamena]

Ahhh, apparently I was supposed to kill the width, not the height. So I can't kill the dotcrawl and correct the deinterlacing at the same time then.

Is it just those two fades on the title cards that are created at +24fps or is there more post-processing in the full episodes?

[killerteengohan]

So whats the kill dot crawl part? That way i can just worry about the dot crawl. Ill worry about the deinterlacing myself if I choose too.

I personally didn't care about the interlacing or the fades of that title card. My main concern was to get rid of the things that look like dot crawl on the episode that chekmate wouldn't get rid of. The title card was merely to make it more visible to peoples eyes incase they couldnt see it in the episode.

As for those fades, I only noticed it at the title card and end credits. not really a bother to me.

[ndjamena]

You'd just need to add turnright after the source filter and turnleft at the end. QTGMC does provide temporal filtering, you could try replacing Jagabo's "filtered" variable with QTGMC's output but it may be too different from the rest of the picture for a masked overlay.

I don't know what exactly you're looking at to judge these things but it does seem a lot better to me.

TurnRight()
InterLeave(SelectEven().Spline36Resize(last.width, last.height/2, 0, 0, 0, -1), SelectOdd().Spline36Resize(last.width, last.height/2, 0, 1, 0, 0))
AssumeFieldBased()
AssumeTFF()
Weave()
QTGMC()
TurnLeft()

A straight Spline36Resize(width/2,height).Spline36Resize(width,height) doesn't seem to help the title card much.

[newpball]

I don't see how it's useless. All that's different is it's 640x480 instead of 720x480.

You are cutting the resolution before you do processing by more than 10%, why do that?

Always process first!

Any video player worth its salt can handle anamorphic video so the question is why resize anyway?

Alternatively you can scale to 720x540!

[jagabo]

Not only is he cutting the resolution he's performing an IVTC and re-encoding with a lossy codec. Each of those things makes dot crawl harder to remove. IVTC filters rely on the spacial and temporal properties of dot crawl artifacts. For example NTSC dot crawl alternate with every frame and repeats every other frame. If you IVTC you no longer have that pattern. And he (or someone else) did IVTC even though he claims not to have. How else did the samples become 23.976 fps?

[ndjamena]

I'm working on the assumption it was someone else, he's insisting it's progressive, even though it's agreed the fades are still interlaced...

[killerteengohan]

I'm working on the assumption it was someone else, he's insisting it's progressive, even though it's agreed the fades are still interlaced...

I'm not insisting it's progressive, that's just what the scan says and what MediaInfo says as well.