Hideous dot crawl

[Mephesto]

https://forum.videohelp.com/attachment.php?attachmentid=10248&stc=1&d=1324931972

I have tried all dot crawl filters, which either did nothing or badly smeared the rest of the picture when it did. It looks to me they weren't designed to fix porey dot crawl as large as what you see in the screenshot. Any ideas?

Video: http://www.sendspace.com/file/jhaeu0

[edDV]

Better to cap S-Video or composite with a good 2D/3D comb filter. Difficult to fix after A/D.

[vhelp]

i see no dot-crawl in that image. maybe you mean alias around edges. other than that the image looks clean to me.

[Mephesto]

i see no dot-crawl in that image. maybe you mean alias around edges. other than that the image looks clean to me.

Yeah, except that alias... crawls. Check the video out and see. Focus on the kid with the blue toque.

edDV, I am reluctant to process the interlaced video as that usually makes it harder to deinterlace after. And I don't dig your analog jargon, I only have the DVD, this video was not captured from the TV.

[aedipuss]

you're trying to fix a studio dvd? you should have the original mpeg-2 if you used something like vob2mpg.

[jagabo]

There are too many generations of crap in the video. I suspect the original cap was done from an analog source via composite video. Then it went through a poor NTSC to PAL conversion. Normal dot crawl filters will not work because the dots have been messed up. I hope that's not a commercial DVD.

[Mephesto]

There are too many generations of crap in the video. I suspect the original cap was done from an analog source via composite video. Then it went through a poor NTSC to PAL conversion. Normal dot crawl filters will not work because the dots have been messed up. I hope that's not a commercial DVD.

It is. This is exactly why we don't do DVD rips. Unfortunately, we can't get our hands on a better source. There is no blu-ray available for the early seasons.

[edDV]

edDV, I am reluctant to process the interlaced video as that usually makes it harder to deinterlace after. And I don't dig your analog jargon, I only have the DVD, this video was not captured from the TV.

The artifacts of dot crawl are in the image



If this was from a commercial DVD it had an extremely poor capture and conversion to 720x576. The analog jargon means dot crawl can be avoided with S-Video capture.

But most 80-90s animation was done in Korea and those chaps were too cheap to master to component D1. The ones I observed in Seoul were all using Sony 1" Type C in stop frame. Some were planning composite D2.

The production process described in http://en.wikipedia.org/wiki/South_Park#Animation
says stop frame was only used for the pilot and several shorts.

Construction paper and traditional stop motion cutout animation techniques were used in the original animated shorts and in the pilot episode.

After green light they switched to computer animation ...

When the show began using computers, the cardboard cutouts were scanned and re-drawn with CorelDRAW, then imported into PowerAnimator, which was used with SGI workstations to animate the characters.[46][48] The workstations were linked to a 54-processor render farm that could render 10 to 15 shots an hour.[46] Beginning with season five, the animators began using Maya instead of PowerAnimator.[74] The studio now runs a 120-processor render farm that can produce 30 or more shots an hour.[46]

That means componet RGB masters must exist for making the Blu-Ray version. But that begs the question when they changed rendering to >640x480 and 16:9?

[jagabo]

The oldest South Park episode all have dot crawl here on our cable service. Even on the HD channel (they're obviously just upscaled SD). I seem to recall the NTSC DVDs have the same dot crawl artifacts.

[Mephesto]

The analog jargon means dot crawl can be avoided with S-Video capture

These DVDs are my only source, how would I do an S-Video capture?

And yes we know they are incompetent fucks. The South Park movie, which I have on Blu-ray was produced purely digitally, yet for some reason they captured it from the 35mm master, full of noise, projector wobble and film damage all over the place. On top of that they seemed to use some shitty automated spot-remover which fucked up Cartman's cross-eyes a few times... and despite the filter there still are spots and scratches every here and there.

Not that this is new to me. Lots of BD "remasters" are self-satisfying jokes, many being worse quality than the DVD versions, which is why I always find myself or my homeboy (he's way better at video restoration than I) having to personally touch it up all the time.

[jagabo]

The analog jargon means dot crawl can be avoided with S-Video capture

These DVDs are my only source, how would I do an S-Video capture?

You can't. You would have to have the original analog tapes. And even those probably have the dot crawl in them. You might as well get some construction paper and recreate the episodes.

[edDV]

The S-video comment was intended for future analog captures. By the time the video reaches the DVD it is too late.

[poisondeathray]

I have tried all dot crawl filters, which either did nothing or badly smeared the rest of the picture when it did. It looks to me they weren't designed to fix porey dot crawl as large as what you see in the screenshot. Any ideas?

Did you apply the dot crawl filters first? They are supposed to be applied while still interlaced, before all other filters


A general approach is to use dot crawl filters, then bob, field deblend, then use +/- AA filters, awarpsharp to thin the lines and get rid of residual dot crawl, then use line darkeners , sharpeners

You might want to apply filters selectively (maybe different strengths in different sections), because the strengths required may be too damaging for your tastes (you have more room in animation)

"Cheap" animation like this is actually much easier to fix manually than live action, because of simple textures and frame similarities. It's easier to clone & motion track elements from neighboring frames into other frames





These suggestions are just meant as a starting point - of course you'll have to fiddle with the filters and settings, but it certainly reduces the remaining amount of manual work you have to do later

I didn't bother resizing to 720x480 in this example (probably should have since it was "srestored"), or with any other cleanup or color correction, but some of the "side effects" of some of the filters like QTGMC can be helpful like denoising

Sample video is attached below. This is a "heavy" filter example (so details are less) , but it gets rid of all the dot crawl. What I usually do is a "heavy" filtered , and a "light filtered" version, then you can mix & match frames (take most of the frames from the light filtered version) . There is an avisynth function called clipclop, or replaceframes from stickboy. If you want even better resluts, then use better restoration software or even after effects, where you can just limit the "heavy" filtered version more effectively to parts of frames instead of whole frames (e.g. circle the repairs, feathered masks), and you have other better tools to fix some of the other stuff

AssumeTFF()
TComb(fthreshL=8, fthreshC=10, othreshL=10, othreshC=12)
Checkmate()
QTGMC(preset="slower", sharpness=0.9)
Srestore()
Santiag(3,3)
Awarpsharp2(thresh=128, depth=10)
Toon(0.8)
Destripe(rad=2, offset=1, thr=32)
LSFmod(strength=50)

[Mephesto]

poisondeathray, a living neural-network machine like always. I couldn't recognize half those filters in your script. You kick ass. Your restored video looks ******* great.

Sad to say, I couldn't reproduce the results. I used the same script as yours except I had to manually load the awarpsharp2 plugin as it appeared to be conflicting with another one, but I couldn't remove the other one because Toon was dependent on it.

The chroma after-image artifact looks precisely as when I applied the dot crawl removers before deinterlacing, but even moving them below the QTGMC and Srestore, the artifact remained.

Script and screenshot:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\awarpsharp2\aWarpSharp.dll")

avisource("J:\spi.avi")

AssumeTFF()
TComb(fthreshL=8, fthreshC=10, othreshL=10, othreshC=12)
Checkmate()
QTGMC(preset="slower", sharpness=0.9)
Srestore()
Santiag(3,3)
Awarpsharp2(thresh=128, depth=10)
Toon(0.8)
Destripe(rad=2, offset=1, thr=32)
LSFmod(strength=50)

[poisondeathray]

What is your "spi.avi" ? Has it been processed in any way beforehand?

Did you do a linear encode (beginning to end of that segment)? or is that "chroma ghosting" screenshot taken randomly previewing the script? srestore can pick different choices if you start in different places - so you can get different results

Also note, srestore isn't perfect, so you're bound to get some remenants of blended fields in a few frames. If you start with the NTSC DVD version, at least that wouldn't happen




BTW this is a review of the 3rd season DVD, presumably NTSC version

Similar to the Simpsons early series (at some point I’ll stop mentioning the Simpsons I promise) these episodes were “mastered” in the video realm on equipment that was…well…NTSC. This means that built-into the picture are artifacts like scan-line aliasing and “twitter” on fine horizontal detail. You’ve even got dot-crawl on your screen for a few scenes here and there that must have taken a trip though composite-video land and back again.

http://www.hometheaterforum.com/t/162062/htf-review-south-park-the-complete-third-season

[jagabo]

What is your "spi.avi" ?

Hes, chroma blending is often caused by incorrect handling of interlaced YV12 in VirtualDub.

[Mephesto]

What is your "spi.avi" ?

South Park interlaced.

Has it been processed in any way beforehand?

No, converted to a lossless format. (first 140 frames deleted to not waste time processing that stupid disclaimer at the beginning when testing tools/avisynth scripts.)

Has it been processed in any way beforehand?Did you do a linear encode (beginning to end of that segment)? or is that "chroma ghosting" screenshot taken randomly previewing the script? srestore can pick different choices if you start in different places - so you can get different results

I did a linear encode, but the first 140 frames were trimmed which proved to be the problem. I just encoded that original MPG I sent you and there was no chroma ghosting.

What would be a safe way to avoid this when editing? Should I make sure whatever frames I edit out is a multiple of 25?

BTW this is a review of the 3rd season DVD, presumably NTSC version

Similar to the Simpsons early series (at some point I’ll stop mentioning the Simpsons I promise) these episodes were “mastered” in the video realm on equipment that was…well…NTSC. This means that built-into the picture are artifacts like scan-line aliasing and “twitter” on fine horizontal detail. You’ve even got dot-crawl on your screen for a few scenes here and there that must have taken a trip though composite-video land and back again.

http://www.hometheaterforum.com/t/162062/htf-review-south-park-the-complete-third-season

I know. This deformed dot crawl is only present on the first two seasons. My season 3 DVDs and above look fine.

Thanks alot for the help. My encoding buddy is gonna freak when he sees this.

[jagabo]

No, converted to a lossless format.

Lossless may not be lossless if a colorspace conversion took place. And in incorrect colorspace conversion will result in chroma blending between the two fields. Use DgIndex, build a d2v file, and use Mpeg2Sourc() to open the video.

[jmac698]

Nobody realizes dot crawl can be perfectly removed in animation? Just average the chroma from two adjacent frames of edges of non-moving areas. The result is exactly as svideo would look. The dots are exactly oppositive every frame.
Let me try..

[jmac698]

ugh that's horrible.. the pal conversion makes it tough

[Mephesto]

jagabo, I know. It's kind of a habit. I could've used Trim with avisynth, it's just that no matter how many years I've been using Avisynth, I can't subconsciously accept working with video from a command-line. My impulse is to Vdub. I'm a ******* unrehabilitated convict in this department.

jmac698, good luck. This dot crawl is about 4 pixels thick and there is plenty of it on moving areas.

[jagabo]

Nobody realizes dot crawl can be perfectly removed in animation? Just average the chroma from two adjacent frames of edges of non-moving areas.

Dot crawl artifacts are in the luma channel, not the chroma channel. PAL dot crawl artifacts don't seem to be exactly opposite in each pair of frames. I think they have a four frame cycle.

[darkdream787]

checkmate()

will fix that no problem.

Just make sure you put it before your deinterlacing part of the script.

[poisondeathray]

checkmate()

will fix that no problem.

Just make sure you put it before your deinterlacing part of the script.

No it won't ... This is an atypical source and has been poorly processed

[jagabo]

I thought "Hideous Dot Crawl" was the name of a cheesy horror flick.

[killerteengohan]

checkmate()

will fix that no problem.

Just make sure you put it before your deinterlacing part of the script.

No it won't ... This is an atypical source and has been poorly processed

I dont get it though, whats atypical? and also it looks like you said to use checkmate() or used it yourself in a script example above.

[poisondeathray]

checkmate()

will fix that no problem.

Just make sure you put it before your deinterlacing part of the script.

No it won't ... This is an atypical source and has been poorly processed

I dont get it though, whats atypical? and also it looks like you said to use checkmate() or used it yourself in a script example above.

Did you bother to look at the sample? You cannot determine the extent of dot crawl from looking at a single image. Download the video sample and see for yourself. It has gone through too many conversions (resizing, field blending, ntsc<=>pal conversions), so typical dotcrawl filters won't pick it up completely - in fact the op says he already tried many dot crawl filters

I said checkmate() alone won't fix it, and earlier I gave instructions to use typical dot crawl filters properly - i.e they must be applied before all other filters, while still interlaced - whereas many people don't do this way and complain they don't work.

[Dogway]

DotCrawl is b*** to fix, checkmate should work, although it creates artifacts ridden by checkmate's filtering strength, and sometimes as in your case it doesn't work at all. In my experience use dotcrawl removers with some intelligent masking (to limit the artifacting) and you can also use it in conjuction with a recent own find out; if you use mdegrain use it with a median prefilter, mdegrain will catch most of the dotcrawl as noise-that-must-be-temporal-denoised.