VideoHelp Forum
+ Reply to Thread
Results 1 to 26 of 26
Thread
  1. Hi, I've recently picked up encoding but I've encounter a dot crawl issue I can't seem to fix. I've added samples at the end of the post. More or less as the guy gets up you can clearly see dot crawl along the bottom of his shirt and a bit on his right sleeve. I've tried just about every filter I can think of (DeDot, LUTDeCrawl, Checkmate, DeCrawl, TComb), some in conjunction with others, and nothing seems to work. The source is the NTSC DVD. I'm currently using the following for my script:

    Code:
    TComb(mode=0,fthreshC=28,othreshC=29)
    AnimeIVTC(mode=1)
    ChubbyRain2(th=20, radius=10, show=false, sft=23, interlaced=false)
    crop(10, 0, -6, 0)
    Spline36Resize(704,480) # Spline36 (Neutral)
    Undot() # Minimal Noise
    TTempSmooth()
    SharpAAMCmod(aatype="EEDI2")
    TComb does a good job cleaning up most of the dot crawl but it's not good with movement and I'm having a tough time finding a combination that is without a huge loss of detail. ChubbyRain2 took care of all the rainbows that were present something TComb seemed unable to do no matter what settings I used. I have an encode of this done years ago from someone else and the dot crawl is not present on their encode; somehow they got it out (mind you they ruined the color in quite a number of frames).

    Any help/suggestions would be much appreciated!
    Last edited by Elegant; 22nd Jul 2014 at 15:24. Reason: Removed dead links.
    Quote Quote  
  2. I'm a Super Moderator johns0's Avatar
    Join Date: Jun 2002
    Location: canada
    Search Comp PM
    Is there dot crawl on the original dvd?Dot crawl is caused by capturing with composite connections.
    I think,therefore i am a hamster.
    Quote Quote  
  3. There's one sure fire way to remove dot crawl but you probably won't want to use it on a sharp source like this (it's fine for VHS caps): reduce the width by half then scale back to the original width:

    Code:
    Spline64Resize(width/2,height)  # half width
    nnedi3_rpow2(2, cshift="Spline64Resize", fwidth=width*2, fheight=height) # back to full width
    Sharpen(0.3,0.0) # a little more sharpening
    Quote Quote  
  4. Dot crawl is caused by residual color subcarrier in the video and would show most at the borders of different colors. I think your problem is more to do with lack of resolution. When the borderline is almost horizontal the number of vertical pixels it crosses is small and the steps become more visible. You can see the same effect on the headband and other near horizontal contrast changes.

    A dot crawl filter will try to reduce transitions at colour subcarier rate (3.579MHz or 4.433MHz) and the ones you are seeing are at much lower frequencies.

    Brian.
    Quote Quote  
  5. Originally Posted by johns0 View Post
    Is there dot crawl on the original dvd?Dot crawl is caused by capturing with composite connections.
    Yep, this particular boxset seems to have quite a bit of it.

    Actually I should note the video included is with the avs script.

    @jagabo, this does indeed resolve the issue but is it necessary to sharpen it further given that i'm using SharpAAMCmod?
    Last edited by Elegant; 22nd Jul 2014 at 15:24. Reason: Removed dead links.
    Quote Quote  
  6. Originally Posted by Elegant View Post
    @jagabo, this does indeed resolve the issue but is it necessary to sharpen it further given that i'm using SharpAAMCmod?
    The downscale/upscale leaves the video blurrier than when it started so I usually sharpen a little on the horizontal axis afterward. You can try leaving out the Sharpen() and see if SharpAAMCmod() can make up for it.

    But notice how small details are blurred away. For example, at the jagged hairline on the character's neck. I'm not sure this is a good tradeoff given how light the residual dot crawl is (the video was obviously passed through a careful 3d comb filter when captured).
    Quote Quote  
  7. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by Elegant View Post

    Actually I should note the video included is with the avs script.
    Pretty obvious, since it's not mpeg2

    Why don't you upload a native sample ? You might get some other suggestions or approaches
    Quote Quote  
  8. @poisondeathray Fair point, I'll post it in here and in my OP.

    @jagabo Yeah I can see exactly what you're talking about now. I zoomed in on other areas like his shoes and can clearly see his laces have blended together XD
    Last edited by Elegant; 22nd Jul 2014 at 15:25. Reason: Removed dead links.
    Quote Quote  
  9. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Originally Posted by Elegant View Post
    @poisondeathray Fair point, I'll post it in here and in my OP. Sample clip from source
    Your NTSC DVD source is mkv's?
    - My sister Ann's brother
    Quote Quote  
  10. I'm a Super Moderator johns0's Avatar
    Join Date: Jun 2002
    Location: canada
    Search Comp PM
    So basically you have a dvd with mkv on it,not an authored dvd-video,sound like a captured vhs to avi then converted to mkv.
    I think,therefore i am a hamster.
    Quote Quote  
  11. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    It's MPEG2 in the 2nd MKV . He probably used makemkv , or mkvmerge to cut it. 1st one had AVC, that' s what I was referring to - the compression, not the container
    Quote Quote  
  12. I can convert it to avi/vob and link that instead but it's still MPEG2. The source is the official DVD not a bootleg. I actually used Avidemux and saved it as an mkv, kind of my go to output, I can change it around.
    Last edited by Elegant; 22nd Jul 2014 at 15:22.
    Quote Quote  
  13. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    i was about to post that I looked in more detail and found the mkv was plain ol' MPEG2 in an mkv box, but -- oops! -- you beat me to it.

    On the other hand, DDGindex can get a quickie .m2v lickety-split, without a costume change. But you've provided more than enough.
    - My sister Ann's brother
    Quote Quote  
  14. It looks like TemporalDegrain just destroyed all the dot crawl I had. It might not be the best for encode times but it's GONE. This also eliminates the point of TComb, in fact leaving it on produces some rainbows.
    Last edited by Elegant; 18th Jul 2014 at 12:21.
    Quote Quote  
  15. What settings did you use in TemporalDegrain? The default settings removed lots of the spots and fine detail on the ground when the camera panned.
    Last edited by jagabo; 19th Jul 2014 at 05:16.
    Quote Quote  
  16. I was using the default one. Yeah, I came across those issues after a couple episodes of playback. I was experimenting with AAA() as well. It gets the job done but I do feel that resizing is not an appropriate solution. Upon further search I came across the following:

    Code:
    input=last
    mask=input.mt_edge("sobel",7,7,5,5).mt_inflate()
    aa_clip=input.Lanczos4Resize(width(input)*2,height(input)*2).TurnLeft().SangNom().TurnRight().SangNom().Lanczos4Resize(width(input),height(input)).MergeChroma(input) 
    mt_merge(input,aa_clip,mask)
    It works, though I am probably going to use Spline36Resize as it feel more comfortable with it. Apart from that though the mask seems to do an excellent job at resolving the issue.

    I'm still going to dig through TemporalDegrain though (not so much using it but figuring out what section of it I need). Below I have a screenshot (from the source) where one of the guys is rolling but you can clearly see some kind of crawl-like-effect in his orange hair as he is on the ground. TemporalDegrain removed that so I need to figure out what section specifically removed it.
    Last edited by Elegant; 22nd Jul 2014 at 15:22.
    Quote Quote  
  17. Thanks for the SangNom technique.
    Quote Quote  
  18. No problem, this technique is also part of the AnimeIVTC pack with the function name maa() and a faster version known as maa2(). Funny how I found out about this then found it sitting in the IVTC script I was already using XD
    Last edited by Elegant; 20th Jul 2014 at 00:48.
    Quote Quote  
  19. Although maa2() solves most of the dotcrawl on black lines having dotcrawl on other regions seems near impossible to fix. Sample clip and screenshot of trouble area. The new script I'm using at the moment helps to reduce that area but I can't seem to remove it altogether.

    Code:
    TComb(mode=0)
    Checkmate(thr=0, max=1, tthr2=5)
    AnimeIVTC(mode=1)
    crop(10, 0, -6, 0)
    Undot()
    maa2()
    MergeChroma(RemoveDirtMC,0.80)
    ChubbyRain2()
    DDComb()
    TTempSmooth()
    LSFmod(strength=100)
    Quote Quote  
  20. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Less dot crawl than it is aliasing, changes only every couple of frames or so. http://forum.doom9.org/showpost.php?p=1687778&postcount=11
    Attached Files
    Last edited by LMotlow; 23rd Jul 2014 at 22:04.
    - My sister Ann's brother
    Quote Quote  
  21. That has the same loss of small detail problem that my earlier downscale/upscale script had.
    Before/After (4x PointResize):
    Name:  zoom.jpg
Views: 94
Size:  24.0 KB
    Quote Quote  
  22. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Originally Posted by jagabo View Post
    That has the same loss of small detail problem that my earlier downscale/upscale script had.
    Before/After (4x PointResize):
    Of course. Well, not quite. I wonder what people expected? My take on the O.P.'s source: analog tape captured to DV. Even if that guess isn't correct, that's sure what it looks like IMO. Loss is always the result you'll get with (a) that type of processing, or (b) that type of bad edges, whatever caused it. The resizing and aa make for some grayed-out, thicker, fuzzy lines that I thinned out. But the other scripts with 11 plugins and over smoothing wipes out a lot of background stuff, the image starts looking like soft plastic, you end up with banding. and you still have the aggravation you started with.

    If resizing is that bad, don't use maa2.
    Last edited by LMotlow; 24th Jul 2014 at 04:17.
    - My sister Ann's brother
    Quote Quote  
  23. Yes, filtering to remove heavy artifacts is always a compromise. Replacing CheckMate() with TComb() in Elegant's script will eliminate most of the buzzing edges without blurring small details (though it does leave most of the dot crawl on the explosion spikes on the guys head). He then might lighten up on the other filtering.
    Last edited by jagabo; 24th Jul 2014 at 07:20.
    Quote Quote  
  24. The stubborn dot crawl on the "rays" are hard to remove with the traditional dot crawl filters because the rays flicker off and on with each frame. It occurred to me that a motion mask might help with that. So I used mt_masktools to create a motion mask to select between lightly filtered (maa2) and heavily filtered (downsize.santiag.upsize) videos:

    Code:
    Mpeg2Source("D:\Downloads\Sample(1).d2v", CPU2="ooooxx", Info=3) 
    TComb()
    AnimeIVTC(mode=1)
    
    light = maa2()
    heavy = Spline36Resize(width/2,height).Santiag().Spline36Resize(width,height)
    mmask = mt_motion(thY1=10, thY2=10).mt_expand().GreyScale().Blur(1.0)
    Overlay(light, heavy, mask=mmask)
    
    StackVertical(StackHorizontal(light,heavy), StackHorizontal(mmask,last))
    So areas where there isn't motion (where dot crawl filters work well) get the lightly filtered video, areas where there is motion (where dot crawl filters don't work well) get the more heavily filtered video. That worked pretty well on the short sample. The dot crawl on the rays is almost gone, and the small detail on the guys thigh (pictured earlier) is retained. A sample showing the lightly filtered video at the top left, the heavily filtered video at the top right, the motion mask at the lower left, and the final result at the lower right, is attached. The OP should check to verify this doesn't screw up other parts of the video. Then he can fine tune the mask and filtering.
    Attached Files
    Last edited by jagabo; 24th Jul 2014 at 09:33. Reason: clarification
    Quote Quote  
  25. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Sounds like a good idea, jagabo. Clever. The mkv seems a good compromise. Thanks for that tip.
    - My sister Ann's brother
    Quote Quote  
  26. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    If there are only a few problem areas on a few frames, and are concerned about the detail loss you can do very light filtering and some manual edits, or some combination. Depends on how much effort you want to put in to fix it and retain as much detail as possible
    Quote Quote