Help with Dot Crawl or whatever this is...

[low-fat-al]

I have researched and experimented til I am exhausted. So I am posting this here to see what real experts could help with this.

This is a VOB ripped from a commercial DVD. Here is the Media info in case that is needed in this:

Format : MPEG-PS
File size : 307 MiB
Duration : 5mn 34s
Overall bit rate mode : Variable
Overall bit rate : 7 689 Kbps

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Duration : 5mn 34s
Bit rate mode : Variable
Bit rate : 7 344 Kbps
Maximum bit rate : 9 600 Kbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.708
Stream size : 293 MiB (95%)


Here is a screenshot of what I am trying to fix.



I attached a 20 second sample of that VOB.

[poisondeathray]

You can use the usual dotcrawl filters (checkmate, tcomb, etc...), but they can be quite damaging at the strengths required to eliminate the dot crawl in this sample

One approach is to use a heavily filtered version and apply through an edgemask, so only the regions around edges are filtered preferentially , less damage to the other areas

I didn't address any of the other issues ; just suggesting one way you might approach the dotcrawl . As usual adjust to your tastes

main=MPEG2Source("VoulezVous[PAL][Sample].d2v")

main
checkmate
bicubicresize(480,360)
spline36resize(720,576)
lsfmod
filtered=last

main
mt_edge().mt_expand
mymask=last

mt_merge(main, filtered, mymask)

[low-fat-al]

That looks good! I will give that shot.

A little more information on the process I do.

I am taking the VOB... improving quality, cropping, resizing to square aspect (640x480), and exporting to CineForm. I mainly use VirtualDub and TMPGEnc Video Mastering Works 5. If I have a video with no real quality issue, I just use TMPGEnc... otherwise... VirtualDub.

with that in mind...
is there a reason for the resize on your script?

I usually do the cropping and resizing in VirtualDub instead of the AVS file.

[low-fat-al]

I tried it and it works great! I also use NeatVideo to clean up the grainy footage as well.

I will post a sample of a finished version and screenshot soon.

[low-fat-al]

Using the above script along with NeatVideo.

Before:


After:


I have also attached a 23 second AVI video sample. It is in CineForm Codec.

[mike20021969]

Judging by the images, I'm sure the end video looks pretty good.

But unfortunately I cant watch it due to some obscure video codec used. I can hear it though.

[sanlyn]

I can't see it, either. No use for Cineform on my 'puter.

Your original mpg Sample1 is progressive video encoded as interlaced ? ?

[low-fat-al]

The original video is interlaced.

CineForm codec is free. Google it.


I uploaded it in MPEG-2 codec in this post for you non-cineform users.

[sanlyn]

Mm, let me see if I understand. I refer to your post #1, "VoulezVous[PAL][Sample].mpg (21.10 MBz)". That video is progressive, encoded as interlaced. If you separate the fields in each frame, the top field and the bottom field are the same image, from the same moment in time, twice in every frame. If you play the video field-by-field, the same action is repeated, every 2 fields.

[low-fat-al]

Ok... but that wasn't my issue initially. It was the dot crawl. So what then per your observation? How do you get a true progressive video?

[sanlyn]

Sorry, low-fat-al, I had a PC repair client pull me away from the forum for a while. Back in a moment.

Progressive video improperly deimnterlacedn or encoded as interlaced often has subtle interlace artifacts, which I'll demonstrate when I return. But I managed to clean up some of it with this:

QTGMC(InputType=1,preset="faster",TR2=2)

Sorry for the delay. Be back in a moment.

[sanlyn]

I worked up a quick mkv from your original post <gasp> Seems every time I see an interesting thread, interruptions pop up. So I didn't have time to clean the mkv's borders as I would want. I learned much from poisondeathray's script (been trying to get a handle on masktools for months!). Your sample2.mpg was of interest because it illustrates why many people dislike NeatVideo. Complaints about fine detail destruction and plastic effects are common. I use it, too, but not the way it was used in sample2.mpg. And even NeatVideo needs a little prep beforehand.

The images are 1280x480. Click on 'em to display full-size.

sample2.mpg on the left, the mkv attached below on the right: the mpg wasn't prepped for some annoying artifacts/problems before hitting it with NeatVideo, so the effects are crushed and mottled darks, loss of fine skin contours and detail, rough edges, mosquito noise, chroma bleed, oversharpened edges that look blurred, oversaturation (blooming) and posterization effects.


Many darks are crushed beyond repair, but fixing levels helped rescue some of the dark stuff. Trying to remove 100% grain is not a good idea all video, as it destroys fine detail and contours, and looks rather unnatural or animated. The mkv frame on the right shows how much dark and inner detail is missing from sample2.mpg, which is starting to look like anime. This image and the one below it also show some serious halo and ringing effects from oversharpening, much of which are subdued in the mkv. Color blooming (oversaturation) is another problem, and looks worse on TV than on computer.



These images look darker in a browser than they do in a good player (I used VLC), and they will also be brighter on TV. The color balance is a matter of guesswork, but crushed blacks and day-glo chroma blooming are no-no's for digital video.

[sanlyn]

It took a while to clean up my very messy avs script, attached below. Thanks to poisondeathray for the dot crawl idea.

I took my NeatVideo noise sample from the blue right-hand side of frame 182. Here are the NV noise reduction settings I used, which are fairly typical in my projects. Some low-power cleanup was all that was needed after running the script:

[vhelp]

i found a cineform decoder download here --> http://cineform.com/gopro-cineform-decoder

direct link to decoder --> http://software.gopro.com/PC/GoProCineFormDecoders-1.2.0.127.zip

installed. the directshow will work fine in avisynth/virtualdub without problems since cineform is installed and being used.

they no longer go by neoplayer, its gopro now. anyway. all worked ok.

[mike20021969]

CineForm codec is free.

Wow. The video quality is awesome.

[2Bdecided]

Progressive video improperly deimnterlacedn or encoded as interlaced often has subtle interlace artifacts, which I'll demonstrate when I return.

The vast majority of 25p content in "PAL" countries is broadcast as 50i. DVDs get it right more often than not, but when the source is a convention analogue video tape format with 25p stored as 50i, it might not always be a great idea to encode it straight to 25p without some restoration or processing; the fields are sometimes misaligned.

You're right that any treatment of 25p as 50i can cause interlacing-like artefacts to appear. In the days of interlaced displays, this was barely visible. Most progressive TV displays will assume that content may be interlaced and so hopefully process it OK. It's only by viewing the content in a "forced progressive" mode (e.g. on a PC with deinterlacing disabled) where you may see the issues. They're far more visible when viewing a single frame as a still picture, but you can get obsessed with all kinds of faults that will never be noticed during normal viewing by analysing videos in this way.

Cheers,
David.

[sanlyn]

.....you can get obsessed with all kinds of faults that will never be noticed during normal viewing by analysing videos in this way.

Sometimes yes, sometimes no. Depends on the source, I'd say. The Cineform and the mpg display the same faults and processing glitches on my PC and both TV's, through hard drive and disc play. The extreme contrast and over sharpening look almost like anime, and one player had a problem with "flashing" frames (and it's not a cheap $30 Coby player). But that's the trend nowadays. I'd prefer something closer to standard that doesn't look this weird. But that's just moi.